API Reference

Auto-generated from docstrings on the workspace's public Python surface. Source of truth: each package's __init__.py (__all__) and the modules referenced below.

Core schemas and exceptions

openral core schemas and exceptions — Pydantic v2 contracts shared across all layers.

Public surface is everything imported here. Private modules use leading underscore.

BenchmarkName: TypeAlias = Literal['aloha_insertion', 'aloha_transfer_cube', 'gr1_tabletop', 'libero_10', 'libero_goal', 'libero_object', 'libero_spatial', 'maniskill3_franka_pick_cube', 'maniskill3_pick_place', 'metaworld_mt50', 'pusht', 'robocasa_pnp', 'simpler_env_widowx'] module-attribute

Canonical benchmark suite ids — one per benchmarks/<id>.yaml in tree.

Used as keys in RSkillManifest.benchmarks. Each value is the headline success rate [0.0, 1.0] the skill achieves on that suite. The full breakdown lives in the matching rskills/<id>/eval/<key>.json.

CollisionShape: TypeAlias = CapsuleShape | SphereShape module-attribute

Discriminated union of convex collision primitives (ADR-0030).

The discriminator field is shape. Used by :class:LinkCollisionGeometry (robot links) and :class:WorldCollisionPrimitive (world obstacles). Mesh primitives are intentionally excluded — the allocation-free safety kernel checks only convex analytic shapes; mesh-accurate collision stays a planning-layer concern.

EmbodimentTag: TypeAlias = Literal['aloha', 'custom', 'franka_panda', 'g1', 'google_robot', 'gr1', 'h1', 'mobile_base', 'openarm', 'panda_mobile', 'pusht', 'rizon4', 'sawyer', 'so100_follower', 'so101_follower', 'ur10e', 'ur5e', 'widowx'] module-attribute

Canonical embodiment tags — one per robots/<id>/robot.yaml shipped in tree, plus "custom" as the explicit "I know what I'm doing" escape hatch.

"mobile_base" is a CLASS tag (not a specific robot): any robot with a planar base + body_twist actuator declares it so base-only rSkills (Nav2 NavigateToPose, etc.) can target the whole class without naming each specific mobile platform. Robot-specific tags (e.g. "panda_mobile") coexist on the same RobotDescription for skills that DO depend on the specific composition.

The RSkillManifest.embodiment_tags field is restricted to this set so a typo or framework hint (lerobot, libero) cannot land in a manifest where the loader's compat check would silently never match. When "custom" is used, the manifest MUST also populate embodiment_extra (see :class:EmbodimentExtra); the cross-validator on :class:RSkillManifest enforces this. ADR-0013.

FailureEvidence: TypeAlias = TimeoutEvidence | ForceEvidence | WorkspaceEvidence | PerceptionStaleEvidence | CriticEvidence | ControllerEvidence | SelfVerifyEvidence | HumanEvidence | WamEvidence | ReasonerTimeoutEvidence | CollisionEvidence | SuppressedSummaryEvidence module-attribute

Discriminated union for FailureTrigger.evidence_json payloads.

The discriminator field is kind (a string Literal on each variant). Consumers decode an incoming evidence_json string with::

from pydantic import TypeAdapter
from openral_core import FailureEvidence

evidence = TypeAdapter(FailureEvidence).validate_json(msg.evidence_json)

Producers serialize via evidence.model_dump_json().

The union mirrors the KIND_* constants on openral_msgs/msg/FailureTrigger (ADR-0018 F3). Adding a new variant requires adding a new KIND_* constant to the IDL.

GripperConvention: TypeAlias = Literal['normalized_open_unit', 'normalized_open_symmetric', 'binary_close_one', 'raw_joint_rad', 'width_meters'] module-attribute

Per-skill gripper action encoding (Gap 2 of the rSkill self-containment audit).

Required on :class:ControlModeSemantics whenever the parent :class:ActuatorRequirement.kind is :attr:ControlMode.GRIPPER_BINARY or :attr:ControlMode.GRIPPER_POSITION — silently mis-encoding the gripper slot between scenes / robots is a top observed mis-actuation failure.

Conventions:

• normalized_open_unit — 0.0 = fully closed, 1.0 = fully open. Most lerobot / SmolVLA / pi0.5 LIBERO checkpoints.
• normalized_open_symmetric — -1.0 = fully closed, +1.0 = fully open. Some MetaWorld checkpoints.
• binary_close_one — 0.0 = open, 1.0 = close (single bit, RoboCasa-style).
• raw_joint_rad — raw per-finger joint angle in radians (Fourier dexhands, some humanoid skills).
• width_meters — physical gripper width in metres (Franka FCI native).

JointRole: TypeAlias = Literal['arm', 'base', 'gripper', 'torso', 'leg', 'head', 'neck', 'wheel', 'unknown'] module-attribute

Structural classification of a :class:JointSpec (ADR-0028a).

Carries the joint's purpose in the embodiment's morphology — what the runner, safety kernel, and dataset bridge need to identify a channel without relying on name-substring heuristics (e.g. "gripper" in name.lower() in rskill_runner_node._build_joint_permutation which silently misclassifies any joint with "gripper" in the name).

"unknown" is the default so legacy manifests load unchanged; the fleet annotates incrementally as ADR-0028 sub-PRs land.

ModelFamily: TypeAlias = Literal['smolvla', 'pi05', 'xvla', 'act', 'diffusion', 'rldx', 'molmoact2', 'gr00t'] module-attribute

VLA / policy family the skill belongs to.

Used by the eval / runner adapters to dispatch to the right openral_sim.adapters.<family> policy adapter without string-matching the skill name. Adding a family here means landing the matching adapter under python/sim/src/openral_sim/adapters/.

gr00t (NVIDIA Isaac GR00T N1.x / N2) runs out-of-process via a ZMQ sidecar in an isolated Python 3.10 venv, sharing the architecture of the rldx adapter (RLDX-1 is itself a GR00T-N1.5 finetune) — see ADR-0046.

PerceptionEventMetadata: TypeAlias = MotionMetadata | ObjectsMetadata | OcrMetadata | SceneChangeMetadata module-attribute

Discriminated union for PromptStamped.metadata_json payloads on the /openral/perception/<kind> topics (ADR-0018 F6).

The discriminator field is kind (a string Literal on each variant). Consumers decode an incoming metadata_json string with::

from pydantic import TypeAdapter
from openral_core import PerceptionEventMetadata

metadata = TypeAdapter(PerceptionEventMetadata).validate_json(msg.metadata_json)

Producers serialise via metadata.model_dump_json().

The four variants map 1:1 onto the four per-kind topics fixed in ADR-0018 §3 (motion, objects, ocr, scene_change). Adding a new variant requires adding a new /openral/perception/<kind> topic to the contract — by design, new kinds get new topics, not a schema bump, so subscribers can subscribe to exactly the kinds they care about.

RSkillKind: TypeAlias = Literal['vla', 'wam', 'ros_action', 'ros_service', 'detector', 'vlm', 'reward'] module-attribute

Discriminator selecting how an rSkill is instantiated at the loader.

• "vla" — learnable Vision-Language-Action policy. Requires :attr:RSkillManifest.model_family and :attr:RSkillManifest.weights_uri; resolved by the policy adapter dispatch in openral_rskill (the pre-existing path used by every in-tree rSkill prior to this discriminator landing).
• "wam" — World Action Model (planning-layer mental-simulation / failure-anticipation component per CLAUDE.md §3). Reserved so the discriminator is forward-compatible; the loader / runner branch is not implemented yet and kind: wam manifests are rejected at resolve time with :class:~openral_core.exceptions.ROSConfigError.
• "ros_action" — wraps an existing ROS 2 action server. Requires a :class:RosIntegration block. model_family and weights_uri are forbidden. chunk_size is pinned to 1 so the safety supervisor's per-row check sees every commanded position.
• "ros_service" — wraps an existing ROS 2 service. Same constraints as "ros_action".
• "detector" — perception producer that runs an exported detection model (RT-DETR / D-FINE ONNX) on the camera tee and publishes :class:~openral_core.schemas.ObjectsMetadata; emits no :class:~openral_core.schemas.Action. Requires a :class:DetectorContract block and :attr:RSkillManifest.weights_uri (the exported ONNX / TensorRT engine). model_family and action_contract / state_contract are forbidden. ADR-0037.
• "vlm" — vision/video-language model used as a scene-understanding perception component (e.g. Qwen3.5-4B NF4). Accepts RGB image or video frames and a natural-language query; returns a text answer. Emits no actions or bounding boxes. Runs at S2 rate (role: "s2"), so it is surfaced to the reasoner as a read-only scene-query tool, never as an ExecuteSkill policy. weights_uri REQUIRED; actuators_required MUST be empty; action_contract, state_contract, detector, ros_integration, processors, image_preprocessing, n_action_steps, and starting_pose are FORBIDDEN. model_family is OPTIONAL metadata. ADR-0047.

ReasonerToolCall: TypeAlias = ExecuteRskillTool | ReloadGstPipelineTool | LifecycleTransitionTool | EmitPromptTool | RecallObjectTool | ResolvePlaceTool | LocateInViewTool | QuerySceneTool | QueryTaskProgressTool module-attribute

Discriminated union over the reasoner tool variants (ADR-0018 §4; ADR-0039).

The discriminator field is tool (a string Literal on each variant). Consumers decode an LLM tool-use payload with::

from pydantic import TypeAdapter
from openral_core import ReasonerToolCall

call = TypeAdapter(ReasonerToolCall).validate_json(payload)

Producers (LLM clients) serialise via call.model_dump_json().

The first four variants are the actuation/effect palette ADR-0018 §4 commits to. ADR-0039 adds two read-only query variants — :class:RecallObjectTool and :class:ResolvePlaceTool — that only read the ADR-0038 spatial memory (no actuation authority). Extending the palette requires (a) a new variant here, (b) the corresponding ROS-side dispatch in openral_reasoner_ros.reasoner_node, (c) a CLAUDE.md §6.2 / §7.6 amendment if the new tool shifts the reasoner's authority surface. The two query variants' dispatch + result-return path is ADR-0039 Phase 2; until then they are a typed contract not yet exposed in the live provider palette.

StateLayout: TypeAlias = Literal['smolvla_9d', 'human300_16d', 'gr1', 'rc365', 'simpler_widowx', 'simpler_google'] module-attribute

Closed set of per-checkpoint proprioception layouts. ADR-0014 + ADR-0027.

A layout names the SHAPE the checkpoint was trained on — field order, frame convention, gripper encoding, quaternion handedness. The per-robot SOURCE bindings (which TF frame is "the EE", which joint names are "the gripper") live on :class:StateContractBindings. The openral_state_adapter registry maps each literal to an assembler function that joins shape + bindings + live JointState + live TF.

WRAPPED_TASK_SPACE_LAYOUTS: frozenset[StateLayout] = frozenset({'rc365', 'human300_16d'}) module-attribute

Layouts that are TASK-space composites (Cartesian poses + gripper widths), NOT one-scalar-per-joint. These layouts REQUIRE :attr:StateContract.bindings to name the source TF frames + JointState entries — the cross-validator on :class:StateContract enforces this at manifest load. The remaining layouts (smolvla_9d, gr1, simpler_*) are joint-space slices (potentially across multiple controller groups, as in GR1's 29-D waist+arms+hands composite) that the runner serves verbatim from observation.joint_state.position; a robot.yaml with the matching joint count dispatches them without an assembler.

Action

Bases: BaseModel

A single action step or chunk produced by a Skill.

Attributes:

Name	Type	Description
control_mode	ControlMode	Target action space.
horizon	int	Number of steps (1 = single step, H = chunk).
joint_targets	list[list[float]] | None	Joint position targets, shape (H, N).
joint_velocities	list[list[float]] | None	Joint velocity targets, shape (H, N).
joint_torques	list[list[float]] | None	Joint torque targets, shape (H, N).
cartesian_pose	list[Pose6D] | None	EE pose targets.
cartesian_delta	list[tuple[float, ...]] | None	EE pose deltas.
cartesian_twist	list[tuple[float, ...]] | None	EE velocity targets.
body_twist	list[tuple[float, float, float, float, float, float]] | None	Base twist targets.
foot_placements	list[dict[str, object]] | None	Discrete footstep targets.
gripper	list[float] | None	Gripper commands in [0, 1].
dex_hand_joints	list[list[float]] | None	Dexterous hand joint targets.
confidence	float	rSkill confidence in [0, 1].
stamp_ns	int	Action timestamp in nanoseconds.
ee_name	str | None	Target end-effector name.
frame_id	str | None	Reference frame for Cartesian actions.
safety_overrides	dict[str, object]	Operator-approved safety override tokens.

ActionContract

Bases: BaseModel

Per-rSkill action-vector contract (ADR-0019 PR-revert).

Mirrors :class:StateContract for the action side. Carries the output dimensionality the checkpoint emits so the dataset bridge (and any downstream consumer) can bind the LeRobot v3 action feature shape without consulting the sim or hardware adapter.

Per ADR-0007, the sim-specific action contract belongs on the per-checkpoint rSkill manifest, not on the physical :class:RobotDescription (the same Franka emits 7-D delta-EEF on LIBERO vs 8-D joint pos on a hardware deploy).

Attributes:

Name	Type	Description
dim	int	Dimensionality of the action vector emitted by Skill.step() / PolicyAdapter.step(). Required when this contract is set.
representation	ActionRepresentation | None	Optional named representation (mirrors :attr:ActionSpec.representation). When set, downstream consumers can map between equivalent representations (e.g. joint_positions → delta_ee_6d_plus_gripper).
slots	list[ActionSlot] | None	ADR-0028b — declarative slot layout. When set, every index in [0, dim) must be covered by exactly one :class:ActionSlot (no gaps, no overlaps). The skill_runner reads this to dispatch slices of the policy vector onto typed :class:Action objects. When None, the runner falls back to the legacy single-Action path (one implicit JOINT_POSITION slot covering the whole vector). Manifests carrying slots are exempt from the ADR-0028a dim <= len(robot.joints) invariant because the slot decoder gives a typed contract per slice instead.

ActionRepresentation

Bases: str, Enum

Action vector representation format.

ActionSlot

Bases: BaseModel

One contiguous slice of an rSkill's action vector (ADR-0028b).

The skill_runner reads ActionContract.slots and emits one typed :class:Action per non-discard slot per step. All actions inherit the parent step's trace_id so the safety supervisor and downstream telemetry can join them post-hoc.

Attributes:

Name	Type	Description
range	tuple[int, int]	Inclusive [start, end] indices into the flat policy action vector. range[0] must be ≤ range[1]; both must fall within [0, ActionContract.dim).
control_mode	ControlMode | None	The :class:ControlMode the slice is routed to. The HAL whitelist on the target robot must include this mode (the palette filter rejects the rSkill at install time otherwise). None only when :attr:discard is True.
discard	bool	When True the slice is dropped silently — used for dataset artefacts like RoboCasa365's torso placeholder dim or paired gripper channels. The slot still occupies its range so coverage validation works; no :class:Action is emitted.
ee	str | None	End-effector name from the robot's :attr:RobotDescription.end_effectors / :attr:RobotDescription.joints. REQUIRED for CARTESIAN_* modes (the pose is computed in the named EE's frame) and for GRIPPER_* modes (names the actuator). FORBIDDEN for BODY_TWIST and JOINT_POSITION.
frame	str | None	tf2 frame name. REQUIRED for cartesian + body-twist modes (the slice's bytes are expressed in this frame). FORBIDDEN for joint-position and gripper modes.
joint_names	list[str]	Robot joint names this slice targets, in slot order. REQUIRED for JOINT_POSITION / JOINT_VELOCITY / JOINT_TORQUE when the slot covers fewer than all robot joints; FORBIDDEN for non-joint modes. Length must equal range[1] - range[0] + 1.

ActionSpec

Bases: BaseModel

VLA action configuration for this robot.

Attributes:

Name	Type	Description
dim	int	Dimensionality of the action vector.
representation	ActionRepresentation | None	How the action vector is encoded.
control_freq_hz	float | None	Control frequency the actions are executed at.
chunk_size	int | None	Number of action steps per inference call (chunk size H).

ActuatorRequirement

Bases: BaseModel

One actuator slot an rSkill emits actions for.

Symmetric with :class:SensorRequirement on the action side. The compatibility check resolves each entry against a :class:RobotDescription's action_spec and rejects pairings whose declared :class:ControlMode is not advertised by the robot.

For predefined embodiments (one of the 9 canonical robots/<id>/robot.yaml slugs), n_dof and vla_action_key are optional — the loader auto-fills them from the robot YAML at compatibility-check time. For the "custom" embodiment escape hatch they MUST be set on the manifest; the :class:RSkillManifest cross-validator enforces this.

Attributes:

Name	Type	Description
kind	ControlMode	The action interface the skill emits (e.g. ControlMode.JOINT_POSITION, ControlMode.CARTESIAN_DELTA). Reuses the canonical ControlMode enum so the manifest, the safety kernel, Action.control_mode, and RobotDescription.action_spec all share one source of truth.
n_dof	int | None	Optional degrees of freedom the skill emits. Auto-filled from the robot YAML for predefined embodiments; REQUIRED on the manifest when "custom" is in RSkillManifest.embodiment_tags.
vla_action_key	str | None	Optional slot name the policy emits, e.g. "action.joints.arm_left". Auto-filled for predefined embodiments; REQUIRED on the manifest when "custom" is in RSkillManifest.embodiment_tags.
control_mode_semantics	ControlModeSemantics	Action-space semantics for this slot. REQUIRED — closes Gap 2 of the rSkill self-containment audit. Cross-validators enforce: gripper kinds require gripper_convention; cartesian kinds require reference_frame; other kinds forbid both.

Example

a = ActuatorRequirement( ... kind=ControlMode.JOINT_POSITION, ... control_mode_semantics=ControlModeSemantics(mode="absolute"), ... ) a.kind is ControlMode.JOINT_POSITION True a.n_dof is None and a.vla_action_key is None True

ApproachViewpoint

Bases: BaseModel

A camera-facing standoff pose for viewing/manipulating an object (ADR-0038 §6).

Attributes:

Name	Type	Description
pose	Pose6D	Base/EE goal pose (map frame) at a standoff from the object, oriented so the gripper-mounted camera faces it.
standoff_m	float	Standoff distance from the object, in metres.
camera_frame_id	str	tf2 frame of the camera the viewpoint orients toward.

AssetRefs

Bases: BaseModel

The unified description-asset block on :class:RobotDescription.

ADR-0058 §4. Replaces the scattered urdf_path / mjcf_uri / srdf_path (+ ADR-0027 URDF-root fields) with one block whose refs share the :func:openral_core.assets.resolve_asset grammar.

Attributes:

Name	Type	Description
urdf	UrdfAsset | None	URDF asset (with optional robot_state_publisher wiring), or None when the robot ships no URDF.
mjcf	str | None	MJCF asset ref (MuJoCo wiring), or None.
srdf	str | None	SRDF asset ref whose disable_collisions block seeds :attr:RobotDescription.allowed_collision_pairs, or None.

Example

AssetRefs(urdf=UrdfAsset(ref="rd:panda_description")).urdf.ref 'rd:panda_description'

BenchmarkMetadata

Bases: BaseModel

Provenance block required on every BenchmarkScene.

paper / honest_scope are the published-protocol citation + a one-sentence "what this eval actually measured" statement.

display_name / simulator are optional paper-comparison labels (ADR-0042) that surface into RSkillEvalResult.benchmark when present — display_name becomes benchmark.name and simulator becomes benchmark.simulator. Pre-ADR-0042 these lived in a free-form dict on the deleted BenchmarkSpec; moving them per-scene keeps them with their provenance and lets the aggregator emit identical JSON whether driven by run_benchmark (suite) or run_benchmark_scene (single scene).

BenchmarkScene

Bases: SimScene

Full benchmark eval spec for openral benchmark.

n_episodes, seed, and metadata are required with no defaults — they must match the published evaluation protocol. The task must supply success_key and max_steps.

from_yaml(path: str) -> BenchmarkScene classmethod

Load and validate a BenchmarkScene YAML from disk.

Source code in python/core/src/openral_core/schemas.py

@classmethod
def from_yaml(cls, path: str) -> BenchmarkScene:
    """Load and validate a ``BenchmarkScene`` YAML from disk."""
    import yaml  # noqa: PLC0415

    with open(path, encoding="utf-8") as fh:
        data = yaml.safe_load(fh)
    return cls.model_validate(data)

CapsuleShape

Bases: BaseModel

Capsule collision primitive — a segment swept by a radius.

The central segment runs along the local +Z axis from -length_m / 2 to +length_m / 2 (the MJCF / URDF capsule convention); it is placed and oriented by the owning frame (:attr:LinkCollisionGeometry.origin_xyz_rpy for a link, :attr:WorldCollisionPrimitive.pose for an obstacle). Capsules bound most robot links tightly, so the safety check stays conservative (ADR-0030 §2).

Attributes:

Name	Type	Description
shape	Literal['capsule']	Discriminator (always "capsule").
radius_m	float	Capsule radius in metres.
length_m	float	Length of the central segment in metres (the cylinder portion; the total span is length_m + 2 * radius_m). 0.0 degenerates to a sphere.

Example

CapsuleShape(radius_m=0.04, length_m=0.3).length_m 0.3

CollisionEvidence

Bases: _FailureEvidenceBase

Evidence for KIND_COLLISION — a proposed motion would collide (ADR-0030).

Attributes:

Name	Type	Description
kind	Literal['collision']	Discriminator (always "collision").
collision_kind	Literal['self', 'world']	"self" (link vs link) or "world" (link vs obstacle).
link_a	str	Robot link whose collision volume tripped the check.
link_b_or_object	str	The other robot link (self-collision) or the world object / occupancy region (world-collision).
horizon_step	int	Chunk step (horizon index) where the collision was first detected.
min_distance_m	float	Signed minimum distance at detection, in metres (negative means interpenetration).

ControlMode

Bases: str, Enum

Action space / control interface exposed by a robot or skill.

ControlModeSemantics

Bases: BaseModel

Action-space semantics declared on each :class:ActuatorRequirement.

Closes Gap 2 of the rSkill self-containment audit: kind alone ("joint_position") doesn't say whether the policy emits absolute targets or per-step deltas, in which joint order, in which reference frame, or how it encodes the gripper. Today these are assumed at the adapter level and silently broken when porting a skill between embodiments.

The block is REQUIRED on every actuator entry of a manifest. For the canonical embodiments the loader still auto-fills n_dof / vla_action_key from the robot YAML; semantics must be declared explicitly on the manifest because they are a property of the trained checkpoint, not the robot.

Attributes:

Name	Type	Description
mode	Literal['absolute', 'delta']	Whether the action vector represents absolute targets (joint positions, end-effector pose) or deltas from the current state. Required.
gripper_convention	GripperConvention | None	Encoding of the gripper slot in the action vector. Required when the parent :attr:ActuatorRequirement.kind is :attr:ControlMode.GRIPPER_BINARY or :attr:ControlMode.GRIPPER_POSITION; forbidden otherwise.
joint_order	list[str] | None	Ordered list of joint names matching the channels the policy emits. Optional for canonical embodiments (the robot YAML's :attr:ActionSpec.joint_names is the source of truth); REQUIRED for "custom" embodiments.
reference_frame	str | None	tf2 frame the action is expressed in. Required when the parent :attr:ActuatorRequirement.kind is one of :attr:ControlMode.CARTESIAN_POSE, :attr:ControlMode.CARTESIAN_DELTA, or :attr:ControlMode.CARTESIAN_TWIST; forbidden otherwise.

Example

sem = ControlModeSemantics(mode="absolute") sem.mode 'absolute' sem.gripper_convention is None and sem.reference_frame is None True

ControllerEvidence

Bases: _FailureEvidenceBase

Evidence for KIND_CONTROLLER — a ros2_control controller faulted.

Attributes:

Name	Type	Description
kind	Literal['controller']	Discriminator (always "controller").
controller_name	str	Failing controller (e.g. "joint_trajectory_controller").
state	str	Reported controller state (e.g. "inactive", "error").
detail	str	Free-form detail from the controller manager.

CriticEvidence

Bases: _FailureEvidenceBase

Evidence for KIND_CRITIC — a critic flagged an action below threshold.

Attributes:

Name	Type	Description
kind	Literal['critic']	Discriminator (always "critic").
critic_id	str	Identifier of the critic that fired (model id or hand-rolled).
score	float	Critic output in the critic's native range.
threshold	float	Configured pass threshold.

DeadlineOverrunPolicy

Bases: str, Enum

What the inference runner does when a tick exceeds the deadline.

warn logs + records an OTel attribute but still sends the action (mirrors lerobot's record loop). drop skips the action for this tick — used when stale actions are worse than no action (e.g. velocity control). raise is test-only: raises :class:ROSDeadlineMissed.

DeployScene

Bases: BaseModel

Environment-only scene for openral deploy run.

Carries the physics world and optional robot mount. No task, no eval config — this is a playground for the full OpenRAL stack.

from_yaml(path: str) -> DeployScene classmethod

Load and validate a DeployScene YAML from disk.

Source code in python/core/src/openral_core/schemas.py

@classmethod
def from_yaml(cls, path: str) -> DeployScene:
    """Load and validate a ``DeployScene`` YAML from disk."""
    import yaml  # noqa: PLC0415

    with open(path, encoding="utf-8") as fh:
        data = yaml.safe_load(fh)
    return cls.model_validate(data)

DetectedObject

Bases: BaseModel

A detected object in the scene.

Attributes:

Name	Type	Description
label	str	Semantic class label.
confidence	float	Detection confidence in [0, 1].
pose	Pose6D	6D pose in the world or camera frame.
bbox_3d	tuple[float, float, float, float, float, float] | None	3D bounding box (x_min, y_min, z_min, x_max, y_max, z_max).
track_id	int | None	Persistent track ID across frames.

DetectorContract

Bases: BaseModel

Manifest contract for kind: "detector" rSkills (ADR-0037).

Carries the configuration the runtime :class:~openral_core.schemas.ObjectsDetector needs to instantiate an exported detection model (RT-DETR / D-FINE ONNX) and match its output indices to semantic class labels.

Required when :attr:RSkillManifest.kind is "detector"; forbidden for all other kinds (enforced by :meth:RSkillManifest._check_kind_consistency).

Attributes:

Name	Type	Description
labels	list[str]	Ordered class-label list; the integer class-id the model emits is used as an index into this list. At least one label is required.
input_size	tuple[int, int]	Model input resolution as (width, height) in pixels. Both dimensions must be > 0. Default (640, 640) matches the RT-DETR / D-FINE default export resolution.
score_threshold	float	Detections with confidence below this value are discarded before publishing. Must be in [0.0, 1.0]. Default 0.5.
engine	DetectorEngine | None	Optional explicit backend selector (:class:DetectorEngine). None (default) keeps the legacy runtime-keyed dispatch. Set it to disambiguate backends that share a runtime — e.g. zeroshot_hf for an in-process Transformers open-vocabulary detector, which is runtime: pytorch like the VLM sidecar.
mode	DetectorMode	Invocation mode (:class:DetectorMode; default continuous). Declares whether the detector is an always-on background producer (output reaches the reasoner via world state) or an on-demand prompted locator (surfaces the locate_in_view tool). ADR-0051.

Example

c = DetectorContract( ... labels=["person", "bicycle", "car"], ... input_size=(640, 640), ... score_threshold=0.4, ... ) c.labels[0] 'person' c.mode

DetectorEngine

Bases: str, Enum

Backend that executes a kind: "detector" rSkill (ADR-0037).

Selects which runtime detector class :func:build_manifest_detector constructs. None (the default on :class:DetectorContract) preserves the legacy runtime-keyed dispatch: runtime: onnx/tensorrt → RT-DETR ONNX, runtime: pytorch → the LocateAnything VLM sidecar. Set it explicitly to opt into a backend that the runtime value alone cannot disambiguate (e.g. an in-process Transformers open-vocabulary detector, which is also runtime: pytorch).

Attributes:

Name	Type	Description
RTDETR_ONNX		Fixed-label RT-DETR / D-FINE ONNX export (CPU / NVMM tiers). Equivalent to leaving engine unset with an onnx/tensorrt runtime.
VLM_SIDECAR		Out-of-process open-vocabulary visual-grounding VLM (LocateAnything-3B). Equivalent to leaving engine unset with a pytorch runtime. Query-driven (prompted).
ZEROSHOT_HF		In-process Transformers open-vocabulary detector (AutoModelForZeroShotObjectDetection — e.g. OmDet-Turbo). Runs against a fixed class vocabulary (the manifest labels) every frame, so it behaves like a large closed-vocabulary detector that needs no prompting — an unprompted background producer that populates the world object list with far more than the 80 COCO classes (ADR-0037 2026-06-12 amendment).

Example

DetectorEngine.ZEROSHOT_HF.value 'zeroshot_hf'

DetectorMode

Bases: str, Enum

Invocation mode of a kind: "detector" rSkill (ADR-0051).

The axis orthogonal to :class:DetectorEngine: where engine says how the model runs, mode says when the reasoner invokes it and therefore how its output reaches the LLM.

Attributes:

Name	Type	Description
CONTINUOUS		An always-on background producer. Runs on the camera tee every frame and streams ObjectsMetadata into WorldState.detected_objects; the reasoner reads it passively (via world state / recall_object) and never prompts it. It is not an ExecuteSkill-dispatchable tool and carries no actuation authority. RT-DETR (closed vocab) and OmDet-Turbo (frozen open vocab) are continuous. The reasoner may still toggle it via LifecycleTransitionTool to free VRAM (ADR-0050).
ON_DEMAND		A prompted locator the reasoner invokes only when it needs to find a specific object right now. Surfaces the read-only locate_in_view tool (ADR-0043) backed by an open-vocabulary detector; it is not run continuously. LocateAnything is on-demand.

The two modes cleanly separate "open-vocabulary" from "prompting": continuous detectors cover a fixed bank of classes the reasoner reads for free; the on-demand locator handles the long tail that bank does not cover.

Example

DetectorMode.CONTINUOUS.value 'continuous' DetectorMode.ON_DEMAND.value 'on_demand'

DeviceInfo

Bases: BaseModel

Snapshot of the host compute capabilities used for runtime selection.

Attributes:

Name	Type	Description
device_str	str	PyTorch-style device string ("cpu", "cuda:0", "mps").
gpu_memory_bytes	int	Total GPU VRAM in bytes (0 if CPU-only).
cuda_compute_capability	tuple[int, int] | None	CUDA compute capability major/minor pair.
cpu_count	int	Logical CPU count.
arch	str	CPU architecture string.

Example

info = DeviceInfo(device_str="cpu") info.gpu_memory_bytes 0

EmbodimentExtra

Bases: BaseModel

Sensor + actuator surface for a "custom" embodiment.

Required when "custom" appears in :attr:RSkillManifest.embodiment_tags; forbidden otherwise. This is the explicit "I know what I'm doing" hatch for skill manifests targeting embodiments that do not have a canonical robots/<id>/robot.yaml in tree.

Reuses :class:SensorRequirement and :class:ActuatorRequirement on the assumption that the natural shape of "what this embodiment offers" mirrors "what the skill needs" — the loader just runs the same compat check it would run against a real robot YAML.

Attributes:

Name	Type	Description
sensors	list[SensorRequirement]	Sensor slots this custom embodiment exposes (≥1).
actuators	list[ActuatorRequirement]	Actuator slots this custom embodiment exposes (≥1).

Example

from openral_core import SensorModality, ControlMode EmbodimentExtra( ... sensors=[SensorRequirement(modality=SensorModality.RGB)], ... actuators=[ ... ActuatorRequirement( ... kind=ControlMode.JOINT_POSITION, ... n_dof=6, ... vla_action_key="action.joints.arm", ... control_mode_semantics=ControlModeSemantics(mode="absolute"), ... ) ... ], ... ) # doctest: +ELLIPSIS EmbodimentExtra(sensors=[...], actuators=[...])

EmbodimentKind

Bases: str, Enum

Top-level kinematic class of a robot body.

EmitPromptTool

Bases: _ReasonerToolBase

Tool variant — republish a PromptStamped onto another topic.

Dispatch: publish on target_topic (typically /openral/prompt for self-cascading, but any PromptStamped topic is valid). Lets the reasoner stage multi-step plans, talk to a peer reasoner, or feed a downstream prompt-aware skill without going through an ExecuteSkill goal.

Attributes:

Name	Type	Description
tool	Literal['emit_prompt']	Discriminator (always "emit_prompt").
target_topic	str	Absolute ROS topic name (must start with "/"). The reasoner's own subscription on /openral/prompt plus the prompt-router's FIFO queue (ADR-0018 F10) handle the cascade.
text	str	Human-readable prompt body forwarded as PromptStamped.text.
metadata_json	str	Free-form JSON forwarded as PromptStamped.metadata_json. Empty string when no structured metadata is needed.

EndEffectorSpec

Bases: BaseModel

End-effector specification.

Attributes:

Name	Type	Description
name	str	End-effector name.
kind	Literal['parallel_gripper', 'suction', 'dexterous_hand', 'tool', 'none']	Type of end-effector.
hand	Hand	Laterality.
n_dof	int	Number of controllable degrees of freedom.
max_grip_force_n	float | None	Maximum grip force in Newtons.
max_payload_kg	float | None	Maximum payload in kg.
workspace_radius_m	float | None	Reach radius in meters.
tactile_sensors	list[str]	List of tactile SensorSpec names attached to this EE.
actuated	bool	Whether the end-effector is driven by an actuator (ADR-0028a). False for passive tools (inert flanges, magnetic plates without electromagnet, kinematic-only mounts). When False, the safety kernel rejects any chunk addressed at this EE — the chunk routes to a no-op rather than risking unintended motion. Default True: every actuated gripper / dexterous hand / suction cup we ship is driven.

ExecuteRskillTool

Bases: _ReasonerToolBase

Tool variant — invoke an installed, capability-matched rSkill.

Dispatch: action goal on /openral/execute_skill (the openral_msgs/action/ExecuteSkill action server in F1's rskill_runner_node). The chunk path that follows is Skill → /openral/candidate_action → safety_node → /openral/safe_action → HAL per ADR-0018 §3.

Attributes:

Name	Type	Description
tool	Literal['execute_rskill']	Discriminator (always "execute_rskill").
rskill_id	str	RSkillManifest.name of an installed, capable skill. Validated against the local registry by the reasoner at palette-build time; an unknown id raises :class:ROSReasonerInvalidPlan.
prompt	str	Natural-language prompt forwarded to the skill's ExecuteSkill goal alongside the rskill_id. For VLA skills this is the policy's task-conditioning signal (SmolVLA writes it into observation["task"]); for wrapped-ROS skills it's carried for trace / log context but the actual goal is built from goal_params_json merged over the manifest's default_goal_json.
goal_params_json	str	ADR-0026 — serialised JSON object carrying per-skill typed parameters the LLM produces against the skill's :attr:RSkillManifest.goal_params_schema. Empty string disables the merge (today's behaviour). Wrapped-ROS skills (kind: ros_action / ros_service) deep-merge it over ros_integration.default_goal_json at configure-time; VLA skills accept the field and ignore it (their prompt is already the structured signal).
deadline_s	float	Hard deadline in seconds for the action server to complete the goal. 0 means "use the skill manifest's default latency budget".

ForceEvidence

Bases: _FailureEvidenceBase

Evidence for KIND_FORCE — measured force exceeded a safety limit.

Attributes:

Name	Type	Description
kind	Literal['force']	Discriminator (always "force").
joint_or_ee	str	Name of the joint or end-effector that tripped the limit.
measured_n	float	Measured force in newtons.
limit_n	float	Configured ceiling in newtons.

FrameEncoding

Bases: str, Enum

How the bytes inside a :class:SensorFrame are interpreted.

The first four values are raw per-pixel layouts. JPEG / PNG are compressed forms; the runner decodes lazily. CUDA_NV12 and RAW mark frames whose payload is an opaque handle (NVMM pointer, DMA-BUF fd) — the data field is empty and the consumer must read via handle.

GripperReadMode

Bases: str, Enum

How :class:openral_hal.MujocoArmHAL reports the gripper qpos.

SUM_OVER_SCALE (default) — clip(sum(qpos[addrs]) / scale, 0, 1). Matches the Franka parallel gripper where two finger qpos are summed and divided by 2 * max_finger_extent (=0.08 m for Panda). Public surface is normalised [0, 1]. AFFINE_LOW_HIGH — (qpos[addrs[0]] - ctrl_range[0]) / (ctrl_range[1] - ctrl_range[0]). Used by the SO-100 menagerie Jaw joint, a 1-DoF revolute with a non-zero closed position (-0.174 rad). Public surface is normalised [0, 1]. PASSTHROUGH — qpos[addrs[0]] reported verbatim, in the same physical units as the MJCF (metres for Aloha prismatic fingers, radians for OpenArm revolute jaws). The public surface is not [0, 1] — Skills must accept the raw range.

GripperWriteMode

Bases: str, Enum

How :class:openral_hal.MujocoArmHAL maps an Action's gripper value to ctrl.

NORMALISED (default) — input is [0, 1], mapped affinely to ctrl_range. low ↔ closed (Action.gripper = 0), high ↔ open (Action.gripper = 1). PASSTHROUGH — input is in the same physical units as ctrl_range (metres / radians), written directly to ctrl. MuJoCo's actuator ctrlrange does the clipping. Used by Aloha (positive-finger metres) and OpenArm (jaw radians).

HalConfig

Bases: BaseModel

Configuration for a HAL adapter instantiation by the inference runner.

Picks which HAL adapter class to use (sim digital twin vs real hardware) and how to reach the robot (serial port, FCI URI, ROS 2 namespace). Adapter-specific fields are forwarded as params.

Attributes:

Name	Type	Description
adapter	str	HAL adapter id, e.g. "so100_follower", "so100_digital_twin", "franka_panda_real", "ur5e_real", "ros_control". Resolved against the HAL registry.
transport	dict[str, object]	Transport-layer parameters. Examples: {"port": "/dev/ttyACM0", "baud": 1_000_000} for SO-100; {"fci_uri": "172.16.0.2"} for Franka; {"namespace": "/ur5e"} for ros2_control.
params	dict[str, object]	Adapter-specific keyword arguments forwarded to the constructor (e.g. calibration overrides).

Example

HalConfig( ... adapter="so100_follower", ... transport={"port": "/dev/ttyACM0", "baud": 1_000_000}, ... ).adapter 'so100_follower'

HalEntrypoints

Bases: BaseModel

Per-robot simulation and real-hardware HAL import strings.

The two HALs a robot can expose, declared independently so the choice of HAL type lives in the manifest (never in environment config or runtime params). Each value is a "module:Attr" import string resolved by :func:openral_hal.build_hal, or None when the robot has no HAL of that kind (sim-only / real-only / scene-only). ADR-0031.

Attributes:

Name	Type	Description
sim	str | None	Import string for the simulation HAL. When None and :attr:RobotDescription.sim is populated, the resolver derives MujocoArmHAL.from_description (ADR-0023) — so every plain arm leaves this null. Set it explicitly only for a non-generic sim HAL (e.g. "openral_hal.panda_mobile:PandaMobileHAL", which has no sim: block to derive from).
real	str | None	Import string for the real-hardware HAL (e.g. "openral_hal.ur_real:UR5eRealHAL"). None for sim-only robots; the resolver raises ROSCapabilityMismatch if mode="real" is requested on a robot whose real is None.
parameters	HalParameters	Per-robot HAL construction defaults (serial port, robot_ip, …) merged into the constructor by :func:openral_hal.build_hal. ADR-0029. Empty by default.

Example

HalEntrypoints(real="openral_hal.ur_real:UR5eRealHAL").sim is None True

HalParameters

Bases: BaseModel

Per-robot HAL construction defaults declared in the manifest (ADR-0029).

Carries the transport / constructor keyword arguments a robot's HAL needs — the SO-100's serial port + baud, a ros2_control arm's robot_ip / fci_ip — so the manifest is the single source of those defaults instead of a per-robot lifecycle-node subclass. This is the schema seam that lets the unified, robot.yaml-driven ManifestHALLifecycleNode (ADR-0032) serve a parameterised HAL without a bespoke _create_hal.

:func:openral_hal.build_hal merges :attr:defaults underneath any explicit transport kwargs (so a deploy run override wins) and then drops every key the target HAL constructor does not accept — exactly the filtering it already applies to transport. Empty by default, so robots that need no construction kwargs (the derived MujocoArmHAL arms) are unaffected.

Attributes:

Name	Type	Description
defaults	dict[str, object]	HAL constructor / transport keyword defaults, e.g. {"port": "/dev/ttyACM0", "baud": 1_000_000} for the SO-100 or {"robot_ip": "192.168.1.10"} for a real UR arm.

Example

HalParameters(defaults={"port": "/dev/ttyACM0"}).defaults["port"] '/dev/ttyACM0'

Hand

Bases: str, Enum

Laterality of an end-effector.

HumanEvidence

Bases: _FailureEvidenceBase

Evidence for KIND_HUMAN — a human triggered an intervention.

Attributes:

Name	Type	Description
kind	Literal['human']	Discriminator (always "human").
actor	str	Identifier of the operator (e.g. "slack:alice", "gui").
reason	str	Free-form reason given by the actor.

ImagePreprocessing

Bases: BaseModel

Per-rSkill image preprocessing contract.

Properties of how the checkpoint was trained against image frames, surfaced on the manifest so the sim adapter does not have to learn them from a YAML override. vla.extra overrides on the eval config still win — see openral_rskill._vla_core.resolve_image_preprocessing.

Attributes:

Name	Type	Description
flip_180	bool	Apply torch.flip(t, dims=[1, 2]) (H+W reversal) to every camera frame before the policy forward. SmolVLA / pi05 LIBERO checkpoints want this on; the RoboCasa checkpoints published by RoMALab / DAVIAN-Robotics want it off.
flip_vertical	bool	Apply img[::-1, :, :] (vertical-only, H-axis reversal) BEFORE any 180° rotation. Mirrors robocasa.wrappers.gym_wrapper.RoboCasaGymEnv.process_img, which the canonical openpi-robocasa eval applies before feeding frames to the policy. Required for robocasa/robocasa365_checkpoints/pi05_pretrain_human300 and its lerobot-converted siblings; the RoMALab MG_300 checkpoint does NOT want this flip.
input_template	str	Format string used to name image tensors in the policy input batch — for example "observation.images.{cam}" for SmolVLA / pi05 RoMALab vs "observation.image.{cam}" for ruiname/pi05-robocasa-10tasks-200k.
aliases	dict[str, str]	Per-checkpoint rename map from the scene / robot raw camera key (e.g. robosuite's robot0_agentview_left_image) to the model's expected input feature name (e.g. agentview). Empty means pass through unchanged. SmolVLA LIBERO usually wants {"camera1": "image", "camera2": "image2"}.
norm_tag	str | None	Normalization statistics tag for policies that use multiple checkpoints or training distributions (e.g. MolmoAct2 with norm_tag="so100_so101_molmoact2" for SO-100/101 finetuned weights). Optional; None means the adapter's default tag applies. Used by predict_action methods to select the right norm_stats entry at inference time.
image_max_crops	int | None	Cap on the number of image tiles a multi-crop image processor (Molmo / MolmoAct2 family) produces per camera frame. Each extra 378px crop adds ~182 pooled image tokens and attention cost is quadratic in the token count, so this is the primary activation-memory lever (the NF4 weights are fixed at ~3.5 GiB; what overflows an 8 GiB card is the crop activations). Optional; None keeps the checkpoint's own default (8 for MolmoAct2). A per-rollout vla.extra["image_max_crops"] (or the OPENRAL_MOLMOACT2_MAX_CROPS env) still overrides this per-checkpoint default. Must be >= 1 when set.

IntrinsicsPinhole

Bases: BaseModel

Pinhole camera intrinsics.

Attributes:

Name	Type	Description
width	int	Image width in pixels.
height	int	Image height in pixels.
fx	float	Focal length in x (pixels).
fy	float	Focal length in y (pixels).
cx	float	Principal point x (pixels).
cy	float	Principal point y (pixels).
distortion_model	Literal['plumb_bob', 'equidistant', 'none']	Distortion model name.
distortion_coeffs	list[float]	Distortion coefficients.

Example

IntrinsicsPinhole(width=640, height=480, fx=600.0, fy=600.0, cx=320.0, cy=240.0) IntrinsicsPinhole(width=640, height=480, fx=600.0, fy=600.0, cx=320.0, cy=240.0, ...)

JointSpec

Bases: BaseModel

URDF-derived joint specification.

Attributes:

Name	Type	Description
name	str	Joint name matching the URDF.
joint_type	JointType	Kinematic joint type.
parent_link	str	Parent link name.
child_link	str	Child link name.
axis_xyz	tuple[float, float, float]	Rotation/translation axis unit vector.
position_limits	tuple[float, float] | None	(min, max) in radians or meters.
velocity_limit	float | None	Maximum velocity.
effort_limit	float | None	Maximum effort (N or Nm).
has_position_sensor	bool	Whether position feedback is available.
has_velocity_sensor	bool	Whether velocity feedback is available.
has_torque_sensor	bool	Whether torque feedback is available.
backlash_estimate	float | None	Estimated backlash in radians.
actuator_kind	Literal['dc', 'bldc', 'stepper', 'servo', 'tendon', 'hydraulic', 'pneumatic'] | None	Type of actuator.
sim_joint_name	str | None	Optional override carrying the MJCF/MuJoCo joint name as it appears in the simulator's compiled model when that name differs from :attr:name. The HAL uses :attr:name for the world-state contract (it's the URDF-shaped logical identifier the safety supervisor sees on every chunk); a separate sim_joint_name lets a sim-adapter look up mj_name2id without hardcoding robosuite / robocasa / MuJoCo naming conventions in backend modules. None means "the MJCF joint name matches :attr:name" — true for every fixed-base manipulator we ship; only mobile bases and humanoids whose MJCF auto-prefixes joints (robosuite's mobilebase0_… namespace, GR-1's robot0_…) need it. See ADR-0025.
role	JointRole	Structural classification (ADR-0028a). The downstream runner / safety / dataset-bridge code identifies grippers and base DoFs by this tag instead of substring-matching the joint name (which silently misclassifies any joint containing "gripper", e.g. "gripper_pose"). Default "unknown" keeps legacy manifests loadable; the fleet annotates incrementally per ADR-0028a.
origin_xyz	tuple[float, float, float]	Fixed translation (metres) of this joint's frame in its parent_link frame — the URDF <joint><origin xyz>. With :attr:origin_rpy and :attr:axis_xyz it gives the kernel everything it needs for forward kinematics (ADR-0030). Default (0, 0, 0); populated by the offline lowering tool (from MJCF/URDF) only for robots that enable self-collision checking — legacy manifests are unaffected.
origin_rpy	tuple[float, float, float]	Fixed orientation (roll, pitch, yaw, radians) of this joint's frame in its parent_link frame — the URDF <joint><origin rpy>. Default (0, 0, 0). ADR-0030.

JointState

Bases: BaseModel

Real-time joint state snapshot.

Attributes:

Name	Type	Description
name	list[str]	Ordered list of joint names.
position	list[float]	Joint positions (rad or m).
velocity	list[float]	Joint velocities (rad/s or m/s).
effort	list[float]	Joint efforts (Nm or N).
stamp_ns	int	ROS 2 timestamp in nanoseconds.

JointType

Bases: str, Enum

Kinematic joint type following URDF conventions.

LifecycleTransitionTool

Bases: _ReasonerToolBase

Tool variant — drive a ROS 2 lifecycle transition on a peer node.

Dispatch: service call on <node>/change_state with the matching Transition id (configure / activate / deactivate / cleanup). Used to bring a HAL back online, restart a faulted node, or stage a controlled shutdown.

Attributes:

Name	Type	Description
tool	Literal['lifecycle_transition']	Discriminator (always "lifecycle_transition").
node	str	Fully-qualified ROS node name (e.g. "/openral/hal/so100").
transition	Literal['configure', 'activate', 'deactivate', 'cleanup']	One of "configure", "activate", "deactivate", "cleanup". Other transitions ("shutdown", error-recovery) are deliberately omitted from the open-core palette per CLAUDE.md §6 Layer 6 — shutdown is owned by the safety supervisor, not the reasoner.

LinkCollisionGeometry

Bases: BaseModel

One convex collision volume rigidly attached to a robot link (ADR-0030).

The lowered, kernel-facing form of a link's collision geometry. Authored by hand, or emitted by the offline lowering tool from a robot's MJCF or URDF + SRDF source; the kernel loads these into pre-sized buffers at on_configure and never parses the source geometry on the hot path. link_name is a :class:JointSpec link (joints stays normative for the kinematic chain — this adds geometry only).

Attributes:

Name	Type	Description
link_name	str	The :attr:JointSpec.child_link (or :attr:JointSpec.parent_link) this volume is attached to.
shape	CollisionShape	The convex primitive (capsule or sphere).
origin_xyz_rpy	tuple[float, float, float, float, float, float]	Pose of the primitive in the link frame — (x, y, z, roll, pitch, yaw) in metres and radians.

Example

g = LinkCollisionGeometry( ... link_name="link_1", ... shape=CapsuleShape(radius_m=0.04, length_m=0.3), ... ) g.origin_xyz_rpy (0.0, 0.0, 0.0, 0.0, 0.0, 0.0)

LocateInViewTool

Bases: _ReasonerToolBase

Tool variant (read-only) — check if an object is in a live camera view (ADR-0043).

The complement to :class:RecallObjectTool: where recall_object recalls a remembered object from the ADR-0038 scene-graph memory, locate_in_view asks a live camera-mounted VLM detector (e.g. LocateAnything, ADR-0037) to look at the current frame right now and report whether the queried object is visible — and where. It runs the detector's open-vocabulary query on demand via the /openral/perception/locate_in_view ROS service and feeds the answer back to the LLM as a re-prompt (the prompt cascade). Like every variant it holds no authority over actuation (ADR-0018 §4) — it only reads a frame; the dispatch never gates the safety kernel.

Attributes:

Name	Type	Description
tool	Literal['locate_in_view']	Discriminator (always "locate_in_view").
query	str	Free-text object/description to look for (e.g. "the red mug"). Sent verbatim as the detector's open-vocabulary query.
camera	str	Optional camera selector. Empty (default) uses the detector's primary camera; otherwise names one of the detector's configured cameras so the reasoner can pick a viewpoint. Not a hardcoded name — the detector is camera-agnostic and maps the id to a topic.
detector	str	Optional on-demand locator selector (ADR-0056). Empty (default) uses the deployment's default locator; otherwise an rSkill id / short alias of one of the on-demand locators in the graph (e.g. "omdet-turbo-locator" for fast simple "find X", "locateanything-3b" for complex referring expressions). The reasoner routes to /openral/perception/<detector>/locate_in_view. Still read-only — choosing a model does not grant actuation.

MotionMetadata

Bases: _PerceptionEventBase

Perception event for /openral/perception/motion.

Emitted when a frame-difference detector measures sub-frame motion above a configured threshold. region_bbox is set when the detector localises the moving pixels (top-left origin in pixels); None means "motion magnitude only, no localisation."

Attributes:

Name	Type	Description
kind	Literal['motion']	Discriminator (always "motion").
magnitude	float	Mean absolute per-pixel difference, normalised to [0, 1] against the encoding's full-scale range.
threshold	float	Magnitude threshold that fired the event.
region_bbox	tuple[int, int, int, int] | None	Optional bounding box (x_min, y_min, x_max, y_max) of the moving region in pixels.

ObjectDetection2D

Bases: BaseModel

A single 2D detection inside an :class:ObjectsMetadata event.

The 3D :class:DetectedObject lives in :class:WorldState; the 2D form here is what a per-camera detector (nvinfer, tflite, cv2-CPU) emits before any pose lift / fusion.

Attributes:

Name	Type	Description
label	str	Semantic class label as produced by the model.
confidence	float	Detection confidence in [0, 1].
bbox_xyxy	tuple[int, int, int, int]	Axis-aligned bounding box in pixels (x_min, y_min, x_max, y_max); image origin top-left.

ObjectsMetadata

Bases: _PerceptionEventBase

Perception event for /openral/perception/objects.

Emitted by the event leg's detector element (nvinfer on Jetson, tflite on CPU, etc.) when one or more objects are detected.

Attributes:

Name	Type	Description
kind	Literal['objects']	Discriminator (always "objects").
detections	list[ObjectDetection2D]	Per-object 2D detections, ordered by descending confidence.
model_id	str	Identifier of the detector that fired (e.g. "yolov8n", "nvinfer:resnet50").
frame_width	int	Pixel width of the frame the detector ran on; the bbox_xyxy of each detection is in this pixel space (ADR-0035 lift scales it to the sensor's intrinsics resolution).
frame_height	int	Pixel height of that frame.

ObservationSpec

Bases: BaseModel

VLA observation configuration for this robot.

Attributes:

Name	Type	Description
state_key	str	Observation dict key for the state vector.
state_shape	tuple[int, ...]	Shape of the state tensor, e.g. (6,) for 6-D EEF.
state_representation	StateRepresentation | None	How the state vector is encoded.
image_flip_180	bool	Whether camera images need 180° rotation before feeding to the VLA (common for wrist-mounted cameras).

OccupancyGridRef

Bases: BaseModel

Reference to a 2D occupancy grid for mobile-base world-collision (ADR-0030).

Mirrors the nav_msgs/OccupancyGrid metadata that :mod:openral_runner.slam_bridge already decodes. The kernel consumes a bounded, fixed-capacity copy; a grid exceeding the configured capacity or older than the staleness deadline is treated as unavailable (fail-closed). The occupancy bytes are referenced by topic, not inlined.

Attributes:

Name	Type	Description
frame_id	str	tf2 frame the grid origin is expressed in (e.g. "map").
resolution_m	float	Edge length of one cell, in metres.
width	int	Grid width in cells.
height	int	Grid height in cells.
origin	Pose6D	Pose of cell (0, 0)'s lower-left corner.
data_topic	str	ROS 2 topic carrying the nav_msgs/OccupancyGrid.

OcrMetadata

Bases: _PerceptionEventBase

Perception event for /openral/perception/ocr.

Attributes:

Name	Type	Description
kind	Literal['ocr']	Discriminator (always "ocr").
text	str	Recognised text (already stripped of leading / trailing whitespace by the detector).
confidence	float	Recogniser confidence in [0, 1].
region_bbox	tuple[int, int, int, int] | None	Optional bounding box (x_min, y_min, x_max, y_max) of the recognised region in pixels.

PerceptionStaleEvidence

Bases: _FailureEvidenceBase

Evidence for KIND_PERCEPTION — a sensor frame went stale.

Attributes:

Name	Type	Description
kind	Literal['perception']	Discriminator (always "perception").
sensor_id	str	SensorSpec.name of the stale sensor.
staleness_ms	float	Observed age of the last frame, in milliseconds.
threshold_ms	float	Staleness threshold that was crossed, in milliseconds.

PhysicsBackend

Bases: str, Enum

Physics / scene backend used to instantiate a :class:SceneSpec.

Attributes:

Name	Type	Description
MUJOCO		Vanilla MuJoCo (CPU / single-env). Default for LIBERO, MetaWorld.
MUJOCO_MJX		MuJoCo MJX (XLA, GPU-batched headless rollouts).
PYBULLET		PyBullet (legacy adapters, contact-rich tabletop).
ISAACSIM		NVIDIA Isaac Sim (Omniverse, GPU). Future.
GENESIS		Genesis (physics-language unification). Future.
MOCK		In-process mock with no physics — used for wiring smoketests.

Pose6D

Bases: BaseModel

6D pose: position + quaternion.

Attributes:

Name	Type	Description
xyz	tuple[float, float, float]	Position (x, y, z) in meters.
quat_xyzw	tuple[float, float, float, float]	Quaternion (x, y, z, w).
frame_id	str	tf2 reference frame.

ProtocolSpec

Bases: BaseModel

Stand-alone eval-protocol descriptor.

Historically the protocol block on the deleted BenchmarkSpec (ADR-0009). After the Task-10 scene-hierarchy convergence (ADR-0041, June 2026) a benchmark became a list of :class:BenchmarkScenes — each scene carries its own n_episodes / seed / task.success_key / task.max_steps — and ADR-0042 then deleted the BenchmarkSpec wrapper altogether. ProtocolSpec is retained as a public schema for callers that want to describe a protocol independently (e.g. ADR drafts, benchmark-report tooling).

Pins the methodology so two rSkills evaluated under the same benchmark produce apples-to-apples numbers. Authors of a benchmark should set these to match the published protocol of the suite they are reproducing (e.g. LIBERO: 10 episodes per task, fixed seed range, is_success).

Attributes:

Name	Type	Description
n_episodes	int	Number of independent episodes per task. Honest success rates need >= 10 per task for the published LIBERO / MetaWorld protocols.
seeds	list[int]	Seed list applied per task (paired index-wise with the episode index). Length MUST be >= n_episodes; runners slice seeds[:n_episodes] so re-runs are reproducible.
success_key	str	info key on the gym step() return whose truthy value marks task success. Default "is_success" (LIBERO convention); MetaWorld uses "success".
max_steps	int	Per-task episode budget. Adapters may clip to scene-internal limits; the value here is the protocol ceiling.
min_reps	int | None	Minimum number of completed episodes per task before the result JSON is written. Guards against a crash mid-suite producing a misleadingly partial roll-up. None means "require every task x episode".

Example

p = ProtocolSpec(n_episodes=10, seeds=list(range(10)), max_steps=280) p.success_key 'is_success'

model_post_init(_context: object) -> None

Cross-field validation: seeds list must cover n_episodes.

Source code in python/core/src/openral_core/schemas.py

def model_post_init(self, _context: object) -> None:
    """Cross-field validation: seeds list must cover n_episodes."""
    if len(self.seeds) < self.n_episodes:
        raise ValueError(
            f"ProtocolSpec.seeds has {len(self.seeds)} entries but "
            f"n_episodes={self.n_episodes}; provide at least n_episodes "
            f"seeds so re-runs are reproducible."
        )
    if self.min_reps is not None and self.min_reps > self.n_episodes:
        raise ValueError(
            f"ProtocolSpec.min_reps ({self.min_reps}) exceeds "
            f"n_episodes ({self.n_episodes}); a benchmark cannot require "
            f"more completed reps than it schedules."
        )

QuantizationBackend

Bases: str, Enum

Inference backend that will execute the quantized model.

Attributes:

Name	Type	Description
PYTORCH		torch.quantization / bitsandbytes / torchao.
ONNX		onnxruntime with quantization pre-applied at export.
TENSORRT		NVIDIA TensorRT engine (INT8 / FP8 calibrated).
GGUF		llama.cpp / ggml GGUF format (CPU and Metal).
MLX		Apple MLX framework (Apple Silicon only).

Example

QuantizationBackend.PYTORCH.value 'pytorch'

QuantizationConfig

Bases: BaseModel

Full specification of how to quantize a skill's model weights.

Attributes:

Name	Type	Description
dtype	QuantizationDtype	Target numeric format.
backend	QuantizationBackend	Inference backend that will execute the model.
per_channel	bool	Use per-channel (vs per-tensor) quantization.
calibration_dataset	str | None	HuggingFace dataset ID or local path used for post-training calibration (INT8 / TensorRT only).
extra	dict[str, object]	Backend-specific overrides (e.g. {"calibration_steps": 128}).

Example

cfg = QuantizationConfig(dtype=QuantizationDtype.INT8) cfg.backend

QuantizationDtype

Bases: str, Enum

Numeric format used to represent model weights and activations.

Attributes:

Name	Type	Description
FP32		32-bit float (full precision, reference).
FP16		16-bit float (GPU-native, common default).
BF16		Brain-float16 (Ampere+ GPUs, TPUs; better dynamic range than FP16).
INT8		8-bit integer (CPU/GPU; good accuracy/speed trade-off).
INT4		4-bit integer (edge / memory-constrained; some accuracy loss).
FP4_NVFP4		NVIDIA FP4 format (Hopper+; highest throughput).

Example

QuantizationDtype.INT8.value 'int8'

QuerySceneTool

Bases: _ReasonerToolBase

Tool variant (read-only) — ask a scene VLM a question about the live view (ADR-0047).

Backed by a kind: "vlm" rSkill (e.g. Qwen3.5-4B NF4) running in an out-of-process ZMQ sidecar. Where :class:LocateInViewTool answers where an object is (open-vocabulary localization via the detector), query_scene answers open-ended questions about the scene's state — task-progress and success/failure verification the reasoner needs for its replanning ladder: "has the robot grasped the mug?", "is the bowl on the shelf?", "did we drop the object?", "is the table clear?".

It captures the current frame of the requested camera, sends it plus the question to the VLM over the /openral/perception/query_scene ROS service, and feeds the free-text answer back to the LLM as a re-prompt (the prompt cascade). Like every variant it holds no authority over actuation (ADR-0018 §4) — it only reads a frame; the dispatch never gates the safety kernel. It is not a localizer: use :class:LocateInViewTool to find objects.

Attributes:

Name	Type	Description
tool	Literal['query_scene']	Discriminator (always "query_scene").
question	str	Natural-language question about the current scene (e.g. "Has the robot grasped the red mug?"). Sent verbatim to the VLM as the textual prompt alongside the frame.
camera	str	Optional camera selector. Empty (default) uses the perception node's primary camera; otherwise names one of its configured cameras so the reasoner can pick a viewpoint. Not a hardcoded name.

QueryTaskProgressTool

Bases: _ReasonerToolBase

Tool variant (read-only) — ask the reward monitor how the task is going (ADR-0057).

Backed by a kind: "reward" rSkill (Robometer-4B NF4) running in parallel with the active VLA in an out-of-process ZMQ sidecar. Where :class:QuerySceneTool answers open-ended scene questions as free text, query_task_progress returns a quantitative windowed assessment of the current task: normalized progress and success over the last :attr:window_s seconds, plus their trends and a stalled flag.

It calls the /openral/perception/query_task_progress ROS service, which scores the monitor's buffered camera frames against the task instruction and returns progress_now / success_now / progress_trend / success_trend / stalled / succeeded. The reasoner uses it to decide whether to continue, escalate to :class:QuerySceneTool, advance, or enter the replanning ladder. Like every variant it holds no authority over actuation (ADR-0018 §4) — the reward signal is advisory; the dispatch never gates the safety kernel.

Attributes:

Name	Type	Description
tool	Literal['query_task_progress']	Discriminator (always "query_task_progress").
window_s	float	How many seconds of recent frames to assess. Must be > 0; clamped to the monitor's configured frame_window_s.
task	str	Optional task-instruction override. Empty (default) reuses the instruction the monitor was co-activated with (the active VLA's goal).

ROSBTValidationError

Bases: ROSPlanningError

The emitted BehaviorTree XML failed BT.CPP v4 validation.

ROSCapabilityMismatch

Bases: ROSError

A skill requires a capability the target robot does not have.

ROSCollisionImminent

Bases: ROSSafetyViolation

A proposed motion would self-collide or strike a world obstacle.

Raised on the safety path when geometric checking (ADR-0030) finds a chunk whose forward-kinematic sweep brings a robot link within its clearance of another link or a world primitive. Like every :class:ROSSafetyViolation, it is caught only at the safety supervisor boundary, where it triggers an E-stop and a structured incident log.

ROSConfigError

Bases: ROSError

Bad manifest, missing weights, or invalid YAML / URDF.

ROSDeadlineMissed

Bases: ROSFleetError

Cloud RTT exceeded the skill's deadline; fallback was not configured.

ROSDispatchUnavailable

Bases: ROSFleetError

No dispatcher (edge or cloud) is available for the requested skill.

ROSEStopRequested

Bases: ROSSafetyViolation

An emergency stop was requested; all actuation must cease immediately.

ROSError

Bases: Exception

Base class for all OpenRAL errors.

ROSFleetError

Bases: ROSError

A fleet-level or dispatch error.

ROSForceLimitExceeded

Bases: ROSSafetyViolation

Measured or predicted contact force exceeds the configured limit.

ROSGPUMemoryError

Bases: ROSRuntimeError

Out of GPU memory; use a smaller skill variant or quantize further.

ROSInferenceTimeout

Bases: ROSRuntimeError

VLA inference did not complete within its latency budget.

ROSObjectNotInMemory

Bases: ROSPerceptionStale

A spatial-memory query matched no node, or only nodes that are stale.

Raised by the ADR-0038 scene-graph query surface when a RecallObjectQuery / ResolvePlaceQuery cannot be satisfied. The caller degrades by treating the target as unknown (and may trigger active search) — it never fabricates a pose (CLAUDE.md §1.2, §1.4).

ROSPerceptionStale

Bases: ROSError

A sensor reading is older than the configured staleness deadline.

ROSPlanningError

Bases: ROSError

The reasoner failed to produce a valid plan.

ROSReasonerInvalidPlan

Bases: ROSPlanningError

The LLM returned a plan that failed schema or capability validation.

ROSRskillGoalSatisfied

Bases: ROSError

A wrapped-ROS rSkill has finished its goal successfully.

Used as a typed completion signal raised by :meth:openral_rskill.ros_action_rskill.ROSActionRskill._step_impl after the last waypoint of a one-shot planner (e.g. MoveIt) has been emitted, or after a result-only wrapped action (e.g. Nav2 NavigateToPose) reports success. The ExecuteSkill action server catches this specifically and closes the goal with success=True.

This is NOT an error — it inherits :class:ROSError only to stay inside the OpenRAL exception surface so it is greppable and discoverable via the standard hierarchy. It must be caught ONLY at the rskill_runner_node execute-callback boundary; everywhere else it is a programming bug to raise it.

ROSRuntimeError

Bases: ROSError

General runtime failure during skill execution or HAL operation.

ROSSafetyViolation

Bases: ROSError

A safety constraint was violated.

NEVER catch this silently. Catch only at the safety supervisor boundary, trigger an E-stop, and emit a structured incident log entry.

ROSWorkspaceViolation

Bases: ROSSafetyViolation

An action would move the robot outside its allowed workspace.

RSkillAction

Bases: str, Enum

Closed vocabulary of high-level action verbs an rSkill can perform.

Declared on :attr:RSkillManifest.actions so the reasoner's LLM tool palette can present each skill with a structured "what does it do" label, in addition to the free-form description and the slug. The LLM scores tools primarily on natural-language description, but a closed verb vocabulary lets the palette pre-filter and lets the schema be unit-testable. New entries are additive.

Categories (descriptive only — not part of the wire format):

• Manipulation primitives: PICK, PLACE, PICK_AND_PLACE, TRANSFER, GRASP, RELEASE.
• Articulated / contact-rich: OPEN, CLOSE, PUSH, PULL, SLIDE, INSERT, POUR, WIPE, ROTATE.
• Motion: REACH; LOOK (aim a camera at a point, ADR-0044).
• Mobile (for mobile-manipulator embodiments): NAVIGATE.
• Social / expressive: WAVE, SHAKE.
• Generalist marker: GENERALIST for foundation / multi-task checkpoints (e.g. RoboCasa-365, DROID, MetaWorld-MT50). The palette surfaces a generalist skill for goals that don't match a specific verb.

RSkillEvalBenchmark

Bases: BaseModel

The benchmark suite a :class:RSkillEvalResult was measured against.

Attributes:

Name	Type	Description
name	str	Canonical suite name ("LIBERO", "MetaWorld", "ALOHA", "PushT", "RoboCasa" …). Used by openral benchmark report to group rows.
dataset	str | None	Optional HF dataset identifier.
protocol	str	Free-text description of the eval protocol.
robot	str	robot_id the benchmark was evaluated on.
simulator	str	Free-text simulator description.

RSkillEvalResult

Bases: BaseModel

Pydantic model of a rskills/<id>/eval/<benchmark>.json file.

The results and baselines blocks are intentionally benchmark-specific — LIBERO has four sub-suites with success rates, MetaWorld carries an MT50 average, ALOHA has a single cube-transfer rate. We require the metadata blocks (source, benchmark, eval_config) but leave results as a free-form dict so each benchmark can declare its own keys.

Attributes:

Name	Type	Description
schema_version	str	Semver-ish version string for this on-disk format.
source	RSkillEvalSource	Provenance — paper / reproduction state.
benchmark	RSkillEvalBenchmark	Suite identity (name, robot, simulator).
eval_config	dict[str, object]	Free-form configuration the benchmark was run with (chunk size, image size, denoising steps, …).
results	dict[str, object]	Free-form per-task / per-suite success rates.
baselines	dict[str, object]	Optional free-form comparison numbers from prior work.
trace_id	str | None	Hex OTel trace id (32 chars) for the rollout that produced this result. Set by openral benchmark run from the cli.command root span so reviewers can deep-link from rskills/<id>/eval/<benchmark>.json straight to the trace tree in Jaeger / Tempo. Optional — paper-cited numbers (reproduced_locally: false) leave it None.

Example

RSkillEvalResult.model_validate_json(

'{"schema_version": "0.1", "source": {...}, ...}')

from_json(path: str) -> RSkillEvalResult classmethod

Load and validate a rskills/<id>/eval/<benchmark>.json file.

Parameters:

Name	Type	Description	Default
path	str	Filesystem path to the JSON file.	required

Returns:

Type	Description
RSkillEvalResult	A validated :class:RSkillEvalResult.

Raises:

Type	Description
FileNotFoundError	If path does not exist.
ValidationError	If the JSON fails schema validation.

Source code in python/core/src/openral_core/schemas.py

@classmethod
def from_json(cls, path: str) -> RSkillEvalResult:
    """Load and validate a ``rskills/<id>/eval/<benchmark>.json`` file.

    Args:
        path: Filesystem path to the JSON file.

    Returns:
        A validated :class:`RSkillEvalResult`.

    Raises:
        FileNotFoundError: If ``path`` does not exist.
        pydantic.ValidationError: If the JSON fails schema validation.
    """
    import json as _json  # noqa: PLC0415  # reason: deferred import

    with open(path, encoding="utf-8") as fh:
        data = _json.load(fh)
    return cls.model_validate(data)

RSkillEvalSource

Bases: BaseModel

Provenance for a benchmark result block.

Used inside :class:RSkillEvalResult so consumers know where the numbers came from (paper vs local reproduction) without having to read the _comment line.

Attributes:

Name	Type	Description
paper	str	Plain-text title of the source paper / report.
arxiv	str | None	Optional arxiv URL or id.
model_variant	str	Which checkpoint variant the numbers describe (e.g. "SmolVLA (0.45B)").
evaluated_by	str	Free-text — "upstream authors" or a contributor.
reproduced_locally	bool	True only when OpenRAL CI / contributor re-ran the benchmark and the listed numbers match.
reproduction_planned	str | None	Optional plan for closing the gap when reproduced_locally is False.
reproduction_cli	dict[str, object] | str | None	Either a single command string or a structured {description, single_suite_example, all_suites, suite_max_steps, notes} block (matches the LIBERO eval JSON shape).
table	str | None	Optional pointer at a specific table in the source paper.
status	str | None	Optional state marker — "in_progress", "deferred", "reproduced" — for benchmarks that are not yet final.

RSkillInfo

Bases: BaseModel

Snapshot of a Skill's runtime state — published on /skill/<name>/info.

Attributes:

Name	Type	Description
name	str	rSkill name (e.g. "noop_skill").
version	str	SemVer string.
state	RSkillState	Current primary lifecycle state.
weights_loaded	bool	Whether model weights have been loaded into memory.
quantized	bool	Whether weights have been quantized for the target runtime.
warmed_up	bool	Whether the model has been warmed up (first inference run).
embodiment_tags	list[str]	Embodiment tags from the skill manifest.
role	Literal['s0', 's1', 's2']	rSkill role: "s0" (cerebellar), "s1" (fast policy), "s2" (slow reasoning).
latency_budget_ms	float | None	Maximum allowed inference latency in milliseconds.
last_inference_ms	float | None	Actual latency of the most recent step() call.
error_msg	str | None	Human-readable error description when state == "error".
stamp_ns	int	Timestamp of this snapshot in nanoseconds.

Example

info = RSkillInfo(name="hello", version="0.1.0", state=RSkillState.UNCONFIGURED) info.weights_loaded False info.state

RSkillLatencyBudget

Bases: BaseModel

Per-stage latency budget declared in rskill.yaml.

Attributes:

Name	Type	Description
per_chunk_ms	float	End-to-end step() budget. CI fails if exceeded on the reference host (CLAUDE.md §7.4).
warmup_ms	float | None	Maximum allowed warm-up time during activate().
load_ms	float | None	Maximum allowed weight-load time during configure().

RSkillLicensePosture

Bases: str, Enum

License posture surfaced at install time (CLAUDE.md §7.4).

RSkillManifest

Bases: BaseModel

Pydantic model of the rskill.yaml package manifest (V1).

This is the on-disk schema for an rSkill HF Hub repo. An rSkill is loaded by capability-checking against a :class:RobotDescription, selecting a runtime + quantization, then constructing a runtime :class:~openral_rskill.Skill instance.

schema_version is "0.1": the manifest surface has had no backward-incompatible change. Now the repo is published it is versioned for real (CLAUDE.md §1.6) — a backward-incompatible change bumps it and ships a migrator, while backward-compatible additions (ADR-0013/0022/0024) evolve the surface in place.

ADR-0013 added two symmetric guards on top of the initial V1 shape:

1. actuators_required mirrors sensors_required on the output side. Every skill declares at least one :class:ActuatorRequirement; the loader validates it against :attr:RobotDescription.action_spec. n_dof and vla_action_key are auto-filled from the robot YAML for the 9 canonical embodiments; for "custom" they must be set on the manifest.

2. embodiment_extra is the explicit escape hatch for embodiments that do not have a canonical robots/<id>/robot.yaml. Required iff "custom" appears in :attr:embodiment_tags; forbidden otherwise.

V1 already tightened: name / fallback_skill_id must be HF-Hub-shaped, version is SemVer, weights_uri is restricted to hf:// or local://, embodiment_tags is closed to the set of in-tree-supported robots (now including "custom"), and benchmarks replaces the old free-form metadata blob with a typed dict of canonical suite ids → success rate. Commercial-use posture is derived from :attr:license via :attr:is_commercial_use_allowed.

Attributes:

Name	Type	Description
schema_version	Literal['0.1']	On-disk format version. "0.1" today. Backward-compatible extensions (ADR-0013/0022/0024) evolved the surface in place; now the repo is published, a backward-incompatible change bumps this and ships a migrator (CLAUDE.md §1.6).
name	str	HF Hub identifier, e.g. "openral/rskill-pick-cube-so100". Must match <owner>/<repo>.
version	str	SemVer string of the rSkill package itself (not the wrapped weights).
license	RSkillLicensePosture	License posture (surfaced at install time). Drives :attr:is_commercial_use_allowed.
role	Literal['s0', 's1', 's2']	rSkill slot. "s1" for fast policies (the only slot currently loaded via :func:rSkill.from_yaml — 100% of in-tree skills); "s0" reserved for cerebellar realtime (humanoid balance, C++ only); "s2" reserved for slow reasoning (LLM-emitted BehaviorTree XML).
model_family	ModelFamily | None	Closed VLA / policy family. Drives the runner's adapter dispatch.
embodiment_tags	list[EmbodimentTag]	Robot embodiments this rSkill targets. Must intersect a robot's RobotCapabilities.embodiment_tags for the loader to accept. Restricted to the canonical set (:data:EmbodimentTag) so typos / framework hints cannot silently always-miss. "custom" is the explicit hatch (see :attr:embodiment_extra).
embodiment_extra	EmbodimentExtra | None	ADR-0013. When "custom" is in :attr:embodiment_tags, declares the sensor + actuator surface of the custom rig so the loader's compat check still has something to match against. MUST be None when "custom" is not present.
capabilities_required	dict[str, bool | float | int | str]	Per-flag capability requirements (subset of :class:RobotCapabilities boolean fields). The loader rejects installation on a robot that does not satisfy every flag.
sensors_required	list[SensorRequirement]	Sensor inputs the policy expects from the robot. See :class:SensorRequirement.
actuators_required	list[ActuatorRequirement]	ADR-0013. Symmetric output-side contract: at least one :class:ActuatorRequirement entry. The loader matches each entry's :attr:kind against :attr:RobotDescription.action_spec.control_mode and auto-fills :attr:ActuatorRequirement.n_dof / :attr:vla_action_key from the canonical robot YAML (predefined embodiments only — for "custom" they must be set on the manifest).
runtime	RSkillRuntime	Preferred inference runtime.
quantization	QuantizationConfig	Default :class:QuantizationConfig for this rSkill.
weights_uri	str | None	HF Hub revision-pinned URI to weights or local path reference. Production deployments pin a SHA (CLAUDE.md operating principle 8).
chunk_size	int	Action-chunk length the policy emits per :meth:Skill.step. Drives the ChunkedExecutor's overlap schedule and shows up in the trace surface.
latency_budget	RSkillLatencyBudget	Latency contract enforced by CI on the reference host (CLAUDE.md §7.4).
min_vram_gb	dict[QuantizationDtype, float] | None	Optional minimum VRAM (GB) the skill needs per quantization dtype. Informational — the loader uses it for ral skill check / openral doctor; the actually-applied dtype is still pinned by quantization.dtype (CLAUDE.md §11 forbids silent downcast).
fallback_skill_id	str | None	rSkill id to substitute if this one fails (RFC §8.3 replanning ladder). HF Hub shape.
benchmarks	dict[BenchmarkName, float]	Map of canonical benchmark suite id → success rate ([0.0, 1.0]) as measured for this skill. Keys constrained to :data:BenchmarkName. The full per-task breakdown lives in the matching rskills/<id>/eval/<key>.json validated against :class:RSkillEvalResult.
paper_url	str | None	Canonical paper URL for this skill / family.
dataset_uri	str | None	HF Hub URI for the training dataset.
source_repo	str | None	HF Hub URI for the upstream weights repo (often the same as :attr:weights_uri's prefix).
description	str	REQUIRED. Short (1-500 char) human-readable summary surfaced by ral skill list and, more importantly, by the reasoner's per-skill LLM tool description (see :func:openral_reasoner.palette.build_tool_palette). The LLM scores tools primarily on this text; keep it specific to what the skill does (objects, scenes, task type), not how it was trained.
actions	list[RSkillAction]	REQUIRED. Closed-vocabulary list (≥1) of high-level action verbs this skill performs (see :class:RSkillAction). Generalist / foundation checkpoints declare [GENERALIST]; specialist checkpoints list the verbs they were trained on (e.g. [PICK, PLACE]). Surfaced to the reasoner LLM alongside :attr:description so it can pick the right skill for a given goal.
objects	list[str]	Optional free-form keywords for the objects this skill manipulates (e.g. ["cube"], ["pipe"], ["drawer"]). Discriminative hints for the LLM; the long tail (RoboCasa-365 has hundreds of categories) makes a closed enum impractical.
scenes	list[str]	Optional free-form keywords for the scenes / environments this skill targets (e.g. ["tabletop"], ["kitchen"]). Same rationale as :attr:objects.
processors	RSkillProcessors | None	Per-file URIs for the lerobot PolicyProcessorPipeline artefacts (Gap 1 + Gap 3 of the rSkill self-containment audit). REQUIRED when model_family is one of smolvla, pi05, xvla, diffusion, or rldx; OPTIONAL for act (the legacy norm-stats-in-safetensors path is allowed there).
image_preprocessing	ImagePreprocessing | None	Per-checkpoint image-side knobs that cannot be encoded in the processor JSONs (flip, camera renames). Optional; None means schema defaults apply.
state_contract	StateContract | None	Per-checkpoint proprioception layout (named layouts for RoboCasa; explicit dim for everything else).
action_contract	ActionContract | None	Per-checkpoint action vector contract (dim + optional representation). ADR-0019: consumed by the dataset bridge to bind the LeRobot v3 action feature shape without consulting the runtime adapter.
n_action_steps	int | None	Replay cadence (how many actions to consume from a chunk before re-inferring). Omit when equal to chunk_size; the adapter falls through to its family default.

Example

m = RSkillManifest( ... name="openral/rskill-pick-cube-so100", ... version="0.1.0", ... license=RSkillLicensePosture.APACHE_2_0, ... role="s1", ... kind="vla", ... model_family="smolvla", ... embodiment_tags=["so100_follower"], ... runtime=RSkillRuntime.PYTORCH, ... weights_uri="hf://lerobot/smolvla_base", ... chunk_size=16, ... latency_budget=RSkillLatencyBudget(per_chunk_ms=100.0), ... description="Pick a cube on the SO-100 follower arm.", ... actions=[RSkillAction.PICK], ... objects=["cube"], ... scenes=["tabletop"], ... actuators_required=[ ... ActuatorRequirement( ... kind=ControlMode.JOINT_POSITION, ... control_mode_semantics=ControlModeSemantics(mode="absolute"), ... ), ... ], ... processors=RSkillProcessors( ... preprocessor_uri="hf://lerobot/smolvla_base/policy_preprocessor.json", ... postprocessor_uri="hf://lerobot/smolvla_base/policy_postprocessor.json", ... ), ... ) m.is_commercial_use_allowed True m.schema_version '0.1'

is_commercial_use_allowed: bool property

Derive commercial-use posture from :attr:license.

apache-2.0 / mit / bsd → True. Every other posture (including unknown) → False, conservatively. The loader consults this in :meth:rSkill._check_license; users wanting to deploy a non-commercial skill in a research context set OPENRAL_ALLOW_NONCOMMERCIAL=1 per CLAUDE.md §7.4.

from_yaml(path: str) -> RSkillManifest classmethod

Load and validate an rskill.yaml from disk.

Parameters:

Name	Type	Description	Default
path	str	Filesystem path to the manifest YAML.	required

Returns:

Type	Description
RSkillManifest	A validated :class:RSkillManifest.

Raises:

Type	Description
FileNotFoundError	If path does not exist.
ValidationError	If the YAML fails schema validation.

Source code in python/core/src/openral_core/schemas.py

@classmethod
def from_yaml(cls, path: str) -> RSkillManifest:
    """Load and validate an ``rskill.yaml`` from disk.

    Args:
        path: Filesystem path to the manifest YAML.

    Returns:
        A validated :class:`RSkillManifest`.

    Raises:
        FileNotFoundError: If ``path`` does not exist.
        pydantic.ValidationError: If the YAML fails schema validation.
    """
    import yaml  # noqa: PLC0415  # reason: yaml is a runtime dep; deferred to avoid import-time cost

    with open(path, encoding="utf-8") as fh:
        data = yaml.safe_load(fh)
    return cls.model_validate(data)

RSkillProcessors

Bases: BaseModel

Explicit lerobot PolicyProcessorPipeline artefact pointers.

Closes Gap 1 + Gap 3 of the rSkill self-containment audit. Replaces the implicit snapshot_download(repo_id) → make_pre_post_processors(pretrained_path=...) fetch path used today by the SmolVLA / pi05 / xVLA / Diffusion / modern-ACT adapters. Two files are required because lerobot's pipeline ships them as a pair: a preprocessor that normalises the policy input batch and a postprocessor that un-normalises the action chunk.

Each URI must point at a specific file (the implicit-snapshot shape hf://owner/repo with no file tail is rejected). Two URI schemes accepted:

• hf://owner/repo[@rev]/path/to/file.json — file inside an HF Hub repo, revision-pinnable.

Attributes:

Name	Type	Description
preprocessor_uri	str	Per-file URI to the policy preprocessor JSON.
postprocessor_uri	str	Per-file URI to the policy postprocessor JSON.

Example

RSkillProcessors( ... preprocessor_uri="hf://lerobot/smolvla_libero/policy_preprocessor.json", ... postprocessor_uri="hf://lerobot/smolvla_libero/policy_postprocessor.json", ... ) # doctest: +ELLIPSIS RSkillProcessors(preprocessor_uri='hf://...preprocessor.json', postprocessor_uri='hf://...postprocessor.json')

RSkillRuntime

Bases: str, Enum

Inference runtime hint declared in the manifest (RFC §7.3).

RSkillState

Bases: str, Enum

Primary lifecycle states for a Skill node.

Matches the ROS 2 Managed Node state machine with openral extensions.

States

unconfigured Initial state. Weights are not loaded. inactive Configured and weights loaded; not yet warmed up. Accepts no actions. active Ready to execute. step() may be called. finalized Terminal state after shutdown(). Cannot transition further. error Unrecoverable failure. Requires external intervention.

ReasonerTimeoutEvidence

Bases: _FailureEvidenceBase

Evidence for KIND_REASONER_TIMEOUT — the LLM call missed its deadline.

Attributes:

Name	Type	Description
kind	Literal['reasoner_timeout']	Discriminator (always "reasoner_timeout").
model	str	Model identifier (e.g. "claude-opus-4-7").
deadline_s	float	Configured deadline in seconds.
elapsed_s	float	Actual elapsed wall-clock time in seconds.

RecallObjectMatch

Bases: BaseModel

One ranked match from a :class:RecallObjectQuery (ADR-0038 §6).

Attributes:

Name	Type	Description
node_id	str	The matched node's id.
label	str	The matched node's label.
pose	Pose6D	The object's recalled pose (map frame).
score	float	Match score in [0, 1].
last_seen_ns	int	When the object was last observed, in nanoseconds.
approach	ApproachViewpoint | None	Optional computed camera-facing approach viewpoint.
inside_container_id	str | None	Set when the object is inside an occluding container that must be opened first; None otherwise.

RecallObjectQuery

Bases: BaseModel

Read-only query to recall a remembered object (ADR-0038 §6).

At least one of text / label must be non-empty. text is matched against node embeddings when an embedder is configured (ADR-0038 §5), otherwise matching falls back to label.

Attributes:

Name	Type	Description
text	str	Free-text query (open-vocabulary match).
label	str	Exact label match.
near	Pose6D | None	Optional pose to bias results toward (proximity).
max_age_ns	int | None	Optional recency filter — drop nodes whose last_seen_ns is older than this many nanoseconds before "now".
limit	int	Maximum number of matches to return.

RecallObjectResult

Bases: BaseModel

Result of a :class:RecallObjectQuery (ADR-0038 §6).

Attributes:

Name	Type	Description
matches	list[RecallObjectMatch]	Ranked matches (possibly empty — an empty result is how the query reports "unknown"; callers raise / handle :class:~openral_core.exceptions.ROSObjectNotInMemory rather than fabricating a pose).

RecallObjectTool

Bases: _ReasonerToolBase

Tool variant (read-only) — recall a remembered object (ADR-0039).

Queries the ADR-0038 scene-graph spatial memory and returns the object's map-frame pose plus a camera-facing approach viewpoint and any occluding container to the reasoner's next reasoning step. Like every variant it holds no authority over actuation (ADR-0018 §4) — it only reads memory. The dispatch that runs the query and feeds the result back to the LLM is wired in ADR-0039 Phase 2 (this is the typed contract).

Attributes:

Name	Type	Description
tool	Literal['recall_object']	Discriminator (always "recall_object").
query	str	Free-text or label naming the object to recall (e.g. "the red mug"). Mapped to a :class:RecallObjectQuery at dispatch.
limit	int	Maximum number of ranked matches to return.

ReloadGstPipelineTool

Bases: _ReasonerToolBase

Tool variant — swap a sensor's GStreamer pipeline at runtime.

Dispatch: service call on /openral/sensors/<sensor_id>/reload_pipeline with the pipeline YAML payload. Lets the reasoner tune perception (switch resolution, enable an nvinfer leg, swap an RTSP source) without redeploying the runtime.

Attributes:

Name	Type	Description
tool	Literal['reload_gst_pipeline']	Discriminator (always "reload_gst_pipeline").
sensor_id	str	SensorSpec.name of the camera whose pipeline is being reloaded. Validated against the active runtime's sensor catalog at dispatch time.
pipeline_yaml	str	Full YAML body of the new :class:SensorReaderConfig. The sensor node validates this against the Pydantic schema before accepting the swap.

ResolvePlaceQuery

Bases: BaseModel

Read-only query to resolve a place/room/agent reference to a goal (ADR-0038 §6).

Attributes:

Name	Type	Description
reference	str	Free-text, id, or label of the target (e.g. "kitchen", "where I was standing", a node id).
kind	SpatialNodeKind | None	Optional node-kind filter (room / place / agent).

ResolvePlaceResult

Bases: BaseModel

Result of a :class:ResolvePlaceQuery (ADR-0038 §6).

Attributes:

Name	Type	Description
node_id	str	The resolved node's id.
goal	Pose6D	Navigation goal pose (map frame).
path_node_ids	list[str]	Ordered traversable_to path of node ids from the robot's current place to the goal (empty when no path is needed or known).

ResolvePlaceTool

Bases: _ReasonerToolBase

Tool variant (read-only) — resolve a place/room/agent to a goal (ADR-0039).

Queries the ADR-0038 scene-graph memory for a navigation goal pose plus a traversable_to path. Read-only; holds no authority over actuation. Dispatch + result-return are ADR-0039 Phase 2.

Attributes:

Name	Type	Description
tool	Literal['resolve_place']	Discriminator (always "resolve_place").
reference	str	Free-text, id, or label of the target (e.g. "the kitchen", "where I was standing"). Mapped to a :class:ResolvePlaceQuery.

RewardContract

Bases: BaseModel

Manifest contract for kind: "reward" rSkills (ADR-0057).

Carries the configuration a robotic reward / progress-monitor model (e.g. Robometer-4B, a Qwen3-VL-4B reward foundation model) needs to score a rollout. A reward skill runs in parallel with a kind: "vla" policy, continuously ingesting the VLA's camera frames into a rolling window, and emits per-frame progress ∈ progress_range and per-frame success probability. The Reasoner queries it on demand (QueryTaskProgressTool) to decide whether to continue, escalate to a scene VLM, advance, or replan. The signal is advisory only — it never actuates and never gates motors (CLAUDE.md §1.1).

Required when :attr:RSkillManifest.kind is "reward"; forbidden for all other kinds (enforced by :meth:RSkillManifest._check_kind_consistency). Like detector / vlm, a reward skill is a pure perception consumer: it emits no Action chunks and requires no actuators.

Attributes:

Name	Type	Description
progress_range	tuple[float, float]	(min, max) of the normalized per-frame progress scalar. Default (0.0, 1.0) — Robometer's discrete/binned mode (see :attr:num_bins) emits progress already in [0, 1].
success_threshold	float	Per-frame success probability at/above which the frame is considered a task success. In [0.0, 1.0]; default 0.5.
preference	bool	Whether the model also exposes a trajectory-preference head (Robometer does). Default False — the progress/success path is the Reasoner-facing contract; preference is future work.
frame_window_s	float	Length of the rolling frame buffer in seconds. The sidecar evicts frames older than this relative to the newest. Must be > 0.
target_fps	float	Frame-sampling rate fed to the model (Robometer's example uses 3 fps). Must be > 0. This is an S2-cadence monitor, not a per-control-step signal.
num_bins	int	Discrete-mode bin count for the progress head. Robometer's discrete mode yields per-frame normalized progress in [0, 1]; continuous mode yields raw regression values. Must be > 0; default 100.
instruction_required	bool	Whether a natural-language task instruction must accompany the frames (Robometer requires one). Default True.

Example

c = RewardContract(frame_window_s=8.0, target_fps=3.0) c.progress_range (0.0, 1.0) c.success_threshold 0.5 c.num_bins 100

RoboCasaBackendOptions

Bases: BaseModel

Typed validator helper for SceneSpec.backend_options under RoboCasa.

RoboCasa exposes two scenario modes:

• prebuilt — pick one of the ~100 atomic-PnP / door / drawer / navigation tasks shipped with the package by name (e.g. "PnPCounterToCab"). Set :attr:prebuilt_task; leave all procedural keys at their defaults.
• procedural -- author a kitchen by composing :attr:kitchen_style x :attr:layout_id x :attr:fixtures x :attr:spawn_objects x :attr:task_verb. Set :attr:mode to "procedural" and leave :attr:prebuilt_task None.

The model is purely additive: it lives alongside :class:SceneSpec and is constructed by the RoboCasa scene adapter at factory time via RoboCasaBackendOptions.model_validate( scene.backend_options). The parent SceneSpec.backend_options: dict[str, object] field is unchanged, so this class does not constitute a schema migration (ADR-0015, CLAUDE.md §1.6).

See :doc:ADR-0015 </adr/0015-robocasa-isolated-backend-lazy-assets> for the full backend rationale.

Attributes:

Name	Type	Description
mode	Literal['prebuilt', 'procedural']	"prebuilt" (default) or "procedural".
prebuilt_task	str | None	One of RoboCasa's ~100 atomic task names, e.g. "PnPCounterToCab". Only valid when mode="prebuilt".
kitchen_style	int | None	0..9, picks one of RoboCasa's 10 kitchen aesthetic packs. Only valid when mode="procedural".
layout_id	int | None	0..9, picks one of RoboCasa's 10 floor plans. Only valid when mode="procedural".
fixtures	list[str]	Subset of RoboCasa fixture names to spawn. Empty list keeps the layout's default fixtures.
spawn_objects	list[str]	Subset of RoboCasa object asset names to spawn on counters / inside cabinets.
task_verb	Literal['pnp', 'open', 'close', 'press', 'navigate'] | None	Coarse procedural-mode goal: pick-and-place, open, close, press, navigate. Required when mode="procedural".
robots	list[str]	RoboCasa robot composition names, e.g. ["PandaMobile"] (default) or ["GR1"]. Must match entries the upstream RoboCasa env factory accepts.
controller	str	RoboCasa controller name. "OSC_POSE" works for arm-style robots; the adapter validates against the controller config exposed by RoboCasa at import time.
horizon	int	Maximum step budget for the underlying RoboCasa env.

Example

opts = RoboCasaBackendOptions(mode="prebuilt", prebuilt_task="PnPCounterToCab") opts.task_verb is None True

obj_groups: str | None = None class-attribute instance-attribute

Pin the PnP target object to a specific RoboCasa object group / category (e.g. "baguette", "vegetable"). Forwarded to the prebuilt PnP task's obj_groups kwarg so the sampled object is deterministic instead of the default "all" random draw. Only valid for prebuilt PickPlace tasks; leave None for door / drawer / navigation tasks.

RobotCapabilities

Bases: BaseModel

Capability flags used for skill compatibility checking.

Attributes:

Name	Type	Description
locomotion	list[LocomotionKind]	Locomotion types available.
can_lift_kg	float	Maximum payload in kg.
has_dexterous_hands	bool	Whether dexterous hands are present.
has_tactile	bool	Whether tactile sensing is available.
has_force_control	bool	Whether force/impedance control is supported.
has_vision	bool	Whether camera(s) are present.
has_lidar	bool	Whether LiDAR is present.
has_audio	bool	Whether audio I/O is present.
bimanual	bool	Whether the robot has two arms.
onboard_compute_tops	float	Peak onboard compute in TOPS.
onboard_memory_gb	float	Onboard RAM in GB.
gpu_vram_gb	float	Largest single-GPU VRAM in GB (0 when no discrete GPU).
cuda_compute_capability	tuple[int, int] | None	Highest CUDA compute capability (major, minor), e.g. (8, 9) for Ada Lovelace, (10, 0) for Blackwell. None on non-CUDA hosts.
cuda_toolkit_version	str | None	nvcc version string when present.
tensorrt_version	str | None	TensorRT runtime version string when importable.
gpu_supported_runtimes	list[RSkillRuntime]	Inference runtimes the host can execute, used by rSkill.check_capabilities to match RSkillManifest.runtime.
gpu_supported_dtypes	list[QuantizationDtype]	Quantization dtypes the host accelerator supports (derived from CUDA compute capability or platform).
supported_control_modes	list[ControlMode]	List of supported ControlMode values.
supported_vla_embodiments	list[str]	VLA embodiment IDs this robot can run.
embodiment_tags	list[str]	Short tags mapping to VLA heads / dataset splits.
nvmm_available	bool	Whether libnvbufsurface.so is present on the host (Tegra L4T multimedia stack). True enables the NVMM zero-copy sensor-ingest path on Jetson; always False on x86 and on stripped L4T images. Populated by :func:openral_detect.probes.gpu._probe_nvmm_available. Added in v0.5 per ADR-0013.

RobotDescription

Bases: BaseModel

Top-level robot manifest — one per robot, published to HuggingFace Hub.

Attributes:

Name	Type	Description
name	str	Robot name, e.g. "so100_follower".
embodiment_kind	EmbodimentKind	Top-level kinematic class.
assets	AssetRefs	Unified URDF / MJCF / SRDF reference block (ADR-0058). Refs share the :func:openral_core.assets.resolve_asset grammar; the URDF's robot_state_publisher wiring lives on :attr:AssetRefs.urdf (ADR-0027). Empty by default.
base_frame	str	Base link tf2 frame name.
odom_frame	str	Odometry tf2 frame name.
map_frame	str	Map tf2 frame name.
joints	list[JointSpec]	List of joint specifications.
end_effectors	list[EndEffectorSpec]	List of end-effector specifications.
sensors	list[SensorSpec]	List of individual sensor specs.
sensor_bundles	list[SensorBundle]	List of multi-modal sensor bundles.
capabilities	RobotCapabilities	Capability flags for skill matching.
safety	SafetyEnvelope	Safety envelope constraints.
ros2_namespace	str	ROS 2 namespace prefix.
middleware	Literal['fastdds', 'cyclonedds', 'zenoh']	ROS 2 middleware selection.
onboard_compute	dict[str, object]	Onboard compute descriptors.
sdk_kind	Literal['open', 'closed_with_api', 'closed']	Whether the SDK is open or closed.
hal	HalEntrypoints	Simulation + real-hardware HAL import strings. deploy sim constructs hal.sim (or derives MujocoArmHAL from :attr:sim); deploy run constructs hal.real. ADR-0031.
observation_spec	ObservationSpec | None	VLA observation configuration.
action_spec	ActionSpec | None	VLA action configuration.
sim	SimDescription | None	Optional MuJoCo wiring consumed by :class:openral_hal.MujocoArmHAL. When set, the HAL can be constructed entirely from the manifest via :meth:MujocoArmHAL.from_description; no per-robot Python subclass is required.
scene_defaults	SceneDefaults | None	Optional per-robot scene rendering defaults. Today carries :class:TopCameraDefaults consumed by the openarm_tabletop_pnp MJCF composer (see :mod:openral_sim.backends.openarm_robosuite._assets). Added in openral-core 0.6.0 to replace the dataset-specific module-level constants that previously lived in the sim backend.
base_joints	list[str] | None	Optional ordered list of :attr:JointSpec.name references identifying the planar mobile base joints. When the robot has a holonomic / differential / omnidirectional base, the three entries are conventionally [forward_axis, side_axis, yaw_axis]. Consumed by the generic ray-cast helpers in :mod:openral_sim.backends.robocasa and the ROS HAL lifecycle nodes to resolve MJCF joint names without hardcoding robot-specific conventions. None for fixed-base manipulators. ADR-0025.
collision_geometry	list[LinkCollisionGeometry]	Per-link convex collision primitives (capsules / spheres) the safety kernel uses for self- and world-collision checking. Empty by default. ADR-0030.
allowed_collision_pairs	list[tuple[str, str]]	Link-name pairs excluded from self-collision (adjacent links touch by design); the allowed-collision matrix. On real robots this is sourced from the SRDF disable_collisions block (named by :attr:AssetRefs.srdf). ADR-0030.

Example

desc = RobotDescription( ... name="smoke_robot", ... embodiment_kind=EmbodimentKind.MANIPULATOR, ... joints=[ ... JointSpec( ... name="j1", ... joint_type=JointType.REVOLUTE, ... parent_link="base_link", ... child_link="link_1", ... ) ... ], ... capabilities=RobotCapabilities( ... supported_control_modes=[ControlMode.JOINT_POSITION], ... embodiment_tags=["smoke"], ... ), ... safety=SafetyEnvelope(), ... ) assert desc.name == "smoke_robot"

lidar_sensor: SensorSpec | None property

The first declared 2-D LiDAR / scan sensor, or None.

ADR-0025 — the panda_mobile HAL synthesises a sensor_msgs/ LaserScan from MuJoCo ray-casts; its beam count (:attr:SensorSpec.n_channels), range (:attr:SensorSpec.range_min_m / :attr:SensorSpec.range_max_m) and rate (:attr:SensorSpec.rate_hz) live on this descriptor so openral deploy sim and the HAL lifecycle node read one source of truth from robot.yaml instead of each hardcoding scan params.

from_yaml(path: str) -> RobotDescription classmethod

Load and validate a RobotDescription YAML manifest from disk.

Parameters:

Name	Type	Description	Default
path	str	Filesystem path to the robot.yaml file.	required

Returns:

Type	Description
RobotDescription	A validated :class:RobotDescription.

Raises:

Type	Description
FileNotFoundError	If path does not exist.
ValidationError	If the YAML fails schema validation.

Example

RobotDescription.from_yaml("robots/franka_panda/robot.yaml")

Source code in python/core/src/openral_core/schemas.py

@classmethod
def from_yaml(cls, path: str) -> RobotDescription:
    """Load and validate a ``RobotDescription`` YAML manifest from disk.

    Args:
        path: Filesystem path to the ``robot.yaml`` file.

    Returns:
        A validated :class:`RobotDescription`.

    Raises:
        FileNotFoundError: If ``path`` does not exist.
        pydantic.ValidationError: If the YAML fails schema validation.

    Example:
        >>> # RobotDescription.from_yaml("robots/franka_panda/robot.yaml")
    """
    import yaml  # noqa: PLC0415  # reason: deferred to avoid import-time cost

    with open(path, encoding="utf-8") as fh:
        data = yaml.safe_load(fh)
    return cls.model_validate(data)

nav2_param_overrides() -> dict[str, str]

Nav2 param substitutions derived from this robot's base props.

ADR-0025 — lets the Nav2 bringup stay generic: instead of hand-vendoring a per-robot nav2_<robot>.yaml, the launch rewrites a shared base param file with these key→value substitutions. Maps :attr:footprint_radius → robot_radius (collision envelope) and the costmap inflation_radius (footprint_radius + :data:NAV2_INFLATION_CLEARANCE_M, so it stays ≥ the inscribed/circumscribed radius Nav2 derives from the footprint — otherwise Nav2 errors and falls back to slow full-footprint collision checks), and :attr:base_kinematics → the MPPI motion_model. Returns an empty dict for fixed-base arms (no mobile base → no Nav2). Velocity bounds remain Nav2 tuning in the base param file, not robot identity.

Source code in python/core/src/openral_core/schemas.py

def nav2_param_overrides(self) -> dict[str, str]:
    """Nav2 param substitutions derived from this robot's base props.

    ADR-0025 — lets the Nav2 bringup stay generic: instead of
    hand-vendoring a per-robot ``nav2_<robot>.yaml``, the launch
    rewrites a shared base param file with these key→value
    substitutions. Maps :attr:`footprint_radius` → ``robot_radius``
    (collision envelope) **and** the costmap ``inflation_radius``
    (``footprint_radius`` + :data:`NAV2_INFLATION_CLEARANCE_M`, so it
    stays ≥ the inscribed/circumscribed radius Nav2 derives from the
    footprint — otherwise Nav2 errors and falls back to slow
    full-footprint collision checks), and :attr:`base_kinematics` →
    the MPPI ``motion_model``. Returns an empty dict for fixed-base
    arms (no mobile base → no Nav2). Velocity bounds remain Nav2
    tuning in the base param file, not robot identity.
    """
    overrides: dict[str, str] = {}
    if self.footprint_radius is not None:
        overrides["robot_radius"] = str(self.footprint_radius)
        overrides["inflation_radius"] = (
            f"{self.footprint_radius + NAV2_INFLATION_CLEARANCE_M:.3f}"
        )
    if self.base_kinematics is not None:
        overrides["motion_model"] = {
            "omni": "Omni",
            "holonomic": "Omni",
            "differential": "DiffDrive",
            "ackermann": "Ackermann",
        }[self.base_kinematics]
    return overrides

validate_for_e2e_pipeline() -> None

Assert this manifest carries every field the e2e ROS graph needs.

The C++ safety kernel (cpp/openral_safety_kernel, ADR-0020) reads per-joint position_limits / velocity_limit / effort_limit + the global safety: block. Per-joint limit fields are optional on :class:JointSpec for sim-only robots, but they become mandatory the moment you ask openral deploy sim to bring up the kernel against this robot.

This method makes the e2e contract explicit and fails loud listing every missing field. Pair it with openral_safety.envelope_loader.compute_intersection(robot, skill=None) for the actual envelope synthesis — this method only validates.

Raises:

Type	Description
ROSConfigError	If any actuated joint is missing position_limits, velocity_limit, or effort_limit. The error lists every missing field at once so the operator does not have to fix one, re-run, fix the next.

Source code in python/core/src/openral_core/schemas.py

def validate_for_e2e_pipeline(self) -> None:
    """Assert this manifest carries every field the e2e ROS graph needs.

    The C++ safety kernel (``cpp/openral_safety_kernel``, ADR-0020)
    reads per-joint ``position_limits`` / ``velocity_limit`` /
    ``effort_limit`` + the global ``safety:`` block. Per-joint
    limit fields are *optional* on :class:`JointSpec` for sim-only
    robots, but they become mandatory the moment you ask
    ``openral deploy sim`` to bring up the kernel against this robot.

    This method makes the e2e contract explicit and fails loud
    listing every missing field. Pair it with
    ``openral_safety.envelope_loader.compute_intersection(robot,
    skill=None)`` for the actual envelope synthesis — this method
    only validates.

    Raises:
        ROSConfigError: If any actuated joint is missing
            ``position_limits``, ``velocity_limit``, or
            ``effort_limit``. The error lists every missing field
            at once so the operator does not have to fix one,
            re-run, fix the next.
    """
    missing: list[str] = []
    for joint in self.joints:
        if joint.joint_type not in (
            JointType.REVOLUTE,
            JointType.PRISMATIC,
            JointType.CONTINUOUS,
        ):
            continue
        if joint.position_limits is None:
            missing.append(f"joints[{joint.name}].position_limits")
        if joint.velocity_limit is None:
            missing.append(f"joints[{joint.name}].velocity_limit")
        if joint.effort_limit is None:
            missing.append(f"joints[{joint.name}].effort_limit")
    if missing:
        raise ROSConfigError(
            f"RobotDescription({self.name!r}) is missing fields required by "
            f"the e2e safety kernel: {missing}. Set them on the matching "
            "JointSpec(s) in robots/<robot_id>/robot.yaml."
        )

RobotEnvironment

Bases: BaseModel

A full hardware deployment configuration — the openral deploy artefact.

Sibling of :class:SimEnvironment for real hardware. Loaded from YAML by openral_runner (planned, ADR-0010) and consumed by the hardware InferenceRunner. The runner instantiates the HAL adapter, opens every :class:SensorReader, wires the :class:~openral_world_state.aggregator.WorldStateAggregator, constructs the :class:~openral_rskill.Skill from vla.weights_uri (a bare rSkill reference — name, path, or HF repo ID) and ticks at rate_hz.

Cross-field invariants enforced in :meth:model_post_init:

• Every :attr:SensorReaderConfig.sensor_id MUST be unique within :attr:sensors.
• :attr:vla.weights_uri MUST be a bare rSkill reference (no URI scheme) — the rSkill manifest is the contract between robot/sensors/preprocessing and policy weights (CLAUDE.md §6.4).

Attributes:

Name	Type	Description
robot_id	str	ID into the ROBOTS registry — matches a :attr:RobotDescription.name. Examples: "so100_follower", "franka_panda", "ur5e".
hal	HalConfig	HAL adapter + transport configuration.
sensors	list[SensorReaderConfig]	Per-sensor reader backend choices. The runner opens one :class:SensorReader per entry and feeds the :class:WorldStateAggregator.
task	TaskSpec	What the robot should achieve. Reused from :class:SimEnvironment so :attr:TaskSpec.instruction becomes the language prompt handed to the VLA.
vla	VLASpec	Policy that drives the robot. vla.weights_uri MUST be a bare rSkill reference (name, path, or HF repo ID).
safety	SafetyEnvelope | None	Optional :class:SafetyEnvelope override; falls back to the robot's :attr:RobotDescription.safety when None.
rate_hz	float	Foreground tick rate. Default 30 Hz (matches the :class:WorldStateAggregator publish rate).
thumbnail_hz	float	Per-camera rate at which the runner encodes a JPEG thumbnail onto the sensors.read_latest span for the dashboard. Default 25 Hz; decoupled from rate_hz. 0 disables thumbnails. End-to-end dashboard refresh is min(thumbnail_hz, span export flush rate) — see OPENRAL_OTEL_SPAN_SCHEDULE_DELAY_MS.
deadline_overrun_policy	DeadlineOverrunPolicy	Behavior when tick wall-time exceeds 1 / rate_hz.
max_ticks	int | None	Optional cap — the runner exits cleanly after this many ticks. None means "run until task.max_steps or external stop".
save_dir	str | None	Optional directory for traces / video / JSON summaries.
metadata	dict[str, object]	Free-form notes (operator, run id, …).

Example

env = RobotEnvironment( ... robot_id="so100_follower", ... hal=HalConfig(adapter="so100_follower"), ... sensors=[SensorReaderConfig(sensor_id="wrist_rgb")], ... task=TaskSpec( ... id="pick_cube/red", ... scene_id="pick_cube/red", ... instruction="pick up the red cube", ... ), ... vla=VLASpec( ... id="smolvla", ... weights_uri="rskills/smolvla-so100", ... ), ... ) env.rate_hz 30.0

from_yaml(path: str) -> RobotEnvironment classmethod

Load and validate a RobotEnvironment YAML from disk.

Parameters:

Name	Type	Description	Default
path	str	Filesystem path to the YAML config file.	required

Returns:

Type	Description
RobotEnvironment	A validated :class:RobotEnvironment.

Raises:

Type	Description
FileNotFoundError	If path does not exist.
ValidationError	If the YAML fails schema validation.

Source code in python/core/src/openral_core/schemas.py

@classmethod
def from_yaml(cls, path: str) -> RobotEnvironment:
    """Load and validate a ``RobotEnvironment`` YAML from disk.

    Args:
        path: Filesystem path to the YAML config file.

    Returns:
        A validated :class:`RobotEnvironment`.

    Raises:
        FileNotFoundError: If ``path`` does not exist.
        pydantic.ValidationError: If the YAML fails schema validation.
    """
    import yaml  # noqa: PLC0415  # reason: deferred to avoid import-time cost

    with open(path, encoding="utf-8") as fh:
        data = yaml.safe_load(fh)
    return cls.model_validate(data)

model_post_init(_context: object) -> None

Cross-field validation: unique sensors, bare rSkill weights_uri.

Source code in python/core/src/openral_core/schemas.py

def model_post_init(self, _context: object) -> None:
    """Cross-field validation: unique sensors, bare rSkill weights_uri."""
    seen: set[str] = set()
    for sensor in self.sensors:
        if sensor.sensor_id in seen:
            raise ValueError(
                f"RobotEnvironment({self.robot_id!r}) has duplicate "
                f"sensor_id={sensor.sensor_id!r} in sensors; each "
                f"sensor must be configured at most once."
            )
        seen.add(sensor.sensor_id)
    for bad in ("hf://", "local://", "file://", "http://", "https://"):
        if self.vla.weights_uri.startswith(bad):
            raise ValueError(
                f"RobotEnvironment({self.robot_id!r}).vla.weights_uri must "
                f"be a bare rSkill reference (name, path, or HF repo ID), "
                f"got {self.vla.weights_uri!r}; hardware deployments resolve "
                f"weights through an rSkill manifest for reproducibility "
                f"(CLAUDE.md §6.4)."
            )

RosIntegration

Bases: BaseModel

Wiring for an rSkill that wraps an existing ROS 2 action or service.

Populated when :attr:RSkillManifest.kind is "ros_action" or "ros_service". The :class:~openral_rskill.ros_action_rskill.ROSActionRskill adapter reads this block at configure time to build the right action / service client on the host LifecycleNode.

Attributes:

Name	Type	Description
package	str	IDL package name (e.g. "moveit_msgs", "nav2_msgs"). Imported lazily so manifests for un-installed ROS packages still parse — the loader raises :class:~openral_core.exceptions.ROSConfigError only when the skill is actually resolved.
interface_type	str	Action / service type name inside package (e.g. "MoveGroup", "NavigateToPose").
interface_name	str	Fully-qualified ROS path of the running server (e.g. "/move_action", "/navigate_to_pose").
result_trajectory_field	str | None	Dotted accessor pointing at a trajectory_msgs/JointTrajectory inside the action result (e.g. "planned_trajectory.joint_trajectory" for MoveIt's MoveGroup). When set, the adapter replays one waypoint per :meth:~openral_rskill.base.rSkillBase.step call onto /openral/candidate_action (so the safety supervisor + HAL see every position). When None the action is treated as result-only: the adapter awaits the action result, raises :class:~openral_core.exceptions.ROSRskillGoalSatisfied on success, and never emits an :class:~openral_core.schemas.Action chunk. This second mode covers wrapped ROS packages that drive actuators on their own (Nav2's behaviour tree publishes cmd_vel directly).
default_goal_json	str	JSON dict literal used to construct the goal message. v1 hard-codes the target here — the structured-prompt path that lowers LLM-emitted JSON into ExecuteSkill.Goal.prompt_metadata_json is a follow-up. REQUIRED so that a wrapped skill is always invocable end-to-end without per-call schema work.
ros_dependencies	list[str]	Apt / colcon packages the operator must have installed for the wrapped server to be reachable (e.g. "ros-${ROS_DISTRO}-moveit"). Surfaced by ral skill check and quoted in :class:~openral_core.exceptions.ROSConfigError messages when the action client fails to connect.

Example

ri = RosIntegration( ... package="moveit_msgs", ... interface_type="MoveGroup", ... interface_name="/move_action", ... result_trajectory_field="planned_trajectory.joint_trajectory", ... default_goal_json='{"target_joint_positions": [0, 0, 0, 0, 0, 0, 0]}', ... ros_dependencies=["ros-${ROS_DISTRO}-moveit"], ... ) ri.package 'moveit_msgs'

goal_builder: Literal['joint', 'pose', 'look_at'] | None = None class-attribute instance-attribute

ADR-0044 / ADR-0054 — optional goal-lowering adapter over the shared ROSActionRskill MoveGroup engine. None (the default) sends default_goal_json + LLM overrides verbatim (the raw-IDL escape hatch). The named builders consume a typed block from the merged goal and lower it into MoveGroup constraints:

• "joint" — a joint block (positions, joint_names) → joint_constraints (:class:~openral_rskill.joint_goal_rskill.JointGoalRskill).
• "pose" — a pose block (position, orientation as a quaternion array with quaternion_order, link_name) → MoveGroup position + orientation constraints for a Cartesian end-effector goal (:class:~openral_rskill.pose_goal_rskill.PoseGoalRskill).
• "look_at" — a look_at block (target_xyz, camera, …) → the same pose-constraint lowering, with the gaze pose computed from the camera's live TF pose (:class:~openral_rskill.look_at_rskill.LookAtRskill, a specialisation of the pose builder).

RunResult

Bases: BaseModel

Aggregated summary returned by :meth:InferenceRunner.run.

Attributes:

Name	Type	Description
n_ticks	int	Total ticks executed.
success	bool | None	Task success when the runner has a success signal (sim, scripted goal). None for open-ended hardware runs.
budget_violations	int	Count of ticks whose tick_ms exceeded the rSkill manifest's :attr:RSkillLatencyBudget.per_chunk_ms.
avg_inference_ms	float	Mean of :attr:TickResult.inference_ms.
p99_inference_ms	float	99th-percentile of :attr:TickResult.inference_ms.
avg_tick_ms	float	Mean of :attr:TickResult.tick_ms.
p99_tick_ms	float	99th-percentile of :attr:TickResult.tick_ms.
trace_id	str | None	OTel trace id (hex) of the run's root span. None when tracing is not configured.
save_dir	str | None	Directory where artefacts were written, when configured.
metadata	dict[str, object]	Free-form metadata (runner id, host, git SHA, …).

SafetyEnvelope

Bases: BaseModel

Safety constraints enforced by the C++ safety kernel.

Attributes:

Name	Type	Description
workspace_box_min_xyz	tuple[float, float, float] | None	Lower corner of allowed workspace (m).
workspace_box_max_xyz	tuple[float, float, float] | None	Upper corner of allowed workspace (m).
no_go_zones	list[dict[str, object]]	List of polygon definitions (dicts with 'vertices').
max_ee_speed_m_s	float	Maximum end-effector linear speed in m/s. Also used by ADR-0028b's per-mode supervisor as the CARTESIAN_TWIST linear bound.
max_ee_accel_m_s2	float	Maximum end-effector acceleration in m/s².
max_joint_speed_factor	float	Fraction of joint velocity_limit allowed.
max_force_n	float	Maximum contact force in Newtons.
max_torque_nm	float	Maximum joint torque in Nm.
deadman_required	bool	Whether a deadman switch is required.
e_stop_topic	str	ROS 2 topic for E-stop commands.
e_stop_qos	Literal['reliable']	QoS profile for E-stop topic.
contact_force_threshold_n	float	Force threshold for contact detection.
cycle_time_violation_threshold_ms	float	Control cycle time violation threshold.
human_in_loop_required	list[str]	rSkill names requiring human supervision.
max_cartesian_step_m	float | None	ADR-0028b — per-step magnitude bound on CARTESIAN_DELTA's xyz triplet (Euclidean). None means "no per-mode check declared, skip"; today's behaviour preserved. Robots that host OSC-trained checkpoints (panda_mobile, future Franka + π0.7) declare this so the supervisor rejects out-of-distribution arm deltas before they reach the controller.
max_cartesian_step_rad	float | None	ADR-0028b — per-step magnitude bound on CARTESIAN_DELTA's axis-angle triplet (Euclidean). None skips the check.
max_ee_angular_speed_rad_s	float | None	ADR-0028b — angular component bound for CARTESIAN_TWIST (the linear bound reuses :attr:max_ee_speed_m_s). None skips the check.
max_base_linear_speed_m_s	float | None	ADR-0028b — BODY_TWIST linear bound (Euclidean over vx,vy,vz). None skips the check; mobile manipulators / wheeled bases declare it.
max_base_angular_speed_rad_s	float | None	ADR-0028b — BODY_TWIST angular bound (Euclidean over wx,wy,wz; for planar bases only wz is non-zero). None skips the check.

SceneChangeMetadata

Bases: _PerceptionEventBase

Perception event for /openral/perception/scene_change.

Emitted when a histogram / structural-similarity detector measures a frame-to-frame distance above a configured threshold — typically used to wake the reasoner when the scene has changed enough to warrant a re-plan.

Attributes:

Name	Type	Description
kind	Literal['scene_change']	Discriminator (always "scene_change").
distance	float	Frame-to-frame distance in the detector's native metric (e.g. cv2.HISTCMP_CHISQR_ALT distance, 1 - ssim).
threshold	float	Distance threshold that fired the event.
metric	str	Identifier of the distance metric (e.g. "chisqr_alt", "1-ssim", "hellinger").

SceneComposition

Bases: BaseModel

Declarative MJCF scene composition for a manifest-driven HAL (ADR-0029).

Lets a robot whose sim HAL needs a composed MJCF (a bare arm spliced onto a tabletop + props) declare the composer in its manifest instead of a bespoke _create_hal lifecycle subclass (issue #191 Phase 3b). The manifest-driven node calls :attr:composer before constructing the HAL and threads the resulting MJCF path in as the HAL's mjcf_path.

The composer is a "module.path:function" import string. The function is called with **params and MUST return (xml: str, meshdir: Path) — the composed MJCF XML and the mesh directory it references (the node writes the XML next to meshdir so relative mesh paths resolve). Today's only composer is openral_sim.backends.openarm_robosuite._assets:compose_openarm_tabletop_mjcf.

Attributes:

Name	Type	Description
composer	str	"module.path:function" returning (xml, meshdir).
params	dict[str, object]	Keyword arguments for the composer (e.g. robot_lift_z, white_background).

Example

sc = SceneComposition( ... composer="pkg.scenes:compose_tabletop", ... params={"robot_lift_z": 0.36, "white_background": True}, ... ) sc.params["robot_lift_z"] 0.36

SceneDefaults

Bases: BaseModel

Per-robot scene rendering defaults.

These are values that scene composers may consult when the :class:SimScene YAML does not override them. Today the only field is :attr:top_camera, which the openarm_tabletop_pnp backend consumes; future scenes can extend this submodel as new defaults are pulled out of backend hardcodes.

Attributes:

Name	Type	Description
top_camera	TopCameraDefaults | None	Default placement for the scene-overview camera.

Example

sd = SceneDefaults( ... top_camera=TopCameraDefaults( ... pos=(0.2, 0.0, 0.95), ... target=(0.65, 0.0, 0.05), ... fovy=65.0, ... ), ... ) sd.top_camera.pos (0.2, 0.0, 0.95)

SceneGraph

Bases: BaseModel

Persistent hierarchical scene-graph spatial memory (ADR-0038).

The durable, queryable world model the S2 Reasoner consults to recall where objects/places/agents are and how to navigate to them. Distinct from the ephemeral ADR-0030 collision grid and advisory only — never a safety input (CLAUDE.md §1.1).

Invariants (enforced): node ids are unique; every edge references existing nodes.

Attributes:

Name	Type	Description
schema_version	Literal['0.1']	On-disk schema version ("0.1"; no backward-incompatible change yet). Now the repo is published it is versioned for real (CLAUDE.md §1.6): an incompatible change bumps it and ships a migrator.
nodes	list[SpatialNode]	All scene-graph nodes.
edges	list[SpatialEdge]	All directed relations between nodes.

SceneSpec

Bases: BaseModel

A physics scene declaration — the WORLD the robot acts in.

A scene is a deterministic, reproducible MuJoCo / MJX / etc. world: assets, lighting, cameras, fixed objects. Tasks (:class:TaskSpec) are evaluated INSIDE a scene; multiple tasks can share one scene (e.g. libero_spatial has 10 tasks per suite).

Attributes:

Name	Type	Description
id	str	Stable scene identifier used by the eval registry, e.g. "libero_spatial", "metaworld_mt50", "so100_tabletop".
backend	PhysicsBackend	Physics backend used to instantiate the scene.
assets_uri	str | None	Optional URI (file:// or hf://) pointing at scene assets (XML / MJCF / asset bundle). When None, the registered adapter resolves assets internally (LIBERO / MetaWorld pull theirs from their own packages).
observation_height	int	Default render height in pixels for camera obs.
observation_width	int	Default render width in pixels for camera obs.
cameras	list[str]	List of camera names the scene exposes. Adapters use this both to render and to map sensors to VLA feature keys.
backend_options	dict[str, object]	Backend-specific overrides (e.g. {"render_modes": ["rgb_array"]}). Opaque to the eval layer; passed through to the adapter.

Example

s = SceneSpec(id="libero_spatial", backend=PhysicsBackend.MUJOCO) s.observation_width 256

SelfVerifyEvidence

Bases: _FailureEvidenceBase

Evidence for KIND_SELFVERIFY — a self-check failed.

Attributes:

Name	Type	Description
kind	Literal['selfverify']	Discriminator (always "selfverify").
check	str	Short check identifier (e.g. "action_chunk.shape").
expected	str	Expected value as a string.
observed	str	Observed value as a string.

SensorBundle

Bases: BaseModel

Multi-modal sensor group, e.g. RealSense D455 = (RGB, depth, IMU).

Attributes:

Name	Type	Description
bundle_name	str	Unique name for this bundle.
sensors	list[SensorSpec]	List of constituent sensor specs.
sync	Literal['hardware', 'approximate', 'none']	Synchronization strategy.
sync_tolerance_ms	float	Tolerance for approximate synchronization.

SensorFrame

Bases: BaseModel

A single sensor frame — metadata plus an optional inline payload.

A :class:SensorFrame is the carrier passed from a SensorReader into :class:WorldState.image_frames. Exactly one of data, topic, or handle is populated:

• data carries the pixel bytes for in-process delivery and for trace capture. JSON-serialized as base64 by Pydantic, so payloads round-trip through model_dump_json / model_validate_json.
• topic points at a ROS 2 topic — the bytes live on the ROS 2 bus and the consumer subscribes for them. This is the path the Ros2ImageSensorReader uses and what :attr:WorldState.images already carries.
• handle is an opaque integer (e.g. CUDA NVMM pointer, DMA-BUF file descriptor) — in-process only, never serialized. Set by the GStreamerSensorReader when frames stay on the GPU.

Attributes:

Name	Type	Description
sensor_id	str	Sensor name; matches :attr:SensorSpec.name.
stamp_monotonic_ns	int	Capture-time monotonic timestamp in nanoseconds.
stamp_wall_ns	int	Capture-time wall-clock timestamp in nanoseconds.
encoding	FrameEncoding	How to interpret the bytes.
width	int	Image width in pixels.
height	int	Image height in pixels.
channels	int	Number of channels (3 for RGB/BGR, 1 for MONO/DEPTH).
data	bytes | None	Optional inline pixel payload (or compressed bytes for JPEG / PNG). Mutually exclusive with topic / handle.
topic	str | None	Optional ROS 2 topic reference. Mutually exclusive with data / handle.
handle	int | None	Optional opaque in-process handle (NVMM / DMA-BUF). Mutually exclusive with data / topic.
metadata	dict[str, object]	Free-form per-frame metadata (gain, exposure, …).

Example

SensorFrame( ... sensor_id="wrist_rgb", ... stamp_monotonic_ns=1, ... stamp_wall_ns=2, ... encoding=FrameEncoding.RGB8, ... width=640, ... height=480, ... topic="/cameras/wrist_rgb/image_raw", ... ).channels 3

model_post_init(_context: object) -> None

Exactly one of (data, topic, handle) must be set.

Source code in python/core/src/openral_core/schemas.py

def model_post_init(self, _context: object) -> None:
    """Exactly one of (data, topic, handle) must be set."""
    populated = [bool(self.data), self.topic is not None, self.handle is not None]
    n = sum(populated)
    if n != 1:
        raise ValueError(
            f"SensorFrame({self.sensor_id!r}): exactly one of "
            f"(data, topic, handle) must be set; got {n}."
        )

SensorModality

Bases: str, Enum

Physical sensing modality.

SensorReaderBackend

Bases: str, Enum

Which :class:SensorReader implementation to instantiate.

ADR-0010 §SensorReader — four backends are reserved. opencv_thread is the default and mirrors lerobot's per-camera background-thread pattern. ros2_image subscribes to a ROS 2 image topic published by a vendor driver. gstreamer runs a GStreamer pipeline whose appsink delivers frames (NVMM / DMA-BUF on Jetson; CPU bytes on x86).

holoscan is reserved-but-unimplemented: ADR-0010 Amendment 2026-05-12 evaluated NVIDIA Holoscan SDK as a parallel ingest backbone and deferred adoption (lean GStreamer with custom NvBufSurface glue won the comparison). The enum value exists so a future PR can add the backend additively without bumping the schema again; configs that select it today raise ROSConfigError at factory time.

SensorReaderConfig

Bases: BaseModel

Per-sensor backend configuration for the inference runner.

Picks which :class:SensorReader backend services a sensor and how the pipeline is parameterised. The optional publish_to_ros tee lets a GStreamer pipeline publish a downsampled stream to a ROS 2 topic for observability (rosbag2 / rqt_image_view) without putting the hot path through ROS.

Attributes:

Name	Type	Description
sensor_id	str	Sensor name; MUST match a :attr:SensorSpec.name in the target :class:RobotDescription.
backend	SensorReaderBackend	Which :class:SensorReaderBackend to instantiate.
backend_params	dict[str, object]	Backend-specific keyword arguments forwarded to the reader constructor. For gstreamer typically a single "pipeline" string. For opencv_thread typically {"device": "/dev/video0", "fps": 30}.
max_age_ms	int	How stale a frame may be before SensorReader.read_latest raises. Defaults to ~3 frames at 30 Hz.
publish_to_ros	bool	If True, the reader tees a downsampled stream to publish_topic at publish_rate_hz.
publish_topic	str | None	ROS 2 topic to publish to when publish_to_ros is True. Required iff publish_to_ros.
publish_rate_hz	float | None	Downsample rate for the ROS tee.

Example

SensorReaderConfig( ... sensor_id="wrist_rgb", ... backend=SensorReaderBackend.GSTREAMER, ... backend_params={ ... "pipeline": "v4l2src device=/dev/video0 ! " ... "nvv4l2decoder ! nvvideoconvert ! appsink" ... }, ... publish_to_ros=True, ... publish_topic="/cameras/wrist_rgb/image_raw", ... publish_rate_hz=5.0, ... ).backend.value 'gstreamer'

model_post_init(_context: object) -> None

Cross-field validation for the ROS tee.

Source code in python/core/src/openral_core/schemas.py

def model_post_init(self, _context: object) -> None:
    """Cross-field validation for the ROS tee."""
    if self.publish_to_ros and self.publish_topic is None:
        raise ValueError(
            f"SensorReaderConfig({self.sensor_id!r}): publish_to_ros is "
            f"True but publish_topic is unset; a ROS tee needs a topic."
        )
    if self.publish_topic is not None and not self.publish_to_ros:
        raise ValueError(
            f"SensorReaderConfig({self.sensor_id!r}): publish_topic is "
            f"set but publish_to_ros is False; enable publish_to_ros "
            f"explicitly or drop the topic."
        )

SensorRequirement

Bases: BaseModel

One sensor an rSkill needs the robot to provide.

Used by :class:RSkillManifest.sensors_required to declare the inputs the policy expects. The compatibility check resolves each entry against a :class:RobotDescription's sensors list and rejects the pairing if no robot sensor satisfies the requirement.

Resolution rules (in order):

1. If vla_feature_key is set, the robot MUST expose exactly one sensor with that vla_feature_key. The check then verifies modality matches and (if specified) the sensor's intrinsics meet min_width / min_height.
2. Otherwise, the robot must expose at least count sensors of the requested modality (each meeting any specified resolution minimum).

Attributes:

Name	Type	Description
modality	SensorModality	Required physical modality (rgb, depth, imu…).
vla_feature_key	str | None	Optional exact key the VLA expects, e.g. "observation.images.camera1". When set, the robot's matching SensorSpec.vla_feature_key must be identical.
min_width	int | None	Minimum image width in pixels (RGB/depth/IR/stereo only).
min_height	int | None	Minimum image height in pixels.
count	int	Number of robot sensors of this modality required when vla_feature_key is unset. Ignored when vla_feature_key is provided (a key uniquely identifies one sensor).

Example

SensorRequirement( ... modality=SensorModality.RGB, ... vla_feature_key="observation.images.camera1", ... min_width=224, ... min_height=224, ... ) # doctest: +ELLIPSIS SensorRequirement(modality=, ...)

SensorSpec

Bases: BaseModel

Generalizable sensor descriptor — covers all modalities.

Attributes:

Name	Type	Description
name	str	Human-readable sensor name, e.g. "head_rgb", "left_wrist_depth".
modality	SensorModality	Physical sensing modality.
frame_id	str	tf2 frame name.
parent_frame	str | None	tf2 parent frame (for static transform).
static_transform_xyz_rpy	tuple[float, float, float, float, float, float] | None	Static transform from parent to this sensor.
rate_hz	float	Expected publishing rate in Hz.
intrinsics	IntrinsicsPinhole | None	Pinhole camera intrinsics (if applicable).
encoding	str | None	Image encoding, e.g. "rgb8", "16UC1".
vla_feature_key	str | None	VLA observation dict key this sensor maps to, e.g. 'observation.images.camera1'. Used by skill loaders to auto-wire sensors to VLA input_features.
ros2_topic	str | None	ROS 2 topic name. None for non-ROS robots (USB, sim-only).
ros2_msg_type	str | None	ROS 2 message type, e.g. "sensor_msgs/Image". None for non-ROS robots (USB, sim-only).
qos_profile	Literal['sensor_data', 'reliable', 'transient_local', 'parameters']	QoS profile key.
vendor	str | None	Sensor vendor name.
model	str | None	Sensor model, e.g. "RealSense D455".
driver_pkg	str | None	ROS 2 driver package name.
metadata	dict[str, object]	Additional key-value metadata.

SimDescription

Bases: BaseModel

MuJoCo wiring for a single-arm robot, consumed by MujocoArmHAL.

The MJCF itself is named by :attr:RobotDescription.assets.mjcf (ADR-0058); this block carries only the joint↔qpos/qvel/actuator plumbing. All fields are optional with defaults derived from :attr:RobotDescription.joints. The default mapping is "1:1 in joint order, offset by 7 (qpos) / 6 (qvel) if floating_base is True" — which is correct for every robot in the open core today bar minor gripper bookkeeping.

Attributes:

Name	Type	Description
floating_base	bool	If True, the robot's MJCF has a 6-DoF free joint before the actuated joints (humanoids). In that case the default joint_qpos_addr[joints[i].name] = 7 + i and joint_qvel_addr[joints[i].name] = 6 + i.
joint_qpos_addr	dict[str, int] | None	Optional override of the joint→qpos-index map. When omitted, the default 1:1 mapping is used.
joint_qvel_addr	dict[str, int] | None	Optional override of the joint→qvel-index map. When omitted, defaults to joint_qpos_addr (or its default).
actuator_index	dict[str, int] | None	Optional override of the joint→actuator-index map. When omitted, defaults to the 1:1 mapping joints[i].name → i.
grippers	list[SimGripperDescription]	Optional list of :class:SimGripperDescription entries. Each must reference a joint by name that is also present in :attr:RobotDescription.joints. Single-arm robots have one entry (or none); bimanual robots have two (left + right).
settle_steps_default	int	Default number of mj_step calls executed per :meth:MujocoArmHAL.send_action. Defaults to 1.
keyframe_index	int | None	When set, :meth:MujocoArmHAL.connect calls mj_resetDataKeyframe(model, data, keyframe_index) before mj_forward. Required for MJCFs whose default MjData.qpos (zeros) sits outside the actuator ctrlrange — e.g. the gym-aloha parallel jaws, whose fingers have ctrlrange=[0.021, 0.057] and never recover from qpos = 0.
seed_ctrl_from_qpos	bool	When True, :meth:MujocoArmHAL.connect seeds data.ctrl[actuator] = data.qpos[joint_qpos_addr] for every controllable joint so position actuators hold the initial pose on the first mj_step. Required by the OpenArm v2 MJCF (its position actuators with per-class PD gains will drive qpos to ctrl == 0 otherwise).

Example

SimDescription(floating_base=True).floating_base True

SimEnvironment

Bases: BaseModel

A full sim configuration: the swappable (robot x scene x task x VLA) tuple.

The composed runtime form of a :class:SimScene plus an :class:RSkillManifest. openral_sim's CLI builds it; adapters consume it. Not loaded from YAML directly — from_yaml raises.

Attributes:

Name	Type	Description
robot_id	str	ID into the eval ROBOTS registry. Matches a robot's RobotDescription.name or an embodiment shortcut (e.g. "so100_follower", "franka_panda").
scene	SceneSpec	Scene the robot acts in.
task	TaskSpec	Task to evaluate. task.scene_id MUST equal scene.id.
vla	VLASpec	Policy that drives the robot.
base_pose	Pose6D | None	Optional per-episode mounting pose for the robot in the scene's world frame. None means "use the scene adapter's default" (e.g. an URDF's identity placement). Only honoured by free-axis scene adapters; setting it on a scene that registers with a fixed_robot= constraint (LIBERO, MetaWorld, RoboCasa, PushT, ALOHA) is rejected at CLI compose-time. The pose's frame_id is expected to be "world"; adapters anchor on the robot's :class:RobotDescription.base_frame.
seed	int	Global random seed (env reset, action sampling, torch RNG).
n_episodes	int	Number of episodes to run. 1 for a smoketest; ≥ 50 for an honest single-task success rate.
record_video	bool	Whether to ask the runner to record video frames.
save_dir	str | None	Optional directory to write artefacts (video, traces, json summary). None means "discard outputs".
metadata	dict[str, object]	Free-form notes (commit SHA, run owner, etc.).

Example

env = SimEnvironment( ... robot_id="franka_panda", ... scene=SceneSpec(id="libero_spatial"), ... task=TaskSpec( ... id="libero_spatial/task_0", ... scene_id="libero_spatial", ... instruction="pick up the cube", ... ), ... vla=VLASpec(id="smolvla", weights_uri="hf://lerobot/smolvla_libero"), ... ) env.task.scene_id == env.scene.id True

from_yaml(path: str) -> SimEnvironment classmethod

Removed: pass --rskill on the CLI; YAML carries only the scene.

After the feat(core,sim): SceneEnvironment + openral sim run --rskill, no legacy commit, the YAML shape for openral sim run is :class:SimScene (scene + task only; ADR-0041 renamed SceneEnvironment → SimScene) and the policy is supplied via the --rskill CLI flag. SimEnvironment is now the composed runtime form and is built by the CLI; loading it directly from YAML is no longer supported.

Source code in python/core/src/openral_core/schemas.py

@classmethod
def from_yaml(cls, path: str) -> SimEnvironment:  # pragma: no cover - removed
    """Removed: pass ``--rskill`` on the CLI; YAML carries only the scene.

    After the ``feat(core,sim): SceneEnvironment + openral sim run --rskill,
    no legacy`` commit, the YAML shape for ``openral sim run`` is
    :class:`SimScene` (scene + task only; ADR-0041 renamed
    ``SceneEnvironment`` → ``SimScene``) and the policy is
    supplied via the ``--rskill`` CLI flag. ``SimEnvironment`` is now
    the *composed runtime form* and is built by the CLI; loading it
    directly from YAML is no longer supported.
    """
    raise ROSConfigError(
        "SimEnvironment.from_yaml has been removed. "
        "Load the scene + task via SimScene.from_yaml(path) (or "
        "load_scene_strict(path, SimScene)) and supply the rSkill via "
        "--rskill rskills/<id> on the CLI; the CLI composes the "
        "SimEnvironment from those two artefacts."
    )

model_post_init(_context: object) -> None

Cross-field validation: task.scene_id must match scene.id.

Source code in python/core/src/openral_core/schemas.py

def model_post_init(self, _context: object) -> None:
    """Cross-field validation: task.scene_id must match scene.id."""
    if self.task.scene_id != self.scene.id:
        raise ValueError(
            f"SimEnvironment.task.scene_id ({self.task.scene_id!r}) does not "
            f"match scene.id ({self.scene.id!r}); a task can only run in its "
            f"declared scene."
        )

SimGripperDescription

Bases: BaseModel

Gripper wiring inside a MuJoCo MJCF.

Attributes:

Name	Type	Description
joint	str	Name of the gripper joint as it appears in :attr:RobotDescription.joints (the public, lerobot-style name — not the menagerie MJCF name).
ctrl_range	tuple[float, float]	(low, high) raw control range for the gripper actuator. In NORMALISED write mode, low ↔ closed (Action.gripper = 0), high ↔ open (Action.gripper = 1). In PASSTHROUGH write mode this is informational — MuJoCo clips on its own.
qpos_addrs	tuple[int, ...]	qpos indices used to compute the reported gripper position. When the gripper has multiple finger joints (Franka), list every finger's qpos index.
qpos_scale	float	Span used to normalise the summed/raw gripper qpos to [0, 1]. For Franka with two fingers each in [0, 0.04] m, qpos_scale = 0.08. Ignored by AFFINE_LOW_HIGH and PASSTHROUGH read modes but kept for symmetry with the base-class invariant.
read_mode	GripperReadMode	How to report the qpos — see :class:GripperReadMode.
write_mode	GripperWriteMode	How to map an Action's gripper value to ctrl — see :class:GripperWriteMode.
actuator_index	int | None	Explicit MJCF actuator index that receives the (mapped) gripper command. When omitted, defaults to the position of joint in :attr:RobotDescription.joints (i.e. the 1:1 mapping derived by SimDescription for arm joints).
mirror_actuator_index	int | None	Optional second actuator that receives the negation of the (mapped) gripper command. Models the Aloha parallel jaws where one motor drives +x and the other -x to keep the fingers symmetric. None for single-actuator grippers.

SimScene

Bases: DeployScene

Scene + task for openral sim run.

All task fields are overridable at the CLI level. max_steps and success_key are optional — omit them for open-ended experiments. Accepts a BenchmarkScene YAML transparently (the eval-specific fields n_episodes, seed, and metadata simply fill the defaults).

from_yaml(path: str) -> SimScene classmethod

Load and validate a SimScene YAML from disk.

Source code in python/core/src/openral_core/schemas.py

@classmethod
def from_yaml(cls, path: str) -> SimScene:
    """Load and validate a ``SimScene`` YAML from disk."""
    import yaml  # noqa: PLC0415

    with open(path, encoding="utf-8") as fh:
        data = yaml.safe_load(fh)
    return cls.model_validate(data)

SpatialEdge

Bases: BaseModel

A directed relation between two scene-graph nodes (ADR-0038).

Attributes:

Name	Type	Description
src	str	node_id of the source node.
dst	str	node_id of the destination node.
kind	SpatialRelationKind	Relation kind.

SpatialNode

Bases: BaseModel

A persistent, typed node in the scene-graph spatial memory (ADR-0038).

A superset of :class:DetectedObject for kind == OBJECT; also used for places, rooms, and agents. The pose is anchored in a durable, drift-corrected frame (typically the tf2 map frame); consumers resolve it to the live base frame at query time via tf2 — the node never stores a raw transform.

This is advisory world-model state consumed by the S2 Reasoner. It is never a safety input (ADR-0038 §1, CLAUDE.md §1.1): the safety kernel gates only on the live, bounded ADR-0030 geometric world.

Attributes:

Name	Type	Description
node_id	str	Stable identifier, unique within a :class:SceneGraph.
kind	SpatialNodeKind	Node kind (object / place / room / agent).
pose	Pose6D	6D pose, anchored in the durable map frame.
label	str	Semantic label or name (e.g. "fridge", "kitchen").
confidence	float	Confidence in [0, 1] for a perceived node.
bbox_3d	tuple[float, float, float, float, float, float] | None	Optional 3D bounding box (x_min, y_min, z_min, x_max, y_max, z_max).
embedding_ref	str | None	Optional handle into the vector store for open-vocabulary matching (ADR-0038 §5); None → label-only.
is_container	bool	Whether the node can hold other nodes (fridge, cabinet).
occludes_contents	bool	Whether contents are unobservable until the container is opened. Requires is_container.
first_seen_ns	int	Timestamp of first observation, in nanoseconds.
last_seen_ns	int	Timestamp of the most recent observation, in nanoseconds; must be >= first_seen_ns.
observation_count	int	Number of times this node has been observed.

Example

node = SpatialNode( ... node_id="fridge", ... kind=SpatialNodeKind.OBJECT, ... pose=Pose6D(xyz=(3.0, 1.0, 0.9), quat_xyzw=(0.0, 0.0, 0.0, 1.0), frame_id="map"), ... label="fridge", ... is_container=True, ... occludes_contents=True, ... first_seen_ns=1, ... last_seen_ns=2, ... ) node.is_container True

SpatialNodeKind

Bases: str, Enum

Kind of node in the persistent scene-graph spatial memory (ADR-0038).

OBJECT is the foundation (an accumulated :class:DetectedObject); PLACE is a standable navigation waypoint; ROOM is a semantic area grouping places/objects; AGENT is a person or robot with a pose — the requester of a task is an AGENT so "bring it back to me" resolves to a concrete goal.

SpatialRelationKind

Bases: str, Enum

Kind of directed edge between scene-graph nodes (ADR-0038).

CONTAINS links a room/container to what is inside it (a fridge CONTAINS a wine bottle); AT_PLACE links an object/agent to the waypoint to stand at to reach it; TRAVERSABLE_TO is the topological navigation graph between places/rooms; ON / NEAR are incidental object-to-object spatial relations.

SphereShape

Bases: BaseModel

Sphere collision primitive — the simplest convex link/obstacle volume.

Attributes:

Name	Type	Description
shape	Literal['sphere']	Discriminator (always "sphere").
radius_m	float	Sphere radius in metres.

Example

SphereShape(radius_m=0.05).shape 'sphere'

StateContract

Bases: BaseModel

Per-rSkill state-vector contract.

Surfaces the proprioception layout the checkpoint was trained against so the runtime adapter does not have to learn it from a YAML override. ADR-0014 + ADR-0027.

Attributes:

Name	Type	Description
layout	StateLayout | None	Named proprioception layout — see :data:StateLayout. LIBERO / MetaWorld / pusht / aloha leave this None and consume the raw joint-position vector directly.
dim	int | None	Explicit state dimension override. The runtime adapter clips or pads the env state vector to this width before handing it to the policy.
bindings	StateContractBindings | None	Per-robot source bindings — TF frame names + JointState entries the runtime adapter pulls values from. REQUIRED when layout is in :data:WRAPPED_TASK_SPACE_LAYOUTS, FORBIDDEN otherwise (joint-space layouts have no source to parameterise).

StateContractBindings

Bases: BaseModel

Per-robot source bindings for an rSkill's state_contract.layout. ADR-0027.

Symmetric to :class:ControlModeSemantics on the action side (joint_order + reference_frame + gripper_convention): the rSkill manifest names the shape via :attr:StateContract.layout; these bindings name the sources on the deploying robot. The layout-adapter registry (openral_state_adapter) joins shape + bindings + live :class:sensor_msgs/JointState + live /tf into the per-checkpoint state vector.

Bindings are required for layouts in :data:WRAPPED_TASK_SPACE_LAYOUTS and forbidden otherwise (the joint-space layouts have no source to parameterise).

Attributes:

Name	Type	Description
eef_frame	str | None	tf2 link name of the end effector (e.g. "panda_hand"). Required for any layout reading EE position / orientation.
base_frame	str | None	tf2 link name of the mobile base (e.g. "base_link"). Required for any layout reading base position / orientation OR base-relative EE poses.
world_frame	str | None	tf2 root frame the base pose is expressed in (default "map" — slam_toolbox publishes map → odom → base_link). Set to "odom" for deployments without SLAM.
gripper_qpos_joints	list[str]	JointState.name entries whose positions populate the gripper-width slot(s) of the layout, in the order the policy expects (e.g. ["panda_finger_joint1", "panda_finger_joint2"]). Empty for grasper-less robots or layouts without a gripper slot.
quaternion_convention	Literal['xyzw', 'wxyz']	Component order of any quaternion in the assembled vector. ROS / TF2 default is "xyzw" (the geometry_msgs/Quaternion field order). Some upstream checkpoints expect "wxyz" — declare it here so the assembler permutes once, at the boundary.

StateRepresentation

Bases: str, Enum

State vector representation format.

SuppressedSummaryEvidence

Bases: _FailureEvidenceBase

Evidence for KIND_SUPPRESSED_SUMMARY — rolling rate-limit roll-up.

Emitted at ~1 Hz by :class:openral_observability.FailureBusPublisher when one or more (kind, severity) buckets dropped events during the past window.

Attributes:

Name	Type	Description
kind	Literal['suppressed_summary']	Discriminator (always "suppressed_summary").
window_s	float	Length of the summarized window, in seconds.
kinds	list[int]	Parallel array of suppressed KIND_* values.
severities	list[int]	Parallel array of suppressed SEVERITY_* values.
counts	list[int]	Parallel array of dropped-event counts per bucket.

TaskSpec

Bases: BaseModel

A task declaration — WHAT the robot must achieve inside a scene.

Tasks decouple scene assets (SceneSpec) from goal-conditioning. The same scene can host many tasks; the same task can occasionally be run in multiple compatible scenes (rare; usually 1:1 with scene_id).

Success is evaluated by the scene adapter — the reward / success signal comes from the underlying gym env (info['is_success'], info['success'], terminal reward, …). success_key lets manifest authors override which info field the runner reads.

Attributes:

Name	Type	Description
id	str	Stable task identifier, e.g. "libero_spatial/task_0", "metaworld/push-v3". Adapters split on / to resolve.
scene_id	str	ID of the :class:SceneSpec this task runs in.
instruction	str	Natural-language goal handed to the VLA as the "task" text input. Some adapters override this with a description baked into the underlying suite (LIBERO, MetaWorld).
max_steps	int | None	Episode budget. Adapters may clip to scene-internal limits. None means unset; BenchmarkScene enforces a concrete value via its model validator.
success_key	str | None	Key inside info returned by env.step() whose truthy value marks task success. None means unset; :class:BenchmarkScene enforces a concrete value via its model validator (required for paper-comparison benchmarks).
metadata	dict[str, object]	Free-form per-task notes (paper reference, dataset split, …).

Example

t = TaskSpec( ... id="libero_spatial/task_0", ... scene_id="libero_spatial", ... instruction="pick up the black bowl", ... max_steps=200, ... success_key="is_success", ... ) t.max_steps 200 t.success_key 'is_success' TaskSpec(id="libero_spatial/task_0", scene_id="libero_spatial").max_steps is None True

TickResult

Bases: BaseModel

One tick's record returned by :meth:InferenceRunner.tick.

Carries the timing breakdown so the parent OTel span can attach exact sub-stage durations and the latency budget enforcement can flag violations without re-instrumenting.

The hardware fields (sensors_ms..hal_ms, safety_violations, action_applied) are the original (v1) surface used by :class:HardwareRunner. The sim-specific fields (step_idx..truncated) were added by ADR-0010 amendment 1 when :class:SimRunner adopted per-step tick semantics; hardware leaves them at their defaults (None), so a hardware tick serialises identically to v1 under model_dump(exclude_none=True).

Attributes:

Name	Type	Description
stamp_ns	int	Tick wall-clock timestamp in nanoseconds (tick start).
tick_idx	int	0-indexed tick counter within a run.
sensors_ms	float	SensorReader.read_latest total wall-time.
world_state_ms	float	WorldStateAggregator.snapshot wall-time.
inference_ms	float	Skill.step wall-time (the chunk dispatch cost, not the full chunked inference — see :class:ChunkedExecutor).
safety_ms	float	Safety check wall-time.
hal_ms	float	HAL.send_action wall-time.
tick_ms	float	End-to-end tick wall-time including the rate-limiter overhead.
chunk_index	int | None	Index of the action played out from a chunked executor. None for non-chunked skills.
safety_violations	list[str]	List of safety-violation reason strings emitted during this tick. Non-empty implies the action was either clamped or dropped per the safety policy.
action_applied	bool	False when the :class:DeadlineOverrunPolicy was drop and the runner elected not to publish this tick's action, or when a sim tick is the reset-tick between episodes (no inference / env step happened).
step_idx	int | None	0-indexed step within the current episode. Set by :class:SimRunner on step-ticks; None on hardware ticks and on sim reset-ticks.
episode_idx	int | None	0-indexed episode within the current run. Set by :class:SimRunner on every tick (including reset-ticks); None on hardware ticks.
reward	float | None	Env step reward for this tick. Set by :class:SimRunner on step-ticks; None on hardware ticks and reset-ticks.
terminated	bool | None	Whether the env signalled natural termination this tick. Set by :class:SimRunner; None elsewhere.
truncated	bool | None	Whether the env hit its step budget this tick. Set by :class:SimRunner; None elsewhere.
trace_context	str | None	Full W3C traceparent for this tick's rskill.tick span, in the form 00-<trace_id_hex>-<span_id_hex>-<flags_hex>. Optional — set by the runner when an OTel context is active so offline consumers (dataset writers, post-hoc analysers) can resume the trace without re-deriving it from the live span. Default None for byte-identical v1 JSON under model_dump(exclude_none=True).

TimeoutEvidence

Bases: _FailureEvidenceBase

Evidence for KIND_TIMEOUT — an operation missed its deadline.

Attributes:

Name	Type	Description
kind	Literal['timeout']	Discriminator (always "timeout").
operation	str	Short name of the operation that timed out (e.g. "skill.step", "hal.read_state", "reasoner.tick").
deadline_s	float	Configured deadline in seconds.
elapsed_s	float	Actual elapsed wall-clock time in seconds.

TopCameraDefaults

Bases: BaseModel

Default placement for the scene-level "top" (a.k.a. "base") camera.

Per-robot scene defaults consumed by sim backends that render an overview camera (today: the openarm_tabletop_pnp scene). The values describe a look-at camera pointed from pos toward target with vertical field-of-view fovy in degrees.

Backend YAML overrides (scene.backend_options.top_camera_*) still win — this submodel is the default fed to the composer when no override is set, replacing the previous module-level constants in openral_sim.backends.openarm_robosuite._assets.

Attributes:

Name	Type	Description
pos	tuple[float, float, float]	(x, y, z) world-frame camera position in metres.
target	tuple[float, float, float]	(x, y, z) world-frame look-at point in metres.
fovy	float	Vertical field-of-view in degrees.

Example

TopCameraDefaults(pos=(0.2, 0.0, 0.95), target=(0.65, 0.0, 0.05), fovy=65.0).fovy 65.0

UrdfAsset

Bases: BaseModel

A URDF asset reference plus its robot_state_publisher wiring.

ADR-0027 / ADR-0058. The ref is resolved by :func:openral_core.assets.resolve_asset; root_frame and base_to_root_xyz_rpy carry the static transform that bridges a URDF whose root link differs from the robot's base_frame (e.g. Franka's panda_link0 mounted onto a base_link mobile platform).

Attributes:

Name	Type	Description
ref	str	Asset reference (rd:<module>, file:<relpath>, or ros2://robot_description for runtime topic-supplied URDFs).
root_frame	str | None	The URDF's root link name when it differs from :attr:RobotDescription.base_frame. None → the URDF root equals base_frame (no static transform needed).
base_to_root_xyz_rpy	tuple[float, float, float, float, float, float] | None	The 6-DoF transform [x, y, z, roll, pitch, yaw] (metres + radians) published via static_transform_publisher to bridge base_frame to :attr:root_frame. None when :attr:root_frame is None.

Example

UrdfAsset(ref="rd:panda_description", root_frame="panda_link0").root_frame 'panda_link0'

VLASpec

Bases: BaseModel

A VLA / policy declaration — the BRAIN driving the robot.

Lightweight pointer to a policy: either an installed rSkill (referenced by manifest name) or a raw HF Hub repo. The eval registry resolves this to a runtime adapter that returns Action objects.

Attributes:

Name	Type	Description
id	str	Adapter id in the eval registry, e.g. "smolvla", "pi05", "xvla", "random", "zero". Picks the loader.
weights_uri	str	Where to fetch weights from. Pass a bare rSkill reference: a name (smolvla-libero), a path (rskills/smolvla-libero), or a bare HF repo ID (OpenRAL/rskill-smolvla-libero). The :class:openral_sim.SimRunner requires a locally-resolvable reference — raw "hf://" URIs are rejected. Other URI shapes (e.g. "mock://") are still parsed by the schema so unit tests of the eval registries can run without an rSkill, but they will fail-fast if used with the runner.
device	str	Torch device override. "auto" picks cuda:0 if available, otherwise cpu.
runtime	RSkillRuntime | None	Optional runtime override; None means "use whatever the policy/manifest declares".
quantization	QuantizationConfig | None	Optional quantization override for this run.
deterministic	bool	When True, set torch.use_deterministic_algorithms and disable cuDNN benchmarking.
extra	dict[str, object]	Adapter-specific options, e.g. {"chunk_size": 16}.

Example

v = VLASpec(id="smolvla", weights_uri="rskills/smolvla-libero") v.device 'auto'

WamEvidence

Bases: _FailureEvidenceBase

Evidence for KIND_WAM — a world-action-model discrepancy.

Attributes:

Name	Type	Description
kind	Literal['wam']	Discriminator (always "wam").
horizon	int	Rollout horizon at which the discrepancy was measured.
discrepancy	float	Scalar discrepancy in the WAM's native units.
wam_id	str	Identifier of the WAM that fired.

WorkspaceEvidence

Bases: _FailureEvidenceBase

Evidence for KIND_WORKSPACE — EE pose left the safety AABB.

Attributes:

Name	Type	Description
kind	Literal['workspace']	Discriminator (always "workspace").
ee_name	str	End-effector that violated the box.
measured_xyz	tuple[float, float, float]	Measured position in metres.
box_min	tuple[float, float, float]	Box minimum corner (x_min, y_min, z_min).
box_max	tuple[float, float, float]	Box maximum corner (x_max, y_max, z_max).

WorldCollisionPrimitive

Bases: BaseModel

A placed convex obstacle volume in the world (ADR-0030).

The world-frame analogue of :class:LinkCollisionGeometry: a convex primitive plus the pose that places it. Populated by perception / SLAM and consumed by the kernel's world-collision phase against the robot's link capsules. A bounded, capped set is the kernel's world model (mesh obstacles are out of scope for the allocation-free check).

Attributes:

Name	Type	Description
shape	CollisionShape	The convex primitive (capsule or sphere).
pose	Pose6D	Pose of the primitive's local origin in the world frame.
object_id	str | None	Optional stable identifier (e.g. a :attr:DetectedObject.track_id rendered as text) surfaced in :attr:CollisionEvidence.link_b_or_object.

WorldState

Bases: BaseModel

Snapshot consumed by Reasoner and Skills.

Attributes:

Name	Type	Description
stamp_ns	int	Snapshot timestamp in nanoseconds.
joint_state	JointState	Current joint state.
base_pose	Pose6D | None	Base link pose (mobile robots).
base_twist	tuple[float, float, float, float, float, float] | None	Base link twist (vx, vy, vz, wx, wy, wz).
ee_poses	dict[str, Pose6D]	End-effector poses keyed by EE name.
contact_forces	dict[str, tuple[float, ...]]	Contact forces keyed by contact name.
images	dict[str, str]	Sensor name → ROS 2 topic reference (not the raw image).
image_frames	dict[str, SensorFrame] | None	Optional per-sensor :class:SensorFrame snapshot for no-ROS / in-process deployments. When None the consumer reads frames via :attr:images topic refs as before.
point_clouds	dict[str, str]	Sensor name → ROS 2 topic reference.
tactile	dict[str, str]	Sensor name → ROS 2 topic reference.
detected_objects	list[DetectedObject]	List of detected objects.
battery_pct	float | None	Battery percentage in [0, 100].
diagnostics	dict[str, Literal['ok', 'warn', 'error', 'stale']]	Per-component diagnostic status.
collision_primitives	list[WorldCollisionPrimitive]	Bounded set of placed convex obstacle volumes the kernel checks robot links against (world-collision). Empty until a perception / SLAM source populates it. ADR-0030.
occupancy_grid	OccupancyGridRef | None	Optional 2D occupancy grid reference for mobile-base footprint checks. None until populated; an absent or stale grid is treated as unavailable (fail-closed). ADR-0030.

canonical_slots_for_representation(rep: ActionRepresentation, *, dim: int, description: RobotDescription) -> list[ActionSlot] | None

Build the canonical :class:ActionSlot layout for a representation.

ADR-0036. The skill_runner calls this to expand a skill that declares only ActionContract.representation (no explicit slots) into a typed slot layout it can dispatch. Joint representations return None so the caller keeps the legacy whole-vector JOINT_POSITION path; cartesian / gripper representations get one slot per control mode, addressed at the robot's primary end-effector (description.end_effectors[0]).

:class:EndEffectorSpec carries no explicit tf-frame field, so the end-effector's name is used as the slot frame (the EE name is the tf frame the cartesian pose/delta is expressed in for the canonical embodiments).

Parameters:

Name	Type	Description	Default
rep	ActionRepresentation	The VLA's declared action-vector representation.	required
dim	int	Dimensionality of the policy's flat action vector.	required
description	RobotDescription	The target robot — its primary end-effector names the cartesian/gripper slots.	required

Returns:

Type	Description
list[ActionSlot] | None	For joint representations: None (caller keeps the legacy
list[ActionSlot] | None	whole-vector JOINT_POSITION path). Otherwise a list of typed
list[ActionSlot] | None	class:ActionSlot s that satisfy the per-slot validators.

Raises:

Type	Description
ROSConfigError	If the representation needs an end-effector but description.end_effectors is empty, or if dim is too small for the layout (DELTA_EE_6D / CARTESIAN_POSE need dim >= 6; DELTA_EE_6D_PLUS_GRIPPER needs dim >= 7).

Example

desc = RobotDescription( ... name="ex_robot", ... embodiment_kind=EmbodimentKind.MANIPULATOR, ... joints=[ ... JointSpec( ... name="j1", ... joint_type=JointType.REVOLUTE, ... parent_link="base_link", ... child_link="link_1", ... ) ... ], ... end_effectors=[EndEffectorSpec(name="ee0", kind="parallel_gripper")], ... capabilities=RobotCapabilities(), ... safety=SafetyEnvelope(), ... ) slots = canonical_slots_for_representation( ... ActionRepresentation.DELTA_EE_6D, dim=6, description=desc ... ) slots[0].control_mode is ControlMode.CARTESIAN_DELTA True

Source code in python/core/src/openral_core/schemas.py

def canonical_slots_for_representation(
    rep: ActionRepresentation,
    *,
    dim: int,
    description: RobotDescription,
) -> list[ActionSlot] | None:
    """Build the canonical :class:`ActionSlot` layout for a representation.

    ADR-0036. The skill_runner calls this to expand a skill that declares
    only ``ActionContract.representation`` (no explicit ``slots``) into a
    typed slot layout it can dispatch. Joint representations return
    ``None`` so the caller keeps the legacy whole-vector ``JOINT_POSITION``
    path; cartesian / gripper representations get one slot per control
    mode, addressed at the robot's primary end-effector
    (``description.end_effectors[0]``).

    :class:`EndEffectorSpec` carries no explicit tf-frame field, so the
    end-effector's ``name`` is used as the slot ``frame`` (the EE name is
    the tf frame the cartesian pose/delta is expressed in for the
    canonical embodiments).

    Args:
        rep: The VLA's declared action-vector representation.
        dim: Dimensionality of the policy's flat action vector.
        description: The target robot — its primary end-effector names
            the cartesian/gripper slots.

    Returns:
        For joint representations: ``None`` (caller keeps the legacy
        whole-vector ``JOINT_POSITION`` path). Otherwise a list of typed
        :class:`ActionSlot` s that satisfy the per-slot validators.

    Raises:
        ROSConfigError: If the representation needs an end-effector but
            ``description.end_effectors`` is empty, or if ``dim`` is too
            small for the layout (``DELTA_EE_6D`` / ``CARTESIAN_POSE``
            need ``dim >= 6``; ``DELTA_EE_6D_PLUS_GRIPPER`` needs
            ``dim >= 7``).

    Example:
        >>> desc = RobotDescription(
        ...     name="ex_robot",
        ...     embodiment_kind=EmbodimentKind.MANIPULATOR,
        ...     joints=[
        ...         JointSpec(
        ...             name="j1",
        ...             joint_type=JointType.REVOLUTE,
        ...             parent_link="base_link",
        ...             child_link="link_1",
        ...         )
        ...     ],
        ...     end_effectors=[EndEffectorSpec(name="ee0", kind="parallel_gripper")],
        ...     capabilities=RobotCapabilities(),
        ...     safety=SafetyEnvelope(),
        ... )
        >>> slots = canonical_slots_for_representation(
        ...     ActionRepresentation.DELTA_EE_6D, dim=6, description=desc
        ... )
        >>> slots[0].control_mode is ControlMode.CARTESIAN_DELTA
        True
    """
    if rep in (ActionRepresentation.JOINT_POSITIONS, ActionRepresentation.JOINT_VELOCITIES):
        # Legacy whole-vector JOINT path — the caller emits one implicit
        # JOINT_POSITION / JOINT_VELOCITY slot covering the whole vector.
        return None

    if not description.end_effectors:
        raise ROSConfigError(
            f"canonical_slots_for_representation: representation {rep.value!r} addresses "
            "an end-effector but the robot description declares no end_effectors; "
            "cannot build a cartesian/gripper slot layout."
        )
    primary = description.end_effectors[0]
    ee_name = primary.name
    # EndEffectorSpec has no explicit tf-frame field; the EE name is the
    # tf frame the cartesian pose/delta is expressed in.
    ee_frame = ee_name

    has_gripper = rep is ActionRepresentation.DELTA_EE_6D_PLUS_GRIPPER
    min_dim = _EE_6D_WIDTH + 1 if has_gripper else _EE_6D_WIDTH
    if dim < min_dim:
        raise ROSConfigError(
            f"canonical_slots_for_representation: representation {rep.value!r} requires "
            f"dim >= {min_dim} but action_contract.dim={dim} is too small for the layout."
        )

    cart_mode = (
        ControlMode.CARTESIAN_POSE
        if rep is ActionRepresentation.CARTESIAN_POSE
        else ControlMode.CARTESIAN_DELTA
    )
    slots: list[ActionSlot] = [
        ActionSlot(
            range=(0, _EE_6D_WIDTH - 1),
            control_mode=cart_mode,
            ee=ee_name,
            frame=ee_frame,
        )
    ]
    if has_gripper:
        slots.append(
            ActionSlot(
                range=(_EE_6D_WIDTH, dim - 1),
                control_mode=ControlMode.GRIPPER_POSITION,
                ee=ee_name,
            )
        )
    return slots

control_modes_for_representation(rep: ActionRepresentation) -> set[ControlMode]

Map an :class:ActionRepresentation to the :class:ControlMode s it drives.

ADR-0036. Used by the reasoner's deploy-path palette gate: a skill is only offered when the target robot advertises every mode in the returned set.

Parameters:

Name	Type	Description	Default
rep	ActionRepresentation	The VLA's declared action-vector representation.	required

Returns:

Type	Description
set[ControlMode]	The set of :class:ControlMode s the representation maps onto.

Example

control_modes_for_representation(ActionRepresentation.DELTA_EE_6D_PLUS_GRIPPER) == { ... ControlMode.CARTESIAN_DELTA, ... ControlMode.GRIPPER_POSITION, ... } True

Source code in python/core/src/openral_core/schemas.py

def control_modes_for_representation(rep: ActionRepresentation) -> set[ControlMode]:
    """Map an :class:`ActionRepresentation` to the :class:`ControlMode` s it drives.

    ADR-0036. Used by the reasoner's deploy-path palette gate: a skill is
    only offered when the target robot advertises *every* mode in the
    returned set.

    Args:
        rep: The VLA's declared action-vector representation.

    Returns:
        The set of :class:`ControlMode` s the representation maps onto.

    Example:
        >>> control_modes_for_representation(ActionRepresentation.DELTA_EE_6D_PLUS_GRIPPER) == {
        ...     ControlMode.CARTESIAN_DELTA,
        ...     ControlMode.GRIPPER_POSITION,
        ... }
        True
    """
    if rep is ActionRepresentation.JOINT_POSITIONS:
        return {ControlMode.JOINT_POSITION}
    if rep is ActionRepresentation.JOINT_VELOCITIES:
        return {ControlMode.JOINT_VELOCITY}
    if rep is ActionRepresentation.DELTA_EE_6D:
        return {ControlMode.CARTESIAN_DELTA}
    if rep is ActionRepresentation.DELTA_EE_6D_PLUS_GRIPPER:
        return {ControlMode.CARTESIAN_DELTA, ControlMode.GRIPPER_POSITION}
    # CARTESIAN_POSE — the only remaining enum member.
    return {ControlMode.CARTESIAN_POSE}

extract_base_sim_joint_names(description: RobotDescription) -> tuple[str, str, str] | None

Return (forward, side, yaw) MJCF joint names from any mobile-base description.

ADR-0025 — generic, robot-agnostic helper. Consumes the :attr:RobotDescription.base_joints declaration + each referenced :class:JointSpec's :attr:~JointSpec.sim_joint_name override. Works for any robot whose robot.yaml declares both fields:

• base_joints: [<forward>, <side>, <yaw>] at the top level.
• Each of the three referenced joints carries a sim_joint_name: "..." mapping its URDF-shape name to the MJCF/MuJoCo joint name the simulator emits (typically auto-prefixed under a composed scene).

Returns None when the description has no base_joints block, fewer than three entries (the schema permits arbitrary list length so future non-planar bases can declare more — this helper specialises to the planar-base case), or any referenced joint lacks sim_joint_name. Callers should treat None as "fall back to module defaults" — the sim-side ray-cast helpers in :mod:openral_sim.backends.robocasa accept this contract.

Parameters:

Name	Type	Description	Default
description	RobotDescription	The robot's manifest, loaded via :meth:RobotDescription.from_yaml.	required

Returns:

Type	Description
tuple[str, str, str] | None	(forward, side, yaw) MJCF joint names, or None when
tuple[str, str, str] | None	the description doesn't carry a complete mobile-base block.

Example

See robots/panda_mobile/robot.yaml for a real fixture.

desc = RobotDescription( ... name="ex", ... embodiment_kind=EmbodimentKind.MOBILE_MANIPULATOR, ... joints=[ ... JointSpec( ... name="base_x", ... joint_type=JointType.PRISMATIC, ... parent_link="world", ... child_link="base_x_link", ... sim_joint_name="mobilebase0_joint_mobile_forward", ... ), ... JointSpec( ... name="base_y", ... joint_type=JointType.PRISMATIC, ... parent_link="base_x_link", ... child_link="base_y_link", ... sim_joint_name="mobilebase0_joint_mobile_side", ... ), ... JointSpec( ... name="base_yaw", ... joint_type=JointType.REVOLUTE, ... parent_link="base_y_link", ... child_link="base_link", ... sim_joint_name="mobilebase0_joint_mobile_yaw", ... ), ... ], ... capabilities=RobotCapabilities(embodiment_tags=["ex"]), ... safety=SafetyEnvelope(), ... base_joints=["base_x", "base_y", "base_yaw"], ... ) extract_base_sim_joint_names(desc)[0] 'mobilebase0_joint_mobile_forward'

Source code in python/core/src/openral_core/schemas.py

def extract_base_sim_joint_names(
    description: RobotDescription,
) -> tuple[str, str, str] | None:
    """Return ``(forward, side, yaw)`` MJCF joint names from any mobile-base description.

    ADR-0025 — generic, robot-agnostic helper. Consumes the
    :attr:`RobotDescription.base_joints` declaration + each referenced
    :class:`JointSpec`'s :attr:`~JointSpec.sim_joint_name` override.
    Works for any robot whose ``robot.yaml`` declares both fields:

    * ``base_joints: [<forward>, <side>, <yaw>]`` at the top level.
    * Each of the three referenced joints carries a
      ``sim_joint_name: "..."`` mapping its URDF-shape ``name`` to the
      MJCF/MuJoCo joint name the simulator emits (typically
      auto-prefixed under a composed scene).

    Returns ``None`` when the description has no ``base_joints``
    block, fewer than three entries (the schema permits arbitrary
    list length so future non-planar bases can declare more — this
    helper specialises to the planar-base case), or any referenced
    joint lacks ``sim_joint_name``. Callers should treat ``None`` as
    "fall back to module defaults" — the sim-side ray-cast helpers
    in :mod:`openral_sim.backends.robocasa` accept this contract.

    Args:
        description: The robot's manifest, loaded via
            :meth:`RobotDescription.from_yaml`.

    Returns:
        ``(forward, side, yaw)`` MJCF joint names, or ``None`` when
        the description doesn't carry a complete mobile-base block.

    Example:
        >>> # See `robots/panda_mobile/robot.yaml` for a real fixture.
        >>> desc = RobotDescription(
        ...     name="ex",
        ...     embodiment_kind=EmbodimentKind.MOBILE_MANIPULATOR,
        ...     joints=[
        ...         JointSpec(
        ...             name="base_x",
        ...             joint_type=JointType.PRISMATIC,
        ...             parent_link="world",
        ...             child_link="base_x_link",
        ...             sim_joint_name="mobilebase0_joint_mobile_forward",
        ...         ),
        ...         JointSpec(
        ...             name="base_y",
        ...             joint_type=JointType.PRISMATIC,
        ...             parent_link="base_x_link",
        ...             child_link="base_y_link",
        ...             sim_joint_name="mobilebase0_joint_mobile_side",
        ...         ),
        ...         JointSpec(
        ...             name="base_yaw",
        ...             joint_type=JointType.REVOLUTE,
        ...             parent_link="base_y_link",
        ...             child_link="base_link",
        ...             sim_joint_name="mobilebase0_joint_mobile_yaw",
        ...         ),
        ...     ],
        ...     capabilities=RobotCapabilities(embodiment_tags=["ex"]),
        ...     safety=SafetyEnvelope(),
        ...     base_joints=["base_x", "base_y", "base_yaw"],
        ... )
        >>> extract_base_sim_joint_names(desc)[0]
        'mobilebase0_joint_mobile_forward'
    """
    # Planar mobile bases carry exactly three holonomic axes (forward,
    # side, yaw); the schema doesn't constrain `base_joints` length so
    # future non-planar bases can declare richer surfaces, but this
    # helper specialises here.
    planar_base_dof = 3
    if description.base_joints is None or len(description.base_joints) < planar_base_dof:
        return None
    by_name = {j.name: j for j in description.joints}
    sim_names: list[str | None] = []
    for ref in description.base_joints[:planar_base_dof]:
        spec = by_name.get(ref)
        if spec is None:
            return None
        sim_names.append(spec.sim_joint_name)
    if any(n is None for n in sim_names):
        return None
    forward, side, yaw = sim_names
    # Narrow tuple[str | None, ...] → tuple[str, str, str] for type checkers.
    assert isinstance(forward, str)
    assert isinstance(side, str)
    assert isinstance(yaw, str)
    return (forward, side, yaw)

load_benchmark_suite(path: str) -> list[BenchmarkScene]

Load a bare list of :class:BenchmarkScenes from benchmarks/<id>.yaml.

ADR-0042 (June 2026) deleted the BenchmarkSpec wrapper class. A benchmark suite YAML is now a bare YAML list at the root; the suite id is derived from the filename stem (e.g. benchmarks/libero_spatial.yaml has suite id "libero_spatial"). Pre-ADR-0042 the YAML root was a {id, tasks, metadata} mapping wrapping the scenes — this loader rejects that shape with an explicit redirect message.

Per-scene Pydantic validation runs here. Suite-level invariants (uniformity, uniqueness, non-empty) are NOT enforced; call :func:raise_on_invalid_suite separately with the suite id of your choice (typically Path(path).stem). This split lets tests construct invalid in-memory suites without touching the filesystem.

Parameters:

Name	Type	Description	Default
path	str	Filesystem path to a benchmark YAML file.	required

Returns:

Type	Description
list[BenchmarkScene]	Validated list of :class:BenchmarkScenes in YAML order.

Raises:

Type	Description
FileNotFoundError	If path does not exist.
ROSConfigError	If the YAML root is not a list (legacy dict-shape gets an explicit ADR-0042 redirect), or any entry fails :class:BenchmarkScene validation.

Example

from pathlib import Path from openral_core import load_benchmark_suite, raise_on_invalid_suite scenes = load_benchmark_suite("benchmarks/libero_spatial.yaml") suite_id = Path("benchmarks/libero_spatial.yaml").stem raise_on_invalid_suite(scenes, suite_id=suite_id) len(scenes) 10

Source code in python/core/src/openral_core/loaders.py

def load_benchmark_suite(path: str) -> list[BenchmarkScene]:
    """Load a bare list of :class:`BenchmarkScene`s from ``benchmarks/<id>.yaml``.

    ADR-0042 (June 2026) deleted the ``BenchmarkSpec`` wrapper class. A
    benchmark suite YAML is now a bare YAML list at the root; the suite id
    is derived from the filename stem (e.g. ``benchmarks/libero_spatial.yaml``
    has suite id ``"libero_spatial"``). Pre-ADR-0042 the YAML root was a
    ``{id, tasks, metadata}`` mapping wrapping the scenes — this loader
    rejects that shape with an explicit redirect message.

    Per-scene Pydantic validation runs here. Suite-level invariants
    (uniformity, uniqueness, non-empty) are NOT enforced; call
    :func:`raise_on_invalid_suite` separately with the suite id of your
    choice (typically ``Path(path).stem``). This split lets tests
    construct invalid in-memory suites without touching the filesystem.

    Args:
        path: Filesystem path to a benchmark YAML file.

    Returns:
        Validated list of :class:`BenchmarkScene`s in YAML order.

    Raises:
        FileNotFoundError: If ``path`` does not exist.
        ROSConfigError: If the YAML root is not a list (legacy dict-shape
            gets an explicit ADR-0042 redirect), or any entry fails
            :class:`BenchmarkScene` validation.

    Example:
        >>> from pathlib import Path
        >>> from openral_core import load_benchmark_suite, raise_on_invalid_suite
        >>> scenes = load_benchmark_suite("benchmarks/libero_spatial.yaml")
        >>> suite_id = Path("benchmarks/libero_spatial.yaml").stem
        >>> raise_on_invalid_suite(scenes, suite_id=suite_id)
        >>> len(scenes)
        10
    """
    raw_obj = _yaml.safe_load(Path(path).read_text(encoding="utf-8"))

    if isinstance(raw_obj, dict):
        raise ROSConfigError(
            f"{path}: YAML root is a mapping, but ADR-0042 (June 2026) "
            "deleted the BenchmarkSpec wrapper. A benchmark suite YAML is "
            "now a bare list of BenchmarkScene mappings at the root; the "
            "suite id is derived from the filename stem. Remove the "
            "top-level `id:` / `metadata:` block and inline the scenes "
            "directly, promoting `suite` / `simulator` from the suite-level "
            "metadata to per-scene `metadata.display_name` / "
            "`metadata.simulator`."
        )

    if not isinstance(raw_obj, list):
        raise ROSConfigError(
            f"{path}: YAML root must be a list of BenchmarkScene mappings "
            f"(ADR-0042), got {type(raw_obj).__name__}."
        )

    scenes: list[BenchmarkScene] = []
    for i, raw_scene in enumerate(raw_obj):
        if not isinstance(raw_scene, dict):
            raise ROSConfigError(
                f"{path}: entry [{i}] must be a mapping, got {type(raw_scene).__name__}."
            )
        try:
            scenes.append(BenchmarkScene.model_validate(raw_scene))
        except ValidationError as exc:
            raise ROSConfigError(
                f"{path}: entry [{i}] is not a valid BenchmarkScene: {exc}\n"
                "BenchmarkScene requires `n_episodes`, `seed`, "
                "`metadata.paper`, `metadata.honest_scope`, `task.max_steps`, "
                "and `task.success_key`."
            ) from exc
    return scenes

load_scene_strict(path: str, expected: type[DeployScene] | type[SimScene] | type[BenchmarkScene]) -> DeployScene | SimScene | BenchmarkScene

load_scene_strict(
    path: str, expected: type[BenchmarkScene]
) -> BenchmarkScene

load_scene_strict(
    path: str, expected: type[SimScene]
) -> SimScene

load_scene_strict(
    path: str, expected: type[DeployScene]
) -> DeployScene

Load path as exactly expected; reject other tiers with a clear error.

Parameters:

Name	Type	Description	Default
path	str	Filesystem path to a YAML file containing a single scene mapping.	required
expected	type[DeployScene] | type[SimScene] | type[BenchmarkScene]	One of DeployScene, SimScene, BenchmarkScene — the exact tier the calling CLI accepts.	required

Returns:

Type	Description
DeployScene | SimScene | BenchmarkScene	A validated instance of expected.

Raises:

Type	Description
FileNotFoundError	If path does not exist.
ROSConfigError	If the YAML root is not a mapping, or if it loads as a tier other than expected (with a redirect message naming the right CLI command).

Example

from openral_core import SimScene, load_scene_strict scene = load_scene_strict("scenes/sim/tabletop_cube_push.yaml", SimScene) scene.scene.id # scene_id is the @SCENES.register("…") key, not the filename 'tabletop_push'

Source code in python/core/src/openral_core/loaders.py

def load_scene_strict(
    path: str,
    expected: type[DeployScene] | type[SimScene] | type[BenchmarkScene],
) -> DeployScene | SimScene | BenchmarkScene:
    """Load ``path`` as exactly ``expected``; reject other tiers with a clear error.

    Args:
        path: Filesystem path to a YAML file containing a single scene mapping.
        expected: One of ``DeployScene``, ``SimScene``, ``BenchmarkScene`` —
            the exact tier the calling CLI accepts.

    Returns:
        A validated instance of ``expected``.

    Raises:
        FileNotFoundError: If ``path`` does not exist.
        ROSConfigError: If the YAML root is not a mapping, or if it loads as
            a tier other than ``expected`` (with a redirect message naming the
            right CLI command).

    Example:
        >>> from openral_core import SimScene, load_scene_strict
        >>> scene = load_scene_strict("scenes/sim/tabletop_cube_push.yaml", SimScene)
        >>> scene.scene.id  # scene_id is the @SCENES.register("…") key, not the filename
        'tabletop_push'
    """
    raw_obj = _yaml.safe_load(Path(path).read_text(encoding="utf-8"))
    if not isinstance(raw_obj, dict):
        raise ROSConfigError(f"{path}: YAML root must be a mapping, got {type(raw_obj).__name__}")
    raw: dict[str, object] = raw_obj

    if expected is DeployScene:
        return _load_as_deploy(path, raw)
    if expected is SimScene:
        return _load_as_sim(path, raw)
    if expected is BenchmarkScene:
        return _load_as_benchmark(path, raw)
    raise ROSConfigError(
        f"load_scene_strict: unsupported expected type {expected!r} — "
        "must be DeployScene, SimScene, or BenchmarkScene."
    )

raise_on_invalid_suite(scenes: list[BenchmarkScene], *, suite_id: str) -> None

Raise :class:ROSConfigError if scenes violates suite-level invariants.

Originally enforced inside BenchmarkSpec.model_post_init (deleted in ADR-0042). Extracted as a free function so callers can validate freshly loaded suites independently and so tests can exercise the rules without touching the filesystem.

Suite-level invariants

• scenes MUST be non-empty.
• Every scenes[i].task.id MUST be unique within the suite.
• Every scenes[i].robot_id MUST be non-None — benchmark suites always pin an embodiment.
• All scenes[i].robot_id / n_episodes / seed / metadata MUST be equal across the list — a suite is a single fixed protocol applied across (potentially varying) scenes / tasks.

Per-scene task.success_key / task.max_steps MAY differ (e.g. ManiSkill3 pick + stack share one suite but each task has its own step budget).

Parameters:

Name	Type	Description	Default
scenes	list[BenchmarkScene]	The list of :class:BenchmarkScenes to validate.	required
suite_id	str	The suite identifier — embedded in every error message so failures point back to the right benchmarks/<id>.yaml file.	required

Raises:

Type	Description
ROSConfigError	If any invariant is violated. The first violation wins (no batched reporting).

Source code in python/core/src/openral_core/loaders.py

def raise_on_invalid_suite(
    scenes: list[BenchmarkScene],
    *,
    suite_id: str,
) -> None:
    """Raise :class:`ROSConfigError` if ``scenes`` violates suite-level invariants.

    Originally enforced inside ``BenchmarkSpec.model_post_init`` (deleted in
    ADR-0042). Extracted as a free function so callers can validate freshly
    loaded suites independently and so tests can exercise the rules without
    touching the filesystem.

    Suite-level invariants:
        * ``scenes`` MUST be non-empty.
        * Every ``scenes[i].task.id`` MUST be unique within the suite.
        * Every ``scenes[i].robot_id`` MUST be non-``None`` — benchmark
          suites always pin an embodiment.
        * All ``scenes[i].robot_id`` / ``n_episodes`` / ``seed`` /
          ``metadata`` MUST be equal across the list — a suite is a single
          fixed protocol applied across (potentially varying) scenes / tasks.

    Per-scene ``task.success_key`` / ``task.max_steps`` MAY differ (e.g.
    ManiSkill3 pick + stack share one suite but each task has its own step
    budget).

    Args:
        scenes: The list of :class:`BenchmarkScene`s to validate.
        suite_id: The suite identifier — embedded in every error message so
            failures point back to the right ``benchmarks/<id>.yaml`` file.

    Raises:
        ROSConfigError: If any invariant is violated. The first violation
            wins (no batched reporting).
    """
    if not scenes:
        raise ROSConfigError(
            f"benchmark suite {suite_id!r}: scene list is empty; a benchmark "
            f"must declare at least one BenchmarkScene."
        )

    first = scenes[0]
    if first.robot_id is None:
        raise ROSConfigError(
            f"benchmark suite {suite_id!r}: scene {first.task.id!r} has "
            f"robot_id=None; benchmark suites must pin an embodiment "
            f"(add `robot_id: <id>` to the BenchmarkScene YAML)."
        )

    seen: set[str] = set()
    for scene in scenes:
        task_id = scene.task.id
        if task_id in seen:
            raise ROSConfigError(
                f"benchmark suite {suite_id!r}: duplicate task id "
                f"{task_id!r}; task ids must be unique within a suite."
            )
        seen.add(task_id)

        if scene.robot_id is None:
            raise ROSConfigError(
                f"benchmark suite {suite_id!r}: scene {task_id!r} has "
                f"robot_id=None; benchmark suites must pin an embodiment "
                f"(add `robot_id: <id>` to the BenchmarkScene YAML)."
            )
        if scene.robot_id != first.robot_id:
            raise ROSConfigError(
                f"benchmark suite {suite_id!r}: scene {task_id!r} has "
                f"robot_id={scene.robot_id!r} but the suite uses "
                f"{first.robot_id!r}; every scene in a suite must run on "
                f"the same robot."
            )
        if scene.n_episodes != first.n_episodes:
            raise ROSConfigError(
                f"benchmark suite {suite_id!r}: scene {task_id!r} has "
                f"n_episodes={scene.n_episodes} but the suite uses "
                f"{first.n_episodes}; every scene in a suite must share "
                f"the same episode count."
            )
        if scene.seed != first.seed:
            raise ROSConfigError(
                f"benchmark suite {suite_id!r}: scene {task_id!r} has "
                f"seed={scene.seed} but the suite uses {first.seed}; "
                f"every scene in a suite must share the same seed offset."
            )
        if scene.metadata != first.metadata:
            raise ROSConfigError(
                f"benchmark suite {suite_id!r}: scene {task_id!r} has "
                f"metadata={scene.metadata!r} but the suite uses "
                f"{first.metadata!r}; every scene in a suite shares the "
                f"same provenance block (paper, honest_scope, optional "
                f"display_name + simulator)."
            )

scale_intrinsics_to(base: IntrinsicsPinhole, width: int, height: int) -> IntrinsicsPinhole

Linearly rescale pinhole intrinsics to a new render resolution.

For a camera with a fixed field of view, fx/fy/cx/cy scale linearly with the image dimensions: a frame rendered at twice the width has twice the focal length (px) and twice the principal-point x. This is the consistency rule deploy-sim needs — the canonical manifest pins one nominal resolution, but a scene may render the same MuJoCo camera at another (scene.observation_width/height). Publishing the manifest's nominal intrinsics on a different-resolution render would back-project depth pixels and project occupancy voxels with the wrong focal length, corrupting the OctoMap voxels and the 2D→3D object lift. Scaling keeps the published camera model matched to whatever was actually rendered.

The distortion model and coefficients are preserved unchanged (coefficients are resolution-independent for the normalised plumb-bob model). When the target already equals base's resolution the input is returned as-is.

Parameters:

Name	Type	Description	Default
base	IntrinsicsPinhole	Nominal pinhole intrinsics (e.g. from a SensorSpec).	required
width	int	Target render width in pixels (> 0).	required
height	int	Target render height in pixels (> 0).	required

Returns:

Type	Description
IntrinsicsPinhole	Intrinsics scaled so (fx, fy, cx, cy) correspond to ``(width,
IntrinsicsPinhole	height)at the same field of view asbase``.

Raises:

Type	Description
ValueError	If width or height is not strictly positive.

Example

base = IntrinsicsPinhole(width=256, height=256, fx=256.0, fy=256.0, cx=128.0, cy=128.0) hi = scale_intrinsics_to(base, 640, 640) (hi.width, hi.fx, hi.cx) (640, 640.0, 320.0)

Source code in python/core/src/openral_core/schemas.py

def scale_intrinsics_to(base: IntrinsicsPinhole, width: int, height: int) -> IntrinsicsPinhole:
    """Linearly rescale pinhole intrinsics to a new render resolution.

    For a camera with a *fixed* field of view, ``fx``/``fy``/``cx``/``cy`` scale
    linearly with the image dimensions: a frame rendered at twice the width has
    twice the focal length (px) and twice the principal-point x. This is the
    consistency rule deploy-sim needs — the canonical manifest pins one nominal
    resolution, but a scene may render the same MuJoCo camera at another
    (``scene.observation_width``/``height``). Publishing the manifest's nominal
    intrinsics on a different-resolution render would back-project depth pixels
    and project occupancy voxels with the wrong focal length, corrupting the
    OctoMap voxels and the 2D→3D object lift. Scaling keeps the published camera
    model matched to whatever was actually rendered.

    The distortion model and coefficients are preserved unchanged (coefficients
    are resolution-independent for the normalised plumb-bob model). When the
    target already equals ``base``'s resolution the input is returned as-is.

    Args:
        base: Nominal pinhole intrinsics (e.g. from a ``SensorSpec``).
        width: Target render width in pixels (``> 0``).
        height: Target render height in pixels (``> 0``).

    Returns:
        Intrinsics scaled so ``(fx, fy, cx, cy)`` correspond to ``(width,
        height)`` at the same field of view as ``base``.

    Raises:
        ValueError: If ``width`` or ``height`` is not strictly positive.

    Example:
        >>> base = IntrinsicsPinhole(width=256, height=256, fx=256.0, fy=256.0, cx=128.0, cy=128.0)
        >>> hi = scale_intrinsics_to(base, 640, 640)
        >>> (hi.width, hi.fx, hi.cx)
        (640, 640.0, 320.0)
    """
    if width <= 0 or height <= 0:
        raise ValueError(f"target resolution must be positive; got ({width}, {height})")
    if width == base.width and height == base.height:
        return base
    sx = width / base.width
    sy = height / base.height
    return IntrinsicsPinhole(
        width=width,
        height=height,
        fx=base.fx * sx,
        fy=base.fy * sy,
        cx=base.cx * sx,
        cy=base.cy * sy,
        distortion_model=base.distortion_model,
        distortion_coeffs=list(base.distortion_coeffs),
    )

openral schema v0 — normative Pydantic v2 contracts for all layers.

This is the single source of truth for the data contracts between layers. Anything imported from openral_core.init is public API. Breaking changes require a SemVer MAJOR bump (pre-1.0: MINOR) and a migration entry.

BenchmarkName: TypeAlias = Literal['aloha_insertion', 'aloha_transfer_cube', 'gr1_tabletop', 'libero_10', 'libero_goal', 'libero_object', 'libero_spatial', 'maniskill3_franka_pick_cube', 'maniskill3_pick_place', 'metaworld_mt50', 'pusht', 'robocasa_pnp', 'simpler_env_widowx'] module-attribute

Canonical benchmark suite ids — one per benchmarks/<id>.yaml in tree.

Used as keys in RSkillManifest.benchmarks. Each value is the headline success rate [0.0, 1.0] the skill achieves on that suite. The full breakdown lives in the matching rskills/<id>/eval/<key>.json.

CollisionShape: TypeAlias = CapsuleShape | SphereShape module-attribute

Discriminated union of convex collision primitives (ADR-0030).

The discriminator field is shape. Used by :class:LinkCollisionGeometry (robot links) and :class:WorldCollisionPrimitive (world obstacles). Mesh primitives are intentionally excluded — the allocation-free safety kernel checks only convex analytic shapes; mesh-accurate collision stays a planning-layer concern.

EmbodimentTag: TypeAlias = Literal['aloha', 'custom', 'franka_panda', 'g1', 'google_robot', 'gr1', 'h1', 'mobile_base', 'openarm', 'panda_mobile', 'pusht', 'rizon4', 'sawyer', 'so100_follower', 'so101_follower', 'ur10e', 'ur5e', 'widowx'] module-attribute

Canonical embodiment tags — one per robots/<id>/robot.yaml shipped in tree, plus "custom" as the explicit "I know what I'm doing" escape hatch.

"mobile_base" is a CLASS tag (not a specific robot): any robot with a planar base + body_twist actuator declares it so base-only rSkills (Nav2 NavigateToPose, etc.) can target the whole class without naming each specific mobile platform. Robot-specific tags (e.g. "panda_mobile") coexist on the same RobotDescription for skills that DO depend on the specific composition.

The RSkillManifest.embodiment_tags field is restricted to this set so a typo or framework hint (lerobot, libero) cannot land in a manifest where the loader's compat check would silently never match. When "custom" is used, the manifest MUST also populate embodiment_extra (see :class:EmbodimentExtra); the cross-validator on :class:RSkillManifest enforces this. ADR-0013.

FailureEvidence: TypeAlias = TimeoutEvidence | ForceEvidence | WorkspaceEvidence | PerceptionStaleEvidence | CriticEvidence | ControllerEvidence | SelfVerifyEvidence | HumanEvidence | WamEvidence | ReasonerTimeoutEvidence | CollisionEvidence | SuppressedSummaryEvidence module-attribute

Discriminated union for FailureTrigger.evidence_json payloads.

The discriminator field is kind (a string Literal on each variant). Consumers decode an incoming evidence_json string with::

from pydantic import TypeAdapter
from openral_core import FailureEvidence

evidence = TypeAdapter(FailureEvidence).validate_json(msg.evidence_json)

Producers serialize via evidence.model_dump_json().

The union mirrors the KIND_* constants on openral_msgs/msg/FailureTrigger (ADR-0018 F3). Adding a new variant requires adding a new KIND_* constant to the IDL.

GripperConvention: TypeAlias = Literal['normalized_open_unit', 'normalized_open_symmetric', 'binary_close_one', 'raw_joint_rad', 'width_meters'] module-attribute

Per-skill gripper action encoding (Gap 2 of the rSkill self-containment audit).

Required on :class:ControlModeSemantics whenever the parent :class:ActuatorRequirement.kind is :attr:ControlMode.GRIPPER_BINARY or :attr:ControlMode.GRIPPER_POSITION — silently mis-encoding the gripper slot between scenes / robots is a top observed mis-actuation failure.

Conventions:

• normalized_open_unit — 0.0 = fully closed, 1.0 = fully open. Most lerobot / SmolVLA / pi0.5 LIBERO checkpoints.
• normalized_open_symmetric — -1.0 = fully closed, +1.0 = fully open. Some MetaWorld checkpoints.
• binary_close_one — 0.0 = open, 1.0 = close (single bit, RoboCasa-style).
• raw_joint_rad — raw per-finger joint angle in radians (Fourier dexhands, some humanoid skills).
• width_meters — physical gripper width in metres (Franka FCI native).

JointRole: TypeAlias = Literal['arm', 'base', 'gripper', 'torso', 'leg', 'head', 'neck', 'wheel', 'unknown'] module-attribute

Structural classification of a :class:JointSpec (ADR-0028a).

Carries the joint's purpose in the embodiment's morphology — what the runner, safety kernel, and dataset bridge need to identify a channel without relying on name-substring heuristics (e.g. "gripper" in name.lower() in rskill_runner_node._build_joint_permutation which silently misclassifies any joint with "gripper" in the name).

"unknown" is the default so legacy manifests load unchanged; the fleet annotates incrementally as ADR-0028 sub-PRs land.

ModelFamily: TypeAlias = Literal['smolvla', 'pi05', 'xvla', 'act', 'diffusion', 'rldx', 'molmoact2', 'gr00t'] module-attribute

VLA / policy family the skill belongs to.

Used by the eval / runner adapters to dispatch to the right openral_sim.adapters.<family> policy adapter without string-matching the skill name. Adding a family here means landing the matching adapter under python/sim/src/openral_sim/adapters/.

gr00t (NVIDIA Isaac GR00T N1.x / N2) runs out-of-process via a ZMQ sidecar in an isolated Python 3.10 venv, sharing the architecture of the rldx adapter (RLDX-1 is itself a GR00T-N1.5 finetune) — see ADR-0046.

PerceptionEventMetadata: TypeAlias = MotionMetadata | ObjectsMetadata | OcrMetadata | SceneChangeMetadata module-attribute

Discriminated union for PromptStamped.metadata_json payloads on the /openral/perception/<kind> topics (ADR-0018 F6).

The discriminator field is kind (a string Literal on each variant). Consumers decode an incoming metadata_json string with::

from pydantic import TypeAdapter
from openral_core import PerceptionEventMetadata

metadata = TypeAdapter(PerceptionEventMetadata).validate_json(msg.metadata_json)

Producers serialise via metadata.model_dump_json().

The four variants map 1:1 onto the four per-kind topics fixed in ADR-0018 §3 (motion, objects, ocr, scene_change). Adding a new variant requires adding a new /openral/perception/<kind> topic to the contract — by design, new kinds get new topics, not a schema bump, so subscribers can subscribe to exactly the kinds they care about.

RSkillKind: TypeAlias = Literal['vla', 'wam', 'ros_action', 'ros_service', 'detector', 'vlm', 'reward'] module-attribute

Discriminator selecting how an rSkill is instantiated at the loader.

• "vla" — learnable Vision-Language-Action policy. Requires :attr:RSkillManifest.model_family and :attr:RSkillManifest.weights_uri; resolved by the policy adapter dispatch in openral_rskill (the pre-existing path used by every in-tree rSkill prior to this discriminator landing).
• "wam" — World Action Model (planning-layer mental-simulation / failure-anticipation component per CLAUDE.md §3). Reserved so the discriminator is forward-compatible; the loader / runner branch is not implemented yet and kind: wam manifests are rejected at resolve time with :class:~openral_core.exceptions.ROSConfigError.
• "ros_action" — wraps an existing ROS 2 action server. Requires a :class:RosIntegration block. model_family and weights_uri are forbidden. chunk_size is pinned to 1 so the safety supervisor's per-row check sees every commanded position.
• "ros_service" — wraps an existing ROS 2 service. Same constraints as "ros_action".
• "detector" — perception producer that runs an exported detection model (RT-DETR / D-FINE ONNX) on the camera tee and publishes :class:~openral_core.schemas.ObjectsMetadata; emits no :class:~openral_core.schemas.Action. Requires a :class:DetectorContract block and :attr:RSkillManifest.weights_uri (the exported ONNX / TensorRT engine). model_family and action_contract / state_contract are forbidden. ADR-0037.
• "vlm" — vision/video-language model used as a scene-understanding perception component (e.g. Qwen3.5-4B NF4). Accepts RGB image or video frames and a natural-language query; returns a text answer. Emits no actions or bounding boxes. Runs at S2 rate (role: "s2"), so it is surfaced to the reasoner as a read-only scene-query tool, never as an ExecuteSkill policy. weights_uri REQUIRED; actuators_required MUST be empty; action_contract, state_contract, detector, ros_integration, processors, image_preprocessing, n_action_steps, and starting_pose are FORBIDDEN. model_family is OPTIONAL metadata. ADR-0047.

ReasonerToolCall: TypeAlias = ExecuteRskillTool | ReloadGstPipelineTool | LifecycleTransitionTool | EmitPromptTool | RecallObjectTool | ResolvePlaceTool | LocateInViewTool | QuerySceneTool | QueryTaskProgressTool module-attribute

Discriminated union over the reasoner tool variants (ADR-0018 §4; ADR-0039).

The discriminator field is tool (a string Literal on each variant). Consumers decode an LLM tool-use payload with::

from pydantic import TypeAdapter
from openral_core import ReasonerToolCall

call = TypeAdapter(ReasonerToolCall).validate_json(payload)

Producers (LLM clients) serialise via call.model_dump_json().

The first four variants are the actuation/effect palette ADR-0018 §4 commits to. ADR-0039 adds two read-only query variants — :class:RecallObjectTool and :class:ResolvePlaceTool — that only read the ADR-0038 spatial memory (no actuation authority). Extending the palette requires (a) a new variant here, (b) the corresponding ROS-side dispatch in openral_reasoner_ros.reasoner_node, (c) a CLAUDE.md §6.2 / §7.6 amendment if the new tool shifts the reasoner's authority surface. The two query variants' dispatch + result-return path is ADR-0039 Phase 2; until then they are a typed contract not yet exposed in the live provider palette.

StateLayout: TypeAlias = Literal['smolvla_9d', 'human300_16d', 'gr1', 'rc365', 'simpler_widowx', 'simpler_google'] module-attribute

Closed set of per-checkpoint proprioception layouts. ADR-0014 + ADR-0027.

A layout names the SHAPE the checkpoint was trained on — field order, frame convention, gripper encoding, quaternion handedness. The per-robot SOURCE bindings (which TF frame is "the EE", which joint names are "the gripper") live on :class:StateContractBindings. The openral_state_adapter registry maps each literal to an assembler function that joins shape + bindings + live JointState + live TF.

WRAPPED_TASK_SPACE_LAYOUTS: frozenset[StateLayout] = frozenset({'rc365', 'human300_16d'}) module-attribute

Layouts that are TASK-space composites (Cartesian poses + gripper widths), NOT one-scalar-per-joint. These layouts REQUIRE :attr:StateContract.bindings to name the source TF frames + JointState entries — the cross-validator on :class:StateContract enforces this at manifest load. The remaining layouts (smolvla_9d, gr1, simpler_*) are joint-space slices (potentially across multiple controller groups, as in GR1's 29-D waist+arms+hands composite) that the runner serves verbatim from observation.joint_state.position; a robot.yaml with the matching joint count dispatches them without an assembler.

Action

Bases: BaseModel

A single action step or chunk produced by a Skill.

Attributes:

Name	Type	Description
control_mode	ControlMode	Target action space.
horizon	int	Number of steps (1 = single step, H = chunk).
joint_targets	list[list[float]] | None	Joint position targets, shape (H, N).
joint_velocities	list[list[float]] | None	Joint velocity targets, shape (H, N).
joint_torques	list[list[float]] | None	Joint torque targets, shape (H, N).
cartesian_pose	list[Pose6D] | None	EE pose targets.
cartesian_delta	list[tuple[float, ...]] | None	EE pose deltas.
cartesian_twist	list[tuple[float, ...]] | None	EE velocity targets.
body_twist	list[tuple[float, float, float, float, float, float]] | None	Base twist targets.
foot_placements	list[dict[str, object]] | None	Discrete footstep targets.
gripper	list[float] | None	Gripper commands in [0, 1].
dex_hand_joints	list[list[float]] | None	Dexterous hand joint targets.
confidence	float	rSkill confidence in [0, 1].
stamp_ns	int	Action timestamp in nanoseconds.
ee_name	str | None	Target end-effector name.
frame_id	str | None	Reference frame for Cartesian actions.
safety_overrides	dict[str, object]	Operator-approved safety override tokens.

ActionContract

Bases: BaseModel

Per-rSkill action-vector contract (ADR-0019 PR-revert).

Mirrors :class:StateContract for the action side. Carries the output dimensionality the checkpoint emits so the dataset bridge (and any downstream consumer) can bind the LeRobot v3 action feature shape without consulting the sim or hardware adapter.

Per ADR-0007, the sim-specific action contract belongs on the per-checkpoint rSkill manifest, not on the physical :class:RobotDescription (the same Franka emits 7-D delta-EEF on LIBERO vs 8-D joint pos on a hardware deploy).

Attributes:

Name	Type	Description
dim	int	Dimensionality of the action vector emitted by Skill.step() / PolicyAdapter.step(). Required when this contract is set.
representation	ActionRepresentation | None	Optional named representation (mirrors :attr:ActionSpec.representation). When set, downstream consumers can map between equivalent representations (e.g. joint_positions → delta_ee_6d_plus_gripper).
slots	list[ActionSlot] | None	ADR-0028b — declarative slot layout. When set, every index in [0, dim) must be covered by exactly one :class:ActionSlot (no gaps, no overlaps). The skill_runner reads this to dispatch slices of the policy vector onto typed :class:Action objects. When None, the runner falls back to the legacy single-Action path (one implicit JOINT_POSITION slot covering the whole vector). Manifests carrying slots are exempt from the ADR-0028a dim <= len(robot.joints) invariant because the slot decoder gives a typed contract per slice instead.

ActionRepresentation

Bases: str, Enum

Action vector representation format.

ActionSlot

Bases: BaseModel

One contiguous slice of an rSkill's action vector (ADR-0028b).

The skill_runner reads ActionContract.slots and emits one typed :class:Action per non-discard slot per step. All actions inherit the parent step's trace_id so the safety supervisor and downstream telemetry can join them post-hoc.

Attributes:

Name	Type	Description
range	tuple[int, int]	Inclusive [start, end] indices into the flat policy action vector. range[0] must be ≤ range[1]; both must fall within [0, ActionContract.dim).
control_mode	ControlMode | None	The :class:ControlMode the slice is routed to. The HAL whitelist on the target robot must include this mode (the palette filter rejects the rSkill at install time otherwise). None only when :attr:discard is True.
discard	bool	When True the slice is dropped silently — used for dataset artefacts like RoboCasa365's torso placeholder dim or paired gripper channels. The slot still occupies its range so coverage validation works; no :class:Action is emitted.
ee	str | None	End-effector name from the robot's :attr:RobotDescription.end_effectors / :attr:RobotDescription.joints. REQUIRED for CARTESIAN_* modes (the pose is computed in the named EE's frame) and for GRIPPER_* modes (names the actuator). FORBIDDEN for BODY_TWIST and JOINT_POSITION.
frame	str | None	tf2 frame name. REQUIRED for cartesian + body-twist modes (the slice's bytes are expressed in this frame). FORBIDDEN for joint-position and gripper modes.
joint_names	list[str]	Robot joint names this slice targets, in slot order. REQUIRED for JOINT_POSITION / JOINT_VELOCITY / JOINT_TORQUE when the slot covers fewer than all robot joints; FORBIDDEN for non-joint modes. Length must equal range[1] - range[0] + 1.

ActionSpec

Bases: BaseModel

VLA action configuration for this robot.

Attributes:

Name	Type	Description
dim	int	Dimensionality of the action vector.
representation	ActionRepresentation | None	How the action vector is encoded.
control_freq_hz	float | None	Control frequency the actions are executed at.
chunk_size	int | None	Number of action steps per inference call (chunk size H).

ActuatorRequirement

Bases: BaseModel

One actuator slot an rSkill emits actions for.

Symmetric with :class:SensorRequirement on the action side. The compatibility check resolves each entry against a :class:RobotDescription's action_spec and rejects pairings whose declared :class:ControlMode is not advertised by the robot.

For predefined embodiments (one of the 9 canonical robots/<id>/robot.yaml slugs), n_dof and vla_action_key are optional — the loader auto-fills them from the robot YAML at compatibility-check time. For the "custom" embodiment escape hatch they MUST be set on the manifest; the :class:RSkillManifest cross-validator enforces this.

Attributes:

Name	Type	Description
kind	ControlMode	The action interface the skill emits (e.g. ControlMode.JOINT_POSITION, ControlMode.CARTESIAN_DELTA). Reuses the canonical ControlMode enum so the manifest, the safety kernel, Action.control_mode, and RobotDescription.action_spec all share one source of truth.
n_dof	int | None	Optional degrees of freedom the skill emits. Auto-filled from the robot YAML for predefined embodiments; REQUIRED on the manifest when "custom" is in RSkillManifest.embodiment_tags.
vla_action_key	str | None	Optional slot name the policy emits, e.g. "action.joints.arm_left". Auto-filled for predefined embodiments; REQUIRED on the manifest when "custom" is in RSkillManifest.embodiment_tags.
control_mode_semantics	ControlModeSemantics	Action-space semantics for this slot. REQUIRED — closes Gap 2 of the rSkill self-containment audit. Cross-validators enforce: gripper kinds require gripper_convention; cartesian kinds require reference_frame; other kinds forbid both.

Example

a = ActuatorRequirement( ... kind=ControlMode.JOINT_POSITION, ... control_mode_semantics=ControlModeSemantics(mode="absolute"), ... ) a.kind is ControlMode.JOINT_POSITION True a.n_dof is None and a.vla_action_key is None True

ApproachViewpoint

Bases: BaseModel

A camera-facing standoff pose for viewing/manipulating an object (ADR-0038 §6).

Attributes:

Name	Type	Description
pose	Pose6D	Base/EE goal pose (map frame) at a standoff from the object, oriented so the gripper-mounted camera faces it.
standoff_m	float	Standoff distance from the object, in metres.
camera_frame_id	str	tf2 frame of the camera the viewpoint orients toward.

AssetRefs

Bases: BaseModel

The unified description-asset block on :class:RobotDescription.

ADR-0058 §4. Replaces the scattered urdf_path / mjcf_uri / srdf_path (+ ADR-0027 URDF-root fields) with one block whose refs share the :func:openral_core.assets.resolve_asset grammar.

Attributes:

Name	Type	Description
urdf	UrdfAsset | None	URDF asset (with optional robot_state_publisher wiring), or None when the robot ships no URDF.
mjcf	str | None	MJCF asset ref (MuJoCo wiring), or None.
srdf	str | None	SRDF asset ref whose disable_collisions block seeds :attr:RobotDescription.allowed_collision_pairs, or None.

Example

AssetRefs(urdf=UrdfAsset(ref="rd:panda_description")).urdf.ref 'rd:panda_description'

BenchmarkMetadata

Bases: BaseModel

Provenance block required on every BenchmarkScene.

paper / honest_scope are the published-protocol citation + a one-sentence "what this eval actually measured" statement.

display_name / simulator are optional paper-comparison labels (ADR-0042) that surface into RSkillEvalResult.benchmark when present — display_name becomes benchmark.name and simulator becomes benchmark.simulator. Pre-ADR-0042 these lived in a free-form dict on the deleted BenchmarkSpec; moving them per-scene keeps them with their provenance and lets the aggregator emit identical JSON whether driven by run_benchmark (suite) or run_benchmark_scene (single scene).

BenchmarkScene

Bases: SimScene

Full benchmark eval spec for openral benchmark.

n_episodes, seed, and metadata are required with no defaults — they must match the published evaluation protocol. The task must supply success_key and max_steps.

from_yaml(path: str) -> BenchmarkScene classmethod

Load and validate a BenchmarkScene YAML from disk.

Source code in python/core/src/openral_core/schemas.py

@classmethod
def from_yaml(cls, path: str) -> BenchmarkScene:
    """Load and validate a ``BenchmarkScene`` YAML from disk."""
    import yaml  # noqa: PLC0415

    with open(path, encoding="utf-8") as fh:
        data = yaml.safe_load(fh)
    return cls.model_validate(data)

CapsuleShape

Bases: BaseModel

Capsule collision primitive — a segment swept by a radius.

The central segment runs along the local +Z axis from -length_m / 2 to +length_m / 2 (the MJCF / URDF capsule convention); it is placed and oriented by the owning frame (:attr:LinkCollisionGeometry.origin_xyz_rpy for a link, :attr:WorldCollisionPrimitive.pose for an obstacle). Capsules bound most robot links tightly, so the safety check stays conservative (ADR-0030 §2).

Attributes:

Name	Type	Description
shape	Literal['capsule']	Discriminator (always "capsule").
radius_m	float	Capsule radius in metres.
length_m	float	Length of the central segment in metres (the cylinder portion; the total span is length_m + 2 * radius_m). 0.0 degenerates to a sphere.

Example

CapsuleShape(radius_m=0.04, length_m=0.3).length_m 0.3

CollisionEvidence

Bases: _FailureEvidenceBase

Evidence for KIND_COLLISION — a proposed motion would collide (ADR-0030).

Attributes:

Name	Type	Description
kind	Literal['collision']	Discriminator (always "collision").
collision_kind	Literal['self', 'world']	"self" (link vs link) or "world" (link vs obstacle).
link_a	str	Robot link whose collision volume tripped the check.
link_b_or_object	str	The other robot link (self-collision) or the world object / occupancy region (world-collision).
horizon_step	int	Chunk step (horizon index) where the collision was first detected.
min_distance_m	float	Signed minimum distance at detection, in metres (negative means interpenetration).

ControlMode

Bases: str, Enum

Action space / control interface exposed by a robot or skill.

ControlModeSemantics

Bases: BaseModel

Action-space semantics declared on each :class:ActuatorRequirement.

Closes Gap 2 of the rSkill self-containment audit: kind alone ("joint_position") doesn't say whether the policy emits absolute targets or per-step deltas, in which joint order, in which reference frame, or how it encodes the gripper. Today these are assumed at the adapter level and silently broken when porting a skill between embodiments.

The block is REQUIRED on every actuator entry of a manifest. For the canonical embodiments the loader still auto-fills n_dof / vla_action_key from the robot YAML; semantics must be declared explicitly on the manifest because they are a property of the trained checkpoint, not the robot.

Attributes:

Name	Type	Description
mode	Literal['absolute', 'delta']	Whether the action vector represents absolute targets (joint positions, end-effector pose) or deltas from the current state. Required.
gripper_convention	GripperConvention | None	Encoding of the gripper slot in the action vector. Required when the parent :attr:ActuatorRequirement.kind is :attr:ControlMode.GRIPPER_BINARY or :attr:ControlMode.GRIPPER_POSITION; forbidden otherwise.
joint_order	list[str] | None	Ordered list of joint names matching the channels the policy emits. Optional for canonical embodiments (the robot YAML's :attr:ActionSpec.joint_names is the source of truth); REQUIRED for "custom" embodiments.
reference_frame	str | None	tf2 frame the action is expressed in. Required when the parent :attr:ActuatorRequirement.kind is one of :attr:ControlMode.CARTESIAN_POSE, :attr:ControlMode.CARTESIAN_DELTA, or :attr:ControlMode.CARTESIAN_TWIST; forbidden otherwise.

Example

sem = ControlModeSemantics(mode="absolute") sem.mode 'absolute' sem.gripper_convention is None and sem.reference_frame is None True

ControllerEvidence

Bases: _FailureEvidenceBase

Evidence for KIND_CONTROLLER — a ros2_control controller faulted.

Attributes:

Name	Type	Description
kind	Literal['controller']	Discriminator (always "controller").
controller_name	str	Failing controller (e.g. "joint_trajectory_controller").
state	str	Reported controller state (e.g. "inactive", "error").
detail	str	Free-form detail from the controller manager.

CriticEvidence

Bases: _FailureEvidenceBase

Evidence for KIND_CRITIC — a critic flagged an action below threshold.

Attributes:

Name	Type	Description
kind	Literal['critic']	Discriminator (always "critic").
critic_id	str	Identifier of the critic that fired (model id or hand-rolled).
score	float	Critic output in the critic's native range.
threshold	float	Configured pass threshold.

DeadlineOverrunPolicy

Bases: str, Enum

What the inference runner does when a tick exceeds the deadline.

warn logs + records an OTel attribute but still sends the action (mirrors lerobot's record loop). drop skips the action for this tick — used when stale actions are worse than no action (e.g. velocity control). raise is test-only: raises :class:ROSDeadlineMissed.

DeployScene

Bases: BaseModel

Environment-only scene for openral deploy run.

Carries the physics world and optional robot mount. No task, no eval config — this is a playground for the full OpenRAL stack.

from_yaml(path: str) -> DeployScene classmethod

Load and validate a DeployScene YAML from disk.

Source code in python/core/src/openral_core/schemas.py

@classmethod
def from_yaml(cls, path: str) -> DeployScene:
    """Load and validate a ``DeployScene`` YAML from disk."""
    import yaml  # noqa: PLC0415

    with open(path, encoding="utf-8") as fh:
        data = yaml.safe_load(fh)
    return cls.model_validate(data)

DetectedObject

Bases: BaseModel

A detected object in the scene.

Attributes:

Name	Type	Description
label	str	Semantic class label.
confidence	float	Detection confidence in [0, 1].
pose	Pose6D	6D pose in the world or camera frame.
bbox_3d	tuple[float, float, float, float, float, float] | None	3D bounding box (x_min, y_min, z_min, x_max, y_max, z_max).
track_id	int | None	Persistent track ID across frames.

DetectorContract

Bases: BaseModel

Manifest contract for kind: "detector" rSkills (ADR-0037).

Carries the configuration the runtime :class:~openral_core.schemas.ObjectsDetector needs to instantiate an exported detection model (RT-DETR / D-FINE ONNX) and match its output indices to semantic class labels.

Required when :attr:RSkillManifest.kind is "detector"; forbidden for all other kinds (enforced by :meth:RSkillManifest._check_kind_consistency).

Attributes:

Name	Type	Description
labels	list[str]	Ordered class-label list; the integer class-id the model emits is used as an index into this list. At least one label is required.
input_size	tuple[int, int]	Model input resolution as (width, height) in pixels. Both dimensions must be > 0. Default (640, 640) matches the RT-DETR / D-FINE default export resolution.
score_threshold	float	Detections with confidence below this value are discarded before publishing. Must be in [0.0, 1.0]. Default 0.5.
engine	DetectorEngine | None	Optional explicit backend selector (:class:DetectorEngine). None (default) keeps the legacy runtime-keyed dispatch. Set it to disambiguate backends that share a runtime — e.g. zeroshot_hf for an in-process Transformers open-vocabulary detector, which is runtime: pytorch like the VLM sidecar.
mode	DetectorMode	Invocation mode (:class:DetectorMode; default continuous). Declares whether the detector is an always-on background producer (output reaches the reasoner via world state) or an on-demand prompted locator (surfaces the locate_in_view tool). ADR-0051.

Example

c = DetectorContract( ... labels=["person", "bicycle", "car"], ... input_size=(640, 640), ... score_threshold=0.4, ... ) c.labels[0] 'person' c.mode

DetectorEngine

Bases: str, Enum

Backend that executes a kind: "detector" rSkill (ADR-0037).

Selects which runtime detector class :func:build_manifest_detector constructs. None (the default on :class:DetectorContract) preserves the legacy runtime-keyed dispatch: runtime: onnx/tensorrt → RT-DETR ONNX, runtime: pytorch → the LocateAnything VLM sidecar. Set it explicitly to opt into a backend that the runtime value alone cannot disambiguate (e.g. an in-process Transformers open-vocabulary detector, which is also runtime: pytorch).

Attributes:

Name	Type	Description
RTDETR_ONNX		Fixed-label RT-DETR / D-FINE ONNX export (CPU / NVMM tiers). Equivalent to leaving engine unset with an onnx/tensorrt runtime.
VLM_SIDECAR		Out-of-process open-vocabulary visual-grounding VLM (LocateAnything-3B). Equivalent to leaving engine unset with a pytorch runtime. Query-driven (prompted).
ZEROSHOT_HF		In-process Transformers open-vocabulary detector (AutoModelForZeroShotObjectDetection — e.g. OmDet-Turbo). Runs against a fixed class vocabulary (the manifest labels) every frame, so it behaves like a large closed-vocabulary detector that needs no prompting — an unprompted background producer that populates the world object list with far more than the 80 COCO classes (ADR-0037 2026-06-12 amendment).

Example

DetectorEngine.ZEROSHOT_HF.value 'zeroshot_hf'

DetectorMode

Bases: str, Enum

Invocation mode of a kind: "detector" rSkill (ADR-0051).

The axis orthogonal to :class:DetectorEngine: where engine says how the model runs, mode says when the reasoner invokes it and therefore how its output reaches the LLM.

Attributes:

Name	Type	Description
CONTINUOUS		An always-on background producer. Runs on the camera tee every frame and streams ObjectsMetadata into WorldState.detected_objects; the reasoner reads it passively (via world state / recall_object) and never prompts it. It is not an ExecuteSkill-dispatchable tool and carries no actuation authority. RT-DETR (closed vocab) and OmDet-Turbo (frozen open vocab) are continuous. The reasoner may still toggle it via LifecycleTransitionTool to free VRAM (ADR-0050).
ON_DEMAND		A prompted locator the reasoner invokes only when it needs to find a specific object right now. Surfaces the read-only locate_in_view tool (ADR-0043) backed by an open-vocabulary detector; it is not run continuously. LocateAnything is on-demand.

The two modes cleanly separate "open-vocabulary" from "prompting": continuous detectors cover a fixed bank of classes the reasoner reads for free; the on-demand locator handles the long tail that bank does not cover.

Example

DetectorMode.CONTINUOUS.value 'continuous' DetectorMode.ON_DEMAND.value 'on_demand'

DeviceInfo

Bases: BaseModel

Snapshot of the host compute capabilities used for runtime selection.

Attributes:

Name	Type	Description
device_str	str	PyTorch-style device string ("cpu", "cuda:0", "mps").
gpu_memory_bytes	int	Total GPU VRAM in bytes (0 if CPU-only).
cuda_compute_capability	tuple[int, int] | None	CUDA compute capability major/minor pair.
cpu_count	int	Logical CPU count.
arch	str	CPU architecture string.

Example

info = DeviceInfo(device_str="cpu") info.gpu_memory_bytes 0

EmbodimentExtra

Bases: BaseModel

Sensor + actuator surface for a "custom" embodiment.

Required when "custom" appears in :attr:RSkillManifest.embodiment_tags; forbidden otherwise. This is the explicit "I know what I'm doing" hatch for skill manifests targeting embodiments that do not have a canonical robots/<id>/robot.yaml in tree.

Reuses :class:SensorRequirement and :class:ActuatorRequirement on the assumption that the natural shape of "what this embodiment offers" mirrors "what the skill needs" — the loader just runs the same compat check it would run against a real robot YAML.

Attributes:

Name	Type	Description
sensors	list[SensorRequirement]	Sensor slots this custom embodiment exposes (≥1).
actuators	list[ActuatorRequirement]	Actuator slots this custom embodiment exposes (≥1).

Example

from openral_core import SensorModality, ControlMode EmbodimentExtra( ... sensors=[SensorRequirement(modality=SensorModality.RGB)], ... actuators=[ ... ActuatorRequirement( ... kind=ControlMode.JOINT_POSITION, ... n_dof=6, ... vla_action_key="action.joints.arm", ... control_mode_semantics=ControlModeSemantics(mode="absolute"), ... ) ... ], ... ) # doctest: +ELLIPSIS EmbodimentExtra(sensors=[...], actuators=[...])

EmbodimentKind

Bases: str, Enum

Top-level kinematic class of a robot body.

EmitPromptTool

Bases: _ReasonerToolBase

Tool variant — republish a PromptStamped onto another topic.

Dispatch: publish on target_topic (typically /openral/prompt for self-cascading, but any PromptStamped topic is valid). Lets the reasoner stage multi-step plans, talk to a peer reasoner, or feed a downstream prompt-aware skill without going through an ExecuteSkill goal.

Attributes:

Name	Type	Description
tool	Literal['emit_prompt']	Discriminator (always "emit_prompt").
target_topic	str	Absolute ROS topic name (must start with "/"). The reasoner's own subscription on /openral/prompt plus the prompt-router's FIFO queue (ADR-0018 F10) handle the cascade.
text	str	Human-readable prompt body forwarded as PromptStamped.text.
metadata_json	str	Free-form JSON forwarded as PromptStamped.metadata_json. Empty string when no structured metadata is needed.

EndEffectorSpec

Bases: BaseModel

End-effector specification.

Attributes:

Name	Type	Description
name	str	End-effector name.
kind	Literal['parallel_gripper', 'suction', 'dexterous_hand', 'tool', 'none']	Type of end-effector.
hand	Hand	Laterality.
n_dof	int	Number of controllable degrees of freedom.
max_grip_force_n	float | None	Maximum grip force in Newtons.
max_payload_kg	float | None	Maximum payload in kg.
workspace_radius_m	float | None	Reach radius in meters.
tactile_sensors	list[str]	List of tactile SensorSpec names attached to this EE.
actuated	bool	Whether the end-effector is driven by an actuator (ADR-0028a). False for passive tools (inert flanges, magnetic plates without electromagnet, kinematic-only mounts). When False, the safety kernel rejects any chunk addressed at this EE — the chunk routes to a no-op rather than risking unintended motion. Default True: every actuated gripper / dexterous hand / suction cup we ship is driven.

ExecuteRskillTool

Bases: _ReasonerToolBase

Tool variant — invoke an installed, capability-matched rSkill.

Dispatch: action goal on /openral/execute_skill (the openral_msgs/action/ExecuteSkill action server in F1's rskill_runner_node). The chunk path that follows is Skill → /openral/candidate_action → safety_node → /openral/safe_action → HAL per ADR-0018 §3.

Attributes:

Name	Type	Description
tool	Literal['execute_rskill']	Discriminator (always "execute_rskill").
rskill_id	str	RSkillManifest.name of an installed, capable skill. Validated against the local registry by the reasoner at palette-build time; an unknown id raises :class:ROSReasonerInvalidPlan.
prompt	str	Natural-language prompt forwarded to the skill's ExecuteSkill goal alongside the rskill_id. For VLA skills this is the policy's task-conditioning signal (SmolVLA writes it into observation["task"]); for wrapped-ROS skills it's carried for trace / log context but the actual goal is built from goal_params_json merged over the manifest's default_goal_json.
goal_params_json	str	ADR-0026 — serialised JSON object carrying per-skill typed parameters the LLM produces against the skill's :attr:RSkillManifest.goal_params_schema. Empty string disables the merge (today's behaviour). Wrapped-ROS skills (kind: ros_action / ros_service) deep-merge it over ros_integration.default_goal_json at configure-time; VLA skills accept the field and ignore it (their prompt is already the structured signal).
deadline_s	float	Hard deadline in seconds for the action server to complete the goal. 0 means "use the skill manifest's default latency budget".

ForceEvidence

Bases: _FailureEvidenceBase

Evidence for KIND_FORCE — measured force exceeded a safety limit.

Attributes:

Name	Type	Description
kind	Literal['force']	Discriminator (always "force").
joint_or_ee	str	Name of the joint or end-effector that tripped the limit.
measured_n	float	Measured force in newtons.
limit_n	float	Configured ceiling in newtons.

FrameEncoding

Bases: str, Enum

How the bytes inside a :class:SensorFrame are interpreted.

The first four values are raw per-pixel layouts. JPEG / PNG are compressed forms; the runner decodes lazily. CUDA_NV12 and RAW mark frames whose payload is an opaque handle (NVMM pointer, DMA-BUF fd) — the data field is empty and the consumer must read via handle.

GripperReadMode

Bases: str, Enum

How :class:openral_hal.MujocoArmHAL reports the gripper qpos.

SUM_OVER_SCALE (default) — clip(sum(qpos[addrs]) / scale, 0, 1). Matches the Franka parallel gripper where two finger qpos are summed and divided by 2 * max_finger_extent (=0.08 m for Panda). Public surface is normalised [0, 1]. AFFINE_LOW_HIGH — (qpos[addrs[0]] - ctrl_range[0]) / (ctrl_range[1] - ctrl_range[0]). Used by the SO-100 menagerie Jaw joint, a 1-DoF revolute with a non-zero closed position (-0.174 rad). Public surface is normalised [0, 1]. PASSTHROUGH — qpos[addrs[0]] reported verbatim, in the same physical units as the MJCF (metres for Aloha prismatic fingers, radians for OpenArm revolute jaws). The public surface is not [0, 1] — Skills must accept the raw range.

GripperWriteMode

Bases: str, Enum

How :class:openral_hal.MujocoArmHAL maps an Action's gripper value to ctrl.

NORMALISED (default) — input is [0, 1], mapped affinely to ctrl_range. low ↔ closed (Action.gripper = 0), high ↔ open (Action.gripper = 1). PASSTHROUGH — input is in the same physical units as ctrl_range (metres / radians), written directly to ctrl. MuJoCo's actuator ctrlrange does the clipping. Used by Aloha (positive-finger metres) and OpenArm (jaw radians).

HalConfig

Bases: BaseModel

Configuration for a HAL adapter instantiation by the inference runner.

Picks which HAL adapter class to use (sim digital twin vs real hardware) and how to reach the robot (serial port, FCI URI, ROS 2 namespace). Adapter-specific fields are forwarded as params.

Attributes:

Name	Type	Description
adapter	str	HAL adapter id, e.g. "so100_follower", "so100_digital_twin", "franka_panda_real", "ur5e_real", "ros_control". Resolved against the HAL registry.
transport	dict[str, object]	Transport-layer parameters. Examples: {"port": "/dev/ttyACM0", "baud": 1_000_000} for SO-100; {"fci_uri": "172.16.0.2"} for Franka; {"namespace": "/ur5e"} for ros2_control.
params	dict[str, object]	Adapter-specific keyword arguments forwarded to the constructor (e.g. calibration overrides).

Example

HalConfig( ... adapter="so100_follower", ... transport={"port": "/dev/ttyACM0", "baud": 1_000_000}, ... ).adapter 'so100_follower'

HalEntrypoints

Bases: BaseModel

Per-robot simulation and real-hardware HAL import strings.

The two HALs a robot can expose, declared independently so the choice of HAL type lives in the manifest (never in environment config or runtime params). Each value is a "module:Attr" import string resolved by :func:openral_hal.build_hal, or None when the robot has no HAL of that kind (sim-only / real-only / scene-only). ADR-0031.

Attributes:

Name	Type	Description
sim	str | None	Import string for the simulation HAL. When None and :attr:RobotDescription.sim is populated, the resolver derives MujocoArmHAL.from_description (ADR-0023) — so every plain arm leaves this null. Set it explicitly only for a non-generic sim HAL (e.g. "openral_hal.panda_mobile:PandaMobileHAL", which has no sim: block to derive from).
real	str | None	Import string for the real-hardware HAL (e.g. "openral_hal.ur_real:UR5eRealHAL"). None for sim-only robots; the resolver raises ROSCapabilityMismatch if mode="real" is requested on a robot whose real is None.
parameters	HalParameters	Per-robot HAL construction defaults (serial port, robot_ip, …) merged into the constructor by :func:openral_hal.build_hal. ADR-0029. Empty by default.

Example

HalEntrypoints(real="openral_hal.ur_real:UR5eRealHAL").sim is None True

HalParameters

Bases: BaseModel

Per-robot HAL construction defaults declared in the manifest (ADR-0029).

Carries the transport / constructor keyword arguments a robot's HAL needs — the SO-100's serial port + baud, a ros2_control arm's robot_ip / fci_ip — so the manifest is the single source of those defaults instead of a per-robot lifecycle-node subclass. This is the schema seam that lets the unified, robot.yaml-driven ManifestHALLifecycleNode (ADR-0032) serve a parameterised HAL without a bespoke _create_hal.

:func:openral_hal.build_hal merges :attr:defaults underneath any explicit transport kwargs (so a deploy run override wins) and then drops every key the target HAL constructor does not accept — exactly the filtering it already applies to transport. Empty by default, so robots that need no construction kwargs (the derived MujocoArmHAL arms) are unaffected.

Attributes:

Name	Type	Description
defaults	dict[str, object]	HAL constructor / transport keyword defaults, e.g. {"port": "/dev/ttyACM0", "baud": 1_000_000} for the SO-100 or {"robot_ip": "192.168.1.10"} for a real UR arm.

Example

HalParameters(defaults={"port": "/dev/ttyACM0"}).defaults["port"] '/dev/ttyACM0'

Hand

Bases: str, Enum

Laterality of an end-effector.

HumanEvidence

Bases: _FailureEvidenceBase

Evidence for KIND_HUMAN — a human triggered an intervention.

Attributes:

Name	Type	Description
kind	Literal['human']	Discriminator (always "human").
actor	str	Identifier of the operator (e.g. "slack:alice", "gui").
reason	str	Free-form reason given by the actor.

ImagePreprocessing

Bases: BaseModel

Per-rSkill image preprocessing contract.

Properties of how the checkpoint was trained against image frames, surfaced on the manifest so the sim adapter does not have to learn them from a YAML override. vla.extra overrides on the eval config still win — see openral_rskill._vla_core.resolve_image_preprocessing.

Attributes:

Name	Type	Description
flip_180	bool	Apply torch.flip(t, dims=[1, 2]) (H+W reversal) to every camera frame before the policy forward. SmolVLA / pi05 LIBERO checkpoints want this on; the RoboCasa checkpoints published by RoMALab / DAVIAN-Robotics want it off.
flip_vertical	bool	Apply img[::-1, :, :] (vertical-only, H-axis reversal) BEFORE any 180° rotation. Mirrors robocasa.wrappers.gym_wrapper.RoboCasaGymEnv.process_img, which the canonical openpi-robocasa eval applies before feeding frames to the policy. Required for robocasa/robocasa365_checkpoints/pi05_pretrain_human300 and its lerobot-converted siblings; the RoMALab MG_300 checkpoint does NOT want this flip.
input_template	str	Format string used to name image tensors in the policy input batch — for example "observation.images.{cam}" for SmolVLA / pi05 RoMALab vs "observation.image.{cam}" for ruiname/pi05-robocasa-10tasks-200k.
aliases	dict[str, str]	Per-checkpoint rename map from the scene / robot raw camera key (e.g. robosuite's robot0_agentview_left_image) to the model's expected input feature name (e.g. agentview). Empty means pass through unchanged. SmolVLA LIBERO usually wants {"camera1": "image", "camera2": "image2"}.
norm_tag	str | None	Normalization statistics tag for policies that use multiple checkpoints or training distributions (e.g. MolmoAct2 with norm_tag="so100_so101_molmoact2" for SO-100/101 finetuned weights). Optional; None means the adapter's default tag applies. Used by predict_action methods to select the right norm_stats entry at inference time.
image_max_crops	int | None	Cap on the number of image tiles a multi-crop image processor (Molmo / MolmoAct2 family) produces per camera frame. Each extra 378px crop adds ~182 pooled image tokens and attention cost is quadratic in the token count, so this is the primary activation-memory lever (the NF4 weights are fixed at ~3.5 GiB; what overflows an 8 GiB card is the crop activations). Optional; None keeps the checkpoint's own default (8 for MolmoAct2). A per-rollout vla.extra["image_max_crops"] (or the OPENRAL_MOLMOACT2_MAX_CROPS env) still overrides this per-checkpoint default. Must be >= 1 when set.

IntrinsicsPinhole

Bases: BaseModel

Pinhole camera intrinsics.

Attributes:

Name	Type	Description
width	int	Image width in pixels.
height	int	Image height in pixels.
fx	float	Focal length in x (pixels).
fy	float	Focal length in y (pixels).
cx	float	Principal point x (pixels).
cy	float	Principal point y (pixels).
distortion_model	Literal['plumb_bob', 'equidistant', 'none']	Distortion model name.
distortion_coeffs	list[float]	Distortion coefficients.

Example

IntrinsicsPinhole(width=640, height=480, fx=600.0, fy=600.0, cx=320.0, cy=240.0) IntrinsicsPinhole(width=640, height=480, fx=600.0, fy=600.0, cx=320.0, cy=240.0, ...)

JointSpec

Bases: BaseModel

URDF-derived joint specification.

Attributes:

Name	Type	Description
name	str	Joint name matching the URDF.
joint_type	JointType	Kinematic joint type.
parent_link	str	Parent link name.
child_link	str	Child link name.
axis_xyz	tuple[float, float, float]	Rotation/translation axis unit vector.
position_limits	tuple[float, float] | None	(min, max) in radians or meters.
velocity_limit	float | None	Maximum velocity.
effort_limit	float | None	Maximum effort (N or Nm).
has_position_sensor	bool	Whether position feedback is available.
has_velocity_sensor	bool	Whether velocity feedback is available.
has_torque_sensor	bool	Whether torque feedback is available.
backlash_estimate	float | None	Estimated backlash in radians.
actuator_kind	Literal['dc', 'bldc', 'stepper', 'servo', 'tendon', 'hydraulic', 'pneumatic'] | None	Type of actuator.
sim_joint_name	str | None	Optional override carrying the MJCF/MuJoCo joint name as it appears in the simulator's compiled model when that name differs from :attr:name. The HAL uses :attr:name for the world-state contract (it's the URDF-shaped logical identifier the safety supervisor sees on every chunk); a separate sim_joint_name lets a sim-adapter look up mj_name2id without hardcoding robosuite / robocasa / MuJoCo naming conventions in backend modules. None means "the MJCF joint name matches :attr:name" — true for every fixed-base manipulator we ship; only mobile bases and humanoids whose MJCF auto-prefixes joints (robosuite's mobilebase0_… namespace, GR-1's robot0_…) need it. See ADR-0025.
role	JointRole	Structural classification (ADR-0028a). The downstream runner / safety / dataset-bridge code identifies grippers and base DoFs by this tag instead of substring-matching the joint name (which silently misclassifies any joint containing "gripper", e.g. "gripper_pose"). Default "unknown" keeps legacy manifests loadable; the fleet annotates incrementally per ADR-0028a.
origin_xyz	tuple[float, float, float]	Fixed translation (metres) of this joint's frame in its parent_link frame — the URDF <joint><origin xyz>. With :attr:origin_rpy and :attr:axis_xyz it gives the kernel everything it needs for forward kinematics (ADR-0030). Default (0, 0, 0); populated by the offline lowering tool (from MJCF/URDF) only for robots that enable self-collision checking — legacy manifests are unaffected.
origin_rpy	tuple[float, float, float]	Fixed orientation (roll, pitch, yaw, radians) of this joint's frame in its parent_link frame — the URDF <joint><origin rpy>. Default (0, 0, 0). ADR-0030.

JointState

Bases: BaseModel

Real-time joint state snapshot.

Attributes:

Name	Type	Description
name	list[str]	Ordered list of joint names.
position	list[float]	Joint positions (rad or m).
velocity	list[float]	Joint velocities (rad/s or m/s).
effort	list[float]	Joint efforts (Nm or N).
stamp_ns	int	ROS 2 timestamp in nanoseconds.

JointType

Bases: str, Enum

Kinematic joint type following URDF conventions.

LifecycleTransitionTool

Bases: _ReasonerToolBase

Tool variant — drive a ROS 2 lifecycle transition on a peer node.

Dispatch: service call on <node>/change_state with the matching Transition id (configure / activate / deactivate / cleanup). Used to bring a HAL back online, restart a faulted node, or stage a controlled shutdown.

Attributes:

Name	Type	Description
tool	Literal['lifecycle_transition']	Discriminator (always "lifecycle_transition").
node	str	Fully-qualified ROS node name (e.g. "/openral/hal/so100").
transition	Literal['configure', 'activate', 'deactivate', 'cleanup']	One of "configure", "activate", "deactivate", "cleanup". Other transitions ("shutdown", error-recovery) are deliberately omitted from the open-core palette per CLAUDE.md §6 Layer 6 — shutdown is owned by the safety supervisor, not the reasoner.

LinkCollisionGeometry

Bases: BaseModel

One convex collision volume rigidly attached to a robot link (ADR-0030).

The lowered, kernel-facing form of a link's collision geometry. Authored by hand, or emitted by the offline lowering tool from a robot's MJCF or URDF + SRDF source; the kernel loads these into pre-sized buffers at on_configure and never parses the source geometry on the hot path. link_name is a :class:JointSpec link (joints stays normative for the kinematic chain — this adds geometry only).

Attributes:

Name	Type	Description
link_name	str	The :attr:JointSpec.child_link (or :attr:JointSpec.parent_link) this volume is attached to.
shape	CollisionShape	The convex primitive (capsule or sphere).
origin_xyz_rpy	tuple[float, float, float, float, float, float]	Pose of the primitive in the link frame — (x, y, z, roll, pitch, yaw) in metres and radians.

Example

g = LinkCollisionGeometry( ... link_name="link_1", ... shape=CapsuleShape(radius_m=0.04, length_m=0.3), ... ) g.origin_xyz_rpy (0.0, 0.0, 0.0, 0.0, 0.0, 0.0)

LocateInViewTool

Bases: _ReasonerToolBase

Tool variant (read-only) — check if an object is in a live camera view (ADR-0043).

The complement to :class:RecallObjectTool: where recall_object recalls a remembered object from the ADR-0038 scene-graph memory, locate_in_view asks a live camera-mounted VLM detector (e.g. LocateAnything, ADR-0037) to look at the current frame right now and report whether the queried object is visible — and where. It runs the detector's open-vocabulary query on demand via the /openral/perception/locate_in_view ROS service and feeds the answer back to the LLM as a re-prompt (the prompt cascade). Like every variant it holds no authority over actuation (ADR-0018 §4) — it only reads a frame; the dispatch never gates the safety kernel.

Attributes:

Name	Type	Description
tool	Literal['locate_in_view']	Discriminator (always "locate_in_view").
query	str	Free-text object/description to look for (e.g. "the red mug"). Sent verbatim as the detector's open-vocabulary query.
camera	str	Optional camera selector. Empty (default) uses the detector's primary camera; otherwise names one of the detector's configured cameras so the reasoner can pick a viewpoint. Not a hardcoded name — the detector is camera-agnostic and maps the id to a topic.
detector	str	Optional on-demand locator selector (ADR-0056). Empty (default) uses the deployment's default locator; otherwise an rSkill id / short alias of one of the on-demand locators in the graph (e.g. "omdet-turbo-locator" for fast simple "find X", "locateanything-3b" for complex referring expressions). The reasoner routes to /openral/perception/<detector>/locate_in_view. Still read-only — choosing a model does not grant actuation.

MotionMetadata

Bases: _PerceptionEventBase

Perception event for /openral/perception/motion.

Emitted when a frame-difference detector measures sub-frame motion above a configured threshold. region_bbox is set when the detector localises the moving pixels (top-left origin in pixels); None means "motion magnitude only, no localisation."

Attributes:

Name	Type	Description
kind	Literal['motion']	Discriminator (always "motion").
magnitude	float	Mean absolute per-pixel difference, normalised to [0, 1] against the encoding's full-scale range.
threshold	float	Magnitude threshold that fired the event.
region_bbox	tuple[int, int, int, int] | None	Optional bounding box (x_min, y_min, x_max, y_max) of the moving region in pixels.

ObjectDetection2D

Bases: BaseModel

A single 2D detection inside an :class:ObjectsMetadata event.

The 3D :class:DetectedObject lives in :class:WorldState; the 2D form here is what a per-camera detector (nvinfer, tflite, cv2-CPU) emits before any pose lift / fusion.

Attributes:

Name	Type	Description
label	str	Semantic class label as produced by the model.
confidence	float	Detection confidence in [0, 1].
bbox_xyxy	tuple[int, int, int, int]	Axis-aligned bounding box in pixels (x_min, y_min, x_max, y_max); image origin top-left.

ObjectsMetadata

Bases: _PerceptionEventBase

Perception event for /openral/perception/objects.

Emitted by the event leg's detector element (nvinfer on Jetson, tflite on CPU, etc.) when one or more objects are detected.

Attributes:

Name	Type	Description
kind	Literal['objects']	Discriminator (always "objects").
detections	list[ObjectDetection2D]	Per-object 2D detections, ordered by descending confidence.
model_id	str	Identifier of the detector that fired (e.g. "yolov8n", "nvinfer:resnet50").
frame_width	int	Pixel width of the frame the detector ran on; the bbox_xyxy of each detection is in this pixel space (ADR-0035 lift scales it to the sensor's intrinsics resolution).
frame_height	int	Pixel height of that frame.

ObservationSpec

Bases: BaseModel

VLA observation configuration for this robot.

Attributes:

Name	Type	Description
state_key	str	Observation dict key for the state vector.
state_shape	tuple[int, ...]	Shape of the state tensor, e.g. (6,) for 6-D EEF.
state_representation	StateRepresentation | None	How the state vector is encoded.
image_flip_180	bool	Whether camera images need 180° rotation before feeding to the VLA (common for wrist-mounted cameras).

OccupancyGridRef

Bases: BaseModel

Reference to a 2D occupancy grid for mobile-base world-collision (ADR-0030).

Mirrors the nav_msgs/OccupancyGrid metadata that :mod:openral_runner.slam_bridge already decodes. The kernel consumes a bounded, fixed-capacity copy; a grid exceeding the configured capacity or older than the staleness deadline is treated as unavailable (fail-closed). The occupancy bytes are referenced by topic, not inlined.

Attributes:

Name	Type	Description
frame_id	str	tf2 frame the grid origin is expressed in (e.g. "map").
resolution_m	float	Edge length of one cell, in metres.
width	int	Grid width in cells.
height	int	Grid height in cells.
origin	Pose6D	Pose of cell (0, 0)'s lower-left corner.
data_topic	str	ROS 2 topic carrying the nav_msgs/OccupancyGrid.

OcrMetadata

Bases: _PerceptionEventBase

Perception event for /openral/perception/ocr.

Attributes:

Name	Type	Description
kind	Literal['ocr']	Discriminator (always "ocr").
text	str	Recognised text (already stripped of leading / trailing whitespace by the detector).
confidence	float	Recogniser confidence in [0, 1].
region_bbox	tuple[int, int, int, int] | None	Optional bounding box (x_min, y_min, x_max, y_max) of the recognised region in pixels.

PerceptionStaleEvidence

Bases: _FailureEvidenceBase

Evidence for KIND_PERCEPTION — a sensor frame went stale.

Attributes:

Name	Type	Description
kind	Literal['perception']	Discriminator (always "perception").
sensor_id	str	SensorSpec.name of the stale sensor.
staleness_ms	float	Observed age of the last frame, in milliseconds.
threshold_ms	float	Staleness threshold that was crossed, in milliseconds.

PhysicsBackend

Bases: str, Enum

Physics / scene backend used to instantiate a :class:SceneSpec.

Attributes:

Name	Type	Description
MUJOCO		Vanilla MuJoCo (CPU / single-env). Default for LIBERO, MetaWorld.
MUJOCO_MJX		MuJoCo MJX (XLA, GPU-batched headless rollouts).
PYBULLET		PyBullet (legacy adapters, contact-rich tabletop).
ISAACSIM		NVIDIA Isaac Sim (Omniverse, GPU). Future.
GENESIS		Genesis (physics-language unification). Future.
MOCK		In-process mock with no physics — used for wiring smoketests.

Pose6D

Bases: BaseModel

6D pose: position + quaternion.

Attributes:

Name	Type	Description
xyz	tuple[float, float, float]	Position (x, y, z) in meters.
quat_xyzw	tuple[float, float, float, float]	Quaternion (x, y, z, w).
frame_id	str	tf2 reference frame.

ProtocolSpec

Bases: BaseModel

Stand-alone eval-protocol descriptor.

Historically the protocol block on the deleted BenchmarkSpec (ADR-0009). After the Task-10 scene-hierarchy convergence (ADR-0041, June 2026) a benchmark became a list of :class:BenchmarkScenes — each scene carries its own n_episodes / seed / task.success_key / task.max_steps — and ADR-0042 then deleted the BenchmarkSpec wrapper altogether. ProtocolSpec is retained as a public schema for callers that want to describe a protocol independently (e.g. ADR drafts, benchmark-report tooling).

Pins the methodology so two rSkills evaluated under the same benchmark produce apples-to-apples numbers. Authors of a benchmark should set these to match the published protocol of the suite they are reproducing (e.g. LIBERO: 10 episodes per task, fixed seed range, is_success).

Attributes:

Name	Type	Description
n_episodes	int	Number of independent episodes per task. Honest success rates need >= 10 per task for the published LIBERO / MetaWorld protocols.
seeds	list[int]	Seed list applied per task (paired index-wise with the episode index). Length MUST be >= n_episodes; runners slice seeds[:n_episodes] so re-runs are reproducible.
success_key	str	info key on the gym step() return whose truthy value marks task success. Default "is_success" (LIBERO convention); MetaWorld uses "success".
max_steps	int	Per-task episode budget. Adapters may clip to scene-internal limits; the value here is the protocol ceiling.
min_reps	int | None	Minimum number of completed episodes per task before the result JSON is written. Guards against a crash mid-suite producing a misleadingly partial roll-up. None means "require every task x episode".

Example

p = ProtocolSpec(n_episodes=10, seeds=list(range(10)), max_steps=280) p.success_key 'is_success'

model_post_init(_context: object) -> None

Cross-field validation: seeds list must cover n_episodes.

Source code in python/core/src/openral_core/schemas.py

def model_post_init(self, _context: object) -> None:
    """Cross-field validation: seeds list must cover n_episodes."""
    if len(self.seeds) < self.n_episodes:
        raise ValueError(
            f"ProtocolSpec.seeds has {len(self.seeds)} entries but "
            f"n_episodes={self.n_episodes}; provide at least n_episodes "
            f"seeds so re-runs are reproducible."
        )
    if self.min_reps is not None and self.min_reps > self.n_episodes:
        raise ValueError(
            f"ProtocolSpec.min_reps ({self.min_reps}) exceeds "
            f"n_episodes ({self.n_episodes}); a benchmark cannot require "
            f"more completed reps than it schedules."
        )

QuantizationBackend

Bases: str, Enum

Inference backend that will execute the quantized model.

Attributes:

Name	Type	Description
PYTORCH		torch.quantization / bitsandbytes / torchao.
ONNX		onnxruntime with quantization pre-applied at export.
TENSORRT		NVIDIA TensorRT engine (INT8 / FP8 calibrated).
GGUF		llama.cpp / ggml GGUF format (CPU and Metal).
MLX		Apple MLX framework (Apple Silicon only).

Example

QuantizationBackend.PYTORCH.value 'pytorch'

QuantizationConfig

Bases: BaseModel

Full specification of how to quantize a skill's model weights.

Attributes:

Name	Type	Description
dtype	QuantizationDtype	Target numeric format.
backend	QuantizationBackend	Inference backend that will execute the model.
per_channel	bool	Use per-channel (vs per-tensor) quantization.
calibration_dataset	str | None	HuggingFace dataset ID or local path used for post-training calibration (INT8 / TensorRT only).
extra	dict[str, object]	Backend-specific overrides (e.g. {"calibration_steps": 128}).

Example

cfg = QuantizationConfig(dtype=QuantizationDtype.INT8) cfg.backend

QuantizationDtype

Bases: str, Enum

Numeric format used to represent model weights and activations.

Attributes:

Name	Type	Description
FP32		32-bit float (full precision, reference).
FP16		16-bit float (GPU-native, common default).
BF16		Brain-float16 (Ampere+ GPUs, TPUs; better dynamic range than FP16).
INT8		8-bit integer (CPU/GPU; good accuracy/speed trade-off).
INT4		4-bit integer (edge / memory-constrained; some accuracy loss).
FP4_NVFP4		NVIDIA FP4 format (Hopper+; highest throughput).

Example

QuantizationDtype.INT8.value 'int8'

QuerySceneTool

Bases: _ReasonerToolBase

Tool variant (read-only) — ask a scene VLM a question about the live view (ADR-0047).

Backed by a kind: "vlm" rSkill (e.g. Qwen3.5-4B NF4) running in an out-of-process ZMQ sidecar. Where :class:LocateInViewTool answers where an object is (open-vocabulary localization via the detector), query_scene answers open-ended questions about the scene's state — task-progress and success/failure verification the reasoner needs for its replanning ladder: "has the robot grasped the mug?", "is the bowl on the shelf?", "did we drop the object?", "is the table clear?".

It captures the current frame of the requested camera, sends it plus the question to the VLM over the /openral/perception/query_scene ROS service, and feeds the free-text answer back to the LLM as a re-prompt (the prompt cascade). Like every variant it holds no authority over actuation (ADR-0018 §4) — it only reads a frame; the dispatch never gates the safety kernel. It is not a localizer: use :class:LocateInViewTool to find objects.

Attributes:

Name	Type	Description
tool	Literal['query_scene']	Discriminator (always "query_scene").
question	str	Natural-language question about the current scene (e.g. "Has the robot grasped the red mug?"). Sent verbatim to the VLM as the textual prompt alongside the frame.
camera	str	Optional camera selector. Empty (default) uses the perception node's primary camera; otherwise names one of its configured cameras so the reasoner can pick a viewpoint. Not a hardcoded name.

QueryTaskProgressTool

Bases: _ReasonerToolBase

Tool variant (read-only) — ask the reward monitor how the task is going (ADR-0057).

Backed by a kind: "reward" rSkill (Robometer-4B NF4) running in parallel with the active VLA in an out-of-process ZMQ sidecar. Where :class:QuerySceneTool answers open-ended scene questions as free text, query_task_progress returns a quantitative windowed assessment of the current task: normalized progress and success over the last :attr:window_s seconds, plus their trends and a stalled flag.

It calls the /openral/perception/query_task_progress ROS service, which scores the monitor's buffered camera frames against the task instruction and returns progress_now / success_now / progress_trend / success_trend / stalled / succeeded. The reasoner uses it to decide whether to continue, escalate to :class:QuerySceneTool, advance, or enter the replanning ladder. Like every variant it holds no authority over actuation (ADR-0018 §4) — the reward signal is advisory; the dispatch never gates the safety kernel.

Attributes:

Name	Type	Description
tool	Literal['query_task_progress']	Discriminator (always "query_task_progress").
window_s	float	How many seconds of recent frames to assess. Must be > 0; clamped to the monitor's configured frame_window_s.
task	str	Optional task-instruction override. Empty (default) reuses the instruction the monitor was co-activated with (the active VLA's goal).

RSkillAction

Bases: str, Enum

Closed vocabulary of high-level action verbs an rSkill can perform.

Declared on :attr:RSkillManifest.actions so the reasoner's LLM tool palette can present each skill with a structured "what does it do" label, in addition to the free-form description and the slug. The LLM scores tools primarily on natural-language description, but a closed verb vocabulary lets the palette pre-filter and lets the schema be unit-testable. New entries are additive.

Categories (descriptive only — not part of the wire format):

• Manipulation primitives: PICK, PLACE, PICK_AND_PLACE, TRANSFER, GRASP, RELEASE.
• Articulated / contact-rich: OPEN, CLOSE, PUSH, PULL, SLIDE, INSERT, POUR, WIPE, ROTATE.
• Motion: REACH; LOOK (aim a camera at a point, ADR-0044).
• Mobile (for mobile-manipulator embodiments): NAVIGATE.
• Social / expressive: WAVE, SHAKE.
• Generalist marker: GENERALIST for foundation / multi-task checkpoints (e.g. RoboCasa-365, DROID, MetaWorld-MT50). The palette surfaces a generalist skill for goals that don't match a specific verb.

RSkillEvalBenchmark

Bases: BaseModel

The benchmark suite a :class:RSkillEvalResult was measured against.

Attributes:

Name	Type	Description
name	str	Canonical suite name ("LIBERO", "MetaWorld", "ALOHA", "PushT", "RoboCasa" …). Used by openral benchmark report to group rows.
dataset	str | None	Optional HF dataset identifier.
protocol	str	Free-text description of the eval protocol.
robot	str	robot_id the benchmark was evaluated on.
simulator	str	Free-text simulator description.

RSkillEvalResult

Bases: BaseModel

Pydantic model of a rskills/<id>/eval/<benchmark>.json file.

The results and baselines blocks are intentionally benchmark-specific — LIBERO has four sub-suites with success rates, MetaWorld carries an MT50 average, ALOHA has a single cube-transfer rate. We require the metadata blocks (source, benchmark, eval_config) but leave results as a free-form dict so each benchmark can declare its own keys.

Attributes:

Name	Type	Description
schema_version	str	Semver-ish version string for this on-disk format.
source	RSkillEvalSource	Provenance — paper / reproduction state.
benchmark	RSkillEvalBenchmark	Suite identity (name, robot, simulator).
eval_config	dict[str, object]	Free-form configuration the benchmark was run with (chunk size, image size, denoising steps, …).
results	dict[str, object]	Free-form per-task / per-suite success rates.
baselines	dict[str, object]	Optional free-form comparison numbers from prior work.
trace_id	str | None	Hex OTel trace id (32 chars) for the rollout that produced this result. Set by openral benchmark run from the cli.command root span so reviewers can deep-link from rskills/<id>/eval/<benchmark>.json straight to the trace tree in Jaeger / Tempo. Optional — paper-cited numbers (reproduced_locally: false) leave it None.

Example

RSkillEvalResult.model_validate_json(

'{"schema_version": "0.1", "source": {...}, ...}')

from_json(path: str) -> RSkillEvalResult classmethod

Load and validate a rskills/<id>/eval/<benchmark>.json file.

Parameters:

Name	Type	Description	Default
path	str	Filesystem path to the JSON file.	required

Returns:

Type	Description
RSkillEvalResult	A validated :class:RSkillEvalResult.

Raises:

Type	Description
FileNotFoundError	If path does not exist.
ValidationError	If the JSON fails schema validation.

Source code in python/core/src/openral_core/schemas.py

@classmethod
def from_json(cls, path: str) -> RSkillEvalResult:
    """Load and validate a ``rskills/<id>/eval/<benchmark>.json`` file.

    Args:
        path: Filesystem path to the JSON file.

    Returns:
        A validated :class:`RSkillEvalResult`.

    Raises:
        FileNotFoundError: If ``path`` does not exist.
        pydantic.ValidationError: If the JSON fails schema validation.
    """
    import json as _json  # noqa: PLC0415  # reason: deferred import

    with open(path, encoding="utf-8") as fh:
        data = _json.load(fh)
    return cls.model_validate(data)

RSkillEvalSource

Bases: BaseModel

Provenance for a benchmark result block.

Used inside :class:RSkillEvalResult so consumers know where the numbers came from (paper vs local reproduction) without having to read the _comment line.

Attributes:

Name	Type	Description
paper	str	Plain-text title of the source paper / report.
arxiv	str | None	Optional arxiv URL or id.
model_variant	str	Which checkpoint variant the numbers describe (e.g. "SmolVLA (0.45B)").
evaluated_by	str	Free-text — "upstream authors" or a contributor.
reproduced_locally	bool	True only when OpenRAL CI / contributor re-ran the benchmark and the listed numbers match.
reproduction_planned	str | None	Optional plan for closing the gap when reproduced_locally is False.
reproduction_cli	dict[str, object] | str | None	Either a single command string or a structured {description, single_suite_example, all_suites, suite_max_steps, notes} block (matches the LIBERO eval JSON shape).
table	str | None	Optional pointer at a specific table in the source paper.
status	str | None	Optional state marker — "in_progress", "deferred", "reproduced" — for benchmarks that are not yet final.

RSkillInfo

Bases: BaseModel

Snapshot of a Skill's runtime state — published on /skill/<name>/info.

Attributes:

Name	Type	Description
name	str	rSkill name (e.g. "noop_skill").
version	str	SemVer string.
state	RSkillState	Current primary lifecycle state.
weights_loaded	bool	Whether model weights have been loaded into memory.
quantized	bool	Whether weights have been quantized for the target runtime.
warmed_up	bool	Whether the model has been warmed up (first inference run).
embodiment_tags	list[str]	Embodiment tags from the skill manifest.
role	Literal['s0', 's1', 's2']	rSkill role: "s0" (cerebellar), "s1" (fast policy), "s2" (slow reasoning).
latency_budget_ms	float | None	Maximum allowed inference latency in milliseconds.
last_inference_ms	float | None	Actual latency of the most recent step() call.
error_msg	str | None	Human-readable error description when state == "error".
stamp_ns	int	Timestamp of this snapshot in nanoseconds.

Example

info = RSkillInfo(name="hello", version="0.1.0", state=RSkillState.UNCONFIGURED) info.weights_loaded False info.state

RSkillLatencyBudget

Bases: BaseModel

Per-stage latency budget declared in rskill.yaml.

Attributes:

Name	Type	Description
per_chunk_ms	float	End-to-end step() budget. CI fails if exceeded on the reference host (CLAUDE.md §7.4).
warmup_ms	float | None	Maximum allowed warm-up time during activate().
load_ms	float | None	Maximum allowed weight-load time during configure().

RSkillLicensePosture

Bases: str, Enum

License posture surfaced at install time (CLAUDE.md §7.4).

RSkillManifest

Bases: BaseModel

Pydantic model of the rskill.yaml package manifest (V1).

This is the on-disk schema for an rSkill HF Hub repo. An rSkill is loaded by capability-checking against a :class:RobotDescription, selecting a runtime + quantization, then constructing a runtime :class:~openral_rskill.Skill instance.

schema_version is "0.1": the manifest surface has had no backward-incompatible change. Now the repo is published it is versioned for real (CLAUDE.md §1.6) — a backward-incompatible change bumps it and ships a migrator, while backward-compatible additions (ADR-0013/0022/0024) evolve the surface in place.

ADR-0013 added two symmetric guards on top of the initial V1 shape:

1. actuators_required mirrors sensors_required on the output side. Every skill declares at least one :class:ActuatorRequirement; the loader validates it against :attr:RobotDescription.action_spec. n_dof and vla_action_key are auto-filled from the robot YAML for the 9 canonical embodiments; for "custom" they must be set on the manifest.

2. embodiment_extra is the explicit escape hatch for embodiments that do not have a canonical robots/<id>/robot.yaml. Required iff "custom" appears in :attr:embodiment_tags; forbidden otherwise.

V1 already tightened: name / fallback_skill_id must be HF-Hub-shaped, version is SemVer, weights_uri is restricted to hf:// or local://, embodiment_tags is closed to the set of in-tree-supported robots (now including "custom"), and benchmarks replaces the old free-form metadata blob with a typed dict of canonical suite ids → success rate. Commercial-use posture is derived from :attr:license via :attr:is_commercial_use_allowed.

Attributes:

Name	Type	Description
schema_version	Literal['0.1']	On-disk format version. "0.1" today. Backward-compatible extensions (ADR-0013/0022/0024) evolved the surface in place; now the repo is published, a backward-incompatible change bumps this and ships a migrator (CLAUDE.md §1.6).
name	str	HF Hub identifier, e.g. "openral/rskill-pick-cube-so100". Must match <owner>/<repo>.
version	str	SemVer string of the rSkill package itself (not the wrapped weights).
license	RSkillLicensePosture	License posture (surfaced at install time). Drives :attr:is_commercial_use_allowed.
role	Literal['s0', 's1', 's2']	rSkill slot. "s1" for fast policies (the only slot currently loaded via :func:rSkill.from_yaml — 100% of in-tree skills); "s0" reserved for cerebellar realtime (humanoid balance, C++ only); "s2" reserved for slow reasoning (LLM-emitted BehaviorTree XML).
model_family	ModelFamily | None	Closed VLA / policy family. Drives the runner's adapter dispatch.
embodiment_tags	list[EmbodimentTag]	Robot embodiments this rSkill targets. Must intersect a robot's RobotCapabilities.embodiment_tags for the loader to accept. Restricted to the canonical set (:data:EmbodimentTag) so typos / framework hints cannot silently always-miss. "custom" is the explicit hatch (see :attr:embodiment_extra).
embodiment_extra	EmbodimentExtra | None	ADR-0013. When "custom" is in :attr:embodiment_tags, declares the sensor + actuator surface of the custom rig so the loader's compat check still has something to match against. MUST be None when "custom" is not present.
capabilities_required	dict[str, bool | float | int | str]	Per-flag capability requirements (subset of :class:RobotCapabilities boolean fields). The loader rejects installation on a robot that does not satisfy every flag.
sensors_required	list[SensorRequirement]	Sensor inputs the policy expects from the robot. See :class:SensorRequirement.
actuators_required	list[ActuatorRequirement]	ADR-0013. Symmetric output-side contract: at least one :class:ActuatorRequirement entry. The loader matches each entry's :attr:kind against :attr:RobotDescription.action_spec.control_mode and auto-fills :attr:ActuatorRequirement.n_dof / :attr:vla_action_key from the canonical robot YAML (predefined embodiments only — for "custom" they must be set on the manifest).
runtime	RSkillRuntime	Preferred inference runtime.
quantization	QuantizationConfig	Default :class:QuantizationConfig for this rSkill.
weights_uri	str | None	HF Hub revision-pinned URI to weights or local path reference. Production deployments pin a SHA (CLAUDE.md operating principle 8).
chunk_size	int	Action-chunk length the policy emits per :meth:Skill.step. Drives the ChunkedExecutor's overlap schedule and shows up in the trace surface.
latency_budget	RSkillLatencyBudget	Latency contract enforced by CI on the reference host (CLAUDE.md §7.4).
min_vram_gb	dict[QuantizationDtype, float] | None	Optional minimum VRAM (GB) the skill needs per quantization dtype. Informational — the loader uses it for ral skill check / openral doctor; the actually-applied dtype is still pinned by quantization.dtype (CLAUDE.md §11 forbids silent downcast).
fallback_skill_id	str | None	rSkill id to substitute if this one fails (RFC §8.3 replanning ladder). HF Hub shape.
benchmarks	dict[BenchmarkName, float]	Map of canonical benchmark suite id → success rate ([0.0, 1.0]) as measured for this skill. Keys constrained to :data:BenchmarkName. The full per-task breakdown lives in the matching rskills/<id>/eval/<key>.json validated against :class:RSkillEvalResult.
paper_url	str | None	Canonical paper URL for this skill / family.
dataset_uri	str | None	HF Hub URI for the training dataset.
source_repo	str | None	HF Hub URI for the upstream weights repo (often the same as :attr:weights_uri's prefix).
description	str	REQUIRED. Short (1-500 char) human-readable summary surfaced by ral skill list and, more importantly, by the reasoner's per-skill LLM tool description (see :func:openral_reasoner.palette.build_tool_palette). The LLM scores tools primarily on this text; keep it specific to what the skill does (objects, scenes, task type), not how it was trained.
actions	list[RSkillAction]	REQUIRED. Closed-vocabulary list (≥1) of high-level action verbs this skill performs (see :class:RSkillAction). Generalist / foundation checkpoints declare [GENERALIST]; specialist checkpoints list the verbs they were trained on (e.g. [PICK, PLACE]). Surfaced to the reasoner LLM alongside :attr:description so it can pick the right skill for a given goal.
objects	list[str]	Optional free-form keywords for the objects this skill manipulates (e.g. ["cube"], ["pipe"], ["drawer"]). Discriminative hints for the LLM; the long tail (RoboCasa-365 has hundreds of categories) makes a closed enum impractical.
scenes	list[str]	Optional free-form keywords for the scenes / environments this skill targets (e.g. ["tabletop"], ["kitchen"]). Same rationale as :attr:objects.
processors	RSkillProcessors | None	Per-file URIs for the lerobot PolicyProcessorPipeline artefacts (Gap 1 + Gap 3 of the rSkill self-containment audit). REQUIRED when model_family is one of smolvla, pi05, xvla, diffusion, or rldx; OPTIONAL for act (the legacy norm-stats-in-safetensors path is allowed there).
image_preprocessing	ImagePreprocessing | None	Per-checkpoint image-side knobs that cannot be encoded in the processor JSONs (flip, camera renames). Optional; None means schema defaults apply.
state_contract	StateContract | None	Per-checkpoint proprioception layout (named layouts for RoboCasa; explicit dim for everything else).
action_contract	ActionContract | None	Per-checkpoint action vector contract (dim + optional representation). ADR-0019: consumed by the dataset bridge to bind the LeRobot v3 action feature shape without consulting the runtime adapter.
n_action_steps	int | None	Replay cadence (how many actions to consume from a chunk before re-inferring). Omit when equal to chunk_size; the adapter falls through to its family default.

Example

m = RSkillManifest( ... name="openral/rskill-pick-cube-so100", ... version="0.1.0", ... license=RSkillLicensePosture.APACHE_2_0, ... role="s1", ... kind="vla", ... model_family="smolvla", ... embodiment_tags=["so100_follower"], ... runtime=RSkillRuntime.PYTORCH, ... weights_uri="hf://lerobot/smolvla_base", ... chunk_size=16, ... latency_budget=RSkillLatencyBudget(per_chunk_ms=100.0), ... description="Pick a cube on the SO-100 follower arm.", ... actions=[RSkillAction.PICK], ... objects=["cube"], ... scenes=["tabletop"], ... actuators_required=[ ... ActuatorRequirement( ... kind=ControlMode.JOINT_POSITION, ... control_mode_semantics=ControlModeSemantics(mode="absolute"), ... ), ... ], ... processors=RSkillProcessors( ... preprocessor_uri="hf://lerobot/smolvla_base/policy_preprocessor.json", ... postprocessor_uri="hf://lerobot/smolvla_base/policy_postprocessor.json", ... ), ... ) m.is_commercial_use_allowed True m.schema_version '0.1'

is_commercial_use_allowed: bool property

Derive commercial-use posture from :attr:license.

apache-2.0 / mit / bsd → True. Every other posture (including unknown) → False, conservatively. The loader consults this in :meth:rSkill._check_license; users wanting to deploy a non-commercial skill in a research context set OPENRAL_ALLOW_NONCOMMERCIAL=1 per CLAUDE.md §7.4.

from_yaml(path: str) -> RSkillManifest classmethod

Load and validate an rskill.yaml from disk.

Parameters:

Name	Type	Description	Default
path	str	Filesystem path to the manifest YAML.	required

Returns:

Type	Description
RSkillManifest	A validated :class:RSkillManifest.

Raises:

Type	Description
FileNotFoundError	If path does not exist.
ValidationError	If the YAML fails schema validation.

Source code in python/core/src/openral_core/schemas.py

@classmethod
def from_yaml(cls, path: str) -> RSkillManifest:
    """Load and validate an ``rskill.yaml`` from disk.

    Args:
        path: Filesystem path to the manifest YAML.

    Returns:
        A validated :class:`RSkillManifest`.

    Raises:
        FileNotFoundError: If ``path`` does not exist.
        pydantic.ValidationError: If the YAML fails schema validation.
    """
    import yaml  # noqa: PLC0415  # reason: yaml is a runtime dep; deferred to avoid import-time cost

    with open(path, encoding="utf-8") as fh:
        data = yaml.safe_load(fh)
    return cls.model_validate(data)

RSkillProcessors

Bases: BaseModel

Explicit lerobot PolicyProcessorPipeline artefact pointers.

Closes Gap 1 + Gap 3 of the rSkill self-containment audit. Replaces the implicit snapshot_download(repo_id) → make_pre_post_processors(pretrained_path=...) fetch path used today by the SmolVLA / pi05 / xVLA / Diffusion / modern-ACT adapters. Two files are required because lerobot's pipeline ships them as a pair: a preprocessor that normalises the policy input batch and a postprocessor that un-normalises the action chunk.

Each URI must point at a specific file (the implicit-snapshot shape hf://owner/repo with no file tail is rejected). Two URI schemes accepted:

• hf://owner/repo[@rev]/path/to/file.json — file inside an HF Hub repo, revision-pinnable.

Attributes:

Name	Type	Description
preprocessor_uri	str	Per-file URI to the policy preprocessor JSON.
postprocessor_uri	str	Per-file URI to the policy postprocessor JSON.

Example

RSkillProcessors( ... preprocessor_uri="hf://lerobot/smolvla_libero/policy_preprocessor.json", ... postprocessor_uri="hf://lerobot/smolvla_libero/policy_postprocessor.json", ... ) # doctest: +ELLIPSIS RSkillProcessors(preprocessor_uri='hf://...preprocessor.json', postprocessor_uri='hf://...postprocessor.json')

RSkillRuntime

Bases: str, Enum

Inference runtime hint declared in the manifest (RFC §7.3).

RSkillState

Bases: str, Enum

Primary lifecycle states for a Skill node.

Matches the ROS 2 Managed Node state machine with openral extensions.

States

unconfigured Initial state. Weights are not loaded. inactive Configured and weights loaded; not yet warmed up. Accepts no actions. active Ready to execute. step() may be called. finalized Terminal state after shutdown(). Cannot transition further. error Unrecoverable failure. Requires external intervention.

ReasonerTimeoutEvidence

Bases: _FailureEvidenceBase

Evidence for KIND_REASONER_TIMEOUT — the LLM call missed its deadline.

Attributes:

Name	Type	Description
kind	Literal['reasoner_timeout']	Discriminator (always "reasoner_timeout").
model	str	Model identifier (e.g. "claude-opus-4-7").
deadline_s	float	Configured deadline in seconds.
elapsed_s	float	Actual elapsed wall-clock time in seconds.

RecallObjectMatch

Bases: BaseModel

One ranked match from a :class:RecallObjectQuery (ADR-0038 §6).

Attributes:

Name	Type	Description
node_id	str	The matched node's id.
label	str	The matched node's label.
pose	Pose6D	The object's recalled pose (map frame).
score	float	Match score in [0, 1].
last_seen_ns	int	When the object was last observed, in nanoseconds.
approach	ApproachViewpoint | None	Optional computed camera-facing approach viewpoint.
inside_container_id	str | None	Set when the object is inside an occluding container that must be opened first; None otherwise.

RecallObjectQuery

Bases: BaseModel

Read-only query to recall a remembered object (ADR-0038 §6).

At least one of text / label must be non-empty. text is matched against node embeddings when an embedder is configured (ADR-0038 §5), otherwise matching falls back to label.

Attributes:

Name	Type	Description
text	str	Free-text query (open-vocabulary match).
label	str	Exact label match.
near	Pose6D | None	Optional pose to bias results toward (proximity).
max_age_ns	int | None	Optional recency filter — drop nodes whose last_seen_ns is older than this many nanoseconds before "now".
limit	int	Maximum number of matches to return.

RecallObjectResult

Bases: BaseModel

Result of a :class:RecallObjectQuery (ADR-0038 §6).

Attributes:

Name	Type	Description
matches	list[RecallObjectMatch]	Ranked matches (possibly empty — an empty result is how the query reports "unknown"; callers raise / handle :class:~openral_core.exceptions.ROSObjectNotInMemory rather than fabricating a pose).

RecallObjectTool

Bases: _ReasonerToolBase

Tool variant (read-only) — recall a remembered object (ADR-0039).

Queries the ADR-0038 scene-graph spatial memory and returns the object's map-frame pose plus a camera-facing approach viewpoint and any occluding container to the reasoner's next reasoning step. Like every variant it holds no authority over actuation (ADR-0018 §4) — it only reads memory. The dispatch that runs the query and feeds the result back to the LLM is wired in ADR-0039 Phase 2 (this is the typed contract).

Attributes:

Name	Type	Description
tool	Literal['recall_object']	Discriminator (always "recall_object").
query	str	Free-text or label naming the object to recall (e.g. "the red mug"). Mapped to a :class:RecallObjectQuery at dispatch.
limit	int	Maximum number of ranked matches to return.

ReloadGstPipelineTool

Bases: _ReasonerToolBase

Tool variant — swap a sensor's GStreamer pipeline at runtime.

Dispatch: service call on /openral/sensors/<sensor_id>/reload_pipeline with the pipeline YAML payload. Lets the reasoner tune perception (switch resolution, enable an nvinfer leg, swap an RTSP source) without redeploying the runtime.

Attributes:

Name	Type	Description
tool	Literal['reload_gst_pipeline']	Discriminator (always "reload_gst_pipeline").
sensor_id	str	SensorSpec.name of the camera whose pipeline is being reloaded. Validated against the active runtime's sensor catalog at dispatch time.
pipeline_yaml	str	Full YAML body of the new :class:SensorReaderConfig. The sensor node validates this against the Pydantic schema before accepting the swap.

ResolvePlaceQuery

Bases: BaseModel

Read-only query to resolve a place/room/agent reference to a goal (ADR-0038 §6).

Attributes:

Name	Type	Description
reference	str	Free-text, id, or label of the target (e.g. "kitchen", "where I was standing", a node id).
kind	SpatialNodeKind | None	Optional node-kind filter (room / place / agent).

ResolvePlaceResult

Bases: BaseModel

Result of a :class:ResolvePlaceQuery (ADR-0038 §6).

Attributes:

Name	Type	Description
node_id	str	The resolved node's id.
goal	Pose6D	Navigation goal pose (map frame).
path_node_ids	list[str]	Ordered traversable_to path of node ids from the robot's current place to the goal (empty when no path is needed or known).

ResolvePlaceTool

Bases: _ReasonerToolBase

Tool variant (read-only) — resolve a place/room/agent to a goal (ADR-0039).

Queries the ADR-0038 scene-graph memory for a navigation goal pose plus a traversable_to path. Read-only; holds no authority over actuation. Dispatch + result-return are ADR-0039 Phase 2.

Attributes:

Name	Type	Description
tool	Literal['resolve_place']	Discriminator (always "resolve_place").
reference	str	Free-text, id, or label of the target (e.g. "the kitchen", "where I was standing"). Mapped to a :class:ResolvePlaceQuery.

RewardContract

Bases: BaseModel

Manifest contract for kind: "reward" rSkills (ADR-0057).

Carries the configuration a robotic reward / progress-monitor model (e.g. Robometer-4B, a Qwen3-VL-4B reward foundation model) needs to score a rollout. A reward skill runs in parallel with a kind: "vla" policy, continuously ingesting the VLA's camera frames into a rolling window, and emits per-frame progress ∈ progress_range and per-frame success probability. The Reasoner queries it on demand (QueryTaskProgressTool) to decide whether to continue, escalate to a scene VLM, advance, or replan. The signal is advisory only — it never actuates and never gates motors (CLAUDE.md §1.1).

Required when :attr:RSkillManifest.kind is "reward"; forbidden for all other kinds (enforced by :meth:RSkillManifest._check_kind_consistency). Like detector / vlm, a reward skill is a pure perception consumer: it emits no Action chunks and requires no actuators.

Attributes:

Name	Type	Description
progress_range	tuple[float, float]	(min, max) of the normalized per-frame progress scalar. Default (0.0, 1.0) — Robometer's discrete/binned mode (see :attr:num_bins) emits progress already in [0, 1].
success_threshold	float	Per-frame success probability at/above which the frame is considered a task success. In [0.0, 1.0]; default 0.5.
preference	bool	Whether the model also exposes a trajectory-preference head (Robometer does). Default False — the progress/success path is the Reasoner-facing contract; preference is future work.
frame_window_s	float	Length of the rolling frame buffer in seconds. The sidecar evicts frames older than this relative to the newest. Must be > 0.
target_fps	float	Frame-sampling rate fed to the model (Robometer's example uses 3 fps). Must be > 0. This is an S2-cadence monitor, not a per-control-step signal.
num_bins	int	Discrete-mode bin count for the progress head. Robometer's discrete mode yields per-frame normalized progress in [0, 1]; continuous mode yields raw regression values. Must be > 0; default 100.
instruction_required	bool	Whether a natural-language task instruction must accompany the frames (Robometer requires one). Default True.

Example

c = RewardContract(frame_window_s=8.0, target_fps=3.0) c.progress_range (0.0, 1.0) c.success_threshold 0.5 c.num_bins 100

RoboCasaBackendOptions

Bases: BaseModel

Typed validator helper for SceneSpec.backend_options under RoboCasa.

RoboCasa exposes two scenario modes:

• prebuilt — pick one of the ~100 atomic-PnP / door / drawer / navigation tasks shipped with the package by name (e.g. "PnPCounterToCab"). Set :attr:prebuilt_task; leave all procedural keys at their defaults.
• procedural -- author a kitchen by composing :attr:kitchen_style x :attr:layout_id x :attr:fixtures x :attr:spawn_objects x :attr:task_verb. Set :attr:mode to "procedural" and leave :attr:prebuilt_task None.

The model is purely additive: it lives alongside :class:SceneSpec and is constructed by the RoboCasa scene adapter at factory time via RoboCasaBackendOptions.model_validate( scene.backend_options). The parent SceneSpec.backend_options: dict[str, object] field is unchanged, so this class does not constitute a schema migration (ADR-0015, CLAUDE.md §1.6).

See :doc:ADR-0015 </adr/0015-robocasa-isolated-backend-lazy-assets> for the full backend rationale.

Attributes:

Name	Type	Description
mode	Literal['prebuilt', 'procedural']	"prebuilt" (default) or "procedural".
prebuilt_task	str | None	One of RoboCasa's ~100 atomic task names, e.g. "PnPCounterToCab". Only valid when mode="prebuilt".
kitchen_style	int | None	0..9, picks one of RoboCasa's 10 kitchen aesthetic packs. Only valid when mode="procedural".
layout_id	int | None	0..9, picks one of RoboCasa's 10 floor plans. Only valid when mode="procedural".
fixtures	list[str]	Subset of RoboCasa fixture names to spawn. Empty list keeps the layout's default fixtures.
spawn_objects	list[str]	Subset of RoboCasa object asset names to spawn on counters / inside cabinets.
task_verb	Literal['pnp', 'open', 'close', 'press', 'navigate'] | None	Coarse procedural-mode goal: pick-and-place, open, close, press, navigate. Required when mode="procedural".
robots	list[str]	RoboCasa robot composition names, e.g. ["PandaMobile"] (default) or ["GR1"]. Must match entries the upstream RoboCasa env factory accepts.
controller	str	RoboCasa controller name. "OSC_POSE" works for arm-style robots; the adapter validates against the controller config exposed by RoboCasa at import time.
horizon	int	Maximum step budget for the underlying RoboCasa env.

Example

opts = RoboCasaBackendOptions(mode="prebuilt", prebuilt_task="PnPCounterToCab") opts.task_verb is None True

obj_groups: str | None = None class-attribute instance-attribute

Pin the PnP target object to a specific RoboCasa object group / category (e.g. "baguette", "vegetable"). Forwarded to the prebuilt PnP task's obj_groups kwarg so the sampled object is deterministic instead of the default "all" random draw. Only valid for prebuilt PickPlace tasks; leave None for door / drawer / navigation tasks.

RobotCapabilities

Bases: BaseModel

Capability flags used for skill compatibility checking.

Attributes:

Name	Type	Description
locomotion	list[LocomotionKind]	Locomotion types available.
can_lift_kg	float	Maximum payload in kg.
has_dexterous_hands	bool	Whether dexterous hands are present.
has_tactile	bool	Whether tactile sensing is available.
has_force_control	bool	Whether force/impedance control is supported.
has_vision	bool	Whether camera(s) are present.
has_lidar	bool	Whether LiDAR is present.
has_audio	bool	Whether audio I/O is present.
bimanual	bool	Whether the robot has two arms.
onboard_compute_tops	float	Peak onboard compute in TOPS.
onboard_memory_gb	float	Onboard RAM in GB.
gpu_vram_gb	float	Largest single-GPU VRAM in GB (0 when no discrete GPU).
cuda_compute_capability	tuple[int, int] | None	Highest CUDA compute capability (major, minor), e.g. (8, 9) for Ada Lovelace, (10, 0) for Blackwell. None on non-CUDA hosts.
cuda_toolkit_version	str | None	nvcc version string when present.
tensorrt_version	str | None	TensorRT runtime version string when importable.
gpu_supported_runtimes	list[RSkillRuntime]	Inference runtimes the host can execute, used by rSkill.check_capabilities to match RSkillManifest.runtime.
gpu_supported_dtypes	list[QuantizationDtype]	Quantization dtypes the host accelerator supports (derived from CUDA compute capability or platform).
supported_control_modes	list[ControlMode]	List of supported ControlMode values.
supported_vla_embodiments	list[str]	VLA embodiment IDs this robot can run.
embodiment_tags	list[str]	Short tags mapping to VLA heads / dataset splits.
nvmm_available	bool	Whether libnvbufsurface.so is present on the host (Tegra L4T multimedia stack). True enables the NVMM zero-copy sensor-ingest path on Jetson; always False on x86 and on stripped L4T images. Populated by :func:openral_detect.probes.gpu._probe_nvmm_available. Added in v0.5 per ADR-0013.

RobotDescription

Bases: BaseModel

Top-level robot manifest — one per robot, published to HuggingFace Hub.

Attributes:

Name	Type	Description
name	str	Robot name, e.g. "so100_follower".
embodiment_kind	EmbodimentKind	Top-level kinematic class.
assets	AssetRefs	Unified URDF / MJCF / SRDF reference block (ADR-0058). Refs share the :func:openral_core.assets.resolve_asset grammar; the URDF's robot_state_publisher wiring lives on :attr:AssetRefs.urdf (ADR-0027). Empty by default.
base_frame	str	Base link tf2 frame name.
odom_frame	str	Odometry tf2 frame name.
map_frame	str	Map tf2 frame name.
joints	list[JointSpec]	List of joint specifications.
end_effectors	list[EndEffectorSpec]	List of end-effector specifications.
sensors	list[SensorSpec]	List of individual sensor specs.
sensor_bundles	list[SensorBundle]	List of multi-modal sensor bundles.
capabilities	RobotCapabilities	Capability flags for skill matching.
safety	SafetyEnvelope	Safety envelope constraints.
ros2_namespace	str	ROS 2 namespace prefix.
middleware	Literal['fastdds', 'cyclonedds', 'zenoh']	ROS 2 middleware selection.
onboard_compute	dict[str, object]	Onboard compute descriptors.
sdk_kind	Literal['open', 'closed_with_api', 'closed']	Whether the SDK is open or closed.
hal	HalEntrypoints	Simulation + real-hardware HAL import strings. deploy sim constructs hal.sim (or derives MujocoArmHAL from :attr:sim); deploy run constructs hal.real. ADR-0031.
observation_spec	ObservationSpec | None	VLA observation configuration.
action_spec	ActionSpec | None	VLA action configuration.
sim	SimDescription | None	Optional MuJoCo wiring consumed by :class:openral_hal.MujocoArmHAL. When set, the HAL can be constructed entirely from the manifest via :meth:MujocoArmHAL.from_description; no per-robot Python subclass is required.
scene_defaults	SceneDefaults | None	Optional per-robot scene rendering defaults. Today carries :class:TopCameraDefaults consumed by the openarm_tabletop_pnp MJCF composer (see :mod:openral_sim.backends.openarm_robosuite._assets). Added in openral-core 0.6.0 to replace the dataset-specific module-level constants that previously lived in the sim backend.
base_joints	list[str] | None	Optional ordered list of :attr:JointSpec.name references identifying the planar mobile base joints. When the robot has a holonomic / differential / omnidirectional base, the three entries are conventionally [forward_axis, side_axis, yaw_axis]. Consumed by the generic ray-cast helpers in :mod:openral_sim.backends.robocasa and the ROS HAL lifecycle nodes to resolve MJCF joint names without hardcoding robot-specific conventions. None for fixed-base manipulators. ADR-0025.
collision_geometry	list[LinkCollisionGeometry]	Per-link convex collision primitives (capsules / spheres) the safety kernel uses for self- and world-collision checking. Empty by default. ADR-0030.
allowed_collision_pairs	list[tuple[str, str]]	Link-name pairs excluded from self-collision (adjacent links touch by design); the allowed-collision matrix. On real robots this is sourced from the SRDF disable_collisions block (named by :attr:AssetRefs.srdf). ADR-0030.

Example

desc = RobotDescription( ... name="smoke_robot", ... embodiment_kind=EmbodimentKind.MANIPULATOR, ... joints=[ ... JointSpec( ... name="j1", ... joint_type=JointType.REVOLUTE, ... parent_link="base_link", ... child_link="link_1", ... ) ... ], ... capabilities=RobotCapabilities( ... supported_control_modes=[ControlMode.JOINT_POSITION], ... embodiment_tags=["smoke"], ... ), ... safety=SafetyEnvelope(), ... ) assert desc.name == "smoke_robot"

lidar_sensor: SensorSpec | None property

The first declared 2-D LiDAR / scan sensor, or None.

ADR-0025 — the panda_mobile HAL synthesises a sensor_msgs/ LaserScan from MuJoCo ray-casts; its beam count (:attr:SensorSpec.n_channels), range (:attr:SensorSpec.range_min_m / :attr:SensorSpec.range_max_m) and rate (:attr:SensorSpec.rate_hz) live on this descriptor so openral deploy sim and the HAL lifecycle node read one source of truth from robot.yaml instead of each hardcoding scan params.

from_yaml(path: str) -> RobotDescription classmethod

Load and validate a RobotDescription YAML manifest from disk.

Parameters:

Name	Type	Description	Default
path	str	Filesystem path to the robot.yaml file.	required

Returns:

Type	Description
RobotDescription	A validated :class:RobotDescription.

Raises:

Type	Description
FileNotFoundError	If path does not exist.
ValidationError	If the YAML fails schema validation.

Example

RobotDescription.from_yaml("robots/franka_panda/robot.yaml")

Source code in python/core/src/openral_core/schemas.py

@classmethod
def from_yaml(cls, path: str) -> RobotDescription:
    """Load and validate a ``RobotDescription`` YAML manifest from disk.

    Args:
        path: Filesystem path to the ``robot.yaml`` file.

    Returns:
        A validated :class:`RobotDescription`.

    Raises:
        FileNotFoundError: If ``path`` does not exist.
        pydantic.ValidationError: If the YAML fails schema validation.

    Example:
        >>> # RobotDescription.from_yaml("robots/franka_panda/robot.yaml")
    """
    import yaml  # noqa: PLC0415  # reason: deferred to avoid import-time cost

    with open(path, encoding="utf-8") as fh:
        data = yaml.safe_load(fh)
    return cls.model_validate(data)

nav2_param_overrides() -> dict[str, str]

Nav2 param substitutions derived from this robot's base props.

ADR-0025 — lets the Nav2 bringup stay generic: instead of hand-vendoring a per-robot nav2_<robot>.yaml, the launch rewrites a shared base param file with these key→value substitutions. Maps :attr:footprint_radius → robot_radius (collision envelope) and the costmap inflation_radius (footprint_radius + :data:NAV2_INFLATION_CLEARANCE_M, so it stays ≥ the inscribed/circumscribed radius Nav2 derives from the footprint — otherwise Nav2 errors and falls back to slow full-footprint collision checks), and :attr:base_kinematics → the MPPI motion_model. Returns an empty dict for fixed-base arms (no mobile base → no Nav2). Velocity bounds remain Nav2 tuning in the base param file, not robot identity.

Source code in python/core/src/openral_core/schemas.py

def nav2_param_overrides(self) -> dict[str, str]:
    """Nav2 param substitutions derived from this robot's base props.

    ADR-0025 — lets the Nav2 bringup stay generic: instead of
    hand-vendoring a per-robot ``nav2_<robot>.yaml``, the launch
    rewrites a shared base param file with these key→value
    substitutions. Maps :attr:`footprint_radius` → ``robot_radius``
    (collision envelope) **and** the costmap ``inflation_radius``
    (``footprint_radius`` + :data:`NAV2_INFLATION_CLEARANCE_M`, so it
    stays ≥ the inscribed/circumscribed radius Nav2 derives from the
    footprint — otherwise Nav2 errors and falls back to slow
    full-footprint collision checks), and :attr:`base_kinematics` →
    the MPPI ``motion_model``. Returns an empty dict for fixed-base
    arms (no mobile base → no Nav2). Velocity bounds remain Nav2
    tuning in the base param file, not robot identity.
    """
    overrides: dict[str, str] = {}
    if self.footprint_radius is not None:
        overrides["robot_radius"] = str(self.footprint_radius)
        overrides["inflation_radius"] = (
            f"{self.footprint_radius + NAV2_INFLATION_CLEARANCE_M:.3f}"
        )
    if self.base_kinematics is not None:
        overrides["motion_model"] = {
            "omni": "Omni",
            "holonomic": "Omni",
            "differential": "DiffDrive",
            "ackermann": "Ackermann",
        }[self.base_kinematics]
    return overrides

validate_for_e2e_pipeline() -> None

Assert this manifest carries every field the e2e ROS graph needs.

The C++ safety kernel (cpp/openral_safety_kernel, ADR-0020) reads per-joint position_limits / velocity_limit / effort_limit + the global safety: block. Per-joint limit fields are optional on :class:JointSpec for sim-only robots, but they become mandatory the moment you ask openral deploy sim to bring up the kernel against this robot.

This method makes the e2e contract explicit and fails loud listing every missing field. Pair it with openral_safety.envelope_loader.compute_intersection(robot, skill=None) for the actual envelope synthesis — this method only validates.

Raises:

Type	Description
ROSConfigError	If any actuated joint is missing position_limits, velocity_limit, or effort_limit. The error lists every missing field at once so the operator does not have to fix one, re-run, fix the next.

Source code in python/core/src/openral_core/schemas.py

def validate_for_e2e_pipeline(self) -> None:
    """Assert this manifest carries every field the e2e ROS graph needs.

    The C++ safety kernel (``cpp/openral_safety_kernel``, ADR-0020)
    reads per-joint ``position_limits`` / ``velocity_limit`` /
    ``effort_limit`` + the global ``safety:`` block. Per-joint
    limit fields are *optional* on :class:`JointSpec` for sim-only
    robots, but they become mandatory the moment you ask
    ``openral deploy sim`` to bring up the kernel against this robot.

    This method makes the e2e contract explicit and fails loud
    listing every missing field. Pair it with
    ``openral_safety.envelope_loader.compute_intersection(robot,
    skill=None)`` for the actual envelope synthesis — this method
    only validates.

    Raises:
        ROSConfigError: If any actuated joint is missing
            ``position_limits``, ``velocity_limit``, or
            ``effort_limit``. The error lists every missing field
            at once so the operator does not have to fix one,
            re-run, fix the next.
    """
    missing: list[str] = []
    for joint in self.joints:
        if joint.joint_type not in (
            JointType.REVOLUTE,
            JointType.PRISMATIC,
            JointType.CONTINUOUS,
        ):
            continue
        if joint.position_limits is None:
            missing.append(f"joints[{joint.name}].position_limits")
        if joint.velocity_limit is None:
            missing.append(f"joints[{joint.name}].velocity_limit")
        if joint.effort_limit is None:
            missing.append(f"joints[{joint.name}].effort_limit")
    if missing:
        raise ROSConfigError(
            f"RobotDescription({self.name!r}) is missing fields required by "
            f"the e2e safety kernel: {missing}. Set them on the matching "
            "JointSpec(s) in robots/<robot_id>/robot.yaml."
        )

RobotEnvironment

Bases: BaseModel

A full hardware deployment configuration — the openral deploy artefact.

Sibling of :class:SimEnvironment for real hardware. Loaded from YAML by openral_runner (planned, ADR-0010) and consumed by the hardware InferenceRunner. The runner instantiates the HAL adapter, opens every :class:SensorReader, wires the :class:~openral_world_state.aggregator.WorldStateAggregator, constructs the :class:~openral_rskill.Skill from vla.weights_uri (a bare rSkill reference — name, path, or HF repo ID) and ticks at rate_hz.

Cross-field invariants enforced in :meth:model_post_init:

• Every :attr:SensorReaderConfig.sensor_id MUST be unique within :attr:sensors.
• :attr:vla.weights_uri MUST be a bare rSkill reference (no URI scheme) — the rSkill manifest is the contract between robot/sensors/preprocessing and policy weights (CLAUDE.md §6.4).

Attributes:

Name	Type	Description
robot_id	str	ID into the ROBOTS registry — matches a :attr:RobotDescription.name. Examples: "so100_follower", "franka_panda", "ur5e".
hal	HalConfig	HAL adapter + transport configuration.
sensors	list[SensorReaderConfig]	Per-sensor reader backend choices. The runner opens one :class:SensorReader per entry and feeds the :class:WorldStateAggregator.
task	TaskSpec	What the robot should achieve. Reused from :class:SimEnvironment so :attr:TaskSpec.instruction becomes the language prompt handed to the VLA.
vla	VLASpec	Policy that drives the robot. vla.weights_uri MUST be a bare rSkill reference (name, path, or HF repo ID).
safety	SafetyEnvelope | None	Optional :class:SafetyEnvelope override; falls back to the robot's :attr:RobotDescription.safety when None.
rate_hz	float	Foreground tick rate. Default 30 Hz (matches the :class:WorldStateAggregator publish rate).
thumbnail_hz	float	Per-camera rate at which the runner encodes a JPEG thumbnail onto the sensors.read_latest span for the dashboard. Default 25 Hz; decoupled from rate_hz. 0 disables thumbnails. End-to-end dashboard refresh is min(thumbnail_hz, span export flush rate) — see OPENRAL_OTEL_SPAN_SCHEDULE_DELAY_MS.
deadline_overrun_policy	DeadlineOverrunPolicy	Behavior when tick wall-time exceeds 1 / rate_hz.
max_ticks	int | None	Optional cap — the runner exits cleanly after this many ticks. None means "run until task.max_steps or external stop".
save_dir	str | None	Optional directory for traces / video / JSON summaries.
metadata	dict[str, object]	Free-form notes (operator, run id, …).

Example

env = RobotEnvironment( ... robot_id="so100_follower", ... hal=HalConfig(adapter="so100_follower"), ... sensors=[SensorReaderConfig(sensor_id="wrist_rgb")], ... task=TaskSpec( ... id="pick_cube/red", ... scene_id="pick_cube/red", ... instruction="pick up the red cube", ... ), ... vla=VLASpec( ... id="smolvla", ... weights_uri="rskills/smolvla-so100", ... ), ... ) env.rate_hz 30.0

from_yaml(path: str) -> RobotEnvironment classmethod

Load and validate a RobotEnvironment YAML from disk.

Parameters:

Name	Type	Description	Default
path	str	Filesystem path to the YAML config file.	required

Returns:

Type	Description
RobotEnvironment	A validated :class:RobotEnvironment.

Raises:

Type	Description
FileNotFoundError	If path does not exist.
ValidationError	If the YAML fails schema validation.

Source code in python/core/src/openral_core/schemas.py

@classmethod
def from_yaml(cls, path: str) -> RobotEnvironment:
    """Load and validate a ``RobotEnvironment`` YAML from disk.

    Args:
        path: Filesystem path to the YAML config file.

    Returns:
        A validated :class:`RobotEnvironment`.

    Raises:
        FileNotFoundError: If ``path`` does not exist.
        pydantic.ValidationError: If the YAML fails schema validation.
    """
    import yaml  # noqa: PLC0415  # reason: deferred to avoid import-time cost

    with open(path, encoding="utf-8") as fh:
        data = yaml.safe_load(fh)
    return cls.model_validate(data)

model_post_init(_context: object) -> None

Cross-field validation: unique sensors, bare rSkill weights_uri.

Source code in python/core/src/openral_core/schemas.py

def model_post_init(self, _context: object) -> None:
    """Cross-field validation: unique sensors, bare rSkill weights_uri."""
    seen: set[str] = set()
    for sensor in self.sensors:
        if sensor.sensor_id in seen:
            raise ValueError(
                f"RobotEnvironment({self.robot_id!r}) has duplicate "
                f"sensor_id={sensor.sensor_id!r} in sensors; each "
                f"sensor must be configured at most once."
            )
        seen.add(sensor.sensor_id)
    for bad in ("hf://", "local://", "file://", "http://", "https://"):
        if self.vla.weights_uri.startswith(bad):
            raise ValueError(
                f"RobotEnvironment({self.robot_id!r}).vla.weights_uri must "
                f"be a bare rSkill reference (name, path, or HF repo ID), "
                f"got {self.vla.weights_uri!r}; hardware deployments resolve "
                f"weights through an rSkill manifest for reproducibility "
                f"(CLAUDE.md §6.4)."
            )

RosIntegration

Bases: BaseModel

Wiring for an rSkill that wraps an existing ROS 2 action or service.

Populated when :attr:RSkillManifest.kind is "ros_action" or "ros_service". The :class:~openral_rskill.ros_action_rskill.ROSActionRskill adapter reads this block at configure time to build the right action / service client on the host LifecycleNode.

Attributes:

Name	Type	Description
package	str	IDL package name (e.g. "moveit_msgs", "nav2_msgs"). Imported lazily so manifests for un-installed ROS packages still parse — the loader raises :class:~openral_core.exceptions.ROSConfigError only when the skill is actually resolved.
interface_type	str	Action / service type name inside package (e.g. "MoveGroup", "NavigateToPose").
interface_name	str	Fully-qualified ROS path of the running server (e.g. "/move_action", "/navigate_to_pose").
result_trajectory_field	str | None	Dotted accessor pointing at a trajectory_msgs/JointTrajectory inside the action result (e.g. "planned_trajectory.joint_trajectory" for MoveIt's MoveGroup). When set, the adapter replays one waypoint per :meth:~openral_rskill.base.rSkillBase.step call onto /openral/candidate_action (so the safety supervisor + HAL see every position). When None the action is treated as result-only: the adapter awaits the action result, raises :class:~openral_core.exceptions.ROSRskillGoalSatisfied on success, and never emits an :class:~openral_core.schemas.Action chunk. This second mode covers wrapped ROS packages that drive actuators on their own (Nav2's behaviour tree publishes cmd_vel directly).
default_goal_json	str	JSON dict literal used to construct the goal message. v1 hard-codes the target here — the structured-prompt path that lowers LLM-emitted JSON into ExecuteSkill.Goal.prompt_metadata_json is a follow-up. REQUIRED so that a wrapped skill is always invocable end-to-end without per-call schema work.
ros_dependencies	list[str]	Apt / colcon packages the operator must have installed for the wrapped server to be reachable (e.g. "ros-${ROS_DISTRO}-moveit"). Surfaced by ral skill check and quoted in :class:~openral_core.exceptions.ROSConfigError messages when the action client fails to connect.

Example

ri = RosIntegration( ... package="moveit_msgs", ... interface_type="MoveGroup", ... interface_name="/move_action", ... result_trajectory_field="planned_trajectory.joint_trajectory", ... default_goal_json='{"target_joint_positions": [0, 0, 0, 0, 0, 0, 0]}', ... ros_dependencies=["ros-${ROS_DISTRO}-moveit"], ... ) ri.package 'moveit_msgs'

goal_builder: Literal['joint', 'pose', 'look_at'] | None = None class-attribute instance-attribute

ADR-0044 / ADR-0054 — optional goal-lowering adapter over the shared ROSActionRskill MoveGroup engine. None (the default) sends default_goal_json + LLM overrides verbatim (the raw-IDL escape hatch). The named builders consume a typed block from the merged goal and lower it into MoveGroup constraints:

• "joint" — a joint block (positions, joint_names) → joint_constraints (:class:~openral_rskill.joint_goal_rskill.JointGoalRskill).
• "pose" — a pose block (position, orientation as a quaternion array with quaternion_order, link_name) → MoveGroup position + orientation constraints for a Cartesian end-effector goal (:class:~openral_rskill.pose_goal_rskill.PoseGoalRskill).
• "look_at" — a look_at block (target_xyz, camera, …) → the same pose-constraint lowering, with the gaze pose computed from the camera's live TF pose (:class:~openral_rskill.look_at_rskill.LookAtRskill, a specialisation of the pose builder).

RunResult

Bases: BaseModel

Aggregated summary returned by :meth:InferenceRunner.run.

Attributes:

Name	Type	Description
n_ticks	int	Total ticks executed.
success	bool | None	Task success when the runner has a success signal (sim, scripted goal). None for open-ended hardware runs.
budget_violations	int	Count of ticks whose tick_ms exceeded the rSkill manifest's :attr:RSkillLatencyBudget.per_chunk_ms.
avg_inference_ms	float	Mean of :attr:TickResult.inference_ms.
p99_inference_ms	float	99th-percentile of :attr:TickResult.inference_ms.
avg_tick_ms	float	Mean of :attr:TickResult.tick_ms.
p99_tick_ms	float	99th-percentile of :attr:TickResult.tick_ms.
trace_id	str | None	OTel trace id (hex) of the run's root span. None when tracing is not configured.
save_dir	str | None	Directory where artefacts were written, when configured.
metadata	dict[str, object]	Free-form metadata (runner id, host, git SHA, …).

SafetyEnvelope

Bases: BaseModel

Safety constraints enforced by the C++ safety kernel.

Attributes:

Name	Type	Description
workspace_box_min_xyz	tuple[float, float, float] | None	Lower corner of allowed workspace (m).
workspace_box_max_xyz	tuple[float, float, float] | None	Upper corner of allowed workspace (m).
no_go_zones	list[dict[str, object]]	List of polygon definitions (dicts with 'vertices').
max_ee_speed_m_s	float	Maximum end-effector linear speed in m/s. Also used by ADR-0028b's per-mode supervisor as the CARTESIAN_TWIST linear bound.
max_ee_accel_m_s2	float	Maximum end-effector acceleration in m/s².
max_joint_speed_factor	float	Fraction of joint velocity_limit allowed.
max_force_n	float	Maximum contact force in Newtons.
max_torque_nm	float	Maximum joint torque in Nm.
deadman_required	bool	Whether a deadman switch is required.
e_stop_topic	str	ROS 2 topic for E-stop commands.
e_stop_qos	Literal['reliable']	QoS profile for E-stop topic.
contact_force_threshold_n	float	Force threshold for contact detection.
cycle_time_violation_threshold_ms	float	Control cycle time violation threshold.
human_in_loop_required	list[str]	rSkill names requiring human supervision.
max_cartesian_step_m	float | None	ADR-0028b — per-step magnitude bound on CARTESIAN_DELTA's xyz triplet (Euclidean). None means "no per-mode check declared, skip"; today's behaviour preserved. Robots that host OSC-trained checkpoints (panda_mobile, future Franka + π0.7) declare this so the supervisor rejects out-of-distribution arm deltas before they reach the controller.
max_cartesian_step_rad	float | None	ADR-0028b — per-step magnitude bound on CARTESIAN_DELTA's axis-angle triplet (Euclidean). None skips the check.
max_ee_angular_speed_rad_s	float | None	ADR-0028b — angular component bound for CARTESIAN_TWIST (the linear bound reuses :attr:max_ee_speed_m_s). None skips the check.
max_base_linear_speed_m_s	float | None	ADR-0028b — BODY_TWIST linear bound (Euclidean over vx,vy,vz). None skips the check; mobile manipulators / wheeled bases declare it.
max_base_angular_speed_rad_s	float | None	ADR-0028b — BODY_TWIST angular bound (Euclidean over wx,wy,wz; for planar bases only wz is non-zero). None skips the check.

SceneChangeMetadata

Bases: _PerceptionEventBase

Perception event for /openral/perception/scene_change.

Emitted when a histogram / structural-similarity detector measures a frame-to-frame distance above a configured threshold — typically used to wake the reasoner when the scene has changed enough to warrant a re-plan.

Attributes:

Name	Type	Description
kind	Literal['scene_change']	Discriminator (always "scene_change").
distance	float	Frame-to-frame distance in the detector's native metric (e.g. cv2.HISTCMP_CHISQR_ALT distance, 1 - ssim).
threshold	float	Distance threshold that fired the event.
metric	str	Identifier of the distance metric (e.g. "chisqr_alt", "1-ssim", "hellinger").

SceneComposition

Bases: BaseModel

Declarative MJCF scene composition for a manifest-driven HAL (ADR-0029).

Lets a robot whose sim HAL needs a composed MJCF (a bare arm spliced onto a tabletop + props) declare the composer in its manifest instead of a bespoke _create_hal lifecycle subclass (issue #191 Phase 3b). The manifest-driven node calls :attr:composer before constructing the HAL and threads the resulting MJCF path in as the HAL's mjcf_path.

The composer is a "module.path:function" import string. The function is called with **params and MUST return (xml: str, meshdir: Path) — the composed MJCF XML and the mesh directory it references (the node writes the XML next to meshdir so relative mesh paths resolve). Today's only composer is openral_sim.backends.openarm_robosuite._assets:compose_openarm_tabletop_mjcf.

Attributes:

Name	Type	Description
composer	str	"module.path:function" returning (xml, meshdir).
params	dict[str, object]	Keyword arguments for the composer (e.g. robot_lift_z, white_background).

Example

sc = SceneComposition( ... composer="pkg.scenes:compose_tabletop", ... params={"robot_lift_z": 0.36, "white_background": True}, ... ) sc.params["robot_lift_z"] 0.36

SceneDefaults

Bases: BaseModel

Per-robot scene rendering defaults.

These are values that scene composers may consult when the :class:SimScene YAML does not override them. Today the only field is :attr:top_camera, which the openarm_tabletop_pnp backend consumes; future scenes can extend this submodel as new defaults are pulled out of backend hardcodes.

Attributes:

Name	Type	Description
top_camera	TopCameraDefaults | None	Default placement for the scene-overview camera.

Example

sd = SceneDefaults( ... top_camera=TopCameraDefaults( ... pos=(0.2, 0.0, 0.95), ... target=(0.65, 0.0, 0.05), ... fovy=65.0, ... ), ... ) sd.top_camera.pos (0.2, 0.0, 0.95)

SceneGraph

Bases: BaseModel

Persistent hierarchical scene-graph spatial memory (ADR-0038).

The durable, queryable world model the S2 Reasoner consults to recall where objects/places/agents are and how to navigate to them. Distinct from the ephemeral ADR-0030 collision grid and advisory only — never a safety input (CLAUDE.md §1.1).

Invariants (enforced): node ids are unique; every edge references existing nodes.

Attributes:

Name	Type	Description
schema_version	Literal['0.1']	On-disk schema version ("0.1"; no backward-incompatible change yet). Now the repo is published it is versioned for real (CLAUDE.md §1.6): an incompatible change bumps it and ships a migrator.
nodes	list[SpatialNode]	All scene-graph nodes.
edges	list[SpatialEdge]	All directed relations between nodes.

SceneSpec

Bases: BaseModel

A physics scene declaration — the WORLD the robot acts in.

A scene is a deterministic, reproducible MuJoCo / MJX / etc. world: assets, lighting, cameras, fixed objects. Tasks (:class:TaskSpec) are evaluated INSIDE a scene; multiple tasks can share one scene (e.g. libero_spatial has 10 tasks per suite).

Attributes:

Name	Type	Description
id	str	Stable scene identifier used by the eval registry, e.g. "libero_spatial", "metaworld_mt50", "so100_tabletop".
backend	PhysicsBackend	Physics backend used to instantiate the scene.
assets_uri	str | None	Optional URI (file:// or hf://) pointing at scene assets (XML / MJCF / asset bundle). When None, the registered adapter resolves assets internally (LIBERO / MetaWorld pull theirs from their own packages).
observation_height	int	Default render height in pixels for camera obs.
observation_width	int	Default render width in pixels for camera obs.
cameras	list[str]	List of camera names the scene exposes. Adapters use this both to render and to map sensors to VLA feature keys.
backend_options	dict[str, object]	Backend-specific overrides (e.g. {"render_modes": ["rgb_array"]}). Opaque to the eval layer; passed through to the adapter.

Example

s = SceneSpec(id="libero_spatial", backend=PhysicsBackend.MUJOCO) s.observation_width 256

SelfVerifyEvidence

Bases: _FailureEvidenceBase

Evidence for KIND_SELFVERIFY — a self-check failed.

Attributes:

Name	Type	Description
kind	Literal['selfverify']	Discriminator (always "selfverify").
check	str	Short check identifier (e.g. "action_chunk.shape").
expected	str	Expected value as a string.
observed	str	Observed value as a string.

SensorBundle

Bases: BaseModel

Multi-modal sensor group, e.g. RealSense D455 = (RGB, depth, IMU).

Attributes:

Name	Type	Description
bundle_name	str	Unique name for this bundle.
sensors	list[SensorSpec]	List of constituent sensor specs.
sync	Literal['hardware', 'approximate', 'none']	Synchronization strategy.
sync_tolerance_ms	float	Tolerance for approximate synchronization.

SensorFrame

Bases: BaseModel

A single sensor frame — metadata plus an optional inline payload.

A :class:SensorFrame is the carrier passed from a SensorReader into :class:WorldState.image_frames. Exactly one of data, topic, or handle is populated:

• data carries the pixel bytes for in-process delivery and for trace capture. JSON-serialized as base64 by Pydantic, so payloads round-trip through model_dump_json / model_validate_json.
• topic points at a ROS 2 topic — the bytes live on the ROS 2 bus and the consumer subscribes for them. This is the path the Ros2ImageSensorReader uses and what :attr:WorldState.images already carries.
• handle is an opaque integer (e.g. CUDA NVMM pointer, DMA-BUF file descriptor) — in-process only, never serialized. Set by the GStreamerSensorReader when frames stay on the GPU.

Attributes:

Name	Type	Description
sensor_id	str	Sensor name; matches :attr:SensorSpec.name.
stamp_monotonic_ns	int	Capture-time monotonic timestamp in nanoseconds.
stamp_wall_ns	int	Capture-time wall-clock timestamp in nanoseconds.
encoding	FrameEncoding	How to interpret the bytes.
width	int	Image width in pixels.
height	int	Image height in pixels.
channels	int	Number of channels (3 for RGB/BGR, 1 for MONO/DEPTH).
data	bytes | None	Optional inline pixel payload (or compressed bytes for JPEG / PNG). Mutually exclusive with topic / handle.
topic	str | None	Optional ROS 2 topic reference. Mutually exclusive with data / handle.
handle	int | None	Optional opaque in-process handle (NVMM / DMA-BUF). Mutually exclusive with data / topic.
metadata	dict[str, object]	Free-form per-frame metadata (gain, exposure, …).

Example

SensorFrame( ... sensor_id="wrist_rgb", ... stamp_monotonic_ns=1, ... stamp_wall_ns=2, ... encoding=FrameEncoding.RGB8, ... width=640, ... height=480, ... topic="/cameras/wrist_rgb/image_raw", ... ).channels 3

model_post_init(_context: object) -> None

Exactly one of (data, topic, handle) must be set.

Source code in python/core/src/openral_core/schemas.py

def model_post_init(self, _context: object) -> None:
    """Exactly one of (data, topic, handle) must be set."""
    populated = [bool(self.data), self.topic is not None, self.handle is not None]
    n = sum(populated)
    if n != 1:
        raise ValueError(
            f"SensorFrame({self.sensor_id!r}): exactly one of "
            f"(data, topic, handle) must be set; got {n}."
        )

SensorModality

Bases: str, Enum

Physical sensing modality.

SensorReaderBackend

Bases: str, Enum

Which :class:SensorReader implementation to instantiate.

ADR-0010 §SensorReader — four backends are reserved. opencv_thread is the default and mirrors lerobot's per-camera background-thread pattern. ros2_image subscribes to a ROS 2 image topic published by a vendor driver. gstreamer runs a GStreamer pipeline whose appsink delivers frames (NVMM / DMA-BUF on Jetson; CPU bytes on x86).

holoscan is reserved-but-unimplemented: ADR-0010 Amendment 2026-05-12 evaluated NVIDIA Holoscan SDK as a parallel ingest backbone and deferred adoption (lean GStreamer with custom NvBufSurface glue won the comparison). The enum value exists so a future PR can add the backend additively without bumping the schema again; configs that select it today raise ROSConfigError at factory time.

SensorReaderConfig

Bases: BaseModel

Per-sensor backend configuration for the inference runner.

Picks which :class:SensorReader backend services a sensor and how the pipeline is parameterised. The optional publish_to_ros tee lets a GStreamer pipeline publish a downsampled stream to a ROS 2 topic for observability (rosbag2 / rqt_image_view) without putting the hot path through ROS.

Attributes:

Name	Type	Description
sensor_id	str	Sensor name; MUST match a :attr:SensorSpec.name in the target :class:RobotDescription.
backend	SensorReaderBackend	Which :class:SensorReaderBackend to instantiate.
backend_params	dict[str, object]	Backend-specific keyword arguments forwarded to the reader constructor. For gstreamer typically a single "pipeline" string. For opencv_thread typically {"device": "/dev/video0", "fps": 30}.
max_age_ms	int	How stale a frame may be before SensorReader.read_latest raises. Defaults to ~3 frames at 30 Hz.
publish_to_ros	bool	If True, the reader tees a downsampled stream to publish_topic at publish_rate_hz.
publish_topic	str | None	ROS 2 topic to publish to when publish_to_ros is True. Required iff publish_to_ros.
publish_rate_hz	float | None	Downsample rate for the ROS tee.

Example

SensorReaderConfig( ... sensor_id="wrist_rgb", ... backend=SensorReaderBackend.GSTREAMER, ... backend_params={ ... "pipeline": "v4l2src device=/dev/video0 ! " ... "nvv4l2decoder ! nvvideoconvert ! appsink" ... }, ... publish_to_ros=True, ... publish_topic="/cameras/wrist_rgb/image_raw", ... publish_rate_hz=5.0, ... ).backend.value 'gstreamer'

model_post_init(_context: object) -> None

Cross-field validation for the ROS tee.

Source code in python/core/src/openral_core/schemas.py

def model_post_init(self, _context: object) -> None:
    """Cross-field validation for the ROS tee."""
    if self.publish_to_ros and self.publish_topic is None:
        raise ValueError(
            f"SensorReaderConfig({self.sensor_id!r}): publish_to_ros is "
            f"True but publish_topic is unset; a ROS tee needs a topic."
        )
    if self.publish_topic is not None and not self.publish_to_ros:
        raise ValueError(
            f"SensorReaderConfig({self.sensor_id!r}): publish_topic is "
            f"set but publish_to_ros is False; enable publish_to_ros "
            f"explicitly or drop the topic."
        )

SensorRequirement

Bases: BaseModel

One sensor an rSkill needs the robot to provide.

Used by :class:RSkillManifest.sensors_required to declare the inputs the policy expects. The compatibility check resolves each entry against a :class:RobotDescription's sensors list and rejects the pairing if no robot sensor satisfies the requirement.

Resolution rules (in order):

1. If vla_feature_key is set, the robot MUST expose exactly one sensor with that vla_feature_key. The check then verifies modality matches and (if specified) the sensor's intrinsics meet min_width / min_height.
2. Otherwise, the robot must expose at least count sensors of the requested modality (each meeting any specified resolution minimum).

Attributes:

Name	Type	Description
modality	SensorModality	Required physical modality (rgb, depth, imu…).
vla_feature_key	str | None	Optional exact key the VLA expects, e.g. "observation.images.camera1". When set, the robot's matching SensorSpec.vla_feature_key must be identical.
min_width	int | None	Minimum image width in pixels (RGB/depth/IR/stereo only).
min_height	int | None	Minimum image height in pixels.
count	int	Number of robot sensors of this modality required when vla_feature_key is unset. Ignored when vla_feature_key is provided (a key uniquely identifies one sensor).

Example

SensorRequirement( ... modality=SensorModality.RGB, ... vla_feature_key="observation.images.camera1", ... min_width=224, ... min_height=224, ... ) # doctest: +ELLIPSIS SensorRequirement(modality=, ...)

SensorSpec

Bases: BaseModel

Generalizable sensor descriptor — covers all modalities.

Attributes:

Name	Type	Description
name	str	Human-readable sensor name, e.g. "head_rgb", "left_wrist_depth".
modality	SensorModality	Physical sensing modality.
frame_id	str	tf2 frame name.
parent_frame	str | None	tf2 parent frame (for static transform).
static_transform_xyz_rpy	tuple[float, float, float, float, float, float] | None	Static transform from parent to this sensor.
rate_hz	float	Expected publishing rate in Hz.
intrinsics	IntrinsicsPinhole | None	Pinhole camera intrinsics (if applicable).
encoding	str | None	Image encoding, e.g. "rgb8", "16UC1".
vla_feature_key	str | None	VLA observation dict key this sensor maps to, e.g. 'observation.images.camera1'. Used by skill loaders to auto-wire sensors to VLA input_features.
ros2_topic	str | None	ROS 2 topic name. None for non-ROS robots (USB, sim-only).
ros2_msg_type	str | None	ROS 2 message type, e.g. "sensor_msgs/Image". None for non-ROS robots (USB, sim-only).
qos_profile	Literal['sensor_data', 'reliable', 'transient_local', 'parameters']	QoS profile key.
vendor	str | None	Sensor vendor name.
model	str | None	Sensor model, e.g. "RealSense D455".
driver_pkg	str | None	ROS 2 driver package name.
metadata	dict[str, object]	Additional key-value metadata.

SimDescription

Bases: BaseModel

MuJoCo wiring for a single-arm robot, consumed by MujocoArmHAL.

The MJCF itself is named by :attr:RobotDescription.assets.mjcf (ADR-0058); this block carries only the joint↔qpos/qvel/actuator plumbing. All fields are optional with defaults derived from :attr:RobotDescription.joints. The default mapping is "1:1 in joint order, offset by 7 (qpos) / 6 (qvel) if floating_base is True" — which is correct for every robot in the open core today bar minor gripper bookkeeping.

Attributes:

Name	Type	Description
floating_base	bool	If True, the robot's MJCF has a 6-DoF free joint before the actuated joints (humanoids). In that case the default joint_qpos_addr[joints[i].name] = 7 + i and joint_qvel_addr[joints[i].name] = 6 + i.
joint_qpos_addr	dict[str, int] | None	Optional override of the joint→qpos-index map. When omitted, the default 1:1 mapping is used.
joint_qvel_addr	dict[str, int] | None	Optional override of the joint→qvel-index map. When omitted, defaults to joint_qpos_addr (or its default).
actuator_index	dict[str, int] | None	Optional override of the joint→actuator-index map. When omitted, defaults to the 1:1 mapping joints[i].name → i.
grippers	list[SimGripperDescription]	Optional list of :class:SimGripperDescription entries. Each must reference a joint by name that is also present in :attr:RobotDescription.joints. Single-arm robots have one entry (or none); bimanual robots have two (left + right).
settle_steps_default	int	Default number of mj_step calls executed per :meth:MujocoArmHAL.send_action. Defaults to 1.
keyframe_index	int | None	When set, :meth:MujocoArmHAL.connect calls mj_resetDataKeyframe(model, data, keyframe_index) before mj_forward. Required for MJCFs whose default MjData.qpos (zeros) sits outside the actuator ctrlrange — e.g. the gym-aloha parallel jaws, whose fingers have ctrlrange=[0.021, 0.057] and never recover from qpos = 0.
seed_ctrl_from_qpos	bool	When True, :meth:MujocoArmHAL.connect seeds data.ctrl[actuator] = data.qpos[joint_qpos_addr] for every controllable joint so position actuators hold the initial pose on the first mj_step. Required by the OpenArm v2 MJCF (its position actuators with per-class PD gains will drive qpos to ctrl == 0 otherwise).

Example

SimDescription(floating_base=True).floating_base True

SimEnvironment

Bases: BaseModel

A full sim configuration: the swappable (robot x scene x task x VLA) tuple.

The composed runtime form of a :class:SimScene plus an :class:RSkillManifest. openral_sim's CLI builds it; adapters consume it. Not loaded from YAML directly — from_yaml raises.

Attributes:

Name	Type	Description
robot_id	str	ID into the eval ROBOTS registry. Matches a robot's RobotDescription.name or an embodiment shortcut (e.g. "so100_follower", "franka_panda").
scene	SceneSpec	Scene the robot acts in.
task	TaskSpec	Task to evaluate. task.scene_id MUST equal scene.id.
vla	VLASpec	Policy that drives the robot.
base_pose	Pose6D | None	Optional per-episode mounting pose for the robot in the scene's world frame. None means "use the scene adapter's default" (e.g. an URDF's identity placement). Only honoured by free-axis scene adapters; setting it on a scene that registers with a fixed_robot= constraint (LIBERO, MetaWorld, RoboCasa, PushT, ALOHA) is rejected at CLI compose-time. The pose's frame_id is expected to be "world"; adapters anchor on the robot's :class:RobotDescription.base_frame.
seed	int	Global random seed (env reset, action sampling, torch RNG).
n_episodes	int	Number of episodes to run. 1 for a smoketest; ≥ 50 for an honest single-task success rate.
record_video	bool	Whether to ask the runner to record video frames.
save_dir	str | None	Optional directory to write artefacts (video, traces, json summary). None means "discard outputs".
metadata	dict[str, object]	Free-form notes (commit SHA, run owner, etc.).

Example

env = SimEnvironment( ... robot_id="franka_panda", ... scene=SceneSpec(id="libero_spatial"), ... task=TaskSpec( ... id="libero_spatial/task_0", ... scene_id="libero_spatial", ... instruction="pick up the cube", ... ), ... vla=VLASpec(id="smolvla", weights_uri="hf://lerobot/smolvla_libero"), ... ) env.task.scene_id == env.scene.id True

from_yaml(path: str) -> SimEnvironment classmethod

Removed: pass --rskill on the CLI; YAML carries only the scene.

After the feat(core,sim): SceneEnvironment + openral sim run --rskill, no legacy commit, the YAML shape for openral sim run is :class:SimScene (scene + task only; ADR-0041 renamed SceneEnvironment → SimScene) and the policy is supplied via the --rskill CLI flag. SimEnvironment is now the composed runtime form and is built by the CLI; loading it directly from YAML is no longer supported.

Source code in python/core/src/openral_core/schemas.py

@classmethod
def from_yaml(cls, path: str) -> SimEnvironment:  # pragma: no cover - removed
    """Removed: pass ``--rskill`` on the CLI; YAML carries only the scene.

    After the ``feat(core,sim): SceneEnvironment + openral sim run --rskill,
    no legacy`` commit, the YAML shape for ``openral sim run`` is
    :class:`SimScene` (scene + task only; ADR-0041 renamed
    ``SceneEnvironment`` → ``SimScene``) and the policy is
    supplied via the ``--rskill`` CLI flag. ``SimEnvironment`` is now
    the *composed runtime form* and is built by the CLI; loading it
    directly from YAML is no longer supported.
    """
    raise ROSConfigError(
        "SimEnvironment.from_yaml has been removed. "
        "Load the scene + task via SimScene.from_yaml(path) (or "
        "load_scene_strict(path, SimScene)) and supply the rSkill via "
        "--rskill rskills/<id> on the CLI; the CLI composes the "
        "SimEnvironment from those two artefacts."
    )

model_post_init(_context: object) -> None

Cross-field validation: task.scene_id must match scene.id.

Source code in python/core/src/openral_core/schemas.py

def model_post_init(self, _context: object) -> None:
    """Cross-field validation: task.scene_id must match scene.id."""
    if self.task.scene_id != self.scene.id:
        raise ValueError(
            f"SimEnvironment.task.scene_id ({self.task.scene_id!r}) does not "
            f"match scene.id ({self.scene.id!r}); a task can only run in its "
            f"declared scene."
        )

SimGripperDescription

Bases: BaseModel

Gripper wiring inside a MuJoCo MJCF.

Attributes:

Name	Type	Description
joint	str	Name of the gripper joint as it appears in :attr:RobotDescription.joints (the public, lerobot-style name — not the menagerie MJCF name).
ctrl_range	tuple[float, float]	(low, high) raw control range for the gripper actuator. In NORMALISED write mode, low ↔ closed (Action.gripper = 0), high ↔ open (Action.gripper = 1). In PASSTHROUGH write mode this is informational — MuJoCo clips on its own.
qpos_addrs	tuple[int, ...]	qpos indices used to compute the reported gripper position. When the gripper has multiple finger joints (Franka), list every finger's qpos index.
qpos_scale	float	Span used to normalise the summed/raw gripper qpos to [0, 1]. For Franka with two fingers each in [0, 0.04] m, qpos_scale = 0.08. Ignored by AFFINE_LOW_HIGH and PASSTHROUGH read modes but kept for symmetry with the base-class invariant.
read_mode	GripperReadMode	How to report the qpos — see :class:GripperReadMode.
write_mode	GripperWriteMode	How to map an Action's gripper value to ctrl — see :class:GripperWriteMode.
actuator_index	int | None	Explicit MJCF actuator index that receives the (mapped) gripper command. When omitted, defaults to the position of joint in :attr:RobotDescription.joints (i.e. the 1:1 mapping derived by SimDescription for arm joints).
mirror_actuator_index	int | None	Optional second actuator that receives the negation of the (mapped) gripper command. Models the Aloha parallel jaws where one motor drives +x and the other -x to keep the fingers symmetric. None for single-actuator grippers.

SimScene

Bases: DeployScene

Scene + task for openral sim run.

All task fields are overridable at the CLI level. max_steps and success_key are optional — omit them for open-ended experiments. Accepts a BenchmarkScene YAML transparently (the eval-specific fields n_episodes, seed, and metadata simply fill the defaults).

from_yaml(path: str) -> SimScene classmethod

Load and validate a SimScene YAML from disk.

Source code in python/core/src/openral_core/schemas.py

@classmethod
def from_yaml(cls, path: str) -> SimScene:
    """Load and validate a ``SimScene`` YAML from disk."""
    import yaml  # noqa: PLC0415

    with open(path, encoding="utf-8") as fh:
        data = yaml.safe_load(fh)
    return cls.model_validate(data)

SpatialEdge

Bases: BaseModel

A directed relation between two scene-graph nodes (ADR-0038).

Attributes:

Name	Type	Description
src	str	node_id of the source node.
dst	str	node_id of the destination node.
kind	SpatialRelationKind	Relation kind.

SpatialNode

Bases: BaseModel

A persistent, typed node in the scene-graph spatial memory (ADR-0038).

A superset of :class:DetectedObject for kind == OBJECT; also used for places, rooms, and agents. The pose is anchored in a durable, drift-corrected frame (typically the tf2 map frame); consumers resolve it to the live base frame at query time via tf2 — the node never stores a raw transform.

This is advisory world-model state consumed by the S2 Reasoner. It is never a safety input (ADR-0038 §1, CLAUDE.md §1.1): the safety kernel gates only on the live, bounded ADR-0030 geometric world.

Attributes:

Name	Type	Description
node_id	str	Stable identifier, unique within a :class:SceneGraph.
kind	SpatialNodeKind	Node kind (object / place / room / agent).
pose	Pose6D	6D pose, anchored in the durable map frame.
label	str	Semantic label or name (e.g. "fridge", "kitchen").
confidence	float	Confidence in [0, 1] for a perceived node.
bbox_3d	tuple[float, float, float, float, float, float] | None	Optional 3D bounding box (x_min, y_min, z_min, x_max, y_max, z_max).
embedding_ref	str | None	Optional handle into the vector store for open-vocabulary matching (ADR-0038 §5); None → label-only.
is_container	bool	Whether the node can hold other nodes (fridge, cabinet).
occludes_contents	bool	Whether contents are unobservable until the container is opened. Requires is_container.
first_seen_ns	int	Timestamp of first observation, in nanoseconds.
last_seen_ns	int	Timestamp of the most recent observation, in nanoseconds; must be >= first_seen_ns.
observation_count	int	Number of times this node has been observed.

Example

node = SpatialNode( ... node_id="fridge", ... kind=SpatialNodeKind.OBJECT, ... pose=Pose6D(xyz=(3.0, 1.0, 0.9), quat_xyzw=(0.0, 0.0, 0.0, 1.0), frame_id="map"), ... label="fridge", ... is_container=True, ... occludes_contents=True, ... first_seen_ns=1, ... last_seen_ns=2, ... ) node.is_container True

SpatialNodeKind

Bases: str, Enum

Kind of node in the persistent scene-graph spatial memory (ADR-0038).

OBJECT is the foundation (an accumulated :class:DetectedObject); PLACE is a standable navigation waypoint; ROOM is a semantic area grouping places/objects; AGENT is a person or robot with a pose — the requester of a task is an AGENT so "bring it back to me" resolves to a concrete goal.

SpatialRelationKind

Bases: str, Enum

Kind of directed edge between scene-graph nodes (ADR-0038).

CONTAINS links a room/container to what is inside it (a fridge CONTAINS a wine bottle); AT_PLACE links an object/agent to the waypoint to stand at to reach it; TRAVERSABLE_TO is the topological navigation graph between places/rooms; ON / NEAR are incidental object-to-object spatial relations.

SphereShape

Bases: BaseModel

Sphere collision primitive — the simplest convex link/obstacle volume.

Attributes:

Name	Type	Description
shape	Literal['sphere']	Discriminator (always "sphere").
radius_m	float	Sphere radius in metres.

Example

SphereShape(radius_m=0.05).shape 'sphere'

StateContract

Bases: BaseModel

Per-rSkill state-vector contract.

Surfaces the proprioception layout the checkpoint was trained against so the runtime adapter does not have to learn it from a YAML override. ADR-0014 + ADR-0027.

Attributes:

Name	Type	Description
layout	StateLayout | None	Named proprioception layout — see :data:StateLayout. LIBERO / MetaWorld / pusht / aloha leave this None and consume the raw joint-position vector directly.
dim	int | None	Explicit state dimension override. The runtime adapter clips or pads the env state vector to this width before handing it to the policy.
bindings	StateContractBindings | None	Per-robot source bindings — TF frame names + JointState entries the runtime adapter pulls values from. REQUIRED when layout is in :data:WRAPPED_TASK_SPACE_LAYOUTS, FORBIDDEN otherwise (joint-space layouts have no source to parameterise).

StateContractBindings

Bases: BaseModel

Per-robot source bindings for an rSkill's state_contract.layout. ADR-0027.

Symmetric to :class:ControlModeSemantics on the action side (joint_order + reference_frame + gripper_convention): the rSkill manifest names the shape via :attr:StateContract.layout; these bindings name the sources on the deploying robot. The layout-adapter registry (openral_state_adapter) joins shape + bindings + live :class:sensor_msgs/JointState + live /tf into the per-checkpoint state vector.

Bindings are required for layouts in :data:WRAPPED_TASK_SPACE_LAYOUTS and forbidden otherwise (the joint-space layouts have no source to parameterise).

Attributes:

Name	Type	Description
eef_frame	str | None	tf2 link name of the end effector (e.g. "panda_hand"). Required for any layout reading EE position / orientation.
base_frame	str | None	tf2 link name of the mobile base (e.g. "base_link"). Required for any layout reading base position / orientation OR base-relative EE poses.
world_frame	str | None	tf2 root frame the base pose is expressed in (default "map" — slam_toolbox publishes map → odom → base_link). Set to "odom" for deployments without SLAM.
gripper_qpos_joints	list[str]	JointState.name entries whose positions populate the gripper-width slot(s) of the layout, in the order the policy expects (e.g. ["panda_finger_joint1", "panda_finger_joint2"]). Empty for grasper-less robots or layouts without a gripper slot.
quaternion_convention	Literal['xyzw', 'wxyz']	Component order of any quaternion in the assembled vector. ROS / TF2 default is "xyzw" (the geometry_msgs/Quaternion field order). Some upstream checkpoints expect "wxyz" — declare it here so the assembler permutes once, at the boundary.

StateRepresentation

Bases: str, Enum

State vector representation format.

SuppressedSummaryEvidence

Bases: _FailureEvidenceBase

Evidence for KIND_SUPPRESSED_SUMMARY — rolling rate-limit roll-up.

Emitted at ~1 Hz by :class:openral_observability.FailureBusPublisher when one or more (kind, severity) buckets dropped events during the past window.

Attributes:

Name	Type	Description
kind	Literal['suppressed_summary']	Discriminator (always "suppressed_summary").
window_s	float	Length of the summarized window, in seconds.
kinds	list[int]	Parallel array of suppressed KIND_* values.
severities	list[int]	Parallel array of suppressed SEVERITY_* values.
counts	list[int]	Parallel array of dropped-event counts per bucket.

TaskSpec

Bases: BaseModel

A task declaration — WHAT the robot must achieve inside a scene.

Tasks decouple scene assets (SceneSpec) from goal-conditioning. The same scene can host many tasks; the same task can occasionally be run in multiple compatible scenes (rare; usually 1:1 with scene_id).

Success is evaluated by the scene adapter — the reward / success signal comes from the underlying gym env (info['is_success'], info['success'], terminal reward, …). success_key lets manifest authors override which info field the runner reads.

Attributes:

Name	Type	Description
id	str	Stable task identifier, e.g. "libero_spatial/task_0", "metaworld/push-v3". Adapters split on / to resolve.
scene_id	str	ID of the :class:SceneSpec this task runs in.
instruction	str	Natural-language goal handed to the VLA as the "task" text input. Some adapters override this with a description baked into the underlying suite (LIBERO, MetaWorld).
max_steps	int | None	Episode budget. Adapters may clip to scene-internal limits. None means unset; BenchmarkScene enforces a concrete value via its model validator.
success_key	str | None	Key inside info returned by env.step() whose truthy value marks task success. None means unset; :class:BenchmarkScene enforces a concrete value via its model validator (required for paper-comparison benchmarks).
metadata	dict[str, object]	Free-form per-task notes (paper reference, dataset split, …).

Example

t = TaskSpec( ... id="libero_spatial/task_0", ... scene_id="libero_spatial", ... instruction="pick up the black bowl", ... max_steps=200, ... success_key="is_success", ... ) t.max_steps 200 t.success_key 'is_success' TaskSpec(id="libero_spatial/task_0", scene_id="libero_spatial").max_steps is None True

TickResult

Bases: BaseModel

One tick's record returned by :meth:InferenceRunner.tick.

Carries the timing breakdown so the parent OTel span can attach exact sub-stage durations and the latency budget enforcement can flag violations without re-instrumenting.

The hardware fields (sensors_ms..hal_ms, safety_violations, action_applied) are the original (v1) surface used by :class:HardwareRunner. The sim-specific fields (step_idx..truncated) were added by ADR-0010 amendment 1 when :class:SimRunner adopted per-step tick semantics; hardware leaves them at their defaults (None), so a hardware tick serialises identically to v1 under model_dump(exclude_none=True).

Attributes:

Name	Type	Description
stamp_ns	int	Tick wall-clock timestamp in nanoseconds (tick start).
tick_idx	int	0-indexed tick counter within a run.
sensors_ms	float	SensorReader.read_latest total wall-time.
world_state_ms	float	WorldStateAggregator.snapshot wall-time.
inference_ms	float	Skill.step wall-time (the chunk dispatch cost, not the full chunked inference — see :class:ChunkedExecutor).
safety_ms	float	Safety check wall-time.
hal_ms	float	HAL.send_action wall-time.
tick_ms	float	End-to-end tick wall-time including the rate-limiter overhead.
chunk_index	int | None	Index of the action played out from a chunked executor. None for non-chunked skills.
safety_violations	list[str]	List of safety-violation reason strings emitted during this tick. Non-empty implies the action was either clamped or dropped per the safety policy.
action_applied	bool	False when the :class:DeadlineOverrunPolicy was drop and the runner elected not to publish this tick's action, or when a sim tick is the reset-tick between episodes (no inference / env step happened).
step_idx	int | None	0-indexed step within the current episode. Set by :class:SimRunner on step-ticks; None on hardware ticks and on sim reset-ticks.
episode_idx	int | None	0-indexed episode within the current run. Set by :class:SimRunner on every tick (including reset-ticks); None on hardware ticks.
reward	float | None	Env step reward for this tick. Set by :class:SimRunner on step-ticks; None on hardware ticks and reset-ticks.
terminated	bool | None	Whether the env signalled natural termination this tick. Set by :class:SimRunner; None elsewhere.
truncated	bool | None	Whether the env hit its step budget this tick. Set by :class:SimRunner; None elsewhere.
trace_context	str | None	Full W3C traceparent for this tick's rskill.tick span, in the form 00-<trace_id_hex>-<span_id_hex>-<flags_hex>. Optional — set by the runner when an OTel context is active so offline consumers (dataset writers, post-hoc analysers) can resume the trace without re-deriving it from the live span. Default None for byte-identical v1 JSON under model_dump(exclude_none=True).

TimeoutEvidence

Bases: _FailureEvidenceBase

Evidence for KIND_TIMEOUT — an operation missed its deadline.

Attributes:

Name	Type	Description
kind	Literal['timeout']	Discriminator (always "timeout").
operation	str	Short name of the operation that timed out (e.g. "skill.step", "hal.read_state", "reasoner.tick").
deadline_s	float	Configured deadline in seconds.
elapsed_s	float	Actual elapsed wall-clock time in seconds.

TopCameraDefaults

Bases: BaseModel

Default placement for the scene-level "top" (a.k.a. "base") camera.

Per-robot scene defaults consumed by sim backends that render an overview camera (today: the openarm_tabletop_pnp scene). The values describe a look-at camera pointed from pos toward target with vertical field-of-view fovy in degrees.

Backend YAML overrides (scene.backend_options.top_camera_*) still win — this submodel is the default fed to the composer when no override is set, replacing the previous module-level constants in openral_sim.backends.openarm_robosuite._assets.

Attributes:

Name	Type	Description
pos	tuple[float, float, float]	(x, y, z) world-frame camera position in metres.
target	tuple[float, float, float]	(x, y, z) world-frame look-at point in metres.
fovy	float	Vertical field-of-view in degrees.

Example

TopCameraDefaults(pos=(0.2, 0.0, 0.95), target=(0.65, 0.0, 0.05), fovy=65.0).fovy 65.0

UrdfAsset

Bases: BaseModel

A URDF asset reference plus its robot_state_publisher wiring.

ADR-0027 / ADR-0058. The ref is resolved by :func:openral_core.assets.resolve_asset; root_frame and base_to_root_xyz_rpy carry the static transform that bridges a URDF whose root link differs from the robot's base_frame (e.g. Franka's panda_link0 mounted onto a base_link mobile platform).

Attributes:

Name	Type	Description
ref	str	Asset reference (rd:<module>, file:<relpath>, or ros2://robot_description for runtime topic-supplied URDFs).
root_frame	str | None	The URDF's root link name when it differs from :attr:RobotDescription.base_frame. None → the URDF root equals base_frame (no static transform needed).
base_to_root_xyz_rpy	tuple[float, float, float, float, float, float] | None	The 6-DoF transform [x, y, z, roll, pitch, yaw] (metres + radians) published via static_transform_publisher to bridge base_frame to :attr:root_frame. None when :attr:root_frame is None.

Example

UrdfAsset(ref="rd:panda_description", root_frame="panda_link0").root_frame 'panda_link0'

VLASpec

Bases: BaseModel

A VLA / policy declaration — the BRAIN driving the robot.

Lightweight pointer to a policy: either an installed rSkill (referenced by manifest name) or a raw HF Hub repo. The eval registry resolves this to a runtime adapter that returns Action objects.

Attributes:

Name	Type	Description
id	str	Adapter id in the eval registry, e.g. "smolvla", "pi05", "xvla", "random", "zero". Picks the loader.
weights_uri	str	Where to fetch weights from. Pass a bare rSkill reference: a name (smolvla-libero), a path (rskills/smolvla-libero), or a bare HF repo ID (OpenRAL/rskill-smolvla-libero). The :class:openral_sim.SimRunner requires a locally-resolvable reference — raw "hf://" URIs are rejected. Other URI shapes (e.g. "mock://") are still parsed by the schema so unit tests of the eval registries can run without an rSkill, but they will fail-fast if used with the runner.
device	str	Torch device override. "auto" picks cuda:0 if available, otherwise cpu.
runtime	RSkillRuntime | None	Optional runtime override; None means "use whatever the policy/manifest declares".
quantization	QuantizationConfig | None	Optional quantization override for this run.
deterministic	bool	When True, set torch.use_deterministic_algorithms and disable cuDNN benchmarking.
extra	dict[str, object]	Adapter-specific options, e.g. {"chunk_size": 16}.

Example

v = VLASpec(id="smolvla", weights_uri="rskills/smolvla-libero") v.device 'auto'

WamEvidence

Bases: _FailureEvidenceBase

Evidence for KIND_WAM — a world-action-model discrepancy.

Attributes:

Name	Type	Description
kind	Literal['wam']	Discriminator (always "wam").
horizon	int	Rollout horizon at which the discrepancy was measured.
discrepancy	float	Scalar discrepancy in the WAM's native units.
wam_id	str	Identifier of the WAM that fired.

WorkspaceEvidence

Bases: _FailureEvidenceBase

Evidence for KIND_WORKSPACE — EE pose left the safety AABB.

Attributes:

Name	Type	Description
kind	Literal['workspace']	Discriminator (always "workspace").
ee_name	str	End-effector that violated the box.
measured_xyz	tuple[float, float, float]	Measured position in metres.
box_min	tuple[float, float, float]	Box minimum corner (x_min, y_min, z_min).
box_max	tuple[float, float, float]	Box maximum corner (x_max, y_max, z_max).

WorldCollisionPrimitive

Bases: BaseModel

A placed convex obstacle volume in the world (ADR-0030).

The world-frame analogue of :class:LinkCollisionGeometry: a convex primitive plus the pose that places it. Populated by perception / SLAM and consumed by the kernel's world-collision phase against the robot's link capsules. A bounded, capped set is the kernel's world model (mesh obstacles are out of scope for the allocation-free check).

Attributes:

Name	Type	Description
shape	CollisionShape	The convex primitive (capsule or sphere).
pose	Pose6D	Pose of the primitive's local origin in the world frame.
object_id	str | None	Optional stable identifier (e.g. a :attr:DetectedObject.track_id rendered as text) surfaced in :attr:CollisionEvidence.link_b_or_object.

WorldState

Bases: BaseModel

Snapshot consumed by Reasoner and Skills.

Attributes:

Name	Type	Description
stamp_ns	int	Snapshot timestamp in nanoseconds.
joint_state	JointState	Current joint state.
base_pose	Pose6D | None	Base link pose (mobile robots).
base_twist	tuple[float, float, float, float, float, float] | None	Base link twist (vx, vy, vz, wx, wy, wz).
ee_poses	dict[str, Pose6D]	End-effector poses keyed by EE name.
contact_forces	dict[str, tuple[float, ...]]	Contact forces keyed by contact name.
images	dict[str, str]	Sensor name → ROS 2 topic reference (not the raw image).
image_frames	dict[str, SensorFrame] | None	Optional per-sensor :class:SensorFrame snapshot for no-ROS / in-process deployments. When None the consumer reads frames via :attr:images topic refs as before.
point_clouds	dict[str, str]	Sensor name → ROS 2 topic reference.
tactile	dict[str, str]	Sensor name → ROS 2 topic reference.
detected_objects	list[DetectedObject]	List of detected objects.
battery_pct	float | None	Battery percentage in [0, 100].
diagnostics	dict[str, Literal['ok', 'warn', 'error', 'stale']]	Per-component diagnostic status.
collision_primitives	list[WorldCollisionPrimitive]	Bounded set of placed convex obstacle volumes the kernel checks robot links against (world-collision). Empty until a perception / SLAM source populates it. ADR-0030.
occupancy_grid	OccupancyGridRef | None	Optional 2D occupancy grid reference for mobile-base footprint checks. None until populated; an absent or stale grid is treated as unavailable (fail-closed). ADR-0030.

canonical_slots_for_representation(rep: ActionRepresentation, *, dim: int, description: RobotDescription) -> list[ActionSlot] | None

Build the canonical :class:ActionSlot layout for a representation.

ADR-0036. The skill_runner calls this to expand a skill that declares only ActionContract.representation (no explicit slots) into a typed slot layout it can dispatch. Joint representations return None so the caller keeps the legacy whole-vector JOINT_POSITION path; cartesian / gripper representations get one slot per control mode, addressed at the robot's primary end-effector (description.end_effectors[0]).

:class:EndEffectorSpec carries no explicit tf-frame field, so the end-effector's name is used as the slot frame (the EE name is the tf frame the cartesian pose/delta is expressed in for the canonical embodiments).

Parameters:

Name	Type	Description	Default
rep	ActionRepresentation	The VLA's declared action-vector representation.	required
dim	int	Dimensionality of the policy's flat action vector.	required
description	RobotDescription	The target robot — its primary end-effector names the cartesian/gripper slots.	required

Returns:

Type	Description
list[ActionSlot] | None	For joint representations: None (caller keeps the legacy
list[ActionSlot] | None	whole-vector JOINT_POSITION path). Otherwise a list of typed
list[ActionSlot] | None	class:ActionSlot s that satisfy the per-slot validators.

Raises:

Type	Description
ROSConfigError	If the representation needs an end-effector but description.end_effectors is empty, or if dim is too small for the layout (DELTA_EE_6D / CARTESIAN_POSE need dim >= 6; DELTA_EE_6D_PLUS_GRIPPER needs dim >= 7).

Example

desc = RobotDescription( ... name="ex_robot", ... embodiment_kind=EmbodimentKind.MANIPULATOR, ... joints=[ ... JointSpec( ... name="j1", ... joint_type=JointType.REVOLUTE, ... parent_link="base_link", ... child_link="link_1", ... ) ... ], ... end_effectors=[EndEffectorSpec(name="ee0", kind="parallel_gripper")], ... capabilities=RobotCapabilities(), ... safety=SafetyEnvelope(), ... ) slots = canonical_slots_for_representation( ... ActionRepresentation.DELTA_EE_6D, dim=6, description=desc ... ) slots[0].control_mode is ControlMode.CARTESIAN_DELTA True

Source code in python/core/src/openral_core/schemas.py

def canonical_slots_for_representation(
    rep: ActionRepresentation,
    *,
    dim: int,
    description: RobotDescription,
) -> list[ActionSlot] | None:
    """Build the canonical :class:`ActionSlot` layout for a representation.

    ADR-0036. The skill_runner calls this to expand a skill that declares
    only ``ActionContract.representation`` (no explicit ``slots``) into a
    typed slot layout it can dispatch. Joint representations return
    ``None`` so the caller keeps the legacy whole-vector ``JOINT_POSITION``
    path; cartesian / gripper representations get one slot per control
    mode, addressed at the robot's primary end-effector
    (``description.end_effectors[0]``).

    :class:`EndEffectorSpec` carries no explicit tf-frame field, so the
    end-effector's ``name`` is used as the slot ``frame`` (the EE name is
    the tf frame the cartesian pose/delta is expressed in for the
    canonical embodiments).

    Args:
        rep: The VLA's declared action-vector representation.
        dim: Dimensionality of the policy's flat action vector.
        description: The target robot — its primary end-effector names
            the cartesian/gripper slots.

    Returns:
        For joint representations: ``None`` (caller keeps the legacy
        whole-vector ``JOINT_POSITION`` path). Otherwise a list of typed
        :class:`ActionSlot` s that satisfy the per-slot validators.

    Raises:
        ROSConfigError: If the representation needs an end-effector but
            ``description.end_effectors`` is empty, or if ``dim`` is too
            small for the layout (``DELTA_EE_6D`` / ``CARTESIAN_POSE``
            need ``dim >= 6``; ``DELTA_EE_6D_PLUS_GRIPPER`` needs
            ``dim >= 7``).

    Example:
        >>> desc = RobotDescription(
        ...     name="ex_robot",
        ...     embodiment_kind=EmbodimentKind.MANIPULATOR,
        ...     joints=[
        ...         JointSpec(
        ...             name="j1",
        ...             joint_type=JointType.REVOLUTE,
        ...             parent_link="base_link",
        ...             child_link="link_1",
        ...         )
        ...     ],
        ...     end_effectors=[EndEffectorSpec(name="ee0", kind="parallel_gripper")],
        ...     capabilities=RobotCapabilities(),
        ...     safety=SafetyEnvelope(),
        ... )
        >>> slots = canonical_slots_for_representation(
        ...     ActionRepresentation.DELTA_EE_6D, dim=6, description=desc
        ... )
        >>> slots[0].control_mode is ControlMode.CARTESIAN_DELTA
        True
    """
    if rep in (ActionRepresentation.JOINT_POSITIONS, ActionRepresentation.JOINT_VELOCITIES):
        # Legacy whole-vector JOINT path — the caller emits one implicit
        # JOINT_POSITION / JOINT_VELOCITY slot covering the whole vector.
        return None

    if not description.end_effectors:
        raise ROSConfigError(
            f"canonical_slots_for_representation: representation {rep.value!r} addresses "
            "an end-effector but the robot description declares no end_effectors; "
            "cannot build a cartesian/gripper slot layout."
        )
    primary = description.end_effectors[0]
    ee_name = primary.name
    # EndEffectorSpec has no explicit tf-frame field; the EE name is the
    # tf frame the cartesian pose/delta is expressed in.
    ee_frame = ee_name

    has_gripper = rep is ActionRepresentation.DELTA_EE_6D_PLUS_GRIPPER
    min_dim = _EE_6D_WIDTH + 1 if has_gripper else _EE_6D_WIDTH
    if dim < min_dim:
        raise ROSConfigError(
            f"canonical_slots_for_representation: representation {rep.value!r} requires "
            f"dim >= {min_dim} but action_contract.dim={dim} is too small for the layout."
        )

    cart_mode = (
        ControlMode.CARTESIAN_POSE
        if rep is ActionRepresentation.CARTESIAN_POSE
        else ControlMode.CARTESIAN_DELTA
    )
    slots: list[ActionSlot] = [
        ActionSlot(
            range=(0, _EE_6D_WIDTH - 1),
            control_mode=cart_mode,
            ee=ee_name,
            frame=ee_frame,
        )
    ]
    if has_gripper:
        slots.append(
            ActionSlot(
                range=(_EE_6D_WIDTH, dim - 1),
                control_mode=ControlMode.GRIPPER_POSITION,
                ee=ee_name,
            )
        )
    return slots

control_modes_for_representation(rep: ActionRepresentation) -> set[ControlMode]

Map an :class:ActionRepresentation to the :class:ControlMode s it drives.

ADR-0036. Used by the reasoner's deploy-path palette gate: a skill is only offered when the target robot advertises every mode in the returned set.

Parameters:

Name	Type	Description	Default
rep	ActionRepresentation	The VLA's declared action-vector representation.	required

Returns:

Type	Description
set[ControlMode]	The set of :class:ControlMode s the representation maps onto.

Example

control_modes_for_representation(ActionRepresentation.DELTA_EE_6D_PLUS_GRIPPER) == { ... ControlMode.CARTESIAN_DELTA, ... ControlMode.GRIPPER_POSITION, ... } True

Source code in python/core/src/openral_core/schemas.py

def control_modes_for_representation(rep: ActionRepresentation) -> set[ControlMode]:
    """Map an :class:`ActionRepresentation` to the :class:`ControlMode` s it drives.

    ADR-0036. Used by the reasoner's deploy-path palette gate: a skill is
    only offered when the target robot advertises *every* mode in the
    returned set.

    Args:
        rep: The VLA's declared action-vector representation.

    Returns:
        The set of :class:`ControlMode` s the representation maps onto.

    Example:
        >>> control_modes_for_representation(ActionRepresentation.DELTA_EE_6D_PLUS_GRIPPER) == {
        ...     ControlMode.CARTESIAN_DELTA,
        ...     ControlMode.GRIPPER_POSITION,
        ... }
        True
    """
    if rep is ActionRepresentation.JOINT_POSITIONS:
        return {ControlMode.JOINT_POSITION}
    if rep is ActionRepresentation.JOINT_VELOCITIES:
        return {ControlMode.JOINT_VELOCITY}
    if rep is ActionRepresentation.DELTA_EE_6D:
        return {ControlMode.CARTESIAN_DELTA}
    if rep is ActionRepresentation.DELTA_EE_6D_PLUS_GRIPPER:
        return {ControlMode.CARTESIAN_DELTA, ControlMode.GRIPPER_POSITION}
    # CARTESIAN_POSE — the only remaining enum member.
    return {ControlMode.CARTESIAN_POSE}

extract_base_sim_joint_names(description: RobotDescription) -> tuple[str, str, str] | None

Return (forward, side, yaw) MJCF joint names from any mobile-base description.

ADR-0025 — generic, robot-agnostic helper. Consumes the :attr:RobotDescription.base_joints declaration + each referenced :class:JointSpec's :attr:~JointSpec.sim_joint_name override. Works for any robot whose robot.yaml declares both fields:

• base_joints: [<forward>, <side>, <yaw>] at the top level.
• Each of the three referenced joints carries a sim_joint_name: "..." mapping its URDF-shape name to the MJCF/MuJoCo joint name the simulator emits (typically auto-prefixed under a composed scene).

Returns None when the description has no base_joints block, fewer than three entries (the schema permits arbitrary list length so future non-planar bases can declare more — this helper specialises to the planar-base case), or any referenced joint lacks sim_joint_name. Callers should treat None as "fall back to module defaults" — the sim-side ray-cast helpers in :mod:openral_sim.backends.robocasa accept this contract.

Parameters:

Name	Type	Description	Default
description	RobotDescription	The robot's manifest, loaded via :meth:RobotDescription.from_yaml.	required

Returns:

Type	Description
tuple[str, str, str] | None	(forward, side, yaw) MJCF joint names, or None when
tuple[str, str, str] | None	the description doesn't carry a complete mobile-base block.

Example

See robots/panda_mobile/robot.yaml for a real fixture.

desc = RobotDescription( ... name="ex", ... embodiment_kind=EmbodimentKind.MOBILE_MANIPULATOR, ... joints=[ ... JointSpec( ... name="base_x", ... joint_type=JointType.PRISMATIC, ... parent_link="world", ... child_link="base_x_link", ... sim_joint_name="mobilebase0_joint_mobile_forward", ... ), ... JointSpec( ... name="base_y", ... joint_type=JointType.PRISMATIC, ... parent_link="base_x_link", ... child_link="base_y_link", ... sim_joint_name="mobilebase0_joint_mobile_side", ... ), ... JointSpec( ... name="base_yaw", ... joint_type=JointType.REVOLUTE, ... parent_link="base_y_link", ... child_link="base_link", ... sim_joint_name="mobilebase0_joint_mobile_yaw", ... ), ... ], ... capabilities=RobotCapabilities(embodiment_tags=["ex"]), ... safety=SafetyEnvelope(), ... base_joints=["base_x", "base_y", "base_yaw"], ... ) extract_base_sim_joint_names(desc)[0] 'mobilebase0_joint_mobile_forward'

Source code in python/core/src/openral_core/schemas.py

def extract_base_sim_joint_names(
    description: RobotDescription,
) -> tuple[str, str, str] | None:
    """Return ``(forward, side, yaw)`` MJCF joint names from any mobile-base description.

    ADR-0025 — generic, robot-agnostic helper. Consumes the
    :attr:`RobotDescription.base_joints` declaration + each referenced
    :class:`JointSpec`'s :attr:`~JointSpec.sim_joint_name` override.
    Works for any robot whose ``robot.yaml`` declares both fields:

    * ``base_joints: [<forward>, <side>, <yaw>]`` at the top level.
    * Each of the three referenced joints carries a
      ``sim_joint_name: "..."`` mapping its URDF-shape ``name`` to the
      MJCF/MuJoCo joint name the simulator emits (typically
      auto-prefixed under a composed scene).

    Returns ``None`` when the description has no ``base_joints``
    block, fewer than three entries (the schema permits arbitrary
    list length so future non-planar bases can declare more — this
    helper specialises to the planar-base case), or any referenced
    joint lacks ``sim_joint_name``. Callers should treat ``None`` as
    "fall back to module defaults" — the sim-side ray-cast helpers
    in :mod:`openral_sim.backends.robocasa` accept this contract.

    Args:
        description: The robot's manifest, loaded via
            :meth:`RobotDescription.from_yaml`.

    Returns:
        ``(forward, side, yaw)`` MJCF joint names, or ``None`` when
        the description doesn't carry a complete mobile-base block.

    Example:
        >>> # See `robots/panda_mobile/robot.yaml` for a real fixture.
        >>> desc = RobotDescription(
        ...     name="ex",
        ...     embodiment_kind=EmbodimentKind.MOBILE_MANIPULATOR,
        ...     joints=[
        ...         JointSpec(
        ...             name="base_x",
        ...             joint_type=JointType.PRISMATIC,
        ...             parent_link="world",
        ...             child_link="base_x_link",
        ...             sim_joint_name="mobilebase0_joint_mobile_forward",
        ...         ),
        ...         JointSpec(
        ...             name="base_y",
        ...             joint_type=JointType.PRISMATIC,
        ...             parent_link="base_x_link",
        ...             child_link="base_y_link",
        ...             sim_joint_name="mobilebase0_joint_mobile_side",
        ...         ),
        ...         JointSpec(
        ...             name="base_yaw",
        ...             joint_type=JointType.REVOLUTE,
        ...             parent_link="base_y_link",
        ...             child_link="base_link",
        ...             sim_joint_name="mobilebase0_joint_mobile_yaw",
        ...         ),
        ...     ],
        ...     capabilities=RobotCapabilities(embodiment_tags=["ex"]),
        ...     safety=SafetyEnvelope(),
        ...     base_joints=["base_x", "base_y", "base_yaw"],
        ... )
        >>> extract_base_sim_joint_names(desc)[0]
        'mobilebase0_joint_mobile_forward'
    """
    # Planar mobile bases carry exactly three holonomic axes (forward,
    # side, yaw); the schema doesn't constrain `base_joints` length so
    # future non-planar bases can declare richer surfaces, but this
    # helper specialises here.
    planar_base_dof = 3
    if description.base_joints is None or len(description.base_joints) < planar_base_dof:
        return None
    by_name = {j.name: j for j in description.joints}
    sim_names: list[str | None] = []
    for ref in description.base_joints[:planar_base_dof]:
        spec = by_name.get(ref)
        if spec is None:
            return None
        sim_names.append(spec.sim_joint_name)
    if any(n is None for n in sim_names):
        return None
    forward, side, yaw = sim_names
    # Narrow tuple[str | None, ...] → tuple[str, str, str] for type checkers.
    assert isinstance(forward, str)
    assert isinstance(side, str)
    assert isinstance(yaw, str)
    return (forward, side, yaw)

scale_intrinsics_to(base: IntrinsicsPinhole, width: int, height: int) -> IntrinsicsPinhole

Linearly rescale pinhole intrinsics to a new render resolution.

For a camera with a fixed field of view, fx/fy/cx/cy scale linearly with the image dimensions: a frame rendered at twice the width has twice the focal length (px) and twice the principal-point x. This is the consistency rule deploy-sim needs — the canonical manifest pins one nominal resolution, but a scene may render the same MuJoCo camera at another (scene.observation_width/height). Publishing the manifest's nominal intrinsics on a different-resolution render would back-project depth pixels and project occupancy voxels with the wrong focal length, corrupting the OctoMap voxels and the 2D→3D object lift. Scaling keeps the published camera model matched to whatever was actually rendered.

The distortion model and coefficients are preserved unchanged (coefficients are resolution-independent for the normalised plumb-bob model). When the target already equals base's resolution the input is returned as-is.

Parameters:

Name	Type	Description	Default
base	IntrinsicsPinhole	Nominal pinhole intrinsics (e.g. from a SensorSpec).	required
width	int	Target render width in pixels (> 0).	required
height	int	Target render height in pixels (> 0).	required

Returns:

Type	Description
IntrinsicsPinhole	Intrinsics scaled so (fx, fy, cx, cy) correspond to ``(width,
IntrinsicsPinhole	height)at the same field of view asbase``.

Raises:

Type	Description
ValueError	If width or height is not strictly positive.

Example

base = IntrinsicsPinhole(width=256, height=256, fx=256.0, fy=256.0, cx=128.0, cy=128.0) hi = scale_intrinsics_to(base, 640, 640) (hi.width, hi.fx, hi.cx) (640, 640.0, 320.0)

Source code in python/core/src/openral_core/schemas.py

def scale_intrinsics_to(base: IntrinsicsPinhole, width: int, height: int) -> IntrinsicsPinhole:
    """Linearly rescale pinhole intrinsics to a new render resolution.

    For a camera with a *fixed* field of view, ``fx``/``fy``/``cx``/``cy`` scale
    linearly with the image dimensions: a frame rendered at twice the width has
    twice the focal length (px) and twice the principal-point x. This is the
    consistency rule deploy-sim needs — the canonical manifest pins one nominal
    resolution, but a scene may render the same MuJoCo camera at another
    (``scene.observation_width``/``height``). Publishing the manifest's nominal
    intrinsics on a different-resolution render would back-project depth pixels
    and project occupancy voxels with the wrong focal length, corrupting the
    OctoMap voxels and the 2D→3D object lift. Scaling keeps the published camera
    model matched to whatever was actually rendered.

    The distortion model and coefficients are preserved unchanged (coefficients
    are resolution-independent for the normalised plumb-bob model). When the
    target already equals ``base``'s resolution the input is returned as-is.

    Args:
        base: Nominal pinhole intrinsics (e.g. from a ``SensorSpec``).
        width: Target render width in pixels (``> 0``).
        height: Target render height in pixels (``> 0``).

    Returns:
        Intrinsics scaled so ``(fx, fy, cx, cy)`` correspond to ``(width,
        height)`` at the same field of view as ``base``.

    Raises:
        ValueError: If ``width`` or ``height`` is not strictly positive.

    Example:
        >>> base = IntrinsicsPinhole(width=256, height=256, fx=256.0, fy=256.0, cx=128.0, cy=128.0)
        >>> hi = scale_intrinsics_to(base, 640, 640)
        >>> (hi.width, hi.fx, hi.cx)
        (640, 640.0, 320.0)
    """
    if width <= 0 or height <= 0:
        raise ValueError(f"target resolution must be positive; got ({width}, {height})")
    if width == base.width and height == base.height:
        return base
    sx = width / base.width
    sy = height / base.height
    return IntrinsicsPinhole(
        width=width,
        height=height,
        fx=base.fx * sx,
        fy=base.fy * sy,
        cx=base.cx * sx,
        cy=base.cy * sy,
        distortion_model=base.distortion_model,
        distortion_coeffs=list(base.distortion_coeffs),
    )

openral exception hierarchy — use these, do not invent new base classes.

All exceptions derive from ROSError. ROSSafetyViolation and its subclasses must never be caught silently; they are only caught at the safety supervisor boundary where they trigger an E-stop and a structured incident log.

Example

try: ... raise ROSConfigError("missing URDF") ... except ROSError as exc: ... print(type(exc).name, str(exc)) ROSConfigError missing URDF

ROSBTValidationError

Bases: ROSPlanningError

The emitted BehaviorTree XML failed BT.CPP v4 validation.

ROSCapabilityMismatch

Bases: ROSError

A skill requires a capability the target robot does not have.

ROSCollisionImminent

Bases: ROSSafetyViolation

A proposed motion would self-collide or strike a world obstacle.

Raised on the safety path when geometric checking (ADR-0030) finds a chunk whose forward-kinematic sweep brings a robot link within its clearance of another link or a world primitive. Like every :class:ROSSafetyViolation, it is caught only at the safety supervisor boundary, where it triggers an E-stop and a structured incident log.

ROSConfigError

Bases: ROSError

Bad manifest, missing weights, or invalid YAML / URDF.

ROSDeadlineMissed

Bases: ROSFleetError

Cloud RTT exceeded the skill's deadline; fallback was not configured.

ROSDispatchUnavailable

Bases: ROSFleetError

No dispatcher (edge or cloud) is available for the requested skill.

ROSEStopRequested

Bases: ROSSafetyViolation

An emergency stop was requested; all actuation must cease immediately.

ROSError

Bases: Exception

Base class for all OpenRAL errors.

ROSFleetError

Bases: ROSError

A fleet-level or dispatch error.

ROSForceLimitExceeded

Bases: ROSSafetyViolation

Measured or predicted contact force exceeds the configured limit.

ROSGPUMemoryError

Bases: ROSRuntimeError

Out of GPU memory; use a smaller skill variant or quantize further.

ROSInferenceTimeout

Bases: ROSRuntimeError

VLA inference did not complete within its latency budget.

ROSObjectNotInMemory

Bases: ROSPerceptionStale

A spatial-memory query matched no node, or only nodes that are stale.

Raised by the ADR-0038 scene-graph query surface when a RecallObjectQuery / ResolvePlaceQuery cannot be satisfied. The caller degrades by treating the target as unknown (and may trigger active search) — it never fabricates a pose (CLAUDE.md §1.2, §1.4).

ROSPerceptionStale

Bases: ROSError

A sensor reading is older than the configured staleness deadline.

ROSPlanningError

Bases: ROSError

The reasoner failed to produce a valid plan.

ROSQuantizationError

Bases: ROSRuntimeError

Quantization failed or produced an incompatible engine.

ROSReasonerInvalidPlan

Bases: ROSPlanningError

The LLM returned a plan that failed schema or capability validation.

ROSRskillGoalSatisfied

Bases: ROSError

A wrapped-ROS rSkill has finished its goal successfully.

Used as a typed completion signal raised by :meth:openral_rskill.ros_action_rskill.ROSActionRskill._step_impl after the last waypoint of a one-shot planner (e.g. MoveIt) has been emitted, or after a result-only wrapped action (e.g. Nav2 NavigateToPose) reports success. The ExecuteSkill action server catches this specifically and closes the goal with success=True.

This is NOT an error — it inherits :class:ROSError only to stay inside the OpenRAL exception surface so it is greppable and discoverable via the standard hierarchy. It must be caught ONLY at the rskill_runner_node execute-callback boundary; everywhere else it is a programming bug to raise it.

ROSRuntimeError

Bases: ROSError

General runtime failure during skill execution or HAL operation.

ROSSafetyViolation

Bases: ROSError

A safety constraint was violated.

NEVER catch this silently. Catch only at the safety supervisor boundary, trigger an E-stop, and emit a structured incident log entry.

ROSWorkspaceViolation

Bases: ROSSafetyViolation

An action would move the robot outside its allowed workspace.

CLI (openral)

openral CLI entry point — openral command.

Two modes of use:

• One-shot: openral <subcommand> [args...] runs a single command and exits. Use this in scripts and CI. openral --help lists the surface.
• Interactive REPL: openral with no arguments drops into a prompt where subcommands run bare (sim run --config … instead of openral sim run --config …). Type help for the menu, exit or Ctrl-D to leave.

Sub-commands

doctor Diagnose the host environment (Python, OS, ROS 2, GPU, USB). detect Probe hardware and write a full RobotDescription robot.yaml. connect Open a HAL connection to a robot and verify it responds. calibrate camera Calibrate a camera sensor using ros2 camera_calibration. install Install opt-in dependency groups (sim, ros, libero, …) — see ADR-0021. rskill search Find installable rSkills on the OpenRAL HF Hub org (ADR-0055). rskill install Download an rSkill from the HF Hub and register it locally. rskill list List all locally installed rSkills. rskill check Report which installed rSkills will run on the current host. rskill new Scaffold a new local rSkill from rskills/template/. collision lower Lower a robot's URDF/SRDF into its self-collision model (ADR-0030). collision check Fail if a manifest drifts from its lowered collision model.

Run openral --help for full usage.

CheckResult

Bases: NamedTuple

One row in the openral doctor output table.

Attributes:

Name	Type	Description
check	str	Short name of the thing being checked.
status	str	One of ok, fail, missing, absent, info, warn.
details	str	Human-readable detail string (path, version, device list, …).

benchmark_list(benchmarks_dir: Path = typer.Option(Path('benchmarks'), '--benchmarks-dir', help='Search directory for benchmark suite YAMLs.')) -> None

List every benchmark suite id available under benchmarks/*.yaml.

Each entry is a paste-able --suite value for openral benchmark run. No rollout, no GPU.

Source code in python/cli/src/openral_cli/main.py

@benchmark_app.command("list")
def benchmark_list(
    benchmarks_dir: Path = typer.Option(
        Path("benchmarks"),
        "--benchmarks-dir",
        help="Search directory for benchmark suite YAMLs.",
    ),
) -> None:
    """List every benchmark suite id available under ``benchmarks/*.yaml``.

    Each entry is a paste-able ``--suite`` value for ``openral benchmark run``.
    No rollout, no GPU.
    """
    if not benchmarks_dir.is_dir():
        console.print(f"[red]No benchmarks dir at {benchmarks_dir}[/red]")
        raise typer.Exit(code=1)
    suites = sorted(p.stem for p in benchmarks_dir.glob("*.yaml") if p.is_file())
    if not suites:
        print("<none>")
        return
    for suite in suites:
        print(suite)

benchmark_report(rskills_dir: Path = typer.Option(Path('rskills'), '--rskills-dir', help='Directory containing rSkill packages (each with optional eval/*.json).'), json_output: bool = typer.Option(False, '--json', help='Emit a JSON dump instead of the rich-table summary.')) -> None

Walk every <skill>/eval/*.json and print a benchmark roll-up.

Validates each JSON against openral_core.RSkillEvalResult (the same schema the rSkill loader uses at install time) so a rotted file fails loudly instead of silently being skipped.

Example

openral benchmark report

openral benchmark report --json > /tmp/report.json

Source code in python/cli/src/openral_cli/main.py

@benchmark_app.command("report")
def benchmark_report(
    rskills_dir: Path = typer.Option(
        Path("rskills"),
        "--rskills-dir",
        help="Directory containing rSkill packages (each with optional eval/*.json).",
    ),
    json_output: bool = typer.Option(
        False,
        "--json",
        help="Emit a JSON dump instead of the rich-table summary.",
    ),
) -> None:
    """Walk every ``<skill>/eval/*.json`` and print a benchmark roll-up.

    Validates each JSON against `openral_core.RSkillEvalResult`
    (the same schema the rSkill loader uses at install time) so a rotted
    file fails loudly instead of silently being skipped.

    Example:
        >>> # openral benchmark report
        >>> # openral benchmark report --json > /tmp/report.json
    """
    from openral_core import RSkillEvalResult
    from pydantic import ValidationError

    if not rskills_dir.is_dir():
        console.print(f"[red]rskills directory not found: {rskills_dir}[/red]")
        raise typer.Exit(code=1)

    rows: list[dict[str, object]] = []
    for skill_dir in sorted(p for p in rskills_dir.iterdir() if p.is_dir()):
        eval_dir = skill_dir / "eval"
        if not eval_dir.is_dir():
            continue
        for json_path in sorted(eval_dir.glob("*.json")):
            try:
                result = RSkillEvalResult.from_json(str(json_path))
            except (ValidationError, _json.JSONDecodeError) as exc:
                console.print(f"[red]invalid {json_path}:[/red] {exc}")
                raise typer.Exit(code=1) from exc
            rows.append(
                {
                    "rskill": skill_dir.name,
                    "benchmark": result.benchmark.name,
                    "robot": result.benchmark.robot,
                    "simulator": result.benchmark.simulator,
                    "reproduced_locally": result.source.reproduced_locally,
                    "model_variant": result.source.model_variant,
                    "status": result.source.status,
                    "results": result.results,
                    "path": (
                        str(json_path.relative_to(Path.cwd()))
                        if json_path.is_relative_to(Path.cwd())
                        else str(json_path)
                    ),
                }
            )

    if json_output:
        console.print_json(_json.dumps(rows, indent=2, default=str))
        return

    if not rows:
        console.print(f"[yellow]no rskills/<id>/eval/*.json files under {rskills_dir}[/yellow]")
        return

    rows.sort(key=lambda r: (str(r["benchmark"]), str(r["rskill"])))
    table = Table(title=f"rSkill benchmark report — {len(rows)} entries")
    table.add_column("Benchmark", style="cyan")
    table.add_column("rSkill", style="magenta")
    table.add_column("Variant", style="dim")
    table.add_column("Robot", style="dim")
    table.add_column("Repro local?", justify="center")
    table.add_column("Headline result", style="green")
    table.add_column("Status", style="dim")
    for row in rows:
        results = row["results"]
        headline = _summarize_results(results) if isinstance(results, dict) else "—"
        table.add_row(
            str(row["benchmark"]),
            str(row["rskill"]),
            str(row["model_variant"]),
            str(row["robot"]),
            "✓" if row["reproduced_locally"] else "✗",
            headline,
            str(row["status"] or ""),
        )
    console.print(table)

benchmark_run(suite: str = typer.Option(..., '--suite', help='Benchmark suite to evaluate — a bare ``list[BenchmarkScene]`` YAML (ADR-0042). Either a built-in id (resolved to `benchmarks/<id>.yaml`) or a direct YAML path.'), rskill: str = typer.Option(..., '--rskill', help="rSkill reference — a bare name ('smolvla-libero'), a path ('rskills/smolvla-libero'), or an HF Hub repo id ('OpenRAL/rskill-smolvla-libero'). Raw hf:// is rejected (weights must come from a manifest). The policy adapter id is read from the manifest's `model_family` field."), out: Path | None = typer.Option(None, '--out', help='Output path for the RSkillEvalResult JSON. Defaults to rskills/<dir>/eval/<suite_id>.json derived from the rSkill ref.'), device: str | None = typer.Option(None, '--device', help='Torch device override for the policy (cpu, cuda:0, mps, auto).'), save_dir: Path | None = typer.Option(None, '--save-dir', help='Optional adapter-side artefact directory (videos, traces).'), benchmarks_dir: Path = typer.Option(Path('benchmarks'), '--benchmarks-dir', help='Search directory for built-in benchmark suite YAMLs.'), n_episodes: int | None = typer.Option(None, '--n-episodes', help='Override ``BenchmarkScene.n_episodes`` for every scene in the suite (lower for quick smoke runs). The published-protocol value lives in the suite YAML; this flag is for fast iteration, not for paper-comparison numbers.'), dry_run: bool = typer.Option(False, '--dry-run', help='Resolve the suite + VLA and print the planned (task x seed) matrix without running any rollouts. Useful in CI to validate config wiring.'), update_manifest: bool = typer.Option(True, '--update-manifest/--no-update-manifest', help='On success, write the avg_success_rate back into the rSkill manifest at `benchmarks.<suite_id>`. Surgical edit — preserves comments. Only fires for locally-resolvable rSkills. Disable for read-only paper-number runs.'), dashboard: bool = typer.Option(False, '--dashboard', help='Boot `openral dashboard` as a child process, point OTel at it, and shut it down on exit (same semantics as `openral sim run --dashboard`).'), dashboard_port: int = typer.Option(4318, '--dashboard-port', help='Port for the spawned dashboard when --dashboard is set.')) -> None

Run a benchmark suite and write a validated RSkillEvalResult JSON.

The runner iterates scenes x range(seed, seed + n_episodes), delegating each rollout to openral_sim.SimRunner so the rSkill compatibility check, OTel spans, and latency-budget reporting are identical to openral sim run. Each :class:BenchmarkScene carries its own scene + task + robot (ADR-0041 / Task 10); ADR-0042 deleted the BenchmarkSpec wrapper class so the suite is a bare list of scenes whose id is the YAML filename stem.

Example

openral benchmark run --suite libero_spatial \

--rskill smolvla-libero

Source code in python/cli/src/openral_cli/main.py

@benchmark_app.command("run")
def benchmark_run(
    suite: str = typer.Option(
        ...,
        "--suite",
        help=(
            "Benchmark suite to evaluate — a bare ``list[BenchmarkScene]`` YAML "
            "(ADR-0042). Either a built-in id (resolved to "
            "`benchmarks/<id>.yaml`) or a direct YAML path."
        ),
    ),
    rskill: str = typer.Option(
        ...,
        "--rskill",
        help=(
            "rSkill reference — a bare name ('smolvla-libero'), a "
            "path ('rskills/smolvla-libero'), or an HF Hub repo id "
            "('OpenRAL/rskill-smolvla-libero'). "
            "Raw hf:// is rejected (weights must come from a manifest). "
            "The policy adapter id is read from the manifest's `model_family` "
            "field."
        ),
    ),
    out: Path | None = typer.Option(
        None,
        "--out",
        help=(
            "Output path for the RSkillEvalResult JSON. Defaults to "
            "rskills/<dir>/eval/<suite_id>.json derived from the rSkill ref."
        ),
    ),
    device: str | None = typer.Option(
        None,
        "--device",
        help="Torch device override for the policy (cpu, cuda:0, mps, auto).",
    ),
    save_dir: Path | None = typer.Option(
        None,
        "--save-dir",
        help="Optional adapter-side artefact directory (videos, traces).",
    ),
    benchmarks_dir: Path = typer.Option(
        Path("benchmarks"),
        "--benchmarks-dir",
        help="Search directory for built-in benchmark suite YAMLs.",
    ),
    n_episodes: int | None = typer.Option(
        None,
        "--n-episodes",
        help=(
            "Override ``BenchmarkScene.n_episodes`` for every scene in the "
            "suite (lower for quick smoke runs). The published-protocol value "
            "lives in the suite YAML; this flag is for fast iteration, not "
            "for paper-comparison numbers."
        ),
    ),
    dry_run: bool = typer.Option(
        False,
        "--dry-run",
        help=(
            "Resolve the suite + VLA and print the planned (task x seed) "
            "matrix without running any rollouts. Useful in CI to "
            "validate config wiring."
        ),
    ),
    update_manifest: bool = typer.Option(
        True,
        "--update-manifest/--no-update-manifest",
        help=(
            "On success, write the avg_success_rate back into the rSkill "
            "manifest at `benchmarks.<suite_id>`. Surgical edit — "
            "preserves comments. Only fires for locally-resolvable rSkills. "
            "Disable for read-only paper-number runs."
        ),
    ),
    dashboard: bool = typer.Option(
        False,
        "--dashboard",
        help=(
            "Boot `openral dashboard` as a child process, point OTel at it, "
            "and shut it down on exit (same semantics as `openral sim run "
            "--dashboard`)."
        ),
    ),
    dashboard_port: int = typer.Option(
        4318,
        "--dashboard-port",
        help="Port for the spawned dashboard when --dashboard is set.",
    ),
) -> None:
    r"""Run a benchmark suite and write a validated `RSkillEvalResult` JSON.

    The runner iterates ``scenes x range(seed, seed + n_episodes)``,
    delegating each rollout to ``openral_sim.SimRunner`` so the
    rSkill compatibility check, OTel spans, and latency-budget reporting
    are identical to ``openral sim run``. Each :class:`BenchmarkScene`
    carries its own scene + task + robot (ADR-0041 / Task 10); ADR-0042
    deleted the ``BenchmarkSpec`` wrapper class so the suite is a bare
    list of scenes whose id is the YAML filename stem.

    Example:
        >>> # openral benchmark run --suite libero_spatial \\
        >>> #     --rskill smolvla-libero
    """
    scenes, suite_id = _resolve_benchmark_suite(suite, benchmarks_dir)
    vla_spec = _parse_rskill_cli_arg(rskill)

    # Apply --n-episodes override to every scene before dry-run or real run.
    if n_episodes is not None:
        scenes = [s.model_copy(update={"n_episodes": n_episodes}) for s in scenes]

    if dry_run:
        # Suite invariants (openral_core.raise_on_invalid_suite) guarantee
        # every BenchmarkScene shares robot_id / n_episodes / seed; read
        # from scenes[0] for the summary.
        first = scenes[0]
        eff_episodes = first.n_episodes
        # ``robot_id`` is non-None per raise_on_invalid_suite; coerce for printing.
        robot_id = first.robot_id or "<unset>"
        # When every scene shares one scene.id we print it; otherwise
        # show how many distinct scenes the suite covers.
        scene_ids = {scene.scene.id for scene in scenes}
        scene_summary = next(iter(scene_ids)) if len(scene_ids) == 1 else f"{len(scene_ids)} scenes"
        console.print(
            f"[cyan]suite[/cyan] {suite_id} — robot={robot_id} "
            f"scene={scene_summary} tasks={len(scenes)} "
            f"n_episodes={eff_episodes}"
        )
        console.print(f"[cyan]vla[/cyan]   id={vla_spec.id} weights={vla_spec.weights_uri}")
        console.print(
            f"[cyan]plan[/cyan]  {len(scenes) * eff_episodes} "
            f"episodes ({len(scenes)} tasks x {eff_episodes} reps)"
        )
        return

    out_path = out if out is not None else _default_benchmark_out_path(vla_spec, suite_id)
    out_path.parent.mkdir(parents=True, exist_ok=True)

    from openral_observability.dashboard import attached_dashboard
    from openral_sim.benchmark import run_benchmark

    # attached_dashboard is a no-op when enabled=False — wrap
    # unconditionally so the run_benchmark call is un-duplicated.
    with attached_dashboard(enabled=dashboard, port=dashboard_port):
        result, episodes = run_benchmark(
            scenes,
            suite_id=suite_id,
            vla=vla_spec,
            device=device,
            save_dir=str(save_dir) if save_dir is not None else None,
        )

    out_path.write_text(result.model_dump_json(indent=2))
    avg = result.results.get("avg_success_rate", 0.0)
    console.print(
        f"[green]wrote {out_path}[/green] — avg success "
        f"= {float(avg) if isinstance(avg, (int, float)) else avg:.3f} "
        f"over {len(episodes)} episodes"
    )

    if update_manifest:
        from openral_rskill.loader import resolve_rskill_local_dir
        from openral_sim.benchmark import update_rskill_benchmarks

        # Resolve to the in-tree dir so the surgical write hits
        # rskills/<name>/rskill.yaml even when the user supplied a bare
        # name or a Hub-style repo id. Falls through to a cwd-relative
        # path for Hub-only references with no in-tree shim — the
        # update is then a no-op (FileNotFoundError handled below).
        local_dir = resolve_rskill_local_dir(vla_spec.weights_uri)
        skill_dir = str(local_dir) if local_dir is not None else vla_spec.weights_uri

        try:
            manifest_path = update_rskill_benchmarks(
                skill_dir,
                suite_id,
                float(avg) if isinstance(avg, (int, float)) else 0.0,
            )
            console.print(
                f"[green]updated {manifest_path}[/green] — "
                f"benchmarks.{suite_id} = "
                f"{float(avg) if isinstance(avg, (int, float)) else avg:.3f}"
            )
        except FileNotFoundError as exc:
            console.print(
                f"[yellow]skipped manifest update:[/yellow] {exc} (eval JSON was still written)"
            )

benchmark_scene(config: Path = typer.Option(..., '--config', help='Path to a BenchmarkScene YAML (`scenes/benchmark/<id>.yaml`). DeployScene and SimScene YAMLs are rejected with a redirect — `openral benchmark scene` accepts BenchmarkScene only (scene + task + `n_episodes` + `seed` + `metadata.paper` + `metadata.honest_scope`).'), rskill: str = typer.Option(..., '--rskill', help="rSkill reference — a bare name ('smolvla-libero'), a path ('rskills/smolvla-libero'), or an HF Hub repo id ('OpenRAL/rskill-smolvla-libero'). Raw hf:// is rejected (weights must come from a manifest). The policy adapter id is read from the manifest's `model_family` field."), out: Path | None = typer.Option(None, '--out', help='Output path for the RSkillEvalResult JSON. Defaults to rskills/<dir>/eval/scene_<scene_id>.json derived from the rSkill ref.'), device: str | None = typer.Option(None, '--device', help='Torch device override for the policy (cpu, cuda:0, mps, auto).'), save_dir: Path | None = typer.Option(None, '--save-dir', help='Optional adapter-side artefact directory (videos, traces).'), n_episodes: int | None = typer.Option(None, '--n-episodes', help='Override `BenchmarkScene.n_episodes` (lower for quick smoke runs). The published-protocol value lives in the YAML; this flag is for fast iteration, not for paper-comparison numbers.'), view: bool | None = typer.Option(None, '--view/--no-view', help='Open a passive mujoco.viewer window and stream the rollout in real time (parity with `openral sim run --view`). Default (unset): headless — benchmark eval artefacts and CI/deploy runs are unaffected. Pass --view to require a window (errors loud if unsupported), or --no-view to force offscreen. Incompatible with MUJOCO_GL=egl.'), dry_run: bool = typer.Option(False, '--dry-run', help='Resolve the scene + rSkill and print the planned (task x seed) matrix without running any rollouts. Useful in CI to validate config wiring.'), update_manifest: bool = typer.Option(True, '--update-manifest/--no-update-manifest', help='On success, write the avg_success_rate back into the rSkill manifest at `benchmarks.<scene_id>`. Surgical edit — preserves comments. Only fires for locally-resolvable rSkills.'), write_eval: bool = typer.Option(True, '--write-eval/--no-write-eval', help='Persist the RSkillEvalResult JSON to --out (default rskills/<dir>/eval/scene_<scene_id>.json, a tracked path). Pass --no-write-eval for a fully non-mutating smoke run: the rollout still executes and prints its score, but nothing is written to the rSkill package (implies --no-update-manifest).'), dashboard: bool = typer.Option(False, '--dashboard', help='Boot `openral dashboard` as a child process, point OTel at it, and shut it down on exit (same semantics as `openral sim run --dashboard`).'), dashboard_port: int = typer.Option(4318, '--dashboard-port', help='Port for the spawned dashboard when --dashboard is set.')) -> None

Run a single-scene benchmark and write a validated RSkillEvalResult JSON.

Single-scene sibling of openral benchmark run --suite — accepts exactly one :class:BenchmarkScene YAML and emits the same eval JSON shape so openral benchmark report does not need to distinguish the two entrypoints.

Example

openral benchmark scene \

--config scenes/benchmark/pusht.yaml \

--rskill diffusion-pusht

Source code in python/cli/src/openral_cli/main.py

@benchmark_app.command("scene")
def benchmark_scene(
    config: Path = typer.Option(
        ...,
        "--config",
        help=(
            "Path to a BenchmarkScene YAML "
            "(`scenes/benchmark/<id>.yaml`). DeployScene and SimScene "
            "YAMLs are rejected with a redirect — `openral benchmark "
            "scene` accepts BenchmarkScene only (scene + task + "
            "`n_episodes` + `seed` + `metadata.paper` + "
            "`metadata.honest_scope`)."
        ),
    ),
    rskill: str = typer.Option(
        ...,
        "--rskill",
        help=(
            "rSkill reference — a bare name ('smolvla-libero'), a "
            "path ('rskills/smolvla-libero'), or an HF Hub repo id "
            "('OpenRAL/rskill-smolvla-libero'). "
            "Raw hf:// is rejected (weights must come from a manifest). "
            "The policy adapter id is read from the manifest's `model_family` "
            "field."
        ),
    ),
    out: Path | None = typer.Option(
        None,
        "--out",
        help=(
            "Output path for the RSkillEvalResult JSON. Defaults to "
            "rskills/<dir>/eval/scene_<scene_id>.json derived from the "
            "rSkill ref."
        ),
    ),
    device: str | None = typer.Option(
        None,
        "--device",
        help="Torch device override for the policy (cpu, cuda:0, mps, auto).",
    ),
    save_dir: Path | None = typer.Option(
        None,
        "--save-dir",
        help="Optional adapter-side artefact directory (videos, traces).",
    ),
    n_episodes: int | None = typer.Option(
        None,
        "--n-episodes",
        help=(
            "Override `BenchmarkScene.n_episodes` (lower for quick smoke "
            "runs). The published-protocol value lives in the YAML; this "
            "flag is for fast iteration, not for paper-comparison numbers."
        ),
    ),
    view: bool | None = typer.Option(
        None,
        "--view/--no-view",
        help=(
            "Open a passive mujoco.viewer window and stream the rollout in "
            "real time (parity with `openral sim run --view`). Default "
            "(unset): headless — benchmark eval artefacts and CI/deploy "
            "runs are unaffected. Pass --view to require a window (errors "
            "loud if unsupported), or --no-view to force offscreen. "
            "Incompatible with MUJOCO_GL=egl."
        ),
    ),
    dry_run: bool = typer.Option(
        False,
        "--dry-run",
        help=(
            "Resolve the scene + rSkill and print the planned (task x "
            "seed) matrix without running any rollouts. Useful in CI to "
            "validate config wiring."
        ),
    ),
    update_manifest: bool = typer.Option(
        True,
        "--update-manifest/--no-update-manifest",
        help=(
            "On success, write the avg_success_rate back into the rSkill "
            "manifest at `benchmarks.<scene_id>`. Surgical edit — "
            "preserves comments. Only fires for locally-resolvable rSkills."
        ),
    ),
    write_eval: bool = typer.Option(
        True,
        "--write-eval/--no-write-eval",
        help=(
            "Persist the RSkillEvalResult JSON to --out (default "
            "rskills/<dir>/eval/scene_<scene_id>.json, a tracked path). "
            "Pass --no-write-eval for a fully non-mutating smoke run: the "
            "rollout still executes and prints its score, but nothing is "
            "written to the rSkill package (implies --no-update-manifest)."
        ),
    ),
    dashboard: bool = typer.Option(
        False,
        "--dashboard",
        help=(
            "Boot `openral dashboard` as a child process, point OTel at it, "
            "and shut it down on exit (same semantics as `openral sim run "
            "--dashboard`)."
        ),
    ),
    dashboard_port: int = typer.Option(
        4318,
        "--dashboard-port",
        help="Port for the spawned dashboard when --dashboard is set.",
    ),
) -> None:
    r"""Run a single-scene benchmark and write a validated `RSkillEvalResult` JSON.

    Single-scene sibling of ``openral benchmark run --suite`` — accepts
    exactly one :class:`BenchmarkScene` YAML and emits the same eval JSON
    shape so ``openral benchmark report`` does not need to distinguish
    the two entrypoints.

    Example:
        >>> # openral benchmark scene \\
        >>> #   --config scenes/benchmark/pusht.yaml \\
        >>> #   --rskill diffusion-pusht
    """
    from openral_core import BenchmarkScene, load_scene_strict

    scene = load_scene_strict(str(config), BenchmarkScene)
    if n_episodes is not None:
        scene = scene.model_copy(update={"n_episodes": n_episodes})

    if dry_run:
        # Dry-run validates config wiring only — do not touch the Hub or
        # load weights. Print the raw --rskill argument as-typed.
        console.print(
            f"[cyan]scene[/cyan] {scene.scene.id} — robot={scene.robot_id} "
            f"task={scene.task.id} n_episodes={scene.n_episodes} "
            f"seed={scene.seed}"
        )
        console.print(f"[cyan]vla[/cyan]   rskill={rskill}")
        console.print(
            f"[cyan]plan[/cyan]  {scene.n_episodes} episodes (seeds "
            f"{scene.seed}..{scene.seed + scene.n_episodes - 1})"
        )
        return

    vla_spec = _parse_rskill_cli_arg(rskill)
    out_path = out if out is not None else _default_benchmark_scene_out_path(vla_spec, scene)

    from openral_observability.dashboard import attached_dashboard
    from openral_sim.benchmark import run_benchmark_scene

    with attached_dashboard(enabled=dashboard, port=dashboard_port):
        result, episodes = run_benchmark_scene(
            scene,
            vla_spec,
            device=device,
            save_dir=str(save_dir) if save_dir is not None else None,
            config_path=str(config),
            view=view,
        )

    avg = result.results.get("avg_success_rate", 0.0)
    avg_f = float(avg) if isinstance(avg, (int, float)) else 0.0
    if _persist_scene_eval(result, out_path, write_eval=write_eval):
        console.print(
            f"[green]wrote {out_path}[/green] — avg success "
            f"= {avg_f:.3f} over {len(episodes)} episodes"
        )
    else:
        console.print(
            f"[yellow]--no-write-eval:[/yellow] not persisting result — avg success "
            f"= {avg_f:.3f} over {len(episodes)} episodes (nothing written to the rSkill)"
        )

    if not write_eval:
        # Non-mutating smoke run: skip the manifest writeback too.
        return

    if update_manifest and not _scene_id_is_benchmark_suite(scene.scene.id):
        # The rskill.yaml `benchmarks:` block holds canonical SUITE headlines
        # (RSkillManifest.benchmarks is keyed by the BenchmarkName literal).
        # A single scene whose id is not itself a suite id (e.g. 'metaworld',
        # 'robocasa/PickPlaceCounterToCabinet') has no headline slot — writing
        # it would raise ROSConfigError. The per-scene result is already
        # captured in the eval JSON, so we skip the manifest write rather than
        # crash. (Scenes whose id IS a suite id — pusht, libero_spatial — still
        # update the headline below.)
        console.print(
            f"[yellow]skipped manifest update:[/yellow] scene id {scene.scene.id!r} "
            f"is not a canonical benchmark suite id; per-scene result written to "
            f"{out_path} only (rskill.yaml benchmarks: holds suite headlines)."
        )
    elif update_manifest:
        from openral_rskill.loader import resolve_rskill_local_dir
        from openral_sim.benchmark import update_rskill_benchmarks

        local_dir = resolve_rskill_local_dir(vla_spec.weights_uri)
        skill_dir = str(local_dir) if local_dir is not None else vla_spec.weights_uri
        try:
            manifest_path = update_rskill_benchmarks(
                skill_dir,
                scene.scene.id,
                float(avg) if isinstance(avg, (int, float)) else 0.0,
            )
            console.print(
                f"[green]updated {manifest_path}[/green] — "
                f"benchmarks.{scene.scene.id} = "
                f"{float(avg) if isinstance(avg, (int, float)) else avg:.3f}"
            )
        except FileNotFoundError as exc:
            console.print(
                f"[yellow]skipped manifest update:[/yellow] {exc} (eval JSON was still written)"
            )

calibrate_camera(sensor: str = typer.Option(..., '--sensor', '-s', help='Sensor name as it appears in robot.yaml (e.g. head_color).'), topic: str = typer.Option('', '--topic', help='Override the image topic (default: derived from sensor name).'), chessboard_size: str = typer.Option('8x6', '--chessboard-size', help='Internal corners COLSxROWS of the calibration target.'), square_size: float = typer.Option(0.025, '--square-size', help='Physical size of one chessboard square in metres.'), dry_run: bool = typer.Option(False, '--dry-run', help='Print the command instead of executing it.')) -> None

Calibrate a camera sensor using the ROS 2 camera_calibration package.

Builds and optionally runs::

ros2 run camera_calibration cameracalibrator \
    --size COLSxROWS --square SIZE \
    --ros-args -r image:=TOPIC -r camera_info:=INFO_TOPIC

Requires ros2_camera_calibration to be installed and ROS 2 sourced.

Example

openral calibrate camera --sensor head_color --chessboard-size 8x6 --square-size 0.025

openral calibrate camera --sensor head_color --dry-run

Source code in python/cli/src/openral_cli/main.py

@calibrate_app.command("camera")
def calibrate_camera(
    sensor: str = typer.Option(
        ...,
        "--sensor",
        "-s",
        help="Sensor name as it appears in robot.yaml (e.g. head_color).",
    ),
    topic: str = typer.Option(
        "",
        "--topic",
        help="Override the image topic (default: derived from sensor name).",
    ),
    chessboard_size: str = typer.Option(
        "8x6",
        "--chessboard-size",
        help="Internal corners COLSxROWS of the calibration target.",
    ),
    square_size: float = typer.Option(
        0.025,
        "--square-size",
        help="Physical size of one chessboard square in metres.",
    ),
    dry_run: bool = typer.Option(
        False,
        "--dry-run",
        help="Print the command instead of executing it.",
    ),
) -> None:
    r"""Calibrate a camera sensor using the ROS 2 camera_calibration package.

    Builds and optionally runs::

        ros2 run camera_calibration cameracalibrator \
            --size COLSxROWS --square SIZE \
            --ros-args -r image:=TOPIC -r camera_info:=INFO_TOPIC

    Requires ``ros2_camera_calibration`` to be installed and ROS 2 sourced.

    Example:
        >>> # openral calibrate camera --sensor head_color --chessboard-size 8x6 --square-size 0.025
        >>> # openral calibrate camera --sensor head_color --dry-run
    """
    try:
        cols_str, rows_str = chessboard_size.lower().split("x")
        cols, rows = int(cols_str), int(rows_str)
    except ValueError:
        console.print(
            f"[red]Invalid --chessboard-size '{chessboard_size}'. "
            "Expected format: COLSxROWS, e.g. 8x6[/red]"
        )
        raise typer.Exit(code=1)  # noqa: B904

    # Derive topic names from sensor name if not overridden.
    image_topic = topic or f"/{sensor}/image_raw"
    info_topic = image_topic.replace("/image_raw", "/camera_info").replace(
        "/image_rect_raw", "/camera_info"
    )

    cmd = [
        "ros2",
        "run",
        "camera_calibration",
        "cameracalibrator",
        "--size",
        f"{cols}x{rows}",
        "--square",
        str(square_size),
        "--ros-args",
        "-r",
        f"image:={image_topic}",
        "-r",
        f"camera_info:={info_topic}",
    ]

    if dry_run:
        console.print("[bold]openral calibrate camera[/bold] — dry run:")
        console.print(" ".join(cmd))
        return

    ros2_bin = shutil.which("ros2")
    if ros2_bin is None:
        console.print(
            "[red]ros2 not found. Source your ROS 2 installation first:[/red]\n"
            "  source /opt/ros/<distro>/setup.bash"
        )
        raise typer.Exit(code=1)

    console.print(f"[bold]openral calibrate camera[/bold] — sensor: [cyan]{sensor}[/cyan]")
    console.print(f"  image topic : [dim]{image_topic}[/dim]")
    console.print(f"  camera info : [dim]{info_topic}[/dim]")
    console.print(f"  target size : [dim]{cols}x{rows} squares @ {square_size} m[/dim]")
    console.print("Running camera_calibration … (Ctrl+C to abort)")

    result = subprocess.run(cmd, check=False)
    if result.returncode != 0:
        console.print(f"[red]cameracalibrator exited with code {result.returncode}.[/red]")
        raise typer.Exit(code=result.returncode)

connect(robot: str = typer.Option(..., help='Robot type (so100, g1, ur5e, …)'), port: str = typer.Option('', '--port', help='USB/serial port override, e.g. /dev/ttyUSB0')) -> None

Open a HAL connection to a robot, read one joint state, and disconnect.

Exits 0 on success; exits 1 with an error message on failure.

Supported robots: so100

Example

openral connect --robot so100

openral connect --robot so100 --port /dev/ttyUSB1

Source code in python/cli/src/openral_cli/main.py

@app.command()
def connect(
    robot: str = typer.Option(..., help="Robot type (so100, g1, ur5e, …)"),
    port: str = typer.Option("", "--port", help="USB/serial port override, e.g. /dev/ttyUSB0"),
) -> None:
    """Open a HAL connection to a robot, read one joint state, and disconnect.

    Exits 0 on success; exits 1 with an error message on failure.

    Supported robots: so100

    Example:
        >>> # openral connect --robot so100
        >>> # openral connect --robot so100 --port /dev/ttyUSB1
    """
    if robot == "so100":
        _connect_so100(port or "/dev/ttyUSB0")
    else:
        console.print(f"[red]Unknown robot '{robot}'. Supported: so100[/red]")
        raise typer.Exit(code=1)

dashboard(host: str = typer.Option('127.0.0.1', '--host', help='Bind address. Loopback by default; no auth.'), port: int = typer.Option(4318, '--port', help='HTTP port; serves UI, /api/state, /api/stream, and OTLP/HTTP receiver. Defaults to 4318 (the OTLP/HTTP standard port) rather than 8000 (issue #132) — `mkdocs serve` and most FastAPI demos already squat on 8000.'), log_level: str = typer.Option('warning', '--log-level', help='uvicorn log level (debug | info | warning | error).'), inprocess: str | None = typer.Option(None, '--inprocess', help="Optional shell-quoted command to spawn as a child process with OTEL_EXPORTER_OTLP_ENDPOINT pointed at this dashboard. Pass the whole command as one string (shlex-tokenised), e.g. `--inprocess 'openral sim run --config scenes/benchmark/pusht.yaml --rskill diffusion-pusht'`.")) -> None

Serve the OpenRAL live dashboard.

Source code in python/cli/src/openral_cli/main.py

@app.command(
    "dashboard",
    help=(
        "Serve a live debugging dashboard (read-only) at the given port. "
        "The same port also acts as an OTLP/HTTP receiver, so any "
        "`openral sim run` / `openral deploy run` pointed at "
        "OTEL_EXPORTER_OTLP_ENDPOINT=http://<host>:<port> + "
        "OTEL_EXPORTER_OTLP_PROTOCOL=http/protobuf streams in live. "
        "Works without Jaeger/Tempo running."
    ),
)
def dashboard(
    host: str = typer.Option(
        "127.0.0.1",
        "--host",
        help="Bind address. Loopback by default; no auth.",
    ),
    port: int = typer.Option(
        4318,
        "--port",
        help=(
            "HTTP port; serves UI, /api/state, /api/stream, and OTLP/HTTP "
            "receiver. Defaults to 4318 (the OTLP/HTTP standard port) "
            "rather than 8000 (issue #132) — `mkdocs serve` and most "
            "FastAPI demos already squat on 8000."
        ),
    ),
    log_level: str = typer.Option(
        "warning",
        "--log-level",
        help="uvicorn log level (debug | info | warning | error).",
    ),
    inprocess: str | None = typer.Option(
        None,
        "--inprocess",
        help=(
            "Optional shell-quoted command to spawn as a child process with "
            "OTEL_EXPORTER_OTLP_ENDPOINT pointed at this dashboard. Pass the "
            "whole command as one string (shlex-tokenised), e.g. "
            "`--inprocess 'openral sim run --config scenes/benchmark/pusht.yaml"
            " --rskill diffusion-pusht'`."
        ),
    ),
) -> None:
    """Serve the OpenRAL live dashboard."""
    import shlex

    from openral_observability.dashboard import run_dashboard

    inprocess_cmd = shlex.split(inprocess) if inprocess else None
    run_dashboard(
        host=host,
        port=port,
        inprocess_cmd=inprocess_cmd,
        log_level=log_level,
    )

deploy_list() -> None

List every robot config under deployments/*.yaml.

Each entry is a paste-able --config path for openral deploy run. No hardware touch, no GPU.

Source code in python/cli/src/openral_cli/main.py

@deploy_app.command("list")
def deploy_list() -> None:
    """List every robot config under `deployments/*.yaml`.

    Each entry is a paste-able `--config` path for `openral deploy run`.
    No hardware touch, no GPU.
    """
    from openral_rskill.loader import _find_repo_root_from

    repo_root = _find_repo_root_from(Path(__file__))
    if repo_root is None:
        console.print("[red]Could not locate repo root.[/red]")
        raise typer.Exit(code=1)
    robot_examples = repo_root / "deployments"
    if not robot_examples.is_dir():
        print("<none>")
        return
    configs = sorted(robot_examples.rglob("*.yaml"))
    if not configs:
        print("<none>")
        return
    for cfg in configs:
        print(cfg.relative_to(repo_root))

deploy_run(config: Path = typer.Option(..., '--config', '-c', exists=True, readable=True, dir_okay=False, help='Path to a RobotEnvironment YAML; its robot_id + hal.transport drive the launch.'), robot: str | None = typer.Option(None, '--robot', help='Override the robot_id resolved from --config.'), hal: list[str] | None = typer.Option(None, '--hal', help='Override HAL node params, key=value (repeatable), e.g. --hal port=/dev/ttyUSB1.'), dashboard: bool = typer.Option(True, '--dashboard/--no-dashboard', help='Spawn the live dashboard (default on).'), dashboard_port: int = typer.Option(4318, '--dashboard-port', help='Dashboard OTLP port.')) -> None

Run an rSkill on REAL hardware via the production ROS graph (ADR-0032).

Unlike openral deploy sim, this drives the real hardware HAL: it resolves the robot from --config (a RobotEnvironment) and shells the SAME sim_e2e.launch.py graph with hal_mode:=real — the HAL lifecycle node + C++ safety kernel + reasoner + world state (+ SLAM/Nav2 when the robot declares a lidar). The HAL's connect() fails loudly if no hardware is attached; a simulation-only robot raises ROSCapabilityMismatch (use openral deploy sim). The robot's hal.transport (serial port / robot_ip / fci_ip) is forwarded as HAL node params; --hal wins.

Source code in python/cli/src/openral_cli/main.py

@deploy_app.command("run")
def deploy_run(
    config: Path = typer.Option(  # reason: typer Option idiom
        ...,
        "--config",
        "-c",
        exists=True,
        readable=True,
        dir_okay=False,
        help="Path to a RobotEnvironment YAML; its robot_id + hal.transport drive the launch.",
    ),
    robot: str | None = typer.Option(
        None,
        "--robot",
        help="Override the robot_id resolved from --config.",
    ),
    hal: list[str] | None = typer.Option(
        None,
        "--hal",
        help="Override HAL node params, key=value (repeatable), e.g. --hal port=/dev/ttyUSB1.",
    ),
    dashboard: bool = typer.Option(
        True,
        "--dashboard/--no-dashboard",
        help="Spawn the live dashboard (default on).",
    ),
    dashboard_port: int = typer.Option(
        4318,
        "--dashboard-port",
        help="Dashboard OTLP port.",
    ),
) -> None:
    """Run an rSkill on REAL hardware via the production ROS graph (ADR-0032).

    Unlike `openral deploy sim`, this drives the **real** hardware HAL: it
    resolves the robot from `--config` (a RobotEnvironment) and shells the SAME
    `sim_e2e.launch.py` graph with `hal_mode:=real` — the HAL lifecycle node +
    C++ safety kernel + reasoner + world state (+ SLAM/Nav2 when the robot
    declares a lidar). The HAL's `connect()` fails loudly if no hardware is
    attached; a simulation-only robot raises ROSCapabilityMismatch (use
    `openral deploy sim`). The robot's `hal.transport` (serial `port` /
    `robot_ip` / `fci_ip`) is forwarded as HAL node params; `--hal` wins.
    """
    from openral_core import RobotEnvironment  # reason: defer schema import
    from openral_core.exceptions import ROSCapabilityMismatch  # reason: defer

    from openral_cli.deploy_sim import (  # reason: defer heavy CLI import
        _parse_hal_overrides,
        resolve_launch_invocation,
        run_launch_invocation,
    )

    try:
        env = RobotEnvironment.from_yaml(str(config))
    except (FileNotFoundError, ROSConfigError) as exc:
        console.print(f"[red]config error:[/red] {exc}")
        raise typer.Exit(code=1) from exc

    # RobotEnvironment.hal.transport (serial port / robot_ip / fci_ip) + params
    # become HAL node param overrides; an explicit --hal key=value wins.
    overrides: dict[str, object] = {**env.hal.transport, **env.hal.params}
    overrides.update(_parse_hal_overrides(hal))

    try:
        invocation = resolve_launch_invocation(
            config=None,
            robot_override=robot or env.robot_id,
            dashboard_port=dashboard_port,
            reset_to_pose_service=None,
            hal_param_overrides=overrides,
            hal_mode="real",
            enable_dashboard=dashboard,
        )
    except (ROSConfigError, ROSCapabilityMismatch) as exc:
        console.print(f"[red]deploy run:[/red] {exc}")
        raise typer.Exit(code=1) from exc

    console.print(
        f"[cyan]deploy run[/cyan] {invocation.robot_id} → real HAL "
        f"(hal_mode=real); the HAL's connect() requires the robot to be attached."
    )
    returncode = run_launch_invocation(invocation)
    raise typer.Exit(code=returncode)

detect(output: Path = typer.Option(Path('robot.yaml'), '--output', '-o', help='Output robot.yaml path'), report: Path | None = typer.Option(None, '--report', help='Optional path to dump the raw DetectionReport as JSON.'), dds_timeout: float = typer.Option(5.0, '--dds-timeout', help='DDS topic discovery timeout in seconds'), include: str | None = typer.Option(None, '--include', help='Comma-separated probe names to run (default: all). Choices: usb, dds, gpu, cameras_v4l2, cameras_realsense, network.'), no_write: bool = typer.Option(False, '--no-write', help='Print summary and skip writing robot.yaml'), yes: bool = typer.Option(False, '--yes', '-y', help='Overwrite existing file without prompting')) -> None

Probe the host and emit a complete RobotDescription robot.yaml.

Runs the auto-provisioning flow from openral_detect:

1. Probe USB / DDS / GPU / V4L2 / RealSense / network.
2. Identify the rig (USB VID/PID match or DDS topology). If a known robot is detected, load the canonical robots/<name>/robot.yaml directly; otherwise synthesize a minimal scaffold.
3. Reverse-look up each detected sensor in the catalog so its SensorSpec carries real intrinsics, FOV, encoding, rate.
4. Promote detected GPU / Jetson / Apple Silicon caps onto RobotCapabilities so openral rskill check can match RSkillManifest.runtime / quantization.dtype.

Example

openral detect

openral detect --include gpu,network --no-write

Source code in python/cli/src/openral_cli/main.py

@app.command()
def detect(
    output: Path = typer.Option(
        Path("robot.yaml"), "--output", "-o", help="Output robot.yaml path"
    ),
    report: Path | None = typer.Option(
        None,
        "--report",
        help="Optional path to dump the raw DetectionReport as JSON.",
    ),
    dds_timeout: float = typer.Option(
        5.0, "--dds-timeout", help="DDS topic discovery timeout in seconds"
    ),
    include: str | None = typer.Option(
        None,
        "--include",
        help="Comma-separated probe names to run (default: all). "
        "Choices: usb, dds, gpu, cameras_v4l2, cameras_realsense, network.",
    ),
    no_write: bool = typer.Option(
        False, "--no-write", help="Print summary and skip writing robot.yaml"
    ),
    yes: bool = typer.Option(
        False, "--yes", "-y", help="Overwrite existing file without prompting"
    ),
) -> None:
    """Probe the host and emit a complete RobotDescription robot.yaml.

    Runs the auto-provisioning flow from ``openral_detect``:

    1. Probe USB / DDS / GPU / V4L2 / RealSense / network.
    2. Identify the rig (USB VID/PID match or DDS topology).  If a
       known robot is detected, load the canonical
       ``robots/<name>/robot.yaml`` directly; otherwise synthesize a
       minimal scaffold.
    3. Reverse-look up each detected sensor in the catalog so its
       ``SensorSpec`` carries **real** intrinsics, FOV, encoding, rate.
    4. Promote detected GPU / Jetson / Apple Silicon caps onto
       ``RobotCapabilities`` so ``openral rskill check`` can match
       ``RSkillManifest.runtime`` / ``quantization.dtype``.

    Example:
        >>> # openral detect
        >>> # openral detect --include gpu,network --no-write
    """
    import yaml as _yaml
    from openral_detect import (
        assemble_robot_description,
        detect_hardware,
    )

    include_set: set[str] | None = (
        {p.strip() for p in include.split(",") if p.strip()} if include else None
    )

    console.print("[bold]openral detect[/bold] — probing host …")
    detection = detect_hardware(dds_timeout_s=dds_timeout, include=include_set)

    _render_detection_summary(detection)

    if report is not None:
        report.write_text(detection.model_dump_json(indent=2), encoding="utf-8")
        console.print(f"[green]Wrote[/green] {report} (raw DetectionReport)")

    description = assemble_robot_description(detection)
    yaml_text = _yaml.safe_dump(
        description.model_dump(mode="json"),
        sort_keys=False,
        default_flow_style=False,
    )

    if no_write:
        console.print("\n[dim]--no-write set — printing yaml to stdout:[/dim]\n")
        console.print(yaml_text)
        return

    if output.exists() and not yes:
        overwrite = typer.confirm(f"{output} already exists. Overwrite?", default=False)
        if not overwrite:
            console.print("[yellow]Aborted.[/yellow]")
            raise typer.Exit(code=0)

    output.write_text(yaml_text, encoding="utf-8")
    console.print(f"\n[green]Wrote[/green] {output} (RobotDescription, {description.name})")
    console.print(f"[dim]Next step:[/dim] openral rskill check --robot {output}")

doctor(json: bool = typer.Option(False, '--json', help='Output machine-readable JSON')) -> None

Diagnose the host: Python, OS, ROS 2 distro, GPU, USB devices.

Exits 0 when every check is ok, absent, or info; exits 1 if any check has status fail or missing.

Example

openral doctor

openral doctor --json

Source code in python/cli/src/openral_cli/main.py

@app.command()
def doctor(
    json: bool = typer.Option(False, "--json", help="Output machine-readable JSON"),
) -> None:
    """Diagnose the host: Python, OS, ROS 2 distro, GPU, USB devices.

    Exits 0 when every check is ``ok``, ``absent``, or ``info``; exits 1 if
    any check has status ``fail`` or ``missing``.

    Example:
        >>> # openral doctor
        >>> # openral doctor --json
    """
    checks = _gather_checks()

    if json:
        result = [{"check": c.check, "status": c.status, "details": c.details} for c in checks]
        console.print_json(_json.dumps(result))
    else:
        table = Table(title="openral doctor")
        table.add_column("check", style="bold")
        table.add_column("status")
        table.add_column("details")
        for c in checks:
            style = (
                "green" if c.status == "ok" else "yellow" if c.status in _YELLOW_STATUSES else "red"
            )
            table.add_row(c.check, f"[{style}]{c.status}[/{style}]", c.details)
        console.print(table)

    fatal = {"fail", "missing"}
    if any(c.status in fatal for c in checks):
        raise typer.Exit(code=1)

profile_session(action: str = typer.Argument(..., help='One of: start | stop | view.'), output: Path = typer.Option(Path('./lttng-traces'), '--output', '-o', help='LTTng output directory. Used by start (write here) and view (read from here).'), name: str = typer.Option('openral', '--name', '-n', help='LTTng session name.')) -> None

Drive an LTTng session — start, stop, view.

Source code in python/cli/src/openral_cli/main.py

@profile_app.command(
    "session",
    help=(
        "Start, stop, or view an LTTng session for the realtime hot path. "
        "Requires lttng-tools on PATH. Set OPENRAL_ROS2_TRACING=1 on the "
        "agent process to emit tracepoints; the env var is the runtime gate."
    ),
)
def profile_session(
    action: str = typer.Argument(  # reason: typer Argument idiom
        ...,
        help="One of: start | stop | view.",
    ),
    output: Path = typer.Option(  # reason: typer Option idiom
        Path("./lttng-traces"),
        "--output",
        "-o",
        help="LTTng output directory. Used by start (write here) and view (read from here).",
    ),
    name: str = typer.Option(
        "openral",
        "--name",
        "-n",
        help="LTTng session name.",
    ),
) -> None:
    """Drive an LTTng session — start, stop, view."""
    from openral_observability.tracing_lttng import (
        LttngSessionError,
        start_session,
        stop_session,
        view_session,
    )

    try:
        if action == "start":
            session = start_session(name=name, output_dir=output)
            console.print(
                f"[green]openral profile session start:[/green] "
                f"{session.name} → {session.output_dir}"
            )
            console.print(
                "Run your workload with OPENRAL_ROS2_TRACING=1, then "
                "`openral profile session stop` to flush."
            )
        elif action == "stop":
            stop_session(name=name)
            console.print(f"[green]openral profile session stop:[/green] {name}")
        elif action == "view":
            view_session(output_dir=output)
        else:
            console.print(
                f"[red]openral profile session:[/red] unknown action {action!r}; "
                "expected start | stop | view"
            )
            raise typer.Exit(code=2)
    except LttngSessionError as exc:
        console.print(f"[red]openral profile session:[/red] {exc}")
        raise typer.Exit(code=1) from exc

record(out: Path = typer.Option(..., '--out', '-o', help='Output directory passed to `ros2 bag record -o`.'), profile: str = typer.Option('slim', '--profile', help="Recording profile: 'slim' (default) or 'full'."), storage: str = typer.Option('mcap', '--storage', help='rosbag2 storage backend; mcap is the openral default.'), extra_topic: list[str] = typer.Option([], '--extra-topic', help='Additional topic to record verbatim. Repeatable.'), extra_regex: list[str] = typer.Option([], '--extra-regex', help='Additional regex to OR into --regex. Repeatable.'), dry_run: bool = typer.Option(False, '--dry-run', help='Print the composed argv instead of executing.')) -> None

Wrap ros2 bag record with ADR-0018 F7's slim/full topic presets.

Source code in python/cli/src/openral_cli/main.py

@app.command(
    "record",
    help=(
        "Spawn `ros2 bag record` for the ADR-0018 graph with a slim/full profile. "
        "Requires a sourced ROS 2 install. Use `--dry-run` to print the argv "
        "instead of executing."
    ),
)
def record(
    out: Path = typer.Option(  # reason: typer Option idiom
        ...,
        "--out",
        "-o",
        help="Output directory passed to `ros2 bag record -o`.",
    ),
    profile: str = typer.Option(
        "slim",
        "--profile",
        help="Recording profile: 'slim' (default) or 'full'.",
    ),
    storage: str = typer.Option(
        "mcap",
        "--storage",
        help="rosbag2 storage backend; mcap is the openral default.",
    ),
    extra_topic: list[str] = typer.Option(
        [],
        "--extra-topic",
        help="Additional topic to record verbatim. Repeatable.",
    ),
    extra_regex: list[str] = typer.Option(
        [],
        "--extra-regex",
        help="Additional regex to OR into --regex. Repeatable.",
    ),
    dry_run: bool = typer.Option(
        False,
        "--dry-run",
        help="Print the composed argv instead of executing.",
    ),
) -> None:
    """Wrap `ros2 bag record` with ADR-0018 F7's slim/full topic presets."""
    from openral_observability.replay.cli import run_record

    if profile not in {"slim", "full"}:
        console.print(f"[red]openral record:[/red] unknown profile {profile!r}; expected slim|full")
        raise typer.Exit(code=2)
    try:
        argv, completed = run_record(
            profile=profile,  # type: ignore[arg-type] # reason: validated above against the literal set
            output_dir=out,
            storage=storage,
            extra_topics=extra_topic,
            extra_regex=extra_regex,
            dry_run=dry_run,
        )
    except FileNotFoundError as exc:
        console.print(f"[red]openral record:[/red] {exc}")
        raise typer.Exit(code=1) from exc
    if dry_run:
        print(" ".join(argv))
        return
    assert completed is not None
    if completed.returncode != 0:
        raise typer.Exit(code=completed.returncode)

render_banner(version_str: str, *, width: int | None = None) -> RenderableType

Build the REPL welcome box as a rich renderable (Claude-Code style).

Returns a :class:rich.panel.Panel so the same renderable can be printed to the live console and exported to plain text in tests, independent of terminal/TTY/colour state. The white-bordered rounded box carries OPENRAL v<version> inline in the top border and adapts to width:

• Wide (>= _WIDE_MIN): two columns — the logo mark beside the OPENRAL wordmark with the tagline below on the left; the community links above a horizontal divider above the quick-start commands on the right, split by a vertical divider.
• Narrow: a single stacked column keeping every section; the logo sits beside the wordmark while it fits and stacks above it once it does not.

Parameters:

Name	Type	Description	Default
version_str	str	The installed openral-cli version, rendered as vX.Y.Z.	required
width	int | None	Target terminal width in columns; defaults to a wide layout.	None

Example

from io import StringIO from rich.console import Console con = Console(file=StringIO(), width=120) con.print(render_banner("0.1.0", width=120)) "OPENRAL v0.1.0" in con.file.getvalue() True

Source code in python/cli/src/openral_cli/main.py

def render_banner(version_str: str, *, width: int | None = None) -> RenderableType:
    """Build the REPL welcome box as a rich renderable (Claude-Code style).

    Returns a :class:`rich.panel.Panel` so the same renderable can be printed to
    the live console *and* exported to plain text in tests, independent of
    terminal/TTY/colour state. The white-bordered rounded box carries ``OPENRAL
    v<version>`` inline in the top border and adapts to ``width``:

    * **Wide** (``>= _WIDE_MIN``): two columns — the logo mark beside the OPENRAL
      wordmark with the tagline below on the left; the community links above a
      horizontal divider above the quick-start commands on the right, split by a
      vertical divider.
    * **Narrow**: a single stacked column keeping every section; the logo sits
      beside the wordmark while it fits and stacks above it once it does not.

    Args:
        version_str: The installed ``openral-cli`` version, rendered as ``vX.Y.Z``.
        width: Target terminal width in columns; defaults to a wide layout.

    Example:
        >>> from io import StringIO
        >>> from rich.console import Console
        >>> con = Console(file=StringIO(), width=120)
        >>> con.print(render_banner("0.1.0", width=120))
        >>> "OPENRAL v0.1.0" in con.file.getvalue()
        True
    """
    cols = width if width is not None else _WIDE_MIN
    links = _kv_grid(_LINKS, "bold white")
    commands = _kv_grid(_COMMANDS, "bold cyan")

    body: RenderableType
    if cols >= _WIDE_MIN:
        columns = Table(
            box=MINIMAL,
            show_header=False,
            show_edge=False,
            show_lines=False,
            pad_edge=False,
            padding=(0, 2),
            border_style="dim",
            expand=False,
        )
        columns.add_column(vertical="top")
        columns.add_column(vertical="top")
        columns.add_row(
            _identity(stacked=False),
            Group(links, Rule(style="dim"), commands),
        )
        body = columns
    else:
        body = Group(
            _identity(stacked=cols < _SIDE_BY_SIDE_MIN),
            Text(""),
            links,
            Rule(style="dim"),
            commands,
        )

    # Size the box to its content (expand=False) rather than stretching it to the
    # terminal: a compact box keeps slack on wide terminals and only breaks if the
    # window is later dragged narrower than the box itself (printed output cannot
    # reflow). The layout above is chosen so the content always fits ``cols``.
    return Panel(
        body,
        box=ROUNDED,
        border_style="white",
        title=f"OPENRAL v{version_str}",
        title_align="left",
        padding=(1, 2),
        expand=False,
    )

replay(bag: Path = typer.Argument(..., exists=True, readable=True, help='Path to a rosbag2 directory or a bare .mcap file.'), trace: str | None = typer.Option(None, '--trace', help='32-hex-char trace_id to filter on. Defaults to the busiest one in the bag.'), frame: str | None = typer.Option(None, '--frame', help="Pivot from a written LeRobotDataset frame: '<repo_id>/<episode>/<frame>' (e.g. 'openral/dataset-pick/0/12'). Reads that frame's trace_id and uses it as the join key. Requires --dataset-root; mutually exclusive with --trace."), dataset_root: Path | None = typer.Option(None, '--dataset-root', help='Root directory of the LeRobotDataset that --frame refers to.'), dashboard: str | None = typer.Option(None, '--dashboard', help='Dashboard base URL (e.g. http://127.0.0.1:8000). Omit for bag-only.'), out: Path | None = typer.Option(None, '--out', '-o', help='Write the timeline JSON to this file; print to stdout when omitted.')) -> None

Read a bag, join it with spans by trace_id, emit a JSON timeline.

Source code in python/cli/src/openral_cli/main.py

@app.command(
    "replay",
    help=(
        "Join a rosbag2/.mcap file with OTel spans from the live dashboard "
        "(ADR-0018 F7). Prints a chronological JSON timeline keyed by trace_id; "
        "writes to `--out` when given. `--dashboard` may be omitted for a "
        "bag-only timeline."
    ),
)
def replay(
    bag: Path = typer.Argument(  # reason: typer Argument idiom
        ...,
        exists=True,
        readable=True,
        help="Path to a rosbag2 directory or a bare .mcap file.",
    ),
    trace: str | None = typer.Option(
        None,
        "--trace",
        help="32-hex-char trace_id to filter on. Defaults to the busiest one in the bag.",
    ),
    frame: str | None = typer.Option(
        None,
        "--frame",
        help=(
            "Pivot from a written LeRobotDataset frame: '<repo_id>/<episode>/<frame>' "
            "(e.g. 'openral/dataset-pick/0/12'). Reads that frame's trace_id and uses "
            "it as the join key. Requires --dataset-root; mutually exclusive with --trace."
        ),
    ),
    dataset_root: Path | None = typer.Option(
        None,
        "--dataset-root",
        help="Root directory of the LeRobotDataset that --frame refers to.",
    ),
    dashboard: str | None = typer.Option(
        None,
        "--dashboard",
        help="Dashboard base URL (e.g. http://127.0.0.1:8000). Omit for bag-only.",
    ),
    out: Path | None = typer.Option(
        None,
        "--out",
        "-o",
        help="Write the timeline JSON to this file; print to stdout when omitted.",
    ),
) -> None:
    """Read a bag, join it with spans by trace_id, emit a JSON timeline."""
    from openral_observability.replay.cli import run_replay, write_timeline

    # ISSUE-109 pivot — resolve --frame into a concrete trace_id off the
    # dataset before the join. Done here (not in run_replay) so the
    # openral_observability replay module stays free of the lerobot dep.
    if frame is not None:
        if trace is not None:
            console.print("[red]openral replay:[/red] --frame and --trace are mutually exclusive")
            raise typer.Exit(code=2)
        if dataset_root is None:
            console.print("[red]openral replay:[/red] --frame requires --dataset-root")
            raise typer.Exit(code=2)
        trace = _resolve_frame_trace_id(frame, dataset_root)

    result = run_replay(bag_path=bag, trace_id=trace, dashboard_url=dashboard)
    if out is not None:
        write_timeline(result, out)
        console.print(
            f"[green]openral replay:[/green] wrote {len(result.timeline)} entries to {out}"
        )
        if result.trace_id:
            console.print(f"trace_id: {result.trace_id}")
        return
    print(_json.dumps(result.to_json(), indent=2, sort_keys=False))

robot_vendor_urdf(robot_id: str = typer.Argument(..., help='OpenRAL robot id; names the output file (e.g. ur5e → ur5e.urdf).'), upstream: str = typer.Option(..., '--upstream', help="Upstream source: 'rd:<robot_descriptions module>' (xacro, expanded via xacrodoc) or 'file:<path>' to an already-flat URDF."), out: Path = typer.Option(..., '--out', help="Output directory; '<robot_id>.urdf' is written here."), rename: list[str] | None = typer.Option(None, '--rename', help="Joint-name rename as 'PATTERN=>REPL' (regex re.sub). Repeatable — applied in order (so100/so101 take 6, gr1/h1 take 1). Defaults to the per-robot rule (openarm strips its 'openarm_' prefix)."), raw_text: bool = typer.Option(False, '--raw-text/--no-raw-text', help='Copy an already-flat upstream URDF verbatim and apply --rename to the raw XML (no yourdfpy round-trip), preserving package:// mesh paths byte-for-byte (so100/so101/gr1/h1).')) -> None

Expand an upstream description to a flat, committed URDF (ADR-0058).

Source code in python/cli/src/openral_cli/main.py

@robot_app.command("vendor-urdf")
def robot_vendor_urdf(
    robot_id: str = typer.Argument(
        ...,
        help="OpenRAL robot id; names the output file (e.g. ur5e → ur5e.urdf).",
    ),
    upstream: str = typer.Option(
        ...,
        "--upstream",
        help=(
            "Upstream source: 'rd:<robot_descriptions module>' (xacro, expanded "
            "via xacrodoc) or 'file:<path>' to an already-flat URDF."
        ),
    ),
    out: Path = typer.Option(
        ...,
        "--out",
        help="Output directory; '<robot_id>.urdf' is written here.",
    ),
    rename: list[str] | None = typer.Option(
        None,
        "--rename",
        help=(
            "Joint-name rename as 'PATTERN=>REPL' (regex re.sub). Repeatable — "
            "applied in order (so100/so101 take 6, gr1/h1 take 1). Defaults to "
            "the per-robot rule (openarm strips its 'openarm_' prefix)."
        ),
    ),
    raw_text: bool = typer.Option(
        False,
        "--raw-text/--no-raw-text",
        help=(
            "Copy an already-flat upstream URDF verbatim and apply --rename to "
            "the raw XML (no yourdfpy round-trip), preserving package:// mesh "
            "paths byte-for-byte (so100/so101/gr1/h1)."
        ),
    ),
) -> None:
    """Expand an upstream description to a flat, committed URDF (ADR-0058)."""
    from openral_cli.robot import vendor_urdf

    rename_pairs: list[tuple[str, str]] | None = None
    if rename:
        rename_pairs = []
        for spec in rename:
            if "=>" not in spec:
                raise typer.BadParameter("--rename must be 'PATTERN=>REPL'", param_hint="--rename")
            pat, _, repl = spec.partition("=>")
            rename_pairs.append((pat, repl))
    written = vendor_urdf(
        robot_id, upstream=upstream, out_dir=out, rename=rename_pairs, raw_text=raw_text
    )
    typer.echo(f"Wrote {written}")

rskill_check(rskill_id: str | None = typer.Argument(None, metavar='RSKILL_ID', help='rSkill to check — bare in-tree name, path (rskills/<name>), or HF Hub repo id (e.g. OpenRAL/rskill-smolvla-libero). Omit to walk every installed / in-tree rSkill (walk-all mode).'), robot: Path = typer.Option(Path('robot.yaml'), '--robot', help='Path to a RobotDescription yaml (typically the output of `openral detect`).'), rskills_dir: Path = typer.Option(Path('rskills'), '--rskills-dir', help='(Walk-all mode) in-tree rSkills directory to scan in addition to the installed registry. Skipped if it does not exist.'), json_output: bool = typer.Option(False, '--json', help='Output the CompatibilityReport as JSON.')) -> None

Report whether one (or every) rSkill will run on the current host.

Two modes:

• openral rskill check <rskill_id> resolves the id the same way as openral rskill list / openral benchmark run --rskill <id> (in-tree, installed registry, or HF Hub) and prints a per-section breakdown — embodiment, capability flags, GPU runtime, GPU dtype, sensors, actuators.
• openral rskill check (no arg) walks every installed / in-tree rSkill via openral_detect.check_installed_rskills and prints a one-row-per-rSkill compatibility table.

Exits 1 if any non-informational section fails (single-rSkill mode) or if any installed rSkill is incompatible (walk-all mode); exits 0 otherwise.

Example

openral rskill check smolvla-libero

openral rskill check OpenRAL/rskill-smolvla-libero --robot /tmp/robot.yaml --json

openral rskill check # walk-all

Source code in python/cli/src/openral_cli/main.py

@rskill_app.command("check")
def rskill_check(
    rskill_id: str | None = typer.Argument(
        None,
        metavar="RSKILL_ID",
        help=(
            "rSkill to check — bare in-tree name, path (rskills/<name>), "
            "or HF Hub repo id (e.g. OpenRAL/rskill-smolvla-libero). "
            "Omit to walk every installed / in-tree rSkill (walk-all mode)."
        ),
    ),
    robot: Path = typer.Option(
        Path("robot.yaml"),
        "--robot",
        help="Path to a RobotDescription yaml (typically the output of `openral detect`).",
    ),
    rskills_dir: Path = typer.Option(
        Path("rskills"),
        "--rskills-dir",
        help=(
            "(Walk-all mode) in-tree rSkills directory to scan in addition to "
            "the installed registry. Skipped if it does not exist."
        ),
    ),
    json_output: bool = typer.Option(
        False, "--json", help="Output the CompatibilityReport as JSON."
    ),
) -> None:
    """Report whether one (or every) rSkill will run on the current host.

    Two modes:

    * ``openral rskill check <rskill_id>`` resolves the id the same way as
      ``openral rskill list`` / ``openral benchmark run --rskill <id>``
      (in-tree, installed registry, or HF Hub) and prints a per-section
      breakdown — embodiment, capability flags, GPU runtime, GPU dtype,
      sensors, actuators.
    * ``openral rskill check`` (no arg) walks every installed / in-tree
      rSkill via `openral_detect.check_installed_rskills` and prints a
      one-row-per-rSkill compatibility table.

    Exits 1 if any non-informational section fails (single-rSkill mode)
    or if any installed rSkill is incompatible (walk-all mode); exits 0
    otherwise.

    Example:
        >>> # openral rskill check smolvla-libero
        >>> # openral rskill check OpenRAL/rskill-smolvla-libero --robot /tmp/robot.yaml --json
        >>> # openral rskill check                                              # walk-all
    """
    import json as _json_mod

    from openral_core.schemas import RobotDescription
    from openral_detect import check_installed_rskills, check_single_rskill

    if not robot.exists():
        console.print(
            f"[red]Robot description not found:[/red] {robot}\n"
            "Run [bold]openral detect[/bold] first."
        )
        raise typer.Exit(code=1)

    description = RobotDescription.from_yaml(str(robot))

    if rskill_id is not None:
        single_report = check_single_rskill(rskill_id, description)
        single_row = single_report.rows[0]
        if json_output:
            console.print_json(_json_mod.dumps(single_report.model_dump(mode="json")))
        else:
            _render_single_rskill_table(single_row, description.name)
        if not single_row.compatible:
            raise typer.Exit(code=1)
        return

    # ── Walk-all (no-arg) ─────────────────────────────────────────────────────
    walk_dir = rskills_dir if rskills_dir.is_dir() else None
    report = check_installed_rskills(description, rskills_dir=walk_dir)
    if json_output:
        console.print_json(_json_mod.dumps(report.model_dump(mode="json")))
    else:
        _render_walk_all_table(report, description.name)

    if report.incompatible:
        raise typer.Exit(code=1)

rskill_install(hub_id: str = typer.Argument(..., metavar='HUB_ID', help='HF Hub repository, e.g. OpenRAL/rskill-smolvla-libero'), revision: str = typer.Option('', '--revision', '-r', help='Git commit SHA or branch to pin (recommended for reproducibility).'), force: bool = typer.Option(False, '--force', '-f', help='Re-download even if cached files already exist.'), non_commercial: bool = typer.Option(False, '--non-commercial', help='Declare non-commercial research intent (relaxes NVIDIA non-commercial guard).'), yes: bool = typer.Option(False, '--yes', '-y', help='Skip confirmation prompt for proprietary or non-commercial licenses.')) -> None

Download an rSkill from the HF Hub, validate it, and register it locally.

Fetches rskill.yaml from the repository, validates the manifest, surfaces the license to the terminal, then downloads the weights snapshot into the local HF Hub cache (~/.cache/openral/rskills/).

The rSkill is registered in ~/.local/share/openral/rskills.json and can be listed with openral rskill list.

Example

openral rskill install OpenRAL/rskill-smolvla-libero

openral rskill install OpenRAL/rskill-smolvla-libero --revision abc1234

Source code in python/cli/src/openral_cli/main.py

@rskill_app.command("install")
def rskill_install(
    hub_id: str = typer.Argument(
        ...,
        metavar="HUB_ID",
        help="HF Hub repository, e.g. OpenRAL/rskill-smolvla-libero",
    ),
    revision: str = typer.Option(
        "",
        "--revision",
        "-r",
        help="Git commit SHA or branch to pin (recommended for reproducibility).",
    ),
    force: bool = typer.Option(
        False,
        "--force",
        "-f",
        help="Re-download even if cached files already exist.",
    ),
    non_commercial: bool = typer.Option(
        False,
        "--non-commercial",
        help="Declare non-commercial research intent (relaxes NVIDIA non-commercial guard).",
    ),
    yes: bool = typer.Option(
        False,
        "--yes",
        "-y",
        help="Skip confirmation prompt for proprietary or non-commercial licenses.",
    ),
) -> None:
    """Download an rSkill from the HF Hub, validate it, and register it locally.

    Fetches ``rskill.yaml`` from the repository, validates the manifest,
    surfaces the license to the terminal, then downloads the weights snapshot
    into the local HF Hub cache (``~/.cache/openral/rskills/``).

    The rSkill is registered in ``~/.local/share/openral/rskills.json`` and
    can be listed with ``openral rskill list``.

    Example:
        >>> # openral rskill install OpenRAL/rskill-smolvla-libero
        >>> # openral rskill install OpenRAL/rskill-smolvla-libero --revision abc1234
    """
    from openral_rskill.loader import rSkill

    # ── Step 0: a HF repo id needs an `org/name` shape. A bare name (the most
    # common paste mistake) otherwise 404s against a non-existent top-level repo
    # — fail fast with the canonical suggestion instead of a raw Hub stack trace.
    if "/" not in hub_id:
        suggestion = f"{_DEFAULT_RSKILL_ORG}/{hub_id}"
        console.print(
            f"[red]Not a Hub repo id:[/red] '{hub_id}' has no org prefix (expected `org/name`)."
        )
        console.print(f"  Did you mean:  [cyan]openral rskill install {suggestion}[/cyan]")
        console.print(f"  Or find it:    [cyan]openral rskill search {hub_id}[/cyan]")
        raise typer.Exit(code=1)

    console.print(f"[bold]openral rskill install[/bold] — fetching [cyan]{hub_id}[/cyan] …")

    # ── Step 1: fetch manifest only (to surface license before downloading weights)
    try:
        from huggingface_hub import hf_hub_download
        from openral_core.schemas import RSkillManifest
    except ImportError as exc:
        console.print(f"[red]Missing dependency:[/red] {exc}")
        raise typer.Exit(code=1)  # noqa: B904

    try:
        manifest_path = hf_hub_download(
            repo_id=hub_id,
            filename="rskill.yaml",
            revision=revision or None,
        )
        manifest = RSkillManifest.from_yaml(manifest_path)
    except Exception as exc:  # reason: surface any download/parse error to user
        console.print(f"[red]Failed to fetch manifest:[/red] {exc}")
        if "404" in str(exc) or "Repository Not Found" in str(exc):
            bare = hub_id.rsplit("/", 1)[-1]
            console.print(f"  Browse available skills: [cyan]openral rskill search {bare}[/cyan]")
        raise typer.Exit(code=1)  # noqa: B904

    # ── Step 2: display license + confirm if non-permissive
    _display_license_banner(manifest.name, manifest.license.value, manifest.version, console)

    _permissive = {"apache-2.0", "mit", "bsd"}
    if manifest.license.value not in _permissive and not yes:
        confirmed = typer.confirm("Proceed with installation?", default=False)
        if not confirmed:
            console.print("[yellow]Aborted.[/yellow]")
            raise typer.Exit(code=0)

    # ── Step 3: full install (snapshot download + registry)
    console.print("  Downloading weights snapshot …")
    try:
        pkg = rSkill.from_pretrained(
            hub_id,
            revision=revision or None,
            force_download=force,
            commercial_use=not non_commercial,
        )
    except Exception as exc:  # reason: surface ROSConfigError + network errors
        console.print(f"[red]Install failed:[/red] {exc}")
        raise typer.Exit(code=1)  # noqa: B904

    console.print(
        f"[green]Installed[/green] [bold]{pkg.manifest.name}[/bold] "
        f"v{pkg.manifest.version} → {pkg.local_dir}"
    )
    if not revision:
        console.print(
            "[yellow]Tip:[/yellow] Pin a revision for reproducibility: "
            f"openral rskill install {hub_id} --revision <sha>"
        )

rskill_list(json: bool = typer.Option(False, '--json', help='Output machine-readable JSON')) -> None

List every available rSkill — in-tree (rskills/) + HF-Hub-installed.

Each row shows the source so users can tell at a glance which entries are paste-able as --rskill <name> (in-tree) versus which need a HF Hub install first. JSON output keeps the same fields.

Source code in python/cli/src/openral_cli/main.py

@rskill_app.command("list")
def rskill_list(
    json: bool = typer.Option(False, "--json", help="Output machine-readable JSON"),
) -> None:
    """List every available rSkill — in-tree (``rskills/``) + HF-Hub-installed.

    Each row shows the source so users can tell at a glance which entries
    are paste-able as ``--rskill <name>`` (in-tree) versus which
    need a HF Hub install first. JSON output keeps the same fields.
    """
    import json as _json_mod

    from openral_rskill.loader import discover_intree_rskills, rSkill

    intree = discover_intree_rskills()
    try:
        installed = rSkill.list_installed()
    except Exception as exc:  # reason: surface corrupt registry error
        console.print(f"[red]Failed to read installed registry:[/red] {exc}")
        raise typer.Exit(code=1)  # noqa: B904

    def _bare_name_from_repo_id(repo_id: str) -> str:
        """Recover the bare rskill name that the loader would resolve to.

        Mirrors `_candidate_local_paths`: strip the org prefix and the
        `rskill-`/`rskill_` prefix so the URI matches the in-tree form.
        """
        tail = repo_id.rsplit("/", 1)[-1]
        return tail.removeprefix("rskill-").removeprefix("rskill_") or repo_id

    if json:
        data = [
            {
                "source": "in-tree",
                "name": name,
                "repo_id": m.name,
                "version": m.version,
                "model_family": m.model_family,
                "role": str(m.role),
                "license": m.license.value,
                "embodiment_tags": list(m.embodiment_tags),
                "uri": name,
            }
            for name, m in intree
        ] + [
            {
                "source": "installed",
                "name": _bare_name_from_repo_id(e.repo_id),
                "repo_id": e.repo_id,
                "version": e.version,
                "model_family": None,
                "role": e.role,
                "license": e.license,
                "embodiment_tags": list(e.embodiment_tags),
                "uri": _bare_name_from_repo_id(e.repo_id),
                "installed_at": e.installed_at,
            }
            for e in installed
        ]
        console.print_json(_json_mod.dumps(data))
        return

    if not intree and not installed:
        console.print(
            "[dim]No rSkills available. Drop one under rskills/<name>/ or run: "
            "openral rskill install <hub-id>[/dim]"
        )
        return

    table = Table(title="Available rSkills")
    table.add_column("source", style="dim")
    table.add_column("name / repo_id", style="cyan bold")
    table.add_column("version")
    table.add_column("family")
    table.add_column("license")
    table.add_column("embodiment_tags")
    table.add_column("paste-able --rskill")

    _permissive = {"apache-2.0", "mit", "bsd"}
    for name, m in intree:
        lic = m.license.value
        lic_color = "green" if lic in _permissive else "yellow"
        table.add_row(
            "in-tree",
            name,
            m.version,
            m.model_family or "—",
            f"[{lic_color}]{lic}[/{lic_color}]",
            ", ".join(m.embodiment_tags) or "—",
            name,
        )
    for entry in installed:
        lic_color = "green" if entry.license in _permissive else "yellow"
        bare = _bare_name_from_repo_id(entry.repo_id)
        table.add_row(
            "installed",
            entry.repo_id,
            entry.version,
            "—",
            f"[{lic_color}]{entry.license}[/{lic_color}]",
            ", ".join(entry.embodiment_tags) or "—",
            bare,
        )
    console.print(table)

rskill_new(rskill_id: str = typer.Argument(..., metavar='ID', help='Local rSkill id, convention <policy>-<task> e.g. pi05-pick-cube.'), out_dir: Path | None = typer.Option(None, '--out-dir', help='Destination directory. Defaults to rskills/<ID>/ under the cwd.'), owner: str | None = typer.Option(None, '--owner', help="HF Hub owner segment for the manifest 'name' field."), license_: str | None = typer.Option(None, '--license', help='One of: apache-2.0 | mit | bsd | permissive_research | nvidia_non_commercial | proprietary | unknown.'), embodiment_tag: str | None = typer.Option(None, '--embodiment-tag', help='One of the canonical EmbodimentTag literals (see CLAUDE.md §6.4).'), family: str | None = typer.Option(None, '--family', '-f', help="Policy family — one of act | smolvla | pi05 | xvla | diffusion. Sets model_family / chunk_size / dtype / latency budget from the matching in-tree reference manifest so a fresh ACT scaffold doesn't ship pi0.5 numbers. Inferred from --from-hf when set; interactively prompted otherwise."), from_hf: str | None = typer.Option(None, '--from-hf', help="HF Hub repo id (e.g. 'Deepkar/libero-test-act' or 'hf://Deepkar/libero-test-act'). Fetches the checkpoint's config.json, infers the family, and pre-fills chunk_size, sensors_required, state_contract.dim, image_preprocessing aliases, and weights_uri. Eliminates manual rewriting after scaffold."), yes: bool = typer.Option(False, '--yes', '-y', help='Skip interactive prompts and accept the defaults (for scripting / CI).'), overwrite: bool = typer.Option(False, '--overwrite', help='Replace an existing destination directory instead of refusing.')) -> None

Scaffold a new local rSkill from rskills/template/.

Three modes:

• --from-hf <owner/repo> — most intuitive. Fetches the checkpoint's config.json and pre-fills model_family / chunk_size / sensors_required / state_contract / image_preprocessing.aliases / weights_uri from the real Hub-side contract. No more hand-rewriting after scaffold.
• --family <act|smolvla|pi05|xvla|diffusion> — family-aware defaults without Hub introspection. Use when you know the family but the weights live somewhere else (private repo, local mirror).
• Neither — interactive. Prompts for --owner / --license / --embodiment-tag / --family (and offers --from-hf as a one-shot alternative). Pass --yes to skip all prompts and accept the defaults (your-org / apache-2.0 / franka_panda / no-family).

The generated manifest is round-tripped through RSkillManifest.from_yaml and rSkill.from_yaml so a malformed scaffold fails at scaffold-time, not on first load.

Example

openral rskill new act-libero --from-hf Deepkar/libero-test-act

openral rskill new pi05-pick-cube --family pi05 --embodiment-tag franka_panda

openral rskill new act-aloha-insertion --owner foo --embodiment-tag aloha

Source code in python/cli/src/openral_cli/main.py

@rskill_app.command("new")
def rskill_new(
    rskill_id: str = typer.Argument(
        ...,
        metavar="ID",
        help="Local rSkill id, convention <policy>-<task> e.g. pi05-pick-cube.",
    ),
    out_dir: Path | None = typer.Option(
        None,
        "--out-dir",
        help="Destination directory. Defaults to rskills/<ID>/ under the cwd.",
    ),
    owner: str | None = typer.Option(
        None,
        "--owner",
        help="HF Hub owner segment for the manifest 'name' field.",
    ),
    license_: str | None = typer.Option(
        None,
        "--license",
        help=(
            "One of: apache-2.0 | mit | bsd | permissive_research | "
            "nvidia_non_commercial | proprietary | unknown."
        ),
    ),
    embodiment_tag: str | None = typer.Option(
        None,
        "--embodiment-tag",
        help="One of the canonical EmbodimentTag literals (see CLAUDE.md §6.4).",
    ),
    family: str | None = typer.Option(
        None,
        "--family",
        "-f",
        help=(
            "Policy family — one of act | smolvla | pi05 | xvla | diffusion. "
            "Sets model_family / chunk_size / dtype / latency budget from the "
            "matching in-tree reference manifest so a fresh ACT scaffold "
            "doesn't ship pi0.5 numbers. Inferred from --from-hf when set; "
            "interactively prompted otherwise."
        ),
    ),
    from_hf: str | None = typer.Option(
        None,
        "--from-hf",
        help=(
            "HF Hub repo id (e.g. 'Deepkar/libero-test-act' or "
            "'hf://Deepkar/libero-test-act'). Fetches the checkpoint's "
            "config.json, infers the family, and pre-fills chunk_size, "
            "sensors_required, state_contract.dim, image_preprocessing aliases, "
            "and weights_uri. Eliminates manual rewriting after scaffold."
        ),
    ),
    yes: bool = typer.Option(
        False,
        "--yes",
        "-y",
        help="Skip interactive prompts and accept the defaults (for scripting / CI).",
    ),
    overwrite: bool = typer.Option(
        False,
        "--overwrite",
        help="Replace an existing destination directory instead of refusing.",
    ),
) -> None:
    """Scaffold a new local rSkill from ``rskills/template/``.

    Three modes:

    - ``--from-hf <owner/repo>`` — most intuitive. Fetches the
      checkpoint's ``config.json`` and pre-fills ``model_family`` /
      ``chunk_size`` / ``sensors_required`` / ``state_contract`` /
      ``image_preprocessing.aliases`` / ``weights_uri`` from the real
      Hub-side contract. No more hand-rewriting after scaffold.
    - ``--family <act|smolvla|pi05|xvla|diffusion>`` — family-aware
      defaults without Hub introspection. Use when you know the family
      but the weights live somewhere else (private repo, local mirror).
    - Neither — interactive. Prompts for ``--owner`` / ``--license`` /
      ``--embodiment-tag`` / ``--family`` (and offers ``--from-hf`` as
      a one-shot alternative). Pass ``--yes`` to skip all prompts and
      accept the defaults (your-org / apache-2.0 / franka_panda /
      no-family).

    The generated manifest is round-tripped through
    `RSkillManifest.from_yaml` and `rSkill.from_yaml`
    so a malformed scaffold fails at scaffold-time, not on first load.

    Example:
        >>> # openral rskill new act-libero --from-hf Deepkar/libero-test-act
        >>> # openral rskill new pi05-pick-cube --family pi05 --embodiment-tag franka_panda
        >>> # openral rskill new act-aloha-insertion --owner foo --embodiment-tag aloha
    """
    from typing import get_args

    from openral_core.schemas import EmbodimentTag, RSkillLicensePosture

    from openral_cli._rskill_scaffolder import scaffold_rskill

    valid_tags = list(get_args(EmbodimentTag))
    valid_licenses = [v.value for v in RSkillLicensePosture]

    resolved_owner = _resolve_or_prompt(
        owner,
        prompt=f"HF Hub owner (e.g. your username or org) [{_DEFAULT_OWNER}]",
        default=_DEFAULT_OWNER,
        skip_prompt=yes,
    )
    resolved_license = _resolve_or_prompt(
        license_,
        prompt=f"License posture [{_DEFAULT_LICENSE}]",
        default=_DEFAULT_LICENSE,
        skip_prompt=yes,
    )
    resolved_embodiment = _resolve_or_prompt(
        embodiment_tag,
        prompt=f"Embodiment tag (canonical robot id) [{_DEFAULT_EMBODIMENT}]",
        default=_DEFAULT_EMBODIMENT,
        skip_prompt=yes,
    )

    try:
        license_enum = RSkillLicensePosture(resolved_license)
    except ValueError as exc:
        console.print(
            f"[red]Invalid license:[/red] {resolved_license!r}. "
            f"Valid values: {', '.join(valid_licenses)}"
        )
        raise typer.Exit(code=1) from exc

    if resolved_embodiment not in valid_tags:
        console.print(
            f"[red]Invalid embodiment_tag:[/red] {resolved_embodiment!r}. "
            f"Valid values: {', '.join(valid_tags)}"
        )
        raise typer.Exit(code=1)

    resolved_family, intel_patch = _resolve_family_and_patch(
        family=family, from_hf=from_hf, yes=yes
    )

    resolved_out = out_dir if out_dir is not None else Path("rskills") / rskill_id

    try:
        result = scaffold_rskill(
            rskill_id,
            out_dir=resolved_out,
            owner=resolved_owner,
            license_=license_enum,
            embodiment_tag=cast(EmbodimentTag, resolved_embodiment),
            family=resolved_family,
            patch=intel_patch,
            overwrite=overwrite,
        )
    except ROSConfigError as exc:
        console.print(f"[red]Scaffold failed:[/red] {exc}")
        raise typer.Exit(code=1) from exc

    console.print(f"[green]Scaffolded[/green] [cyan]{rskill_id}[/cyan] → {result}")
    if intel_patch is not None:
        console.print(
            "[dim]Next steps: edit description / README.md, optionally adjust "
            "image_preprocessing.flip_180 for your scene, add eval/<benchmark>.json "
            "results, then publish with tools/rskill_publisher.py.[/dim]"
        )
    else:
        console.print(
            "[dim]Next steps: edit rskill.yaml (weights_uri / chunk_size / "
            "sensors_required / image_preprocessing), update README.md, add "
            "eval/<benchmark>.json results, then publish with "
            "tools/rskill_publisher.py. Tip: pass `--from-hf <owner/repo>` next "
            "time to auto-fill the manifest from a published checkpoint.[/dim]"
        )

rskill_search(query: str = typer.Argument('', metavar='[QUERY]', help='Free-text query matched against rSkill repo ids on the Hub.'), kind: str = typer.Option('', '--kind', help='Filter by manifest kind (vla, ros_action, detector, …).'), role: str = typer.Option('', '--role', help='Filter by control role (s0, s1, s2).'), embodiment: str = typer.Option('', '--embodiment', help='Filter by embodiment tag (e.g. franka_panda).'), license_: str = typer.Option('', '--license', help='Filter by license posture (e.g. apache-2.0).'), limit: int = typer.Option(50, '--limit', help='Max OpenRAL repos to inspect.'), json: bool = typer.Option(False, '--json', help='Output machine-readable JSON.')) -> None

Search the OpenRAL HF Hub org for installable rSkills (ADR-0055 D4).

Lists every OpenRAL/* repo whose rskill.yaml manifest validates and matches the optional facet filters, so the printed ids are paste-able into openral rskill install <repo_id>. The HF Hub is the index — there is no bespoke catalog service. Repos without a valid manifest are skipped and the count surfaced.

Example

openral rskill search aloha

openral rskill search --kind detector --license apache-2.0

Source code in python/cli/src/openral_cli/main.py

@rskill_app.command("search")
def rskill_search(
    query: str = typer.Argument(
        "",
        metavar="[QUERY]",
        help="Free-text query matched against rSkill repo ids on the Hub.",
    ),
    kind: str = typer.Option(
        "", "--kind", help="Filter by manifest kind (vla, ros_action, detector, …)."
    ),
    role: str = typer.Option("", "--role", help="Filter by control role (s0, s1, s2)."),
    embodiment: str = typer.Option(
        "", "--embodiment", help="Filter by embodiment tag (e.g. franka_panda)."
    ),
    license_: str = typer.Option(
        "", "--license", help="Filter by license posture (e.g. apache-2.0)."
    ),
    limit: int = typer.Option(50, "--limit", help="Max OpenRAL repos to inspect."),
    json: bool = typer.Option(False, "--json", help="Output machine-readable JSON."),
) -> None:
    """Search the OpenRAL HF Hub org for installable rSkills (ADR-0055 D4).

    Lists every ``OpenRAL/*`` repo whose ``rskill.yaml`` manifest validates and
    matches the optional facet filters, so the printed ids are paste-able into
    ``openral rskill install <repo_id>``. The HF Hub is the index — there is no
    bespoke catalog service. Repos without a valid manifest are skipped and the
    count surfaced.

    Example:
        >>> # openral rskill search aloha
        >>> # openral rskill search --kind detector --license apache-2.0
    """
    import json as _json_mod

    try:
        from huggingface_hub import HfApi
    except ImportError as exc:
        console.print(f"[red]Missing dependency:[/red] {exc}")
        raise typer.Exit(code=1)  # noqa: B904

    try:
        models = list(
            HfApi().list_models(author=_DEFAULT_RSKILL_ORG, search=query or None, limit=limit)
        )
    except Exception as exc:  # reason: surface Hub/network errors to the user
        console.print(f"[red]Search failed:[/red] {exc}")
        raise typer.Exit(code=1)  # noqa: B904

    rows: list[tuple[str, RSkillManifest]] = []
    skipped = 0
    for model in models:
        manifest = _load_hub_rskill_manifest(model.id)
        if manifest is None:
            skipped += 1
        elif _rskill_matches_filters(
            manifest, kind=kind, role=role, embodiment=embodiment, license_=license_
        ):
            rows.append((model.id, manifest))

    if json:
        console.print_json(
            _json_mod.dumps(
                [
                    {
                        "repo_id": repo_id,
                        "name": m.name,
                        "version": m.version,
                        "kind": m.kind,
                        "role": m.role,
                        "license": m.license.value,
                        "embodiment_tags": list(m.embodiment_tags),
                        "description": m.description,
                    }
                    for repo_id, m in rows
                ]
            )
        )
        return

    _render_rskill_search_results(rows, skipped, query)

sensor_list(vendor: str = typer.Option('', '--vendor', help='Filter by vendor (lowercase, e.g. intel, orbbec, livox).'), modality: str = typer.Option('', '--modality', help='Filter by sensor modality (e.g. rgb, depth, lidar_2d, point_cloud).'), kind: str = typer.Option('', '--kind', help="Filter by kind: 'sensor' or 'bundle'."), json_output: bool = typer.Option(False, '--json', help='Emit machine-readable JSON instead of a table.')) -> None

List every sensor registered in the openral sensor catalog.

Example

openral sensor list

openral sensor list --vendor intel

openral sensor list --modality lidar_2d --json

Source code in python/cli/src/openral_cli/main.py

@sensor_app.command("list")
def sensor_list(
    vendor: str = typer.Option(
        "",
        "--vendor",
        help="Filter by vendor (lowercase, e.g. intel, orbbec, livox).",
    ),
    modality: str = typer.Option(
        "",
        "--modality",
        help="Filter by sensor modality (e.g. rgb, depth, lidar_2d, point_cloud).",
    ),
    kind: str = typer.Option(
        "",
        "--kind",
        help="Filter by kind: 'sensor' or 'bundle'.",
    ),
    json_output: bool = typer.Option(
        False,
        "--json",
        help="Emit machine-readable JSON instead of a table.",
    ),
) -> None:
    """List every sensor registered in the openral sensor catalog.

    Example:
        >>> # openral sensor list
        >>> # openral sensor list --vendor intel
        >>> # openral sensor list --modality lidar_2d --json
    """
    # Imported lazily so `openral --help` doesn't pay the side-effect import cost.
    from openral_core.schemas import SensorModality
    from openral_sensors import CATALOG

    modality_enum: SensorModality | None = None
    if modality:
        try:
            modality_enum = SensorModality(modality)
        except ValueError:
            valid = ", ".join(m.value for m in SensorModality)
            console.print(f"[red]Unknown --modality {modality!r}.  Valid: {valid}[/red]")
            raise typer.Exit(code=1) from None

    kind_filter: str | None = None
    if kind:
        if kind not in ("sensor", "bundle"):
            console.print(f"[red]--kind must be 'sensor' or 'bundle', got {kind!r}.[/red]")
            raise typer.Exit(code=1)
        kind_filter = kind

    entries = CATALOG.filter(
        vendor=vendor or None,
        modality=modality_enum,
        kind=kind_filter,  # type: ignore[arg-type]  # reason: narrowed above
    )

    if json_output:
        payload = [
            {
                "id": e.id,
                "vendor": e.vendor,
                "model": e.model,
                "kind": e.kind,
                "modalities": [m.value for m in e.modalities],
                "description": e.description,
                "docs_url": e.docs_url,
            }
            for e in entries
        ]
        console.print_json(_json.dumps(payload))
        return

    if not entries:
        console.print("[yellow]No sensors match the requested filters.[/yellow]")
        return

    table = Table(title=f"openral sensor catalog ({len(entries)} entries)")
    table.add_column("id", style="cyan", no_wrap=True)
    table.add_column("kind")
    table.add_column("modalities", style="magenta")
    table.add_column("description")
    for e in entries:
        table.add_row(
            e.id,
            e.kind,
            ",".join(m.value for m in e.modalities),
            e.description,
        )
    console.print(table)

sensor_show(sensor_id: str = typer.Argument(..., metavar='SENSOR_ID', help='Catalog id, e.g. intel/realsense_d435i or slamtec/rplidar_a2'), name: str = typer.Option('sensor', '--name', help='Instance name passed to the factory (used as topic / frame prefix).'), parent_frame: str = typer.Option('base_link', '--parent-frame', help='tf2 parent frame for the static transform.'), json_output: bool = typer.Option(False, '--json', help='Emit the resolved SensorSpec / SensorBundle as JSON.')) -> None

Resolve a catalog entry to a concrete SensorSpec / SensorBundle.

Example

openral sensor show intel/realsense_d435i --name head

openral sensor show slamtec/rplidar_a2 --json

Source code in python/cli/src/openral_cli/main.py

@sensor_app.command("show")
def sensor_show(
    sensor_id: str = typer.Argument(
        ...,
        metavar="SENSOR_ID",
        help="Catalog id, e.g. intel/realsense_d435i or slamtec/rplidar_a2",
    ),
    name: str = typer.Option(
        "sensor",
        "--name",
        help="Instance name passed to the factory (used as topic / frame prefix).",
    ),
    parent_frame: str = typer.Option(
        "base_link",
        "--parent-frame",
        help="tf2 parent frame for the static transform.",
    ),
    json_output: bool = typer.Option(
        False,
        "--json",
        help="Emit the resolved SensorSpec / SensorBundle as JSON.",
    ),
) -> None:
    """Resolve a catalog entry to a concrete ``SensorSpec`` / ``SensorBundle``.

    Example:
        >>> # openral sensor show intel/realsense_d435i --name head
        >>> # openral sensor show slamtec/rplidar_a2 --json
    """
    from openral_sensors import CATALOG

    try:
        entry = CATALOG.get(sensor_id)
    except KeyError as exc:
        console.print(f"[red]{exc.args[0]}[/red]")
        raise typer.Exit(code=1) from None

    resolved = entry.factory(name=name, parent_frame=parent_frame)

    if json_output:
        console.print_json(resolved.model_dump_json(indent=2))
        return

    console.print(f"[bold cyan]{entry.id}[/bold cyan]  ({entry.kind})")
    console.print(f"  vendor      : [magenta]{entry.vendor}[/magenta]")
    console.print(f"  model       : [magenta]{entry.model}[/magenta]")
    console.print(f"  modalities  : {', '.join(m.value for m in entry.modalities)}")
    console.print(f"  description : [dim]{entry.description}[/dim]")
    if entry.docs_url:
        console.print(f"  docs_url    : [dim]{entry.docs_url}[/dim]")
    console.print()
    console.print("[bold]Resolved:[/bold]")
    console.print_json(resolved.model_dump_json(indent=2))

Hardware Abstraction Layer

openral HAL — Hardware Abstraction Layer public API.

Public surface: - HAL: structural Protocol every adapter must satisfy (RFC §8.2). - RosControlHAL: ros2_control-backed adapter. - SO100FollowerHAL: lerobot SO-100 follower arm adapter. - SO100_DESCRIPTION / so100_with_sensors: canonical SO-100 description and a catalog-backed factory that resolves a sensor loadout (issue #23). - SO100DigitalTwin / SO100DigitalTwinConfig: in-process simulator. - UR5eHAL / UR5e_DESCRIPTION / ur5e_with_sensors. - UR10eHAL / UR10e_DESCRIPTION / ur10e_with_sensors. - UR5eRealHAL / UR5e_REAL_DESCRIPTION: real-hardware UR5e via ros2_control + ur_robot_driver (URCap / RTDE). - UR10eRealHAL / UR10e_REAL_DESCRIPTION: real-hardware UR10e via the same driver. - FrankaPandaHAL / FRANKA_PANDA_DESCRIPTION / franka_panda_with_sensors. - FrankaPandaRealHAL / FRANKA_PANDA_REAL_DESCRIPTION: real-hardware adapter over franka_ros2 / FCI (issue #56). - SawyerRealHAL / SAWYER_DESCRIPTION / SAWYER_REAL_DESCRIPTION: real-hardware adapter over intera_sdk / sawyer_robot (issue #57). - AlohaHAL / ALOHA_DESCRIPTION / ALOHA_REAL_DESCRIPTION: real-hardware adapter over the Trossen Interbotix XS SDK (issue #58). - AlohaMujocoHAL: MuJoCo-backed digital twin for the bimanual ALOHA, driving gym-aloha's bimanual_viperx_transfer_cube.xml with the same 14-DoF action layout as AlohaHAL. - SO100MujocoHAL: MuJoCo-backed digital twin for the SO-100 follower, driving the mujoco_menagerie MJCF with the same 6-DoF action layout as SO100FollowerHAL. - G1MujocoHAL / G1_DESCRIPTION: MuJoCo-backed digital twin for the Unitree G1 humanoid (29-DoF, no S0 cerebellum — contract validator only; the robot falls without gravity disabled). Real-HW G1 HAL is planned under the M2 milestone (CLAUDE.md §6.2). - H1MujocoHAL / H1_DESCRIPTION: MuJoCo-backed digital twin for the Unitree H1 humanoid (19-DoF — predecessor to the G1 with a simpler 5-DoF per leg, 1-DoF torso, 4-DoF per arm layout). Same contract-validator scope as G1MujocoHAL; real-HW H1 HAL also waits on the M2 S0 cerebellum. - Rizon4MujocoHAL / RIZON4_DESCRIPTION: MuJoCo-backed digital twin for the Flexiv Rizon 4 (7-DoF cobot with whole-body force sensitivity). Structurally identical to the UR / Franka sim HALs. - OpenArmMujocoHAL / OPENARM_DESCRIPTION: MuJoCo-backed digital twin for the Enactic OpenArm v2 bimanual (2 x (7-DoF arm + 1 gripper) = 16-DoF action). Fresh HALBase subclass because the bimanual layout doesn't fit MujocoArmHAL, but otherwise trivial — v2's native <position> actuators (per-class PD baked into the MJCF) let the HAL just write target → ctrl and step. The v2 MJCF is fetched lazily by openral_hal._openarm_v2_assets; will simplify back to robot_descriptions once upstream bumps its pin. - SimTransport: typed in-memory ros2_control transport for unit tests.

All *_REAL_DESCRIPTION constants are derived from their sim siblings via openral_hal._real_description.make_real_description; they share the same hal entrypoints (hal.sim / hal.real, ADR-0031) and differ only in sdk_kind.

PANDA_MOBILE_BASE_JOINT_NAMES: list[str] = list(PANDA_MOBILE_DESCRIPTION.base_joints or []) module-attribute

Base joints (forward, side, yaw), in order — from robots/panda_mobile/robot.yaml base_joints.

PANDA_MOBILE_JOINT_NAMES: list[str] = [*PANDA_MOBILE_BASE_JOINT_NAMES, *_PANDA_MOBILE_ARM_JOINT_NAMES, _PANDA_MOBILE_GRIPPER_JOINT_NAME] module-attribute

Full 11-DoF joint order: base (3) + arm (7) + gripper (1). Matches robots/panda_mobile/robot.yaml after ADR-0028a.

AlohaHAL(*, left_arm_controller: str = _DEFAULT_LEFT_ARM_CONTROLLER, right_arm_controller: str = _DEFAULT_RIGHT_ARM_CONTROLLER, left_gripper_controller: str = _DEFAULT_LEFT_GRIPPER_CONTROLLER, right_gripper_controller: str = _DEFAULT_RIGHT_GRIPPER_CONTROLLER, joint_state_topic: str = _DEFAULT_ALOHA_JOINT_STATE_TOPIC, estop_topic: str = _DEFAULT_ALOHA_ESTOP_TOPIC, publish_fn: _PublishFn | None = None, state_fn: _StateFn | None = None, staleness_limit_s: float = 0.2)

Bases: HALBase

HAL adapter for a physical Trossen ALOHA bimanual setup.

The 14-DoF action vector is split internally:

• indices 0:6 → left arm joint trajectory (left_arm controller)
• index 6 → left gripper position (left gripper controller)
• indices 7:13 → right arm joint trajectory (right_arm controller)
• index 13 → right gripper position (right gripper controller)

The split allows the per-arm Interbotix controllers to handle each side independently (trajectory smoothing, gravity comp), while openral upstream layers see one unified 14-DoF action.

Parameters:

Name	Type	Description	Default
left_arm_controller	str	ros2_control joint trajectory controller for the left ViperX. Defaults to "left_arm/arm_controller".	_DEFAULT_LEFT_ARM_CONTROLLER
right_arm_controller	str	same for the right arm.	_DEFAULT_RIGHT_ARM_CONTROLLER
left_gripper_controller	str	gripper position controller for the left gripper. Defaults to "left_arm/gripper_controller".	_DEFAULT_LEFT_GRIPPER_CONTROLLER
right_gripper_controller	str	same for the right gripper.	_DEFAULT_RIGHT_GRIPPER_CONTROLLER
joint_state_topic	str	ROS 2 topic publishing aggregated joint state.	_DEFAULT_ALOHA_JOINT_STATE_TOPIC
estop_topic	str	ROS 2 topic the safety supervisor publishes to on estop(). A watchdog node downstream is expected to call the per-arm torque-disable service.	_DEFAULT_ALOHA_ESTOP_TOPIC
publish_fn	_PublishFn | None	Callable forwarding messages to ROS 2 topics. Production use injects the lifecycle node's publisher; tests inject :class:SimTransport.publish.	None
state_fn	_StateFn | None	Callable returning the latest raw joint state as a dict. Production use injects the lifecycle node's subscriber callback; tests inject :class:SimTransport.state.	None
staleness_limit_s	float	Maximum age of a read_state() reading before :class:ROSPerceptionStale is raised.	0.2

Example

from openral_hal.aloha import AlohaHAL from openral_hal.sim_transport import SimTransport transport = SimTransport(n_joints=14) hal = AlohaHAL( ... publish_fn=transport.publish, ... state_fn=transport.state, ... ) hal.connect() hal.description.name 'aloha_bimanual' hal.disconnect()

Initialise the adapter; no transport is opened until connect().

Source code in python/hal/src/openral_hal/aloha.py

def __init__(
    self,
    *,
    left_arm_controller: str = _DEFAULT_LEFT_ARM_CONTROLLER,
    right_arm_controller: str = _DEFAULT_RIGHT_ARM_CONTROLLER,
    left_gripper_controller: str = _DEFAULT_LEFT_GRIPPER_CONTROLLER,
    right_gripper_controller: str = _DEFAULT_RIGHT_GRIPPER_CONTROLLER,
    joint_state_topic: str = _DEFAULT_ALOHA_JOINT_STATE_TOPIC,
    estop_topic: str = _DEFAULT_ALOHA_ESTOP_TOPIC,
    publish_fn: _PublishFn | None = None,
    state_fn: _StateFn | None = None,
    staleness_limit_s: float = 0.2,
) -> None:
    """Initialise the adapter; no transport is opened until ``connect()``."""
    self.description: RobotDescription = ALOHA_REAL_DESCRIPTION
    self._left_arm_controller = left_arm_controller
    self._right_arm_controller = right_arm_controller
    self._left_gripper_controller = left_gripper_controller
    self._right_gripper_controller = right_gripper_controller
    self._joint_state_topic = joint_state_topic
    self._estop_topic = estop_topic
    self._publish_fn: _PublishFn = publish_fn or _default_publish
    self._state_fn: _StateFn | None = state_fn
    self._staleness_limit_s = staleness_limit_s

    self._connected: bool = False
    self._last_state_time: float = 0.0
    self._joint_names: list[str] = list(_ALOHA_JOINT_NAMES)

connect() -> None

Open the transport to the four Interbotix ros2_control controllers.

Raises:

Type	Description
ROSRuntimeError	If already connected.

Source code in python/hal/src/openral_hal/aloha.py

def connect(self) -> None:
    """Open the transport to the four Interbotix ros2_control controllers.

    Raises:
        ROSRuntimeError: If already connected.
    """
    if self._connected:
        raise ROSRuntimeError("AlohaHAL is already connected.")
    log.info(
        "hal.connect",
        robot=self.description.name,
        left_arm=self._left_arm_controller,
        right_arm=self._right_arm_controller,
    )
    self._connected = True
    self._last_state_time = time.monotonic()

disconnect() -> None

Close the transport. Idempotent.

Source code in python/hal/src/openral_hal/aloha.py

def disconnect(self) -> None:
    """Close the transport.  Idempotent."""
    if not self._connected:
        return
    log.info("hal.disconnect", robot=self.description.name)
    self._connected = False

estop() -> None

Trigger an emergency stop on both arms.

Publishes a structured estop message; downstream watchdog node calls the per-arm Interbotix torque-disable service.

Raises:

Type	Description
ROSEStopRequested	Always.

Source code in python/hal/src/openral_hal/aloha.py

def estop(self) -> None:
    """Trigger an emergency stop on both arms.

    Publishes a structured estop message; downstream watchdog node
    calls the per-arm Interbotix torque-disable service.

    Raises:
        ROSEStopRequested: Always.
    """
    log.critical(
        "hal.estop",
        robot=self.description.name,
        estop_topic=self._estop_topic,
    )
    with contextlib.suppress(Exception):
        self._publish_fn(
            self._estop_topic,
            {"reason": "openral_estop", "robot": self.description.name},
        )
    self._connected = False
    raise ROSEStopRequested(
        f"Emergency stop triggered on ALOHA bimanual ('{self.description.name}')."
    )

read_state() -> JointState

Return the latest joint state for all 14 description joints.

Raises:

Type	Description
ROSRuntimeError	If not connected.
ROSPerceptionStale	If the last reading is older than staleness_limit_s.

Source code in python/hal/src/openral_hal/aloha.py

def read_state(self) -> JointState:
    """Return the latest joint state for all 14 description joints.

    Raises:
        ROSRuntimeError: If not connected.
        ROSPerceptionStale: If the last reading is older than
            ``staleness_limit_s``.
    """
    self._require_connected("read_state")
    age = time.monotonic() - self._last_state_time
    if age > self._staleness_limit_s:
        raise ROSPerceptionStale(
            f"Joint state is {age:.3f} s old (limit {self._staleness_limit_s} s)."
        )

    n = len(self._joint_names)
    raw: dict[str, object] = {} if self._state_fn is None else self._state_fn()

    def _floats(key: str) -> list[float]:
        val = raw.get(key)
        if isinstance(val, list):
            return [float(v) for v in val]
        return [0.0] * n

    return JointState(
        name=list(self._joint_names),
        position=_floats("position"),
        velocity=_floats("velocity"),
        effort=_floats("effort"),
        stamp_ns=int(time.time_ns()),
    )

send_action(action: Action) -> None

Forward an action chunk to the four Interbotix controllers.

The 14-D action is split four ways: each arm gets a 6-D joint trajectory and each gripper gets a 1-D position command.

Raises:

Type	Description
ROSRuntimeError	If not connected.
ROSConfigError	If action.control_mode is not JOINT_POSITION or the joint count is wrong.

Source code in python/hal/src/openral_hal/aloha.py

def send_action(self, action: Action) -> None:
    """Forward an action chunk to the four Interbotix controllers.

    The 14-D action is split four ways: each arm gets a 6-D joint
    trajectory and each gripper gets a 1-D position command.

    Raises:
        ROSRuntimeError: If not connected.
        ROSConfigError: If ``action.control_mode`` is not
            ``JOINT_POSITION`` or the joint count is wrong.
    """
    self._require_connected("send_action")
    if action.control_mode != ControlMode.JOINT_POSITION:
        raise ROSConfigError(
            f"AlohaHAL only supports JOINT_POSITION; got {action.control_mode!r}."
        )
    n = len(self._joint_names)
    if not action.joint_targets:
        raise ROSConfigError("AlohaHAL.send_action requires non-empty joint_targets.")
    for step_idx, step in enumerate(action.joint_targets):
        if len(step) != n:
            raise ROSConfigError(
                f"action.joint_targets[{step_idx}] has {len(step)} values "
                f"but ALOHA exposes {n} joints."
            )

    # Index layout — see class docstring.
    left_arm_chunk = [step[0:6] for step in action.joint_targets]
    right_arm_chunk = [step[7:13] for step in action.joint_targets]
    left_gripper_targets = [step[6] for step in action.joint_targets]
    right_gripper_targets = [step[13] for step in action.joint_targets]

    self._publish_fn(
        f"/{self._left_arm_controller}/joint_trajectory",
        {
            "control_mode": action.control_mode,
            "horizon": action.horizon,
            "joint_targets": left_arm_chunk,
            "stamp_ns": action.stamp_ns,
        },
    )
    self._publish_fn(
        f"/{self._right_arm_controller}/joint_trajectory",
        {
            "control_mode": action.control_mode,
            "horizon": action.horizon,
            "joint_targets": right_arm_chunk,
            "stamp_ns": action.stamp_ns,
        },
    )
    self._publish_fn(
        f"/{self._left_gripper_controller}/command",
        {"position": left_gripper_targets[-1], "stamp_ns": action.stamp_ns},
    )
    self._publish_fn(
        f"/{self._right_gripper_controller}/command",
        {"position": right_gripper_targets[-1], "stamp_ns": action.stamp_ns},
    )
    log.debug(
        "hal.send_action",
        robot=self.description.name,
        horizon=action.horizon,
    )

AlohaMujocoHAL(*, mjcf_path: str | None = None, settle_steps: int = 1, gravity_enabled: bool = True, staleness_limit_s: float = 0.5)

Bases: MujocoArmHAL

HAL adapter for the Trossen ALOHA bimanual setup (MuJoCo digital twin).

Thin manifest-driven wrapper around :class:MujocoArmHAL; all wiring (MJCF URI, joint→qpos/actuator maps, two PASSTHROUGH grippers with mirror_actuator_index for the antisymmetric finger pair, keyframe seeding) lives in :data:ALOHA_DESCRIPTION.sim (ADR-0023).

Public surface mirrors :class:AlohaHAL: a 14-DoF :class:openral_core.Action with the left arm 6 + left gripper 1 + right arm 6 + right gripper 1 layout. Gripper values are positive-finger metres in [0.021, 0.057] (passthrough); MuJoCo's ctrlrange clips out-of-range commands.

Parameters:

Name	Type	Description	Default
mjcf_path	str | None	Optional override for the MJCF file. When None, the file is resolved through the gym_aloha: URI scheme from :data:ALOHA_DESCRIPTION.assets.mjcf.	None
settle_steps	int	Number of MuJoCo physics steps per :meth:send_action call.	1
gravity_enabled	bool	When False, gravity is zeroed at connect() time for deterministic closed-loop tests.	True
staleness_limit_s	float	Maximum age of a cached state.	0.5

Example

from openral_hal import AlohaMujocoHAL # doctest: +SKIP hal = AlohaMujocoHAL(gravity_enabled=False) # doctest: +SKIP hal.connect() # doctest: +SKIP state = hal.read_state() # doctest: +SKIP len(state.position) # 14 joints # doctest: +SKIP 14 hal.disconnect() # doctest: +SKIP

Initialise the adapter; the MJCF is not loaded until connect().

Source code in python/hal/src/openral_hal/aloha.py

def __init__(
    self,
    *,
    mjcf_path: str | None = None,
    settle_steps: int = 1,
    gravity_enabled: bool = True,
    staleness_limit_s: float = 0.5,
) -> None:
    """Initialise the adapter; the MJCF is not loaded until ``connect()``."""
    self._init_from_description(
        ALOHA_DESCRIPTION,
        mjcf_path=mjcf_path,
        settle_steps=settle_steps,
        gravity_enabled=gravity_enabled,
        staleness_limit_s=staleness_limit_s,
    )

FrankaPandaHAL(*, mjcf_path: str | None = None, settle_steps: int = 1, gravity_enabled: bool = True, staleness_limit_s: float = 0.5)

Bases: MujocoArmHAL

HAL adapter for the Franka Emika Panda (MuJoCo-backed simulation).

The HAL exposes 8 joints to upper layers — the 7 arm joints plus a synthetic panda_gripper channel reported in [0, 1] (0 = closed, 1 = fully open). Internally the gripper command is mapped to the MJCF tendon actuator's [0, 255] range, and the reported gripper position is the sum of the two finger qpos values normalised by 0.08 m.

Parameters:

Name	Type	Description	Default
mjcf_path	str | None	Optional override for the MJCF file path. When None, the file is fetched lazily from robot_descriptions (mujoco_menagerie/franka_emika_panda/panda.xml).	None
settle_steps	int	Number of MuJoCo physics steps performed in :meth:send_action.	1
gravity_enabled	bool	When False, gravity is zeroed at connect() time for deterministic closed-loop tests.	True
staleness_limit_s	float	Maximum age of a cached state.	0.5

Example

from openral_hal import FrankaPandaHAL # doctest: +SKIP hal = FrankaPandaHAL(gravity_enabled=False) # doctest: +SKIP hal.connect() # doctest: +SKIP state = hal.read_state() # doctest: +SKIP len(state.position) # 7 arm + 1 gripper # doctest: +SKIP 8 hal.disconnect() # doctest: +SKIP

Initialise the Panda HAL; no MuJoCo state is created until connect().

All wiring (MJCF URI, joint indices, gripper config) lives in :data:FRANKA_PANDA_DESCRIPTION.sim (ADR-0023).

Source code in python/hal/src/openral_hal/franka_panda.py

def __init__(
    self,
    *,
    mjcf_path: str | None = None,
    settle_steps: int = 1,
    gravity_enabled: bool = True,
    staleness_limit_s: float = 0.5,
) -> None:
    """Initialise the Panda HAL; no MuJoCo state is created until ``connect()``.

    All wiring (MJCF URI, joint indices, gripper config) lives in
    :data:`FRANKA_PANDA_DESCRIPTION.sim` (ADR-0023).
    """
    self._init_from_description(
        FRANKA_PANDA_DESCRIPTION,
        mjcf_path=mjcf_path,
        settle_steps=settle_steps,
        gravity_enabled=gravity_enabled,
        staleness_limit_s=staleness_limit_s,
    )

FrankaPandaRealHAL(*, fci_ip: str = '172.16.0.2', controller_name: str = _DEFAULT_FRANKA_CONTROLLER, joint_state_topic: str = _DEFAULT_FRANKA_JOINT_STATE_TOPIC, command_topic: str | None = None, error_recovery_topic: str = _DEFAULT_FRANKA_ESTOP_TOPIC, publish_fn: _PublishFn | None = None, state_fn: _StateFn | None = None, staleness_limit_s: float = 0.2)

HAL adapter for a physical Franka Emika Panda over the FCI.

The adapter wraps :class:RosControlHAL and adds Franka-specific configuration: the FCI hostname, the franka_ros2 controller name, and an explicit error-recovery topic used by the safety supervisor after estop().

Parameters:

Name	Type	Description	Default
fci_ip	str	Hostname or IP of the Franka FCI (the robot's "control" port, typically 172.16.0.2 for the default lab subnet). Required; libfranka will refuse to connect without it. Stored as metadata only — the actual TCP connection is opened by franka_hardware inside the lifecycle node.	'172.16.0.2'
controller_name	str	Name of the ros2_control joint trajectory controller exposed by franka_ros2. Defaults to "franka_arm_controller".	_DEFAULT_FRANKA_CONTROLLER
joint_state_topic	str	ROS 2 topic publishing sensor_msgs/JointState. Defaults to "/joint_states".	_DEFAULT_FRANKA_JOINT_STATE_TOPIC
command_topic	str | None	ROS 2 topic for joint trajectory commands. Defaults to "/<controller_name>/joint_trajectory" (set by :class:RosControlHAL).	None
error_recovery_topic	str	ROS 2 topic the safety supervisor publishes to after handling an :class:ROSEStopRequested so the FCI clears its reflex state. Defaults to "/error_recovery/goal".	_DEFAULT_FRANKA_ESTOP_TOPIC
publish_fn	_PublishFn | None	Callable forwarding messages to ROS 2 topics. Production use injects the lifecycle node's publisher; tests inject :class:SimTransport.publish.	None
state_fn	_StateFn | None	Callable returning the latest raw joint state as a dict. Production use injects the lifecycle node's subscriber callback; tests inject :class:SimTransport.state.	None
staleness_limit_s	float	Maximum age of a read_state() reading before :class:ROSPerceptionStale is raised. Defaults to 0.2 s (tighter than the RosControlHAL default because the FCI feedback lands at 1 kHz).	0.2

Raises:

Type	Description
ROSConfigError	If fci_ip is empty / whitespace.

Example

from openral_hal.franka_panda_real import FrankaPandaRealHAL from openral_hal.sim_transport import SimTransport transport = SimTransport(n_joints=8) hal = FrankaPandaRealHAL( ... fci_ip="172.16.0.2", ... publish_fn=transport.publish, ... state_fn=transport.state, ... ) hal.connect() hal.description.name 'franka_panda' hal.disconnect()

Initialise the adapter; no TCP connection is opened until connect().

Source code in python/hal/src/openral_hal/franka_panda_real.py

def __init__(
    self,
    *,
    fci_ip: str = "172.16.0.2",
    controller_name: str = _DEFAULT_FRANKA_CONTROLLER,
    joint_state_topic: str = _DEFAULT_FRANKA_JOINT_STATE_TOPIC,
    command_topic: str | None = None,
    error_recovery_topic: str = _DEFAULT_FRANKA_ESTOP_TOPIC,
    publish_fn: _PublishFn | None = None,
    state_fn: _StateFn | None = None,
    staleness_limit_s: float = 0.2,
) -> None:
    """Initialise the adapter; no TCP connection is opened until ``connect()``."""
    if not fci_ip or not fci_ip.strip():
        raise ROSConfigError(
            "FrankaPandaRealHAL requires a non-empty fci_ip "
            "(the robot's FCI hostname or IP, e.g. '172.16.0.2')."
        )
    self._fci_ip = fci_ip
    self._controller_name = controller_name
    self._error_recovery_topic = error_recovery_topic
    self._publish_fn: _PublishFn | None = publish_fn

    self._inner = RosControlHAL(
        FRANKA_PANDA_REAL_DESCRIPTION,
        controller_name=controller_name,
        joint_state_topic=joint_state_topic,
        command_topic=command_topic,
        publish_fn=publish_fn,
        state_fn=state_fn,
        staleness_limit_s=staleness_limit_s,
    )

controller_name: str property

Name of the ros2_control joint trajectory controller.

description: RobotDescription property

Normative :class:RobotDescription for the Franka Panda.

fci_ip: str property

Hostname / IP of the FCI; consumed by franka_hardware.

connect() -> None

Open the ROS 2 transport to the franka_ros2 controller.

The actual libfranka TCP socket is owned by franka_hardware inside the lifecycle node; this call only attaches the adapter to the injected publisher / subscriber pair.

Raises:

Type	Description
ROSRuntimeError	If already connected.

Source code in python/hal/src/openral_hal/franka_panda_real.py

def connect(self) -> None:
    """Open the ROS 2 transport to the ``franka_ros2`` controller.

    The actual ``libfranka`` TCP socket is owned by ``franka_hardware``
    inside the lifecycle node; this call only attaches the adapter to
    the injected publisher / subscriber pair.

    Raises:
        ROSRuntimeError: If already connected.
    """
    log.info(
        "hal.connect",
        robot=self.description.name,
        fci_ip=self._fci_ip,
        controller=self._controller_name,
    )
    self._inner.connect()

disconnect() -> None

Close the ROS 2 transport. Idempotent.

Source code in python/hal/src/openral_hal/franka_panda_real.py

def disconnect(self) -> None:
    """Close the ROS 2 transport.  Idempotent."""
    self._inner.disconnect()

estop() -> None

Trigger an emergency stop on the FCI.

Publishes a zero-velocity hold to the controller, marks the inner adapter disconnected, and raises :class:ROSEStopRequested so the safety supervisor can log the incident. The supervisor is responsible for calling error_recovery (via error_recovery_topic) before re-arming the robot.

Raises:

Type	Description
ROSEStopRequested	Always.

Source code in python/hal/src/openral_hal/franka_panda_real.py

def estop(self) -> None:
    """Trigger an emergency stop on the FCI.

    Publishes a zero-velocity hold to the controller, marks the inner
    adapter disconnected, and raises :class:`ROSEStopRequested` so the
    safety supervisor can log the incident.  The supervisor is
    responsible for calling ``error_recovery`` (via
    ``error_recovery_topic``) before re-arming the robot.

    Raises:
        ROSEStopRequested: Always.
    """
    log.critical(
        "hal.estop",
        robot=self.description.name,
        fci_ip=self._fci_ip,
        recovery_topic=self._error_recovery_topic,
    )
    if self._publish_fn is not None:
        with contextlib.suppress(Exception):
            self._publish_fn(
                self._error_recovery_topic,
                {"reason": "openral_estop", "robot": self.description.name},
            )
    # Mirror SO100/UR estop semantics: drop the connection then raise so
    # subsequent ``read_state`` / ``send_action`` calls fail fast.
    with contextlib.suppress(Exception):
        self._inner.disconnect()
    raise ROSEStopRequested(
        f"Emergency stop triggered on Franka Panda at FCI {self._fci_ip!r}."
    )

read_state() -> JointState

Return the latest joint state for all 8 description joints.

Raises:

Type	Description
ROSRuntimeError	If not connected.
ROSPerceptionStale	If the last reading is older than staleness_limit_s.

Source code in python/hal/src/openral_hal/franka_panda_real.py

def read_state(self) -> JointState:
    """Return the latest joint state for all 8 description joints.

    Raises:
        ROSRuntimeError: If not connected.
        ROSPerceptionStale: If the last reading is older than
            ``staleness_limit_s``.
    """
    return self._inner.read_state()

send_action(action: Action) -> None

Forward an action chunk to the franka_ros2 controller.

Parameters:

Name	Type	Description	Default
action	Action	The :class:Action produced by the Skill or safety shaper.	required

Raises:

Type	Description
ROSRuntimeError	If not connected.
ROSConfigError	If action.control_mode is not in the description's supported_control_modes.

Source code in python/hal/src/openral_hal/franka_panda_real.py

def send_action(self, action: Action) -> None:
    """Forward an action chunk to the ``franka_ros2`` controller.

    Args:
        action: The :class:`Action` produced by the Skill or safety
            shaper.

    Raises:
        ROSRuntimeError: If not connected.
        ROSConfigError: If ``action.control_mode`` is not in the
            description's ``supported_control_modes``.
    """
    self._inner.send_action(action)

G1MujocoHAL(*, mjcf_path: str | None = None, settle_steps: int = 1, gravity_enabled: bool = True, staleness_limit_s: float = 0.5)

Bases: MujocoArmHAL

HAL adapter for the Unitree G1 humanoid (MuJoCo digital twin).

Drives the 29 actuated joints of the menagerie unitree_g1 MJCF through MuJoCo's position-controlled actuators. Exposes a 29-D :class:openral_core.Action matching the joint order in :data:G1_DESCRIPTION (left leg 6 → right leg 6 → waist 3 → left arm 7 → right arm 7).

.. warning::

This HAL does not provide balance. Without an S0 cerebellar controller (CLAUDE.md §6.2, M2 milestone) the robot will fall over under gravity. The closed-loop convergence tests run with gravity_enabled=False for that reason. Use this HAL to validate the action contract / joint indexing / lifecycle, not to roll out a humanoid policy.

Parameters:

Name	Type	Description	Default
mjcf_path	str | None	Optional override for the MJCF file path. When None, the file is fetched lazily from robot_descriptions (mujoco_menagerie/unitree_g1/g1.xml).	None
settle_steps	int	Number of MuJoCo physics steps performed in :meth:send_action. Defaults to 1; raise it in tests that assert the body has converged at the commanded pose.	1
gravity_enabled	bool	When False, gravity is zeroed at connect() time — required for the contract-validation tests because the floating base falls otherwise.	True
staleness_limit_s	float	Maximum age of a cached state.	0.5

Example

from openral_hal import G1MujocoHAL # doctest: +SKIP hal = G1MujocoHAL(gravity_enabled=False) # doctest: +SKIP hal.connect() # doctest: +SKIP state = hal.read_state() # doctest: +SKIP len(state.position) # 29 actuated joints # doctest: +SKIP 29 hal.disconnect() # doctest: +SKIP

Initialise the G1 HAL; no MuJoCo state is created until connect().

All wiring (MJCF URI, floating-base offsets) lives in :data:G1_DESCRIPTION.sim (ADR-0023).

Source code in python/hal/src/openral_hal/g1.py

def __init__(
    self,
    *,
    mjcf_path: str | None = None,
    settle_steps: int = 1,
    gravity_enabled: bool = True,
    staleness_limit_s: float = 0.5,
) -> None:
    """Initialise the G1 HAL; no MuJoCo state is created until ``connect()``.

    All wiring (MJCF URI, floating-base offsets) lives in
    :data:`G1_DESCRIPTION.sim` (ADR-0023).
    """
    self._init_from_description(
        G1_DESCRIPTION,
        mjcf_path=mjcf_path,
        settle_steps=settle_steps,
        gravity_enabled=gravity_enabled,
        staleness_limit_s=staleness_limit_s,
    )

H1MujocoHAL(*, mjcf_path: str | None = None, settle_steps: int = 1, gravity_enabled: bool = True, staleness_limit_s: float = 0.5)

Bases: MujocoArmHAL

HAL adapter for the Unitree H1 humanoid (MuJoCo digital twin).

Drives the 19 actuated joints of the menagerie unitree_h1 MJCF through MuJoCo's position-controlled actuators. Exposes a 19-D :class:openral_core.Action matching the joint order in :data:H1_DESCRIPTION (left leg 5 → right leg 5 → torso 1 → left arm 4 → right arm 4).

.. warning::

This HAL does not provide balance. Without an S0 cerebellar controller (CLAUDE.md §6.2, M2 milestone) the robot will fall over under gravity. The closed-loop convergence tests run with gravity_enabled=False for that reason. Use this HAL to validate the action contract / joint indexing / lifecycle, not to roll out a humanoid policy.

Parameters:

Name	Type	Description	Default
mjcf_path	str | None	Optional override for the MJCF file path. When None, the file is fetched lazily from robot_descriptions (mujoco_menagerie/unitree_h1/h1.xml).	None
settle_steps	int	Number of MuJoCo physics steps performed in :meth:send_action. Defaults to 1; raise it in tests that assert the body has converged at the commanded pose.	1
gravity_enabled	bool	When False, gravity is zeroed at connect() time — required for the contract-validation tests because the floating base falls otherwise.	True
staleness_limit_s	float	Maximum age of a cached state.	0.5

Example

from openral_hal import H1MujocoHAL # doctest: +SKIP hal = H1MujocoHAL(gravity_enabled=False) # doctest: +SKIP hal.connect() # doctest: +SKIP state = hal.read_state() # doctest: +SKIP len(state.position) # 19 actuated joints # doctest: +SKIP 19 hal.disconnect() # doctest: +SKIP

Initialise the H1 HAL; no MuJoCo state is created until connect().

All MuJoCo wiring (MJCF URI, floating-base offsets) lives in :data:H1_DESCRIPTION.sim (ADR-0023). The software PD gains stay here because they are H1-specific cerebellar substitute behavior, not arm-data.

Source code in python/hal/src/openral_hal/h1.py

def __init__(
    self,
    *,
    mjcf_path: str | None = None,
    settle_steps: int = 1,
    gravity_enabled: bool = True,
    staleness_limit_s: float = 0.5,
) -> None:
    """Initialise the H1 HAL; no MuJoCo state is created until ``connect()``.

    All MuJoCo wiring (MJCF URI, floating-base offsets) lives in
    :data:`H1_DESCRIPTION.sim` (ADR-0023).  The software PD gains
    stay here because they are H1-specific cerebellar substitute
    behavior, not arm-data.
    """
    self._init_from_description(
        H1_DESCRIPTION,
        mjcf_path=mjcf_path,
        settle_steps=settle_steps,
        gravity_enabled=gravity_enabled,
        staleness_limit_s=staleness_limit_s,
    )
    self._pd_gains: dict[str, tuple[float, float]] = _h1_pd_gains()

HAL

Bases: Protocol

Structural protocol that every HAL adapter must satisfy.

The hot path (read_state / send_action) is synchronous and must complete within the robot's control cycle budget. Blocking I/O or memory allocation inside these methods is forbidden.

Implementors must raise ROSEStopRequested (a ROSSafetyViolation) from estop() so that the safety supervisor can catch it at the boundary and trigger an incident log entry.

Attributes:

Name	Type	Description
description	RobotDescription	Normative RobotDescription manifest for this robot.

connect() -> None

Open the connection to the robot hardware or simulator.

Raises:

Type	Description
ROSConfigError	If the URDF, controller name, or topic cannot be resolved.
ROSRuntimeError	If the underlying transport fails to initialise.

Source code in python/hal/src/openral_hal/protocol.py

def connect(self) -> None:
    """Open the connection to the robot hardware or simulator.

    Raises:
        ROSConfigError: If the URDF, controller name, or topic cannot be
            resolved.
        ROSRuntimeError: If the underlying transport fails to initialise.
    """
    ...

disconnect() -> None

Close the connection and release all resources gracefully.

Must be idempotent — calling disconnect() on an already-disconnected HAL must not raise.

Source code in python/hal/src/openral_hal/protocol.py

def disconnect(self) -> None:
    """Close the connection and release all resources gracefully.

    Must be idempotent — calling ``disconnect()`` on an already-disconnected
    HAL must not raise.
    """
    ...

estop() -> None

Trigger an emergency stop.

Raises:

Type	Description
ROSEStopRequested	Always. Callers must NOT catch this silently. Only the safety supervisor boundary may catch it.

Source code in python/hal/src/openral_hal/protocol.py

def estop(self) -> None:
    """Trigger an emergency stop.

    Raises:
        ROSEStopRequested: Always.  Callers must NOT catch this silently.
            Only the safety supervisor boundary may catch it.
    """
    ...

read_state() -> JointState

Return the latest joint state snapshot.

Raises:

Type	Description
ROSRuntimeError	If the HAL is not connected.
ROSPerceptionStale	If the most recent reading exceeds the staleness deadline configured in the RobotDescription.

Returns:

Type	Description
JointState	The latest JointState for all joints listed in
JointState	description.joints.

Source code in python/hal/src/openral_hal/protocol.py

def read_state(self) -> JointState:
    """Return the latest joint state snapshot.

    Raises:
        ROSRuntimeError: If the HAL is not connected.
        ROSPerceptionStale: If the most recent reading exceeds the staleness
            deadline configured in the ``RobotDescription``.

    Returns:
        The latest ``JointState`` for all joints listed in
        ``description.joints``.
    """
    ...

send_action(action: Action) -> None

Forward an action chunk to the underlying controller.

The HAL is responsible only for forwarding the typed action; safety clamping happens in the C++ safety kernel before this call is made.

Parameters:

Name	Type	Description	Default
action	Action	The Action produced by a Skill or the safety shaper.	required

Raises:

Type	Description
ROSRuntimeError	If the HAL is not connected.
ROSConfigError	If the action's control_mode is incompatible with the robot's supported_control_modes.

Source code in python/hal/src/openral_hal/protocol.py

def send_action(self, action: Action) -> None:
    """Forward an action chunk to the underlying controller.

    The HAL is responsible only for forwarding the typed action; safety
    clamping happens in the C++ safety kernel before this call is made.

    Args:
        action: The ``Action`` produced by a Skill or the safety shaper.

    Raises:
        ROSRuntimeError: If the HAL is not connected.
        ROSConfigError: If the action's ``control_mode`` is incompatible
            with the robot's ``supported_control_modes``.
    """
    ...

OpenArmMujocoHAL(*, mjcf_path: str | None = None, settle_steps: int = 1, gravity_enabled: bool = True, staleness_limit_s: float = 0.5)

Bases: MujocoArmHAL

HAL adapter for the Enactic OpenArm v2 (MuJoCo digital twin).

Thin manifest-driven wrapper around :class:MujocoArmHAL; all wiring (MJCF URI via the openarm_v2: scheme, joint→qpos map that skips the passive follower fingers, two PASSTHROUGH grippers, seed_ctrl_from_qpos to hold the initial pose under the v2 PD actuators) lives in :data:OPENARM_DESCRIPTION.sim (ADR-0023).

Public 16-DoF surface (7 arm + 1 gripper per side, left then right) matches what a future OpenArmRealHAL wrapping the LeRobot OpenArm driver will accept. Gripper commands are passthrough radians (left jaw [0, 0.7854], right jaw [-0.7854, 0]).

Parameters:

Name	Type	Description	Default
mjcf_path	str | None	Optional override for the MJCF file path. When None, the v2 bimanual MJCF is fetched lazily through :func:openral_hal._openarm_v2_assets.ensure_openarm_v2_mjcf via the openarm_v2:bimanual URI scheme.	None
settle_steps	int	Number of MuJoCo physics steps performed in :meth:send_action.	1
gravity_enabled	bool	When False, gravity is zeroed at connect() time for deterministic closed-loop tests.	True
staleness_limit_s	float	Maximum age of a cached state.	0.5

Example

from openral_hal import OpenArmMujocoHAL # doctest: +SKIP hal = OpenArmMujocoHAL(gravity_enabled=False) # doctest: +SKIP hal.connect() # doctest: +SKIP state = hal.read_state() # doctest: +SKIP len(state.position) # 7 + 1 + 7 + 1 # doctest: +SKIP 16 hal.disconnect() # doctest: +SKIP

Initialise the adapter; the MJCF is not loaded until connect().

OpenArm has no per-robot connect() override: any starting pose a Skill needs is carried by the rSkill manifest's starting_pose: and applied by rskill_runner_node via :meth:MujocoArmHAL.reset_to_pose before the first inference tick (ADR-0023 bimanual amendment).

Source code in python/hal/src/openral_hal/openarm.py

def __init__(
    self,
    *,
    mjcf_path: str | None = None,
    settle_steps: int = 1,
    gravity_enabled: bool = True,
    staleness_limit_s: float = 0.5,
) -> None:
    """Initialise the adapter; the MJCF is not loaded until ``connect()``.

    OpenArm has no per-robot ``connect()`` override: any starting pose
    a Skill needs is carried by the rSkill manifest's
    ``starting_pose:`` and applied by ``rskill_runner_node`` via
    :meth:`MujocoArmHAL.reset_to_pose` before the first inference
    tick (ADR-0023 bimanual amendment).
    """
    self._init_from_description(
        OPENARM_DESCRIPTION,
        mjcf_path=mjcf_path,
        settle_steps=settle_steps,
        gravity_enabled=gravity_enabled,
        staleness_limit_s=staleness_limit_s,
    )

PandaMobileHAL(*, initial_pose: list[float] | None = None, dt_s: float = 0.05)

In-process digital-twin HAL for the panda_mobile embodiment.

Maintains 10-DoF qpos state in memory. Routing per :attr:Action.control_mode:

• :attr:ControlMode.BODY_TWIST — Euler-integrates base pose using the planar components of the twist (linear x, linear y, angular z). Each send_action call advances by dt_s seconds (default 0.05 — 20 Hz nav control rate).
• :attr:ControlMode.JOINT_POSITION — sets the seven arm joints directly when the action carries seven targets; sets all ten slots when ten targets are supplied (base position + arm).

Parameters:

Name	Type	Description	Default
initial_pose	list[float] | None	Optional override of the start state. Length must be either 3 (base only) or 10 (base + arm). Defaults to all-zero.	None
dt_s	float	Integration timestep for body-twist commands. Defaults to 0.05 s (20 Hz). The Nav2 controller's default cmd_vel rate.	0.05

Latch initial 10-DoF state. No I/O until :meth:connect.

Source code in python/hal/src/openral_hal/panda_mobile.py

def __init__(
    self,
    *,
    initial_pose: list[float] | None = None,
    dt_s: float = 0.05,
) -> None:
    """Latch initial 10-DoF state. No I/O until :meth:`connect`."""
    if initial_pose is None:
        self._qpos: list[float] = [0.0] * len(PANDA_MOBILE_JOINT_NAMES)
    elif len(initial_pose) == len(PANDA_MOBILE_BASE_JOINT_NAMES):
        self._qpos = [*initial_pose, *([0.0] * len(_PANDA_MOBILE_ARM_JOINT_NAMES))]
    elif len(initial_pose) == len(PANDA_MOBILE_JOINT_NAMES):
        self._qpos = list(initial_pose)
    else:
        raise ROSConfigError(
            f"PandaMobileHAL.initial_pose must have length 3 (base only) "
            f"or 10 (base+arm); got {len(initial_pose)}."
        )
    self._dt_s = float(dt_s)
    self._connected: bool = False
    self._estop_latched: bool = False
    # Last commanded base body twist (vx, vy, vz, wx, wy, wz) in the
    # base_link frame — published as the /odom twist by the lifecycle
    # node. The base integrates this exactly, so it is the velocity.
    # Zeroed on any non-BODY_TWIST action.
    self._last_body_twist: tuple[float, float, float, float, float, float] = (
        0.0,
        0.0,
        0.0,
        0.0,
        0.0,
        0.0,
    )
    # `HALLifecycleNodeBase._publish_joint_state` reads
    # `self._hal.description` for OTel span attributes + per-joint
    # limit population. Bind the canonical RobotDescription so
    # downstream observability surfaces the right names / limits
    # without a separate registry lookup.
    self.description: RobotDescription = PANDA_MOBILE_DESCRIPTION

base_pose: tuple[float, float, float] property

Current base (x, y, yaw) for tests / odom publishers.

base_twist: tuple[float, float, float, float, float, float] property

Last commanded base body twist (vx, vy, vz, wx, wy, wz).

In the base_link frame — published as the /odom twist. Zeroed once a non-BODY_TWIST action is sent.

estop_latched: bool property

True while the estop latch is set.

connect() -> None

Idempotent — flips the connected flag.

Source code in python/hal/src/openral_hal/panda_mobile.py

def connect(self) -> None:
    """Idempotent — flips the connected flag."""
    self._connected = True

disconnect() -> None

Idempotent — clears the connected flag.

Source code in python/hal/src/openral_hal/panda_mobile.py

def disconnect(self) -> None:
    """Idempotent — clears the connected flag."""
    self._connected = False

estop() -> None

Latch the estop flag. Subsequent send_action calls no-op.

The latch can only be cleared by calling :meth:reset_estop, mirroring the supervisor's recovery contract: estops never auto-clear.

Source code in python/hal/src/openral_hal/panda_mobile.py

def estop(self) -> None:
    """Latch the estop flag. Subsequent ``send_action`` calls no-op.

    The latch can only be cleared by calling :meth:`reset_estop`,
    mirroring the supervisor's recovery contract: estops never
    auto-clear.
    """
    self._estop_latched = True

read_state() -> JointState

Return a fresh 10-DoF :class:JointState snapshot.

Source code in python/hal/src/openral_hal/panda_mobile.py

def read_state(self) -> JointState:
    """Return a fresh 10-DoF :class:`JointState` snapshot."""
    if not self._connected:
        raise ROSConfigError("PandaMobileHAL.read_state called before connect().")
    import time  # noqa: PLC0415

    return JointState(
        name=list(PANDA_MOBILE_JOINT_NAMES),
        position=list(self._qpos),
        velocity=[0.0] * len(PANDA_MOBILE_JOINT_NAMES),
        effort=[0.0] * len(PANDA_MOBILE_JOINT_NAMES),
        stamp_ns=time.time_ns(),
    )

reset_estop() -> None

Clear the estop latch. Caller asserts the cause has been resolved.

Source code in python/hal/src/openral_hal/panda_mobile.py

def reset_estop(self) -> None:
    """Clear the estop latch. Caller asserts the cause has been resolved."""
    self._estop_latched = False

send_action(action: Action) -> None

Apply the action to the in-memory state. Branches on control_mode.

ADR-0028c — accepts the four surfaces the slot dispatcher emits: JOINT_POSITION, BODY_TWIST, CARTESIAN_DELTA, GRIPPER_POSITION. Each reads its mode-specific payload from the matching :class:Action field (joint_targets, body_twist, cartesian_delta, gripper) — NOT joint_targets for every mode, which was the pre-0028c lie that conflated all surfaces onto one field.

Raises:

Type	Description
ROSConfigError	If the action's control_mode is not in the accepted set, or the mode-specific payload field is missing / mis-shaped.

Source code in python/hal/src/openral_hal/panda_mobile.py

def send_action(self, action: Action) -> None:
    """Apply the action to the in-memory state. Branches on control_mode.

    ADR-0028c — accepts the four surfaces the slot dispatcher
    emits: ``JOINT_POSITION``, ``BODY_TWIST``, ``CARTESIAN_DELTA``,
    ``GRIPPER_POSITION``. Each reads its mode-specific payload
    from the matching :class:`Action` field (joint_targets,
    body_twist, cartesian_delta, gripper) — NOT joint_targets for
    every mode, which was the pre-0028c lie that conflated all
    surfaces onto one field.

    Raises:
        ROSConfigError: If the action's ``control_mode`` is not in
            the accepted set, or the mode-specific payload field
            is missing / mis-shaped.
    """
    if not self._connected:
        raise ROSConfigError("PandaMobileHAL.send_action called before connect().")
    if self._estop_latched:
        # ADR-0025 / CLAUDE.md §1.1 — don't silently honour actions
        # after an estop; the supervisor reset path must clear the
        # latch explicitly.
        return

    mode = action.control_mode
    # Default: base not velocity-commanded. BODY_TWIST overwrites via
    # _apply_body_twist below; every other mode leaves it zeroed so
    # /odom doesn't report a stale base velocity.
    self._last_body_twist = (0.0, 0.0, 0.0, 0.0, 0.0, 0.0)
    if mode is ControlMode.JOINT_POSITION:
        if not action.joint_targets:
            raise ROSConfigError(
                "PandaMobileHAL.send_action: empty Action.joint_targets for JOINT_POSITION."
            )
        self._apply_joint_position(list(action.joint_targets[0]))
    elif mode is ControlMode.BODY_TWIST:
        if not action.body_twist:
            raise ROSConfigError(
                "PandaMobileHAL.send_action: empty Action.body_twist for BODY_TWIST."
            )
        self._apply_body_twist(list(action.body_twist[0]))
    elif mode is ControlMode.CARTESIAN_DELTA:
        # ADR-0028c — apply OSC delta to the cached arm joint
        # vector via a Jacobian-free approximation: treat the
        # cartesian delta as an additive bias on the gripper
        # frame's qpos snapshot. Real motion lives in the
        # sim-attached path (robosuite OSC). The digital-twin
        # tracks the delta for dashboard observability and so
        # tests can verify the chunk reached the HAL inbox.
        if not action.cartesian_delta:
            raise ROSConfigError("PandaMobileHAL.send_action: empty Action.cartesian_delta.")
        self._apply_cartesian_delta(list(action.cartesian_delta[0]))
    elif mode is ControlMode.GRIPPER_POSITION:
        if not action.gripper:
            raise ROSConfigError(
                "PandaMobileHAL.send_action: empty Action.gripper for GRIPPER_POSITION."
            )
        self._apply_gripper_position(float(action.gripper[0]))
    else:
        raise ROSConfigError(
            f"PandaMobileHAL.send_action: unsupported control_mode "
            f"{mode!r}; expected JOINT_POSITION / BODY_TWIST / "
            f"CARTESIAN_DELTA / GRIPPER_POSITION (ADR-0028c)."
        )

Rizon4MujocoHAL(*, mjcf_path: str | None = None, settle_steps: int = 1, gravity_enabled: bool = True, staleness_limit_s: float = 0.5)

Bases: MujocoArmHAL

HAL adapter for the Flexiv Rizon 4 (MuJoCo-backed simulation).

Drives the 7 position-controlled actuators of the menagerie flexiv_rizon4 MJCF through :class:MujocoArmHAL. Exposes a 7-D :class:openral_core.Action matching the joint order in :data:RIZON4_DESCRIPTION (joint1 ... joint7).

Parameters:

Name	Type	Description	Default
mjcf_path	str | None	Optional override for the MJCF file path. When None, the file is fetched lazily from robot_descriptions (mujoco_menagerie/flexiv_rizon4/flexiv_rizon4.xml).	None
settle_steps	int	Number of MuJoCo physics steps performed in :meth:send_action. Defaults to 1; raise it in tests that assert the arm has settled at the commanded pose.	1
gravity_enabled	bool	When False, gravity is zeroed at connect() time for deterministic closed-loop tests.	True
staleness_limit_s	float	Maximum age of a cached state.	0.5

Example

from openral_hal import Rizon4MujocoHAL # doctest: +SKIP hal = Rizon4MujocoHAL(gravity_enabled=False) # doctest: +SKIP hal.connect() # doctest: +SKIP state = hal.read_state() # doctest: +SKIP len(state.position) # 7 arm joints # doctest: +SKIP 7 hal.disconnect() # doctest: +SKIP

Initialise the Rizon 4 HAL; no MuJoCo state is created until connect().

All wiring lives in :data:RIZON4_DESCRIPTION.sim (ADR-0023).

Source code in python/hal/src/openral_hal/flexiv_rizon4.py

def __init__(
    self,
    *,
    mjcf_path: str | None = None,
    settle_steps: int = 1,
    gravity_enabled: bool = True,
    staleness_limit_s: float = 0.5,
) -> None:
    """Initialise the Rizon 4 HAL; no MuJoCo state is created until ``connect()``.

    All wiring lives in :data:`RIZON4_DESCRIPTION.sim` (ADR-0023).
    """
    self._init_from_description(
        RIZON4_DESCRIPTION,
        mjcf_path=mjcf_path,
        settle_steps=settle_steps,
        gravity_enabled=gravity_enabled,
        staleness_limit_s=staleness_limit_s,
    )

RosControlHAL(description: RobotDescription, controller_name: str, *, joint_state_topic: str = '/joint_states', command_topic: str | None = None, publish_fn: _PublishFn | None = None, state_fn: Callable[[], dict[str, object]] | None = None, staleness_limit_s: float = 0.5)

Bases: HALBase

ros2_control-backed HAL adapter.

The adapter does not import rclpy directly so it can be unit-tested without a live ROS 2 installation. In integration / HIL tests, inject a real publisher via the publish_fn parameter.

Parameters:

Name	Type	Description	Default
description	RobotDescription	Normative RobotDescription for the target robot.	required
controller_name	str	Name of the ros2_control joint trajectory controller, e.g. "joint_trajectory_controller".	required
joint_state_topic	str	ROS 2 topic that publishes sensor_msgs/JointState. Defaults to "/joint_states".	'/joint_states'
command_topic	str | None	ROS 2 topic for joint trajectory commands. Defaults to "/<controller_name>/joint_trajectory".	None
publish_fn	_PublishFn | None	Callable used to send messages to ROS 2 topics. Defaults to a no-op logger; replace with a real publisher in integration tests.	None
state_fn	Callable[[], dict[str, object]] | None	Callable that returns the latest raw joint state as a dict. Defaults to None, in which case the adapter returns a zeroed JointState. Replace with a real subscriber callback in integration tests.	None
staleness_limit_s	float	Maximum age (seconds) of a read_state() reading before ROSPerceptionStale is raised. Defaults to 0.5 s.	0.5

Raises:

Type	Description
ROSConfigError	If description.joints is empty.

Initialise the adapter; does not open any connection yet.

Source code in python/hal/src/openral_hal/ros_control.py

def __init__(
    self,
    description: RobotDescription,
    controller_name: str,
    *,
    joint_state_topic: str = "/joint_states",
    command_topic: str | None = None,
    publish_fn: _PublishFn | None = None,
    state_fn: Callable[[], dict[str, object]] | None = None,
    staleness_limit_s: float = 0.5,
) -> None:
    """Initialise the adapter; does not open any connection yet."""
    if not description.joints:
        raise ROSConfigError(
            f"RobotDescription '{description.name}' has no joints; "
            "cannot initialise RosControlHAL."
        )
    self.description = description
    self._controller_name = controller_name
    self._joint_state_topic = joint_state_topic
    self._command_topic = command_topic or f"/{controller_name}/joint_trajectory"
    self._publish_fn: _PublishFn = publish_fn or _default_publish
    self._state_fn = state_fn
    self._staleness_limit_s = staleness_limit_s

    self._connected: bool = False
    self._last_state_time: float = 0.0
    self._joint_names: list[str] = [j.name for j in description.joints]

connect() -> None

Open the connection to the robot hardware or simulator.

Raises:

Type	Description
ROSRuntimeError	If already connected.

Source code in python/hal/src/openral_hal/ros_control.py

def connect(self) -> None:
    """Open the connection to the robot hardware or simulator.

    Raises:
        ROSRuntimeError: If already connected.
    """
    if self._connected:
        raise ROSRuntimeError(f"RosControlHAL('{self.description.name}') is already connected.")
    log.info(
        "hal.connect",
        robot=self.description.name,
        controller=self._controller_name,
        joint_state_topic=self._joint_state_topic,
        command_topic=self._command_topic,
    )
    self._connected = True
    self._last_state_time = time.monotonic()

disconnect() -> None

Close the connection and release all resources.

Idempotent — calling on an already-disconnected HAL is a no-op.

Source code in python/hal/src/openral_hal/ros_control.py

def disconnect(self) -> None:
    """Close the connection and release all resources.

    Idempotent — calling on an already-disconnected HAL is a no-op.
    """
    if not self._connected:
        return
    log.info("hal.disconnect", robot=self.description.name)
    self._connected = False

estop() -> None

Trigger an emergency stop.

Sets the connection state to False before raising so that subsequent calls to read_state or send_action also fail fast.

Raises:

Type	Description
ROSEStopRequested	Always.

Source code in python/hal/src/openral_hal/ros_control.py

def estop(self) -> None:
    """Trigger an emergency stop.

    Sets the connection state to False before raising so that subsequent
    calls to ``read_state`` or ``send_action`` also fail fast.

    Raises:
        ROSEStopRequested: Always.
    """
    log.critical("hal.estop", robot=self.description.name)
    self._connected = False
    raise ROSEStopRequested(f"Emergency stop triggered on robot '{self.description.name}'.")

read_state() -> JointState

Return the latest joint state snapshot.

Raises:

Type	Description
ROSRuntimeError	If not connected.
ROSPerceptionStale	If the last reading is older than staleness_limit_s.

Returns:

Type	Description
JointState	Latest JointState for all joints in description.joints.

Source code in python/hal/src/openral_hal/ros_control.py

def read_state(self) -> JointState:
    """Return the latest joint state snapshot.

    Raises:
        ROSRuntimeError: If not connected.
        ROSPerceptionStale: If the last reading is older than
            ``staleness_limit_s``.

    Returns:
        Latest ``JointState`` for all joints in ``description.joints``.
    """
    self._require_connected("read_state")
    age = time.monotonic() - self._last_state_time
    if age > self._staleness_limit_s:
        raise ROSPerceptionStale(
            f"Joint state is {age:.3f} s old (limit {self._staleness_limit_s} s)."
        )

    n = len(self._joint_names)
    raw: dict[str, object] = {}
    if self._state_fn is not None:
        raw = self._state_fn()

    def _floats(key: str) -> list[float]:
        val = raw.get(key)
        if isinstance(val, list):
            return [float(v) for v in val]
        return [0.0] * n

    return JointState(
        name=self._joint_names,
        position=_floats("position"),
        velocity=_floats("velocity"),
        effort=_floats("effort"),
        stamp_ns=int(time.time_ns()),
    )

send_action(action: Action) -> None

Forward an action chunk to the ros2_control joint trajectory controller.

Parameters:

Name	Type	Description	Default
action	Action	The Action produced by a Skill.	required

Raises:

Type	Description
ROSRuntimeError	If not connected.
ROSConfigError	If action.control_mode is not in the robot's supported_control_modes, or if the joint target dimensions do not match the robot's joint count.

Source code in python/hal/src/openral_hal/ros_control.py

def send_action(self, action: Action) -> None:
    """Forward an action chunk to the ros2_control joint trajectory controller.

    Args:
        action: The ``Action`` produced by a Skill.

    Raises:
        ROSRuntimeError: If not connected.
        ROSConfigError: If ``action.control_mode`` is not in the robot's
            ``supported_control_modes``, or if the joint target dimensions
            do not match the robot's joint count.
    """
    self._require_connected("send_action")
    self._validate_action(action)

    msg: dict[str, object] = {
        "control_mode": action.control_mode,
        "horizon": action.horizon,
        "joint_targets": action.joint_targets,
        "stamp_ns": action.stamp_ns,
    }
    self._publish_fn(self._command_topic, msg)
    log.debug(
        "hal.send_action",
        robot=self.description.name,
        control_mode=action.control_mode,
        horizon=action.horizon,
    )

SO100DigitalTwin(config: SO100DigitalTwinConfig)

Bases: Robot

In-process digital twin for the SO-100 follower arm.

Implements the full lerobot Robot interface without any serial port. Joint state is initialised from config.initial_positions and updated each time send_action() is called, so observation sequences reflect sent commands.

Parameters:

Name	Type	Description	Default
config	SO100DigitalTwinConfig	SO100DigitalTwinConfig with optional initial positions.	required

Example

cfg = SO100DigitalTwinConfig() twin = SO100DigitalTwin(cfg) twin.connect(calibrate=False) twin.is_connected True twin.get_observation()["gripper.pos"] 50.0 twin.disconnect()

Initialise the digital twin; no connection is opened.

Source code in python/hal/src/openral_hal/so100_sim.py

def __init__(self, config: SO100DigitalTwinConfig) -> None:
    """Initialise the digital twin; no connection is opened."""
    super().__init__(config)
    self.config = config
    self._is_connected: bool = False
    self._is_calibrated: bool = True  # pre-calibrated; no wizard needed
    # Internal state: positions in lerobot's native units
    self._positions: dict[str, float] = {
        name: config.initial_positions.get(name, _DEFAULT_POSITIONS[name])
        for name in _JOINT_NAMES
    }

action_features: dict[str, type] cached property

Action feature schema — one float per joint position target.

Returns:

Type	Description
dict[str, type]	Dict mapping "<joint>.pos" keys to float type.

is_calibrated: bool property

Always True — the digital twin requires no calibration.

is_connected: bool property

True if connect() has been called and disconnect() has not.

observation_features: dict[str, type] cached property

Observation feature schema — one float per joint position.

Returns:

Type	Description
dict[str, type]	Dict mapping "<joint>.pos" keys to float type.

calibrate() -> None

No-op — the digital twin requires no interactive calibration.

Source code in python/hal/src/openral_hal/so100_sim.py

def calibrate(self) -> None:
    """No-op — the digital twin requires no interactive calibration."""
    self._is_calibrated = True

configure() -> None

No-op — no bus configuration needed.

Source code in python/hal/src/openral_hal/so100_sim.py

def configure(self) -> None:
    """No-op — no bus configuration needed."""

connect(calibrate: bool = True) -> None

Activate the twin. No serial port is opened.

Parameters:

Name	Type	Description	Default
calibrate	bool	Ignored — the twin is always pre-calibrated.	True

Source code in python/hal/src/openral_hal/so100_sim.py

@check_if_already_connected  # type: ignore[untyped-decorator]
def connect(self, calibrate: bool = True) -> None:
    """Activate the twin.  No serial port is opened.

    Args:
        calibrate: Ignored — the twin is always pre-calibrated.
    """
    self._is_connected = True

disconnect() -> None

Deactivate the twin. Idempotent if called via the decorator.

Source code in python/hal/src/openral_hal/so100_sim.py

@check_if_not_connected  # type: ignore[untyped-decorator]
def disconnect(self) -> None:
    """Deactivate the twin.  Idempotent if called via the decorator."""
    self._is_connected = False

get_observation() -> RobotObservation

Return current joint positions in lerobot's native units.

Returns:

Type	Description
RobotObservation	Dict {"<joint>.pos": float} with revolute joints in degrees
RobotObservation	and gripper in [0, 100].

Raises:

Type	Description
RuntimeError	If not connected (enforced by decorator).

Source code in python/hal/src/openral_hal/so100_sim.py

@check_if_not_connected  # type: ignore[untyped-decorator]
def get_observation(self) -> RobotObservation:
    """Return current joint positions in lerobot's native units.

    Returns:
        Dict ``{"<joint>.pos": float}`` with revolute joints in degrees
        and gripper in [0, 100].

    Raises:
        RuntimeError: If not connected (enforced by decorator).
    """
    return {f"{name}.pos": self._positions[name] for name in _JOINT_NAMES}

send_action(action: RobotAction) -> RobotAction

Apply a position command and update internal state.

Parameters:

Name	Type	Description	Default
action	RobotAction	Dict {"<joint>.pos": float} in lerobot's native units.	required

Returns:

Type	Description
RobotAction	The same action dict (lerobot convention).

Raises:

Type	Description
RuntimeError	If not connected (enforced by decorator).

Source code in python/hal/src/openral_hal/so100_sim.py

@check_if_not_connected  # type: ignore[untyped-decorator]
def send_action(self, action: RobotAction) -> RobotAction:
    """Apply a position command and update internal state.

    Args:
        action: Dict ``{"<joint>.pos": float}`` in lerobot's native units.

    Returns:
        The same action dict (lerobot convention).

    Raises:
        RuntimeError: If not connected (enforced by decorator).
    """
    for name in _JOINT_NAMES:
        key = f"{name}.pos"
        if key in action:
            self._positions[name] = float(action[key])
    return action

SO100DigitalTwinConfig(*, initial_positions: dict[str, float] = (lambda: dict(_DEFAULT_POSITIONS))()) dataclass

Bases: RobotConfig

Configuration for the SO-100 digital twin.

Parameters:

Name	Type	Description	Default
initial_positions	dict[str, float]	Initial joint positions in lerobot's native units (revolute joints in degrees, gripper in [0, 100]). Defaults to all-zero revolute joints and gripper at 50 (half-open).	(lambda: dict(_DEFAULT_POSITIONS))()

Example

cfg = SO100DigitalTwinConfig(initial_positions={"shoulder_pan": 90.0}) cfg.initial_positions["shoulder_pan"] 90.0

SO100FollowerHAL(port: str = '/dev/ttyUSB0', *, calibrate_on_connect: bool = False, max_relative_target: float | dict[str, float] | None = None, staleness_limit_s: float = 0.5, robot: _LeRobotRobot | None = None)

Bases: HALBase

HAL adapter wrapping lerobot's SO-100 follower arm.

The adapter is instantiated without a live robot connection; call connect() to open the USB serial port. No lerobot import happens at module load time so the class can be used in environments where lerobot is not installed (e.g., CI type-checking runs).

For testing without hardware, pass an already-constructed lerobot Robot instance via the robot keyword argument. The injected robot is used directly and no serial port is opened:

>>> from openral_hal.so100_sim import SO100DigitalTwin, SO100DigitalTwinConfig
>>> twin = SO100DigitalTwin(SO100DigitalTwinConfig())
>>> hal = SO100FollowerHAL(robot=twin)
>>> hal.connect()
>>> hal.description.name
'so100_follower'

Parameters:

Name	Type	Description	Default
port	str	USB serial port, e.g. "/dev/ttyUSB0". Ignored when robot is provided. Defaults to "/dev/ttyUSB0" on Linux / "/dev/cu.usbserial-0001" on macOS.	'/dev/ttyUSB0'
calibrate_on_connect	bool	If True (default False), run lerobot's interactive calibration wizard at connect() time. Set to False for automated / HIL use; the stored calibration file is used instead.	False
max_relative_target	float | dict[str, float] | None	Per-joint or scalar cap on goal-position deltas forwarded to the motor bus. None means no capping. Ignored when robot is provided.	None
staleness_limit_s	float	How old (seconds) a read_state() timestamp may be before ROSPerceptionStale is raised. Defaults to 0.5.	0.5
robot	Robot | None	Optional pre-constructed lerobot Robot instance. When provided, connect() calls robot.connect() directly instead of constructing a SO100Follower and opening the serial port. Intended for testing via SO100DigitalTwin.	None

Raises:

Type	Description
ROSConfigError	At connect() time if lerobot is not installed and no robot was injected.

Initialise the adapter; does not open any connection yet.

Source code in python/hal/src/openral_hal/so100_follower.py

def __init__(
    self,
    port: str = "/dev/ttyUSB0",
    *,
    calibrate_on_connect: bool = False,
    max_relative_target: float | dict[str, float] | None = None,
    staleness_limit_s: float = 0.5,
    robot: _LeRobotRobot | None = None,
) -> None:
    """Initialise the adapter; does not open any connection yet."""
    self.description: RobotDescription = SO100_DESCRIPTION
    self._port = port
    self._calibrate_on_connect = calibrate_on_connect
    self._max_relative_target = max_relative_target
    self._staleness_limit_s = staleness_limit_s
    self._injected_robot: _LeRobotRobot | None = robot

    self._robot: _LeRobotRobot | None = None
    self._connected: bool = False
    self._last_obs_time: float = 0.0
    self._last_obs: dict[str, float] = {}

connect() -> None

Open the USB serial connection to the SO-100 arm.

If a robot was passed at construction time, that instance is used directly (no serial port is opened). Otherwise, a SO100Follower is constructed from port and the lerobot package is imported lazily.

Raises:

Type	Description
ROSConfigError	If lerobot is not installed and no robot was injected, or if already connected.
ROSRuntimeError	If the underlying serial open fails.

Source code in python/hal/src/openral_hal/so100_follower.py

def connect(self) -> None:
    """Open the USB serial connection to the SO-100 arm.

    If a ``robot`` was passed at construction time, that instance is used
    directly (no serial port is opened).  Otherwise, a ``SO100Follower``
    is constructed from ``port`` and the lerobot package is imported lazily.

    Raises:
        ROSConfigError: If ``lerobot`` is not installed and no robot was
            injected, or if already connected.
        ROSRuntimeError: If the underlying serial open fails.
    """
    if self._connected:
        raise ROSRuntimeError(f"SO100FollowerHAL(port={self._port!r}) is already connected.")

    if self._injected_robot is not None:
        # Use the pre-constructed robot (e.g. SO100DigitalTwin for testing).
        try:
            self._injected_robot.connect(calibrate=self._calibrate_on_connect)
        except Exception as exc:
            raise ROSRuntimeError(f"Injected robot connect() failed: {exc}") from exc
        self._robot = self._injected_robot
    else:
        try:
            from lerobot.robots.so_follower import (  # noqa: PLC0415
                SO100Follower,
                SOFollowerRobotConfig,
            )
        except ModuleNotFoundError as exc:
            raise ROSConfigError(
                "lerobot is not installed. Install it with: "
                "uv add lerobot --package openral-hal"
            ) from exc

        cfg: _SOFollowerRobotConfig = SOFollowerRobotConfig(
            port=self._port,
            max_relative_target=self._max_relative_target,
            use_degrees=True,  # adapter converts degrees ↔ radians
        )
        try:
            robot = SO100Follower(cfg)
        except ModuleNotFoundError as exc:
            # lerobot's FeetechMotorsBus pulls in `scservo_sdk` at __init__
            # time; surface a typed config error instead of leaking the raw
            # ImportError traceback.
            raise ROSConfigError(
                f"SO-100 driver dependency missing ({exc.name!r}). "
                "Install with: uv add scservo_sdk --package openral-hal"
            ) from exc
        if not os.path.exists(self._port):
            raise ROSConfigError(
                f"SO-100 serial port {self._port!r} does not exist. "
                "Connect the arm via USB, or pass --port /dev/ttyUSBn."
            )
        try:
            robot.connect(calibrate=self._calibrate_on_connect)
        except Exception as exc:
            raise ROSRuntimeError(
                f"Failed to connect to SO-100 on {self._port!r}: {exc}"
            ) from exc
        self._robot = robot

    self._connected = True
    self._last_obs_time = time.monotonic()
    log.info("hal.connect", robot="so100_follower", port=self._port)

disconnect() -> None

Close the USB connection, disabling motor torque. Idempotent.

Source code in python/hal/src/openral_hal/so100_follower.py

def disconnect(self) -> None:
    """Close the USB connection, disabling motor torque.  Idempotent."""
    if not self._connected:
        return
    if self._robot is not None:
        with contextlib.suppress(Exception):
            self._robot.disconnect()
    self._robot = None
    self._connected = False
    log.info("hal.disconnect", robot="so100_follower", port=self._port)

estop() -> None

Trigger an emergency stop: disconnect motors then raise.

Raises:

Type	Description
ROSEStopRequested	Always.

Source code in python/hal/src/openral_hal/so100_follower.py

def estop(self) -> None:
    """Trigger an emergency stop: disconnect motors then raise.

    Raises:
        ROSEStopRequested: Always.
    """
    log.critical("hal.estop", robot="so100_follower", port=self._port)
    if self._robot is not None and self._connected:
        with contextlib.suppress(Exception):
            self._robot.disconnect()
    self._robot = None
    self._connected = False
    raise ROSEStopRequested(f"Emergency stop triggered on SO-100 at port '{self._port}'.")

read_state() -> JointState

Return the latest joint state in radians.

Raises:

Type	Description
ROSRuntimeError	If not connected.
ROSPerceptionStale	If the last observation is too old.

Returns:

Type	Description
JointState	JointState with positions in radians (gripper in [0, 1]).

Source code in python/hal/src/openral_hal/so100_follower.py

def read_state(self) -> JointState:
    """Return the latest joint state in radians.

    Raises:
        ROSRuntimeError: If not connected.
        ROSPerceptionStale: If the last observation is too old.

    Returns:
        ``JointState`` with positions in radians (gripper in [0, 1]).
    """
    self._require_connected("read_state")

    assert self._robot is not None  # guaranteed by _require_connected
    try:
        obs: dict[str, object] = self._robot.get_observation()
    except Exception as exc:
        raise ROSRuntimeError(f"get_observation() failed: {exc}") from exc

    self._last_obs_time = time.monotonic()
    self._last_obs = {k: float(v) for k, v in obs.items() if isinstance(v, (int, float))}

    positions = self._obs_to_positions(self._last_obs)
    return JointState(
        name=_SO100_JOINT_NAMES,
        position=positions,
        stamp_ns=time.time_ns(),
    )

send_action(action: Action) -> None

Forward one action step to the SO-100 motor bus.

Only the first step of the action chunk is sent; lerobot handles single-step commands. Action chunks are executed by calling send_action repeatedly in the skill executor's async loop.

Parameters:

Name	Type	Description	Default
action	Action	The Action produced by a Skill. Must use ControlMode.JOINT_POSITION with joint_targets set.	required

Raises:

Type	Description
ROSRuntimeError	If not connected.
ROSConfigError	If the action cannot be converted.

Source code in python/hal/src/openral_hal/so100_follower.py

def send_action(self, action: Action) -> None:
    """Forward one action step to the SO-100 motor bus.

    Only the **first step** of the action chunk is sent; lerobot handles
    single-step commands.  Action chunks are executed by calling
    ``send_action`` repeatedly in the skill executor's async loop.

    Args:
        action: The ``Action`` produced by a Skill.  Must use
            ``ControlMode.JOINT_POSITION`` with ``joint_targets`` set.

    Raises:
        ROSRuntimeError: If not connected.
        ROSConfigError: If the action cannot be converted.
    """
    self._require_connected("send_action")
    lerobot_action = self._action_to_lerobot(action)
    assert self._robot is not None
    try:
        self._robot.send_action(lerobot_action)
    except Exception as exc:
        raise ROSRuntimeError(f"send_action() failed: {exc}") from exc

SO100MujocoHAL(*, mjcf_path: str | None = None, settle_steps: int = 1, gravity_enabled: bool = True, staleness_limit_s: float = 0.5)

Bases: MujocoArmHAL

HAL adapter for the SO-100 follower arm (MuJoCo-backed simulation).

The HAL exposes the canonical 6 SO-100 joints (5 arm + 1 gripper) on its public surface, using the lerobot-style names from :data:openral_hal.SO100_DESCRIPTION. Internally it drives the Menagerie MJCF's 6 position actuators (Rotation, Pitch, Elbow, Wrist_Pitch, Wrist_Roll, Jaw) and maps the revolute Jaw range [-0.174, 1.75] rad to a normalised [0, 1] gripper channel — so the same 6-DoF action chunk drives the sim twin and the real hardware HAL (:class:openral_hal.SO100FollowerHAL) identically.

The MJCF position limits are tighter than the conservative [-π, π] declared on :data:openral_hal.SO100_DESCRIPTION; commands outside the MJCF range are clipped by MuJoCo's own position controllers. Tests should command targets inside the menagerie range (e.g. Rotation is [-1.92, 1.92]).

Parameters:

Name	Type	Description	Default
mjcf_path	str | None	Optional override for the MJCF file path. When None, the file is fetched lazily from robot_descriptions (mujoco_menagerie/trs_so_arm100/so_arm100.xml).	None
settle_steps	int	Number of MuJoCo physics steps performed in :meth:send_action. Defaults to 1; raise it in tests that assert the arm has settled at the commanded pose.	1
gravity_enabled	bool	When False, gravity is zeroed at connect() time for deterministic closed-loop tests.	True
staleness_limit_s	float	Maximum age of a cached state.	0.5

Example

from openral_hal import SO100MujocoHAL # doctest: +SKIP hal = SO100MujocoHAL(gravity_enabled=False) # doctest: +SKIP hal.connect() # doctest: +SKIP state = hal.read_state() # doctest: +SKIP len(state.position) # 5 arm + 1 gripper # doctest: +SKIP 6 hal.disconnect() # doctest: +SKIP

Initialise the SO-100 MuJoCo HAL; no MuJoCo state is created until connect().

All wiring (MJCF URI, joint indices, Jaw affine_low_high gripper read mode) lives in :data:SO100_DESCRIPTION.sim (ADR-0023).

Source code in python/hal/src/openral_hal/so100_mujoco.py

def __init__(
    self,
    *,
    mjcf_path: str | None = None,
    settle_steps: int = 1,
    gravity_enabled: bool = True,
    staleness_limit_s: float = 0.5,
) -> None:
    """Initialise the SO-100 MuJoCo HAL; no MuJoCo state is created until ``connect()``.

    All wiring (MJCF URI, joint indices, Jaw ``affine_low_high`` gripper
    read mode) lives in :data:`SO100_DESCRIPTION.sim` (ADR-0023).
    """
    self._init_from_description(
        SO100_DESCRIPTION,
        mjcf_path=mjcf_path,
        settle_steps=settle_steps,
        gravity_enabled=gravity_enabled,
        staleness_limit_s=staleness_limit_s,
    )

SawyerRealHAL(*, hostname: str = 'sawyer.local', controller_name: str = _DEFAULT_SAWYER_CONTROLLER, joint_state_topic: str = _DEFAULT_SAWYER_JOINT_STATE_TOPIC, command_topic: str | None = None, estop_topic: str = _DEFAULT_SAWYER_ESTOP_TOPIC, publish_fn: _PublishFn | None = None, state_fn: _StateFn | None = None, staleness_limit_s: float = 0.2)

HAL adapter for a physical Rethink Sawyer over intera_sdk / ROS 2.

Parameters:

Name	Type	Description	Default
hostname	str	Hostname of the Sawyer's onboard PC, typically "sawyer.local" on a lab subnet. Required; the upstream intera_sdk (and sawyer_robot fork) refuses to connect without it. Stored as metadata only — the actual TCP connection is opened by the lifecycle node.	'sawyer.local'
controller_name	str	Name of the ros2_control joint trajectory controller exported by sawyer_robot. Defaults to "sawyer_arm_controller".	_DEFAULT_SAWYER_CONTROLLER
joint_state_topic	str	ROS 2 topic publishing sensor_msgs/JointState. Defaults to "/robot/joint_states" (the legacy intera_sdk topic name).	_DEFAULT_SAWYER_JOINT_STATE_TOPIC
command_topic	str | None	ROS 2 topic for joint trajectory commands. Defaults to "/<controller_name>/joint_trajectory".	None
estop_topic	str	ROS 2 topic the safety supervisor publishes to on estop(). Defaults to "/robot/set_super_stop".	_DEFAULT_SAWYER_ESTOP_TOPIC
publish_fn	_PublishFn | None	Callable forwarding messages to ROS 2 topics. Production use injects the lifecycle node's publisher; tests inject :class:SimTransport.publish.	None
state_fn	_StateFn | None	Callable returning the latest raw joint state as a dict. Production use injects the lifecycle node's subscriber callback; tests inject :class:SimTransport.state.	None
staleness_limit_s	float	Maximum age of a read_state() reading before :class:ROSPerceptionStale is raised. Defaults to 0.2 s (Sawyer's intera_sdk feedback rate is ~100 Hz).	0.2

Raises:

Type	Description
ROSConfigError	If hostname is empty / whitespace.

Example

from openral_hal.sawyer_real import SawyerRealHAL from openral_hal.sim_transport import SimTransport transport = SimTransport(n_joints=7) hal = SawyerRealHAL( ... hostname="sawyer.local", ... publish_fn=transport.publish, ... state_fn=transport.state, ... ) hal.connect() hal.description.name 'sawyer' hal.disconnect()

Initialise the adapter; no TCP connection is opened until connect().

Source code in python/hal/src/openral_hal/sawyer_real.py

def __init__(
    self,
    *,
    hostname: str = "sawyer.local",
    controller_name: str = _DEFAULT_SAWYER_CONTROLLER,
    joint_state_topic: str = _DEFAULT_SAWYER_JOINT_STATE_TOPIC,
    command_topic: str | None = None,
    estop_topic: str = _DEFAULT_SAWYER_ESTOP_TOPIC,
    publish_fn: _PublishFn | None = None,
    state_fn: _StateFn | None = None,
    staleness_limit_s: float = 0.2,
) -> None:
    """Initialise the adapter; no TCP connection is opened until ``connect()``."""
    if not hostname or not hostname.strip():
        raise ROSConfigError(
            "SawyerRealHAL requires a non-empty hostname "
            "(e.g. 'sawyer.local' or the robot's IP)."
        )
    self._hostname = hostname
    self._controller_name = controller_name
    self._estop_topic = estop_topic
    self._publish_fn: _PublishFn | None = publish_fn

    self._inner = RosControlHAL(
        SAWYER_REAL_DESCRIPTION,
        controller_name=controller_name,
        joint_state_topic=joint_state_topic,
        command_topic=command_topic,
        publish_fn=publish_fn,
        state_fn=state_fn,
        staleness_limit_s=staleness_limit_s,
    )

controller_name: str property

Name of the ros2_control joint trajectory controller.

description: RobotDescription property

Normative :class:RobotDescription for the Sawyer.

hostname: str property

Hostname / IP of the Sawyer's onboard PC.

connect() -> None

Open the ROS 2 transport to the sawyer_robot controller.

Raises:

Type	Description
ROSRuntimeError	If already connected.

Source code in python/hal/src/openral_hal/sawyer_real.py

def connect(self) -> None:
    """Open the ROS 2 transport to the ``sawyer_robot`` controller.

    Raises:
        ROSRuntimeError: If already connected.
    """
    log.info(
        "hal.connect",
        robot=self.description.name,
        hostname=self._hostname,
        controller=self._controller_name,
    )
    self._inner.connect()

disconnect() -> None

Close the ROS 2 transport. Idempotent.

Source code in python/hal/src/openral_hal/sawyer_real.py

def disconnect(self) -> None:
    """Close the ROS 2 transport.  Idempotent."""
    self._inner.disconnect()

estop() -> None

Trigger an emergency stop on the Sawyer.

Publishes to the legacy intera_sdk halt topic, marks the inner adapter disconnected, and raises :class:ROSEStopRequested.

Raises:

Type	Description
ROSEStopRequested	Always.

Source code in python/hal/src/openral_hal/sawyer_real.py

def estop(self) -> None:
    """Trigger an emergency stop on the Sawyer.

    Publishes to the legacy intera_sdk halt topic, marks the inner
    adapter disconnected, and raises :class:`ROSEStopRequested`.

    Raises:
        ROSEStopRequested: Always.
    """
    log.critical(
        "hal.estop",
        robot=self.description.name,
        hostname=self._hostname,
        estop_topic=self._estop_topic,
    )
    if self._publish_fn is not None:
        with contextlib.suppress(Exception):
            self._publish_fn(
                self._estop_topic,
                {"reason": "openral_estop", "robot": self.description.name},
            )
    with contextlib.suppress(Exception):
        self._inner.disconnect()
    raise ROSEStopRequested(f"Emergency stop triggered on Sawyer at host {self._hostname!r}.")

read_state() -> JointState

Return the latest joint state for all 7 description joints.

Raises:

Type	Description
ROSRuntimeError	If not connected.
ROSPerceptionStale	If the last reading is older than staleness_limit_s.

Source code in python/hal/src/openral_hal/sawyer_real.py

def read_state(self) -> JointState:
    """Return the latest joint state for all 7 description joints.

    Raises:
        ROSRuntimeError: If not connected.
        ROSPerceptionStale: If the last reading is older than
            ``staleness_limit_s``.
    """
    return self._inner.read_state()

send_action(action: Action) -> None

Forward an action chunk to the sawyer_robot controller.

Raises:

Type	Description
ROSRuntimeError	If not connected.
ROSConfigError	If action.control_mode is not in the description's supported_control_modes.

Source code in python/hal/src/openral_hal/sawyer_real.py

def send_action(self, action: Action) -> None:
    """Forward an action chunk to the ``sawyer_robot`` controller.

    Raises:
        ROSRuntimeError: If not connected.
        ROSConfigError: If ``action.control_mode`` is not in the
            description's ``supported_control_modes``.
    """
    self._inner.send_action(action)

SimTransport(n_joints: int)

In-memory transport simulating a ros2_control joint trajectory controller.

Published joint trajectory commands update internal joint positions, which are then returned by :meth:state. All published messages are recorded for assertion in tests.

Parameters:

Name	Type	Description	Default
n_joints	int	Number of joints to simulate. Initial positions, velocities, and efforts are all 0.0.	required

Example

transport = SimTransport(n_joints=2) transport.publish( ... "/ctrl/traj", ... { ... "joint_targets": [[0.5, -0.5]], ... "control_mode": "joint_position", ... "horizon": 1, ... "stamp_ns": 0, ... }, ... ) tuple(transport.state()["position"]) (0.5, -0.5) transport.call_count 1 topic, msg = transport.last_call # type: ignore[misc] topic '/ctrl/traj'

Initialise zeroed joint state for n_joints joints.

Source code in python/hal/src/openral_hal/sim_transport.py

def __init__(self, n_joints: int) -> None:
    """Initialise zeroed joint state for *n_joints* joints."""
    self._n_joints = n_joints
    self._positions: list[float] = [0.0] * n_joints
    self._velocities: list[float] = [0.0] * n_joints
    self._efforts: list[float] = [0.0] * n_joints
    self._published: list[tuple[str, dict[str, object]]] = []

call_count: int property

Number of times :meth:publish has been called.

calls: list[tuple[str, dict[str, object]]] property

All (topic, msg) pairs in chronological order.

last_call: tuple[str, dict[str, object]] | None property

The most recent (topic, msg) pair, or None.

publish(topic: str, msg: dict[str, object]) -> None

Record msg and apply joint_targets to internal state.

If msg["joint_targets"] is a list of trajectory steps, the last step is applied to positions -- mirroring how a real joint trajectory controller would reach the final waypoint.

Parameters:

Name	Type	Description	Default
topic	str	The ROS 2 topic name.	required
msg	dict[str, object]	The published message dict.	required

Source code in python/hal/src/openral_hal/sim_transport.py

def publish(self, topic: str, msg: dict[str, object]) -> None:
    """Record *msg* and apply ``joint_targets`` to internal state.

    If ``msg["joint_targets"]`` is a list of trajectory steps, the **last
    step** is applied to positions -- mirroring how a real joint trajectory
    controller would reach the final waypoint.

    Args:
        topic: The ROS 2 topic name.
        msg: The published message dict.
    """
    self._published.append((topic, msg))
    targets = msg.get("joint_targets")
    if isinstance(targets, list) and targets:
        last_step = targets[-1]
        if isinstance(last_step, list):
            self._positions = [float(v) for v in last_step]

state() -> dict[str, object]

Return the current simulated joint state.

Returns:

Type	Description
dict[str, object]	Dict with "position", "velocity", and "effort" keys,
dict[str, object]	each containing a list[float] of length n_joints.

Source code in python/hal/src/openral_hal/sim_transport.py

def state(self) -> dict[str, object]:
    """Return the current simulated joint state.

    Returns:
        Dict with ``"position"``, ``"velocity"``, and ``"effort"`` keys,
        each containing a ``list[float]`` of length ``n_joints``.
    """
    return {
        "position": list(self._positions),
        "velocity": list(self._velocities),
        "effort": list(self._efforts),
    }

UR10eHAL(*, mjcf_path: str | None = None, settle_steps: int = 1, gravity_enabled: bool = True, staleness_limit_s: float = 0.5)

Bases: MujocoArmHAL

HAL adapter for the Universal Robots UR10e (MuJoCo-backed simulation).

Args mirror :class:UR5eHAL; the only difference is the underlying MJCF and RobotDescription (different velocity / effort envelopes), both of which now live in :data:UR10e_DESCRIPTION (ADR-0023).

Example

from openral_hal import UR10eHAL # doctest: +SKIP hal = UR10eHAL(gravity_enabled=False) # doctest: +SKIP

Initialise the UR10e HAL; no MuJoCo state is created until connect().

Source code in python/hal/src/openral_hal/ur.py

def __init__(
    self,
    *,
    mjcf_path: str | None = None,
    settle_steps: int = 1,
    gravity_enabled: bool = True,
    staleness_limit_s: float = 0.5,
) -> None:
    """Initialise the UR10e HAL; no MuJoCo state is created until ``connect()``."""
    self._init_from_description(
        UR10e_DESCRIPTION,
        mjcf_path=mjcf_path,
        settle_steps=settle_steps,
        gravity_enabled=gravity_enabled,
        staleness_limit_s=staleness_limit_s,
    )

UR10eRealHAL(*, robot_ip: str | None = None, publish_fn: Callable[[str, dict[str, object]], None] | None = None, state_fn: Callable[[], dict[str, object]] | None = None, staleness_limit_s: float = 0.5)

Bases: _URRealHAL

Real-hardware HAL adapter for the Universal Robots UR10e.

Identical to :class:UR5eRealHAL except for the wrapped :class:RobotDescription (12.5 kg payload, 1.30 m reach, larger torques) — ur_robot_driver itself is the same binary for both arms, differentiated by the URDF and per-joint envelope.

Args mirror :class:UR5eRealHAL.

Example::

>>> from openral_hal.sim_transport import SimTransport
>>> from openral_hal.ur_real import UR10eRealHAL
>>> transport = SimTransport(n_joints=6)
>>> hal = UR10eRealHAL(
...     publish_fn=transport.publish,
...     state_fn=transport.state,
... )
>>> hal.description.name
'ur10e'
>>> hal.description.hal.real
'openral_hal.ur_real:UR10eRealHAL'

Initialise the UR10e real-HW HAL; transport defaults match ur_robot_driver.

Source code in python/hal/src/openral_hal/ur_real.py

def __init__(
    self,
    *,
    robot_ip: str | None = None,
    publish_fn: Callable[[str, dict[str, object]], None] | None = None,
    state_fn: Callable[[], dict[str, object]] | None = None,
    staleness_limit_s: float = 0.5,
) -> None:
    """Initialise the UR10e real-HW HAL; transport defaults match ``ur_robot_driver``."""
    super().__init__(
        UR10e_REAL_DESCRIPTION,
        robot_ip=robot_ip,
        publish_fn=publish_fn,
        state_fn=state_fn,
        staleness_limit_s=staleness_limit_s,
    )

UR5eHAL(*, mjcf_path: str | None = None, settle_steps: int = 1, gravity_enabled: bool = True, staleness_limit_s: float = 0.5)

Bases: MujocoArmHAL

HAL adapter for the Universal Robots UR5e (MuJoCo-backed simulation).

Thin manifest-driven wrapper — every wiring constant lives in :data:UR5e_DESCRIPTION.sim (ADR-0023).

Parameters:

Name	Type	Description	Default
mjcf_path	str | None	Optional override for the MJCF file path. When None, UR5e_DESCRIPTION.assets.mjcf is resolved at construction time (robot_descriptions:ur5e_mj_description).	None
settle_steps	int	Number of MuJoCo physics steps performed in :meth:send_action. Defaults to 1.	1
gravity_enabled	bool	When False, gravity is zeroed at connect() time for deterministic closed-loop tests.	True
staleness_limit_s	float	Maximum age of a cached state.	0.5

Example

from openral_hal import UR5eHAL # doctest: +SKIP hal = UR5eHAL(gravity_enabled=False) # doctest: +SKIP hal.connect() # doctest: +SKIP state = hal.read_state() # doctest: +SKIP hal.disconnect() # doctest: +SKIP

Initialise the UR5e HAL; no MuJoCo state is created until connect().

Source code in python/hal/src/openral_hal/ur.py

def __init__(
    self,
    *,
    mjcf_path: str | None = None,
    settle_steps: int = 1,
    gravity_enabled: bool = True,
    staleness_limit_s: float = 0.5,
) -> None:
    """Initialise the UR5e HAL; no MuJoCo state is created until ``connect()``."""
    self._init_from_description(
        UR5e_DESCRIPTION,
        mjcf_path=mjcf_path,
        settle_steps=settle_steps,
        gravity_enabled=gravity_enabled,
        staleness_limit_s=staleness_limit_s,
    )

UR5eRealHAL(*, robot_ip: str | None = None, publish_fn: Callable[[str, dict[str, object]], None] | None = None, state_fn: Callable[[], dict[str, object]] | None = None, staleness_limit_s: float = 0.5)

Bases: _URRealHAL

Real-hardware HAL adapter for the Universal Robots UR5e.

Drives a real UR5e via ros2_control + ur_robot_driver (URCap + RTDE). The Python adapter itself contains no rclpy import — the transport is injected, so the same class drives the real arm in production and a :class:~openral_hal.sim_transport.SimTransport in unit tests.

Parameters:

Name	Type	Description	Default
robot_ip	str | None	Static IP of the UR5e controller, e.g. "192.168.1.42". Recorded for observability; the driver itself must have been launched against this IP separately.	None
publish_fn	Callable[[str, dict[str, object]], None] | None	Optional injected publish callable for tests.	None
state_fn	Callable[[], dict[str, object]] | None	Optional injected state-read callable for tests.	None
staleness_limit_s	float	Maximum age of a cached state.	0.5

Example::

>>> from openral_hal.sim_transport import SimTransport
>>> from openral_hal.ur_real import UR5eRealHAL
>>> transport = SimTransport(n_joints=6)
>>> hal = UR5eRealHAL(
...     publish_fn=transport.publish,
...     state_fn=transport.state,
... )
>>> hal.description.name
'ur5e'
>>> hal.description.sdk_kind
'closed'
>>> hal.description.hal.real
'openral_hal.ur_real:UR5eRealHAL'

Initialise the UR5e real-HW HAL; transport defaults match ur_robot_driver.

Source code in python/hal/src/openral_hal/ur_real.py

def __init__(
    self,
    *,
    robot_ip: str | None = None,
    publish_fn: Callable[[str, dict[str, object]], None] | None = None,
    state_fn: Callable[[], dict[str, object]] | None = None,
    staleness_limit_s: float = 0.5,
) -> None:
    """Initialise the UR5e real-HW HAL; transport defaults match ``ur_robot_driver``."""
    super().__init__(
        UR5e_REAL_DESCRIPTION,
        robot_ip=robot_ip,
        publish_fn=publish_fn,
        state_fn=state_fn,
        staleness_limit_s=staleness_limit_s,
    )

build_hal(description: RobotDescription, *, mode: HalMode, transport: dict[str, object] | None = None, sim_env_yaml: str | None = None) -> HAL

Construct the simulation or real-hardware HAL for description.

Parameters:

Name	Type	Description	Default
description	RobotDescription	The robot manifest (typically loaded via :meth:RobotDescription.from_yaml).	required
mode	HalMode	"sim" for the simulation HAL (deploy sim / sim run harness), "real" for the real-hardware HAL (deploy run).	required
transport	dict[str, object] | None	Constructor kwargs for the real HAL (serial port, robot_ip, fci_ip, …). Keys the target constructor does not accept are dropped. Ignored by the derived sim path. Merged over the manifest's hal.parameters.defaults (ADR-0029), so an explicit deploy run transport override wins.	None
sim_env_yaml	str | None	Path to a SimScene YAML (ADR-0034; renamed from SceneEnvironment in ADR-0041). When provided with mode="sim", returns a :class:~openral_hal.sim_attached.SimAttachedHAL wrapping the scene's :class:~openral_sim.rollout.SimRollout; bypasses the bare-twin / hal.sim class. Mutually exclusive with mode="real" — raises :class:~openral_core.exceptions.ROSConfigError if both are supplied.	None

Returns:

Type	Description
HAL	An un-connected HAL instance.

Raises:

Type	Description
ROSCapabilityMismatch	The robot has no HAL for mode (sim-only robot asked for "real", or real-only robot asked for "sim" with no sim: block to derive from).
ROSConfigError	mode is invalid, sim_env_yaml is supplied with mode="real", or a declared entrypoint is malformed / unresolvable.

Example

from openral_core import RobotDescription desc = RobotDescription.from_yaml("robots/so100_follower/robot.yaml") # doctest: +SKIP hal = build_hal(desc, mode="sim") # doctest: +SKIP

Source code in python/hal/src/openral_hal/resolver.py

def build_hal(
    description: RobotDescription,
    *,
    mode: HalMode,
    transport: dict[str, object] | None = None,
    sim_env_yaml: str | None = None,
) -> HAL:
    """Construct the simulation or real-hardware HAL for ``description``.

    Args:
        description: The robot manifest (typically loaded via
            :meth:`RobotDescription.from_yaml`).
        mode: ``"sim"`` for the simulation HAL (``deploy sim`` / ``sim run``
            harness), ``"real"`` for the real-hardware HAL (``deploy run``).
        transport: Constructor kwargs for the real HAL (serial ``port``,
            ``robot_ip``, ``fci_ip``, …). Keys the target constructor does not
            accept are dropped. Ignored by the derived sim path. Merged
            **over** the manifest's ``hal.parameters.defaults`` (ADR-0029), so
            an explicit ``deploy run`` transport override wins.
        sim_env_yaml: Path to a SimScene YAML (ADR-0034; renamed from
            SceneEnvironment in ADR-0041). When
            provided with ``mode="sim"``, returns a
            :class:`~openral_hal.sim_attached.SimAttachedHAL` wrapping the
            scene's :class:`~openral_sim.rollout.SimRollout`; bypasses the
            bare-twin / ``hal.sim`` class. Mutually exclusive with
            ``mode="real"`` — raises :class:`~openral_core.exceptions.ROSConfigError`
            if both are supplied.

    Returns:
        An un-connected HAL instance.

    Raises:
        ROSCapabilityMismatch: The robot has no HAL for ``mode`` (sim-only
            robot asked for ``"real"``, or real-only robot asked for
            ``"sim"`` with no ``sim:`` block to derive from).
        ROSConfigError: ``mode`` is invalid, ``sim_env_yaml`` is supplied
            with ``mode="real"``, or a declared entrypoint is malformed /
            unresolvable.

    Example:
        >>> from openral_core import RobotDescription
        >>> desc = RobotDescription.from_yaml("robots/so100_follower/robot.yaml")  # doctest: +SKIP
        >>> hal = build_hal(desc, mode="sim")  # doctest: +SKIP
    """
    # ADR-0029 — the manifest's hal.parameters.defaults supply the HAL's
    # construction kwargs (serial port, robot_ip, …) so a parameterised robot
    # needs no bespoke lifecycle subclass. Explicit ``transport`` overrides
    # them; _construct() then drops any key the constructor does not accept.
    resolved = {**description.hal.parameters.defaults, **(transport or {})}
    if sim_env_yaml is not None and mode != "sim":
        raise ROSConfigError(
            "build_hal: sim_env_yaml is only valid with mode='sim' "
            f"(got mode={mode!r}); a real-hardware HAL never attaches a sim scene."
        )
    if mode == "sim":
        if sim_env_yaml is not None:
            # ADR-0034 — deploy sim attaches the scene's SimRollout behind a
            # SimAttachedHAL (the scene owns physics + pixels). Bypasses the
            # bare twin / hal.sim class. openral_sim imported lazily so
            # openral_hal stays import-safe without the sim group.
            from openral_hal.sim_attached import SimAttachedHAL
            from openral_hal.sim_bringup import build_sim_env_from_yaml

            env, seed = build_sim_env_from_yaml(sim_env_yaml, robot_id_fallback=description.name)
            return SimAttachedHAL(env, description, env_reset_seed=seed)
        entry = description.hal.sim
        if entry is None:
            if description.sim is None:
                raise ROSCapabilityMismatch(
                    f"robot {description.name!r} has no simulation HAL: hal.sim is "
                    "null and there is no `sim:` block to derive MujocoArmHAL from. "
                    "It is real-hardware-only — use `deploy run`."
                )
            return MujocoArmHAL.from_description(description)
        return _construct(_import_object(entry), description, resolved)
    if mode == "real":
        entry = description.hal.real
        if entry is None:
            raise ROSCapabilityMismatch(
                f"robot {description.name!r} has no real-hardware HAL (hal.real is "
                "null); it is simulation-only — use `deploy sim` / `sim run`."
            )
        return _construct(_import_object(entry), description, resolved)
    raise ROSConfigError(f"build_hal: unknown mode {mode!r}; expected 'sim' or 'real'.")

franka_panda_with_sensors(catalog_ids: list[str] | None = None) -> RobotDescription

Return a copy of :data:FRANKA_PANDA_DESCRIPTION with catalog sensors attached.

The Franka Panda research community reference setup uses a wrist-mounted RealSense D435i; pass None to get that default, or override.

Parameters:

Name	Type	Description	Default
catalog_ids	list[str] | None	Catalog ids to resolve, or None for the Franka reference loadout (["intel/realsense_d435i"]).	None

Returns:

Type	Description
RobotDescription	A new :class:RobotDescription with sensors / sensor_bundles
RobotDescription	populated.

Example

desc = franka_panda_with_sensors() desc.sensor_bundles[0].sensors[0].vendor 'Intel'

Source code in python/hal/src/openral_hal/franka_panda.py

def franka_panda_with_sensors(
    catalog_ids: list[str] | None = None,
) -> RobotDescription:
    """Return a copy of :data:`FRANKA_PANDA_DESCRIPTION` with catalog sensors attached.

    The Franka Panda research community reference setup uses a wrist-mounted
    RealSense D435i; pass ``None`` to get that default, or override.

    Args:
        catalog_ids: Catalog ids to resolve, or ``None`` for the Franka
            reference loadout (``["intel/realsense_d435i"]``).

    Returns:
        A new :class:`RobotDescription` with ``sensors`` / ``sensor_bundles``
        populated.

    Example:
        >>> desc = franka_panda_with_sensors()
        >>> desc.sensor_bundles[0].sensors[0].vendor
        'Intel'
    """
    if catalog_ids is None:
        catalog_ids = ["intel/realsense_d435i"]
    return with_sensors(FRANKA_PANDA_DESCRIPTION, catalog_ids)

so100_with_sensors(catalog_ids: list[str] | None = None) -> RobotDescription

Return a copy of :data:SO100_DESCRIPTION with catalog sensors attached.

The reference LeRobot SO-100 setup uses a single Logitech C920 scene camera; pass None to get that default, or pass an explicit list of catalog ids ("logitech/c920", "intel/realsense_d405", …) to override. Pass [] to get an empty sensor loadout.

Parameters:

Name	Type	Description	Default
catalog_ids	list[str] | None	Catalog ids to resolve, or None for the LeRobot reference loadout (["logitech/c920"]).	None

Returns:

Type	Description
RobotDescription	A new :class:RobotDescription with sensors / sensor_bundles
RobotDescription	populated.

Example

desc = so100_with_sensors() desc.sensors[0].vendor 'Logitech'

Source code in python/hal/src/openral_hal/so100_follower.py

def so100_with_sensors(
    catalog_ids: list[str] | None = None,
) -> RobotDescription:
    """Return a copy of :data:`SO100_DESCRIPTION` with catalog sensors attached.

    The reference LeRobot SO-100 setup uses a single Logitech C920 scene
    camera; pass ``None`` to get that default, or pass an explicit list of
    catalog ids (``"logitech/c920"``, ``"intel/realsense_d405"``, …) to
    override.  Pass ``[]`` to get an empty sensor loadout.

    Args:
        catalog_ids: Catalog ids to resolve, or ``None`` for the LeRobot
            reference loadout (``["logitech/c920"]``).

    Returns:
        A new :class:`RobotDescription` with ``sensors`` / ``sensor_bundles``
        populated.

    Example:
        >>> desc = so100_with_sensors()
        >>> desc.sensors[0].vendor
        'Logitech'
    """
    if catalog_ids is None:
        catalog_ids = ["logitech/c920"]
    return with_sensors(SO100_DESCRIPTION, catalog_ids)

ur10e_with_sensors(catalog_ids: list[str] | None = None) -> RobotDescription

Return a copy of :data:UR10e_DESCRIPTION with catalog sensors attached.

Default loadout matches :func:ur5e_with_sensors (D435 + FT-300S); the UR10e shares the same flange interface and is usually paired with the higher-range D435 instead of the D415.

Parameters:

Name	Type	Description	Default
catalog_ids	list[str] | None	Catalog ids to resolve, or None for the UR10e reference loadout (["intel/realsense_d435", "robotiq/ft_300s"]).	None

Returns:

Type	Description
RobotDescription	A new :class:RobotDescription with sensors / sensor_bundles
RobotDescription	populated.

Example

desc = ur10e_with_sensors() desc.sensor_bundles[0].sensors[0].vendor 'Intel'

Source code in python/hal/src/openral_hal/ur.py

def ur10e_with_sensors(
    catalog_ids: list[str] | None = None,
) -> RobotDescription:
    """Return a copy of :data:`UR10e_DESCRIPTION` with catalog sensors attached.

    Default loadout matches :func:`ur5e_with_sensors` (D435 + FT-300S); the
    UR10e shares the same flange interface and is usually paired with the
    higher-range D435 instead of the D415.

    Args:
        catalog_ids: Catalog ids to resolve, or ``None`` for the UR10e
            reference loadout (``["intel/realsense_d435", "robotiq/ft_300s"]``).

    Returns:
        A new :class:`RobotDescription` with ``sensors`` / ``sensor_bundles``
        populated.

    Example:
        >>> desc = ur10e_with_sensors()
        >>> desc.sensor_bundles[0].sensors[0].vendor
        'Intel'
    """
    if catalog_ids is None:
        catalog_ids = ["intel/realsense_d435", "robotiq/ft_300s"]
    return with_sensors(UR10e_DESCRIPTION, catalog_ids)

ur5e_with_sensors(catalog_ids: list[str] | None = None) -> RobotDescription

Return a copy of :data:UR5e_DESCRIPTION with catalog sensors attached.

The reference UR5e research setup uses a flange-mounted RealSense D415 plus a Robotiq FT-300S wrist sensor; pass None to get that default.

Parameters:

Name	Type	Description	Default
catalog_ids	list[str] | None	Catalog ids to resolve, or None for the UR5e reference loadout (["intel/realsense_d415", "robotiq/ft_300s"]).	None

Returns:

Type	Description
RobotDescription	A new :class:RobotDescription with sensors / sensor_bundles
RobotDescription	populated.

Example

desc = ur5e_with_sensors() desc.sensors[0].vendor 'Robotiq'

Source code in python/hal/src/openral_hal/ur.py

def ur5e_with_sensors(
    catalog_ids: list[str] | None = None,
) -> RobotDescription:
    """Return a copy of :data:`UR5e_DESCRIPTION` with catalog sensors attached.

    The reference UR5e research setup uses a flange-mounted RealSense D415
    plus a Robotiq FT-300S wrist sensor; pass ``None`` to get that default.

    Args:
        catalog_ids: Catalog ids to resolve, or ``None`` for the UR5e
            reference loadout (``["intel/realsense_d415", "robotiq/ft_300s"]``).

    Returns:
        A new :class:`RobotDescription` with ``sensors`` / ``sensor_bundles``
        populated.

    Example:
        >>> desc = ur5e_with_sensors()
        >>> desc.sensors[0].vendor
        'Robotiq'
    """
    if catalog_ids is None:
        catalog_ids = ["intel/realsense_d415", "robotiq/ft_300s"]
    return with_sensors(UR5e_DESCRIPTION, catalog_ids)

Sensors

openral Sensors — sensor adapters, catalog and launch generators.

Public surface: - CATALOG: global :class:SensorCatalog populated by every vendor module on import. Used by openral sensor list / show and the (future) catalog: reference field on RobotDescription sensors. - SensorCatalog, SensorCatalogEntry: the registry types. - Per-vendor factories — see each <vendor>.py module: * realsense: D415 / D435 / D435i bundles. * luxonis: OAK-D Pro bundle. * usb_uvc: Logitech C920. * force_torque: Robotiq FT 300-S 6-axis F/T sensor. - Launch helpers (RealSense): bundle_to_node_params, generate_launch_py, calibrate_camera_cmd.

SensorCatalog(_entries: dict[str, SensorCatalogEntry] = dict()) dataclass

In-memory registry of SensorCatalogEntry rows.

The default global instance is :data:CATALOG; tests should create a private :class:SensorCatalog to avoid polluting the global registry.

Example

cat = SensorCatalog() def make() -> SensorSpec: ... return SensorSpec(name="x", modality=SensorModality.RGB, frame_id="x", rate_hz=30.0) _ = cat.register( ... SensorCatalogEntry( ... id="acme/cam", ... vendor="acme", ... model="cam", ... kind="sensor", ... factory=make, ... modalities=(SensorModality.RGB,), ... ) ... ) cat.get("acme/cam").vendor 'acme' cat.list_ids() ['acme/cam']

__contains__(sensor_id: object) -> bool

Membership check by sensor id.

Source code in python/sensors/src/openral_sensors/catalog.py

def __contains__(self, sensor_id: object) -> bool:
    """Membership check by sensor id."""
    return isinstance(sensor_id, str) and sensor_id in self._entries

__iter__() -> object

Iterate over registered ids.

Source code in python/sensors/src/openral_sensors/catalog.py

def __iter__(self) -> object:
    """Iterate over registered ids."""
    return iter(self._entries)

__len__() -> int

Number of registered entries.

Source code in python/sensors/src/openral_sensors/catalog.py

def __len__(self) -> int:
    """Number of registered entries."""
    return len(self._entries)

build(sensor_id: str, **kwargs: object) -> SensorSpec | SensorBundle

Resolve sensor_id and call its factory with **kwargs.

This is the entry point used by the (future) catalog: reference in robot.yaml — the loader passes any per-instance overrides (name, parent_frame, serial_no, …) as keyword arguments.

Source code in python/sensors/src/openral_sensors/catalog.py

def build(self, sensor_id: str, **kwargs: object) -> SensorSpec | SensorBundle:
    """Resolve ``sensor_id`` and call its factory with ``**kwargs``.

    This is the entry point used by the (future) ``catalog:`` reference in
    ``robot.yaml`` — the loader passes any per-instance overrides
    (``name``, ``parent_frame``, ``serial_no``, …) as keyword arguments.
    """
    entry = self.get(sensor_id)
    return entry.factory(**kwargs)

entries() -> list[SensorCatalogEntry]

All entries, sorted alphabetically by id.

Source code in python/sensors/src/openral_sensors/catalog.py

def entries(self) -> list[SensorCatalogEntry]:
    """All entries, sorted alphabetically by id."""
    return [self._entries[k] for k in sorted(self._entries)]

filter(*, vendor: str | None = None, modality: SensorModality | None = None, kind: Literal['sensor', 'bundle'] | None = None) -> list[SensorCatalogEntry]

Return entries that match all provided predicates.

Source code in python/sensors/src/openral_sensors/catalog.py

def filter(
    self,
    *,
    vendor: str | None = None,
    modality: SensorModality | None = None,
    kind: Literal["sensor", "bundle"] | None = None,
) -> list[SensorCatalogEntry]:
    """Return entries that match all provided predicates."""
    out: list[SensorCatalogEntry] = []
    for entry in self.entries():
        if vendor is not None and entry.vendor != vendor:
            continue
        if modality is not None and modality not in entry.modalities:
            continue
        if kind is not None and entry.kind != kind:
            continue
        out.append(entry)
    return out

find_by_signature(signature: SensorSignature) -> SensorCatalogEntry | None

Look up an entry by probe-side signature.

Returns None when no registered entry advertises the signature. Callers (the openral detect assembler) decide between catalog-backed materialisation and a fallback generic SensorSpec.

Example

from openral_sensors import CATALOG from openral_sensors.catalog import SensorSignature entry = CATALOG.find_by_signature(SensorSignature(kind="realsense", value="D435I")) entry is not None and entry.id == "intel/realsense_d435i" True

Source code in python/sensors/src/openral_sensors/catalog.py

def find_by_signature(self, signature: SensorSignature) -> SensorCatalogEntry | None:
    """Look up an entry by probe-side signature.

    Returns ``None`` when no registered entry advertises the signature.
    Callers (the ``openral detect`` assembler) decide between catalog-backed
    materialisation and a fallback generic ``SensorSpec``.

    Example:
        >>> from openral_sensors import CATALOG
        >>> from openral_sensors.catalog import SensorSignature
        >>> entry = CATALOG.find_by_signature(SensorSignature(kind="realsense", value="D435I"))
        >>> entry is not None and entry.id == "intel/realsense_d435i"
        True
    """
    for entry in self._entries.values():
        if signature in entry.signatures:
            return entry
    return None

get(sensor_id: str) -> SensorCatalogEntry

Look up an entry by id. Raises KeyError if absent.

Source code in python/sensors/src/openral_sensors/catalog.py

def get(self, sensor_id: str) -> SensorCatalogEntry:
    """Look up an entry by id.  Raises ``KeyError`` if absent."""
    try:
        return self._entries[sensor_id]
    except KeyError as exc:
        raise KeyError(
            f"Unknown sensor id {sensor_id!r}.  Known ids: {sorted(self._entries)!r}"
        ) from exc

keys() -> list[str]

All registered ids (insertion order).

Source code in python/sensors/src/openral_sensors/catalog.py

def keys(self) -> list[str]:
    """All registered ids (insertion order)."""
    return list(self._entries.keys())

list_ids() -> list[str]

All registered ids, sorted alphabetically.

Source code in python/sensors/src/openral_sensors/catalog.py

def list_ids(self) -> list[str]:
    """All registered ids, sorted alphabetically."""
    return sorted(self._entries)

register(entry: SensorCatalogEntry, *, replace: bool = False) -> SensorCatalogEntry

Register an entry.

Raises KeyError on duplicate id unless replace=True.

Source code in python/sensors/src/openral_sensors/catalog.py

def register(self, entry: SensorCatalogEntry, *, replace: bool = False) -> SensorCatalogEntry:
    """Register an entry.

    Raises ``KeyError`` on duplicate id unless ``replace=True``.
    """
    if not replace and entry.id in self._entries:
        raise KeyError(
            f"Sensor catalog already has an entry for id={entry.id!r}; "
            "pass replace=True to overwrite."
        )
    self._entries[entry.id] = entry
    return entry

register_many(entries: Iterable[SensorCatalogEntry], *, replace: bool = True) -> None

Bulk-register catalog entries.

Used by vendor modules at import time to populate the global CATALOG. Defaults to replace=True because side-effect imports may run more than once in some test setups.

Source code in python/sensors/src/openral_sensors/catalog.py

def register_many(self, entries: Iterable[SensorCatalogEntry], *, replace: bool = True) -> None:
    """Bulk-register catalog entries.

    Used by vendor modules at import time to populate the global CATALOG.
    Defaults to ``replace=True`` because side-effect imports may run more
    than once in some test setups.
    """
    for entry in entries:
        self.register(entry, replace=replace)

unregister(sensor_id: str) -> None

Remove an entry. Idempotent.

Source code in python/sensors/src/openral_sensors/catalog.py

def unregister(self, sensor_id: str) -> None:
    """Remove an entry.  Idempotent."""
    self._entries.pop(sensor_id, None)

SensorCatalogEntry(id: str, vendor: str, model: str, kind: Literal['sensor', 'bundle'], factory: SensorFactory, modalities: tuple[SensorModality, ...], description: str = '', docs_url: str | None = None, signatures: tuple[SensorSignature, ...] = ()) dataclass

One row in the sensor catalog.

Attributes:

Name	Type	Description
id	str	Stable identifier in the form "<vendor>/<model>" — lowercase, slugified. Used as the lookup key and as the value of the future catalog: field in robot.yaml.
vendor	str	Vendor or maintainer name (free-form, lowercase).
model	str	Vendor model identifier (free-form, lowercase / slugified).
kind	Literal['sensor', 'bundle']	Whether the factory returns a single SensorSpec or a multi-sensor SensorBundle.
factory	SensorFactory	Callable that constructs the spec/bundle. Must accept at least the same keyword arguments as the underlying factory; the common convention is name, parent_frame, rate_hz style.
modalities	tuple[SensorModality, ...]	SensorModality values that this sensor exposes.
description	str	One-line human-readable summary.
docs_url	str | None	Optional link to the vendor data sheet or product page.
signatures	tuple[SensorSignature, ...]	Probe-side identifiers that resolve to this entry. Used by openral detect to map a discovered device (RealSense model_id, USB VID/PID, V4L2 product string) to the canonical catalog ID.

SensorSignature(kind: SensorSignatureKind, value: str) dataclass

A probe-side identifier that resolves to one catalog entry.

SensorSignature is the bridge between a live device discovered by openral detect and the canonical SensorCatalogEntry whose factory materialises a fully-populated SensorSpec (with real intrinsics, FOV, encoding, rate, …). Each catalog entry may publish multiple signatures (e.g. a USB UVC camera with several VID/PID rebrands).

Encoding by kind: - "realsense" — pyrealsense2 model_id (e.g. "D435I"). - "orbbec" — Orbbec SDK product name. - "usb_uvc" — "0xVVVV:0xPPPP" lowercase hex. - "v4l2_name" — V4L2 product string (substring match acceptable).

Attributes:

Name	Type	Description
kind	SensorSignatureKind	Probe family that produced this signature.
value	str	Vendor-specific canonical key.

Example

SensorSignature(kind="realsense", value="D435I") SensorSignature(kind='realsense', value='D435I')

bundle_to_node_params(bundle: SensorBundle, serial_no: str = '') -> NodeParams

Map a RealSense SensorBundle to realsense2_camera node parameters.

Only bundles whose sensors carry driver_pkg="realsense2_camera" are supported. Parameters are derived from the first RGB and depth SensorSpec found in the bundle.

Parameters:

Name	Type	Description	Default
bundle	SensorBundle	A SensorBundle produced by :func:realsense_d435_bundle, :func:realsense_d435i_bundle, or :func:realsense_d415_bundle.	required
serial_no	str	Device serial number. Overrides any value stored in sensor metadata.	''

Returns:

Type	Description
NodeParams	A flat parameter dict suitable for the realsense2_camera
NodeParams	composable node.

Raises:

Type	Description
ValueError	If no RGB sensor is found in the bundle.

Example

bundle = realsense_d435_bundle(name="head") params = bundle_to_node_params(bundle) params["camera_name"] 'head' params["rgb_camera.color_profile"] '640x480x30'

Source code in python/sensors/src/openral_sensors/realsense.py

def bundle_to_node_params(bundle: SensorBundle, serial_no: str = "") -> NodeParams:
    """Map a RealSense ``SensorBundle`` to ``realsense2_camera`` node parameters.

    Only bundles whose sensors carry ``driver_pkg="realsense2_camera"`` are
    supported.  Parameters are derived from the first RGB and depth
    ``SensorSpec`` found in the bundle.

    Args:
        bundle: A ``SensorBundle`` produced by :func:`realsense_d435_bundle`,
            :func:`realsense_d435i_bundle`, or :func:`realsense_d415_bundle`.
        serial_no: Device serial number.  Overrides any value stored in
            sensor metadata.

    Returns:
        A flat parameter dict suitable for the ``realsense2_camera``
        composable node.

    Raises:
        ValueError: If no RGB sensor is found in the bundle.

    Example:
        >>> bundle = realsense_d435_bundle(name="head")
        >>> params = bundle_to_node_params(bundle)
        >>> params["camera_name"]
        'head'
        >>> params["rgb_camera.color_profile"]
        '640x480x30'
    """
    rgb = next(
        (s for s in bundle.sensors if s.modality == SensorModality.RGB),
        None,
    )
    if rgb is None:
        raise ValueError(f"SensorBundle '{bundle.bundle_name}' has no RGB sensor.")

    depth = next(
        (s for s in bundle.sensors if s.modality == SensorModality.DEPTH),
        None,
    )
    imu = next(
        (s for s in bundle.sensors if s.modality == SensorModality.IMU),
        None,
    )

    # Resolve serial number: explicit arg > metadata > empty
    sn = serial_no or str(rgb.metadata.get("serial_no", ""))

    rgb_w = rgb.intrinsics.width if rgb.intrinsics else 640
    rgb_h = rgb.intrinsics.height if rgb.intrinsics else 480
    rgb_fps = int(rgb.rate_hz)

    params: NodeParams = {
        "camera_name": bundle.bundle_name,
        "camera_namespace": f"/{bundle.bundle_name}",
        "serial_no": sn,
        "enable_color": True,
        "rgb_camera.color_profile": f"{rgb_w}x{rgb_h}x{rgb_fps}",
        "enable_depth": depth is not None,
        "enable_gyro": imu is not None,
        "enable_accel": imu is not None,
        "unite_imu_method": "1",  # linear interpolation
        "align_depth.enable": True,
        "pointcloud.enable": False,
    }

    if depth is not None:
        d_w = depth.intrinsics.width if depth.intrinsics else 640
        d_h = depth.intrinsics.height if depth.intrinsics else 480
        d_fps = int(depth.rate_hz)
        params["depth_module.depth_profile"] = f"{d_w}x{d_h}x{d_fps}"

    return params

calibrate_camera_cmd(sensor: SensorSpec, chessboard_cols: int = 8, chessboard_rows: int = 6, square_size_m: float = 0.025) -> list[str]

Build a ros2 run camera_calibration cameracalibrator command.

The command can be passed to subprocess.run or printed with --dry-run so the user can inspect it before execution.

Parameters:

Name	Type	Description	Default
sensor	SensorSpec	The RGB SensorSpec to calibrate.	required
chessboard_cols	int	Number of internal corners along the long axis.	8
chessboard_rows	int	Number of internal corners along the short axis.	6
square_size_m	float	Physical size of one chessboard square in metres.	0.025

Returns:

Type	Description
list[str]	A list of command tokens suitable for subprocess.run.

Raises:

Type	Description
ValueError	If sensor.modality is not RGB.

Example

from openral_sensors.realsense import realsense_d435_bundle bundle = realsense_d435_bundle(name="head") rgb_spec = bundle.sensors[0] cmd = calibrate_camera_cmd(rgb_spec, chessboard_cols=8, chessboard_rows=6) cmd[0] 'ros2' "--size" in cmd True "8x6" in cmd True

Source code in python/sensors/src/openral_sensors/realsense.py

def calibrate_camera_cmd(
    sensor: SensorSpec,
    chessboard_cols: int = 8,
    chessboard_rows: int = 6,
    square_size_m: float = 0.025,
) -> list[str]:
    """Build a ``ros2 run camera_calibration cameracalibrator`` command.

    The command can be passed to ``subprocess.run`` or printed with
    ``--dry-run`` so the user can inspect it before execution.

    Args:
        sensor: The RGB ``SensorSpec`` to calibrate.
        chessboard_cols: Number of internal corners along the long axis.
        chessboard_rows: Number of internal corners along the short axis.
        square_size_m: Physical size of one chessboard square in metres.

    Returns:
        A list of command tokens suitable for ``subprocess.run``.

    Raises:
        ValueError: If ``sensor.modality`` is not ``RGB``.

    Example:
        >>> from openral_sensors.realsense import realsense_d435_bundle
        >>> bundle = realsense_d435_bundle(name="head")
        >>> rgb_spec = bundle.sensors[0]
        >>> cmd = calibrate_camera_cmd(rgb_spec, chessboard_cols=8, chessboard_rows=6)
        >>> cmd[0]
        'ros2'
        >>> "--size" in cmd
        True
        >>> "8x6" in cmd
        True
    """
    if sensor.modality != SensorModality.RGB:
        raise ValueError(
            f"Sensor '{sensor.name}' has modality '{sensor.modality}', "
            "expected 'rgb' for camera calibration."
        )

    size_arg = f"{chessboard_cols}x{chessboard_rows}"
    square_arg = str(square_size_m)

    # Derive camera_info topic from image topic convention
    # e.g. /head/color/image_raw → /head/color/camera_info
    topic = sensor.ros2_topic
    if topic is None:
        raise ValueError(
            f"Sensor '{sensor.name}' has no ros2_topic; cannot derive camera_info remap."
        )
    info_topic = topic.replace("/image_raw", "/camera_info").replace(
        "/image_rect_raw", "/camera_info"
    )

    return [
        "ros2",
        "run",
        "camera_calibration",
        "cameracalibrator",
        "--size",
        size_arg,
        "--square",
        square_arg,
        "--ros-args",
        "-r",
        f"image:={topic}",
        "-r",
        f"camera_info:={info_topic}",
    ]

generate_launch_py(bundle: SensorBundle, serial_no: str = '') -> str

Generate a ROS 2 Python launch file string for a RealSense bundle.

The generated file uses the realsense2_camera composable node and does not depend on launch at generation time — this function is fully testable without a ROS 2 installation.

Parameters:

Name	Type	Description	Default
bundle	SensorBundle	A SensorBundle for the target camera.	required
serial_no	str	Optional device serial number forwarded to the node.	''

Returns:

Type	Description
str	A Python string that, when written to <name>.launch.py and
str	executed with ros2 launch, starts the camera driver with the
str	correct parameters.

Example

bundle = realsense_d435_bundle(name="head") src = generate_launch_py(bundle) "realsense2_camera" in src True "head" in src True src.startswith("# Generated by OpenRAL") True

Source code in python/sensors/src/openral_sensors/realsense.py

def generate_launch_py(bundle: SensorBundle, serial_no: str = "") -> str:
    """Generate a ROS 2 Python launch file string for a RealSense bundle.

    The generated file uses the ``realsense2_camera`` composable node and
    does **not** depend on ``launch`` at generation time — this function is
    fully testable without a ROS 2 installation.

    Args:
        bundle: A ``SensorBundle`` for the target camera.
        serial_no: Optional device serial number forwarded to the node.

    Returns:
        A Python string that, when written to ``<name>.launch.py`` and
        executed with ``ros2 launch``, starts the camera driver with the
        correct parameters.

    Example:
        >>> bundle = realsense_d435_bundle(name="head")
        >>> src = generate_launch_py(bundle)
        >>> "realsense2_camera" in src
        True
        >>> "head" in src
        True
        >>> src.startswith("# Generated by OpenRAL")
        True
    """
    params = bundle_to_node_params(bundle, serial_no=serial_no)

    # Format the parameter dict as Python literal lines for the launch file.
    param_lines = "\n".join(f"            {k!r}: {v!r}," for k, v in params.items())

    return textwrap.dedent(
        f"""\
        # Generated by OpenRAL ral sensors — do not edit by hand.
        # Re-generate with: ral launch sensor --bundle {bundle.bundle_name}
        from launch import LaunchDescription
        from launch_ros.actions import ComposableNodeContainer
        from launch_ros.descriptions import ComposableNode


        def generate_launch_description() -> LaunchDescription:
            container = ComposableNodeContainer(
                name="{bundle.bundle_name}_container",
                namespace="/{bundle.bundle_name}",
                package="rclcpp_components",
                executable="component_container",
                composable_node_descriptions=[
                    ComposableNode(
                        package="realsense2_camera",
                        plugin="realsense2_camera::RealSenseNodeFactory",
                        name="{bundle.bundle_name}",
                        parameters=[
                            {{
        {param_lines}
                            }}
                        ],
                        extra_arguments=[{{"use_intra_process_comms": True}}],
                    ),
                ],
                output="screen",
            )
            return LaunchDescription([container])
        """
    )

oak_d_pro_bundle(name: str = 'oak', parent_frame: str = 'base_link', mxid: str = '', rgb_rate_hz: float = 30.0, depth_rate_hz: float = 30.0, imu_rate_hz: float = 400.0, rgb_width: int = 1920, rgb_height: int = 1080, depth_width: int = 1280, depth_height: int = 800) -> SensorBundle

Build a :class:SensorBundle for a Luxonis OAK-D Pro.

The bundle exposes three streams: RGB (IMX378), depth (OV9282 global-shutter stereo) and the on-device BNO086 IMU. Nominal intrinsics are from the Luxonis datasheet at the default resolutions; replace with calibrated values before deploying to hardware.

Parameters:

Name	Type	Description	Default
name	str	Bundle name — used as a prefix for topic names and frame ids (e.g. "oak_top" → /oak_top/rgb/image_raw).	'oak'
parent_frame	str	tf2 parent frame for the static transform.	'base_link'
mxid	str	Optional Luxonis MXID for device targeting (multi-device hosts). Stored in metadata.mxid and forwarded to the ROS driver's i_mxid parameter.	''
rgb_rate_hz	float	RGB stream publish rate.	30.0
depth_rate_hz	float	Depth stream publish rate.	30.0
imu_rate_hz	float	IMU publish rate (BNO086).	400.0
rgb_width	int	RGB stream width. The factory rescales the nominal intrinsics linearly when this differs from the default 1920.	1920
rgb_height	int	RGB stream height.	1080
depth_width	int	Depth stream width.	1280
depth_height	int	Depth stream height.	800

Returns:

Name	Type	Description
A	SensorBundle	class:SensorBundle with sync="hardware" and 5 ms
	SensorBundle	tolerance (RVC2 syncs RGB + depth on-device).

Example

bundle = oak_d_pro_bundle(name="oak_top", parent_frame="box_ceiling") bundle.sensors[1].modality 'depth' bundle.sensors[1].range_max_m 19.0

Source code in python/sensors/src/openral_sensors/luxonis.py

def oak_d_pro_bundle(
    name: str = "oak",
    parent_frame: str = "base_link",
    mxid: str = "",
    rgb_rate_hz: float = 30.0,
    depth_rate_hz: float = 30.0,
    imu_rate_hz: float = 400.0,
    rgb_width: int = 1920,
    rgb_height: int = 1080,
    depth_width: int = 1280,
    depth_height: int = 800,
) -> SensorBundle:
    """Build a :class:`SensorBundle` for a Luxonis OAK-D Pro.

    The bundle exposes three streams: RGB (IMX378), depth (OV9282
    global-shutter stereo) and the on-device BNO086 IMU. Nominal
    intrinsics are from the Luxonis datasheet at the default
    resolutions; replace with calibrated values before deploying to
    hardware.

    Args:
        name: Bundle name — used as a prefix for topic names and frame
            ids (e.g. ``"oak_top"`` → ``/oak_top/rgb/image_raw``).
        parent_frame: tf2 parent frame for the static transform.
        mxid: Optional Luxonis MXID for device targeting (multi-device
            hosts). Stored in ``metadata.mxid`` and forwarded to the
            ROS driver's ``i_mxid`` parameter.
        rgb_rate_hz: RGB stream publish rate.
        depth_rate_hz: Depth stream publish rate.
        imu_rate_hz: IMU publish rate (BNO086).
        rgb_width: RGB stream width. The factory rescales the nominal
            intrinsics linearly when this differs from the default
            1920.
        rgb_height: RGB stream height.
        depth_width: Depth stream width.
        depth_height: Depth stream height.

    Returns:
        A :class:`SensorBundle` with ``sync="hardware"`` and 5 ms
        tolerance (RVC2 syncs RGB + depth on-device).

    Example:
        >>> bundle = oak_d_pro_bundle(name="oak_top", parent_frame="box_ceiling")
        >>> bundle.sensors[1].modality
        'depth'
        >>> bundle.sensors[1].range_max_m
        19.0
    """
    meta: dict[str, object] = {"mxid": mxid} if mxid else {}

    rgb_intr = _scale_intrinsics(_OAK_D_PRO_RGB_INTRINSICS, rgb_width, rgb_height)
    depth_intr = _scale_intrinsics(_OAK_D_PRO_DEPTH_INTRINSICS, depth_width, depth_height)

    rgb = SensorSpec(
        name=f"{name}_color",
        modality=SensorModality.RGB,
        frame_id=f"{name}_rgb_optical_frame",
        parent_frame=parent_frame,
        rate_hz=rgb_rate_hz,
        encoding="rgb8",
        intrinsics=rgb_intr,
        fov_h_deg=95.0,
        fov_v_deg=70.0,
        ros2_topic=f"/{name}/rgb/image_raw",
        ros2_msg_type="sensor_msgs/Image",
        vendor="Luxonis",
        model="OAK-D Pro",
        driver_pkg="depthai_ros_driver",
        metadata=meta,
    )
    depth = SensorSpec(
        name=f"{name}_depth",
        modality=SensorModality.DEPTH,
        frame_id=f"{name}_depth_optical_frame",
        parent_frame=parent_frame,
        rate_hz=depth_rate_hz,
        encoding="16UC1",
        intrinsics=depth_intr,
        fov_h_deg=71.86,
        fov_v_deg=56.0,
        range_min_m=0.20,
        range_max_m=19.0,
        ros2_topic=f"/{name}/stereo/image_raw",
        ros2_msg_type="sensor_msgs/Image",
        vendor="Luxonis",
        model="OAK-D Pro",
        driver_pkg="depthai_ros_driver",
        metadata=meta,
    )
    imu = SensorSpec(
        name=f"{name}_imu",
        modality=SensorModality.IMU,
        frame_id=f"{name}_imu_frame",
        parent_frame=parent_frame,
        rate_hz=imu_rate_hz,
        accel_noise_density=3.0e-3,
        gyro_noise_density=3.5e-3,
        ros2_topic=f"/{name}/imu/data",
        ros2_msg_type="sensor_msgs/Imu",
        vendor="Luxonis",
        model="OAK-D Pro (BNO086 IMU)",
        driver_pkg="depthai_ros_driver",
        metadata=meta,
    )
    return SensorBundle(
        bundle_name=name,
        sensors=[rgb, depth, imu],
        sync="hardware",
        sync_tolerance_ms=5.0,
    )

realsense_d415_bundle(name: str = 'realsense', parent_frame: str = 'base_link', serial_no: str = '', rgb_rate_hz: float = 30.0, depth_rate_hz: float = 30.0) -> SensorBundle

Build a SensorBundle for an Intel RealSense D415.

Rolling-shutter IR-stereo camera, 65°×40° FOV, 0.16 – 10 m range; no IMU.

Example

bundle = realsense_d415_bundle(name="head") bundle.sensors[1].range_min_m 0.16

Source code in python/sensors/src/openral_sensors/realsense.py

def realsense_d415_bundle(
    name: str = "realsense",
    parent_frame: str = "base_link",
    serial_no: str = "",
    rgb_rate_hz: float = 30.0,
    depth_rate_hz: float = 30.0,
) -> SensorBundle:
    """Build a ``SensorBundle`` for an Intel RealSense D415.

    Rolling-shutter IR-stereo camera, 65°×40° FOV, 0.16 – 10 m range; no IMU.

    Example:
        >>> bundle = realsense_d415_bundle(name="head")
        >>> bundle.sensors[1].range_min_m
        0.16
    """
    meta: dict[str, object] = {"serial_no": serial_no} if serial_no else {}
    rgb = SensorSpec(
        name=f"{name}_color",
        modality=SensorModality.RGB,
        frame_id=f"{name}_color_optical_frame",
        parent_frame=parent_frame,
        rate_hz=rgb_rate_hz,
        encoding="rgb8",
        intrinsics=_D415_RGB_INTRINSICS,
        fov_h_deg=65.0,
        fov_v_deg=40.0,
        ros2_topic=f"/{name}/color/image_raw",
        ros2_msg_type="sensor_msgs/Image",
        vendor="Intel",
        model="RealSense D415",
        driver_pkg="realsense2_camera",
        metadata=meta,
    )
    depth = SensorSpec(
        name=f"{name}_depth",
        modality=SensorModality.DEPTH,
        frame_id=f"{name}_depth_optical_frame",
        parent_frame=parent_frame,
        rate_hz=depth_rate_hz,
        encoding="16UC1",
        intrinsics=_D415_DEPTH_INTRINSICS,
        fov_h_deg=65.0,
        fov_v_deg=40.0,
        range_min_m=0.16,
        range_max_m=10.0,
        ros2_topic=f"/{name}/depth/image_rect_raw",
        ros2_msg_type="sensor_msgs/Image",
        vendor="Intel",
        model="RealSense D415",
        driver_pkg="realsense2_camera",
        metadata=meta,
    )
    return SensorBundle(
        bundle_name=name,
        sensors=[rgb, depth],
        sync="hardware",
        sync_tolerance_ms=5.0,
    )

realsense_d435_bundle(name: str = 'realsense', parent_frame: str = 'base_link', serial_no: str = '', rgb_rate_hz: float = 30.0, depth_rate_hz: float = 30.0, imu_rate_hz: float = 400.0) -> SensorBundle

Build a SensorBundle for an Intel RealSense D435.

The bundle contains three sensors: RGB, depth, and IMU. Nominal intrinsics are from the Intel D400 Series data sheet at 640x480; replace them with calibrated values before deploying to hardware.

Parameters:

Name	Type	Description	Default
name	str	Sensor bundle name, used as a prefix for topic names and frame IDs (e.g. "head" → /head/color/image_raw).	'realsense'
parent_frame	str	tf2 parent frame for the static transform.	'base_link'
serial_no	str	Optional serial number for the device. Stored in metadata and forwarded to realsense2_camera as serial_no.	''
rgb_rate_hz	float	RGB stream publish rate.	30.0
depth_rate_hz	float	Depth stream publish rate.	30.0
imu_rate_hz	float	IMU publish rate.	400.0

Returns:

Type	Description
SensorBundle	A SensorBundle with sync="hardware" and 5 ms tolerance.

Example

bundle = realsense_d435_bundle(name="wrist", parent_frame="ee_link") bundle.sensors[0].frame_id 'wrist_color_optical_frame' bundle.sensors[1].modality 'depth' bundle.sensors[2].rate_hz 400.0

Source code in python/sensors/src/openral_sensors/realsense.py

def realsense_d435_bundle(
    name: str = "realsense",
    parent_frame: str = "base_link",
    serial_no: str = "",
    rgb_rate_hz: float = 30.0,
    depth_rate_hz: float = 30.0,
    imu_rate_hz: float = 400.0,
) -> SensorBundle:
    """Build a ``SensorBundle`` for an Intel RealSense D435.

    The bundle contains three sensors: RGB, depth, and IMU.  Nominal
    intrinsics are from the Intel D400 Series data sheet at 640x480; replace
    them with calibrated values before deploying to hardware.

    Args:
        name: Sensor bundle name, used as a prefix for topic names and frame
            IDs (e.g. ``"head"`` → ``/head/color/image_raw``).
        parent_frame: tf2 parent frame for the static transform.
        serial_no: Optional serial number for the device.  Stored in metadata
            and forwarded to ``realsense2_camera`` as ``serial_no``.
        rgb_rate_hz: RGB stream publish rate.
        depth_rate_hz: Depth stream publish rate.
        imu_rate_hz: IMU publish rate.

    Returns:
        A ``SensorBundle`` with ``sync="hardware"`` and 5 ms tolerance.

    Example:
        >>> bundle = realsense_d435_bundle(name="wrist", parent_frame="ee_link")
        >>> bundle.sensors[0].frame_id
        'wrist_color_optical_frame'
        >>> bundle.sensors[1].modality
        'depth'
        >>> bundle.sensors[2].rate_hz
        400.0
    """
    meta: dict[str, object] = {"serial_no": serial_no} if serial_no else {}
    rgb = SensorSpec(
        name=f"{name}_color",
        modality=SensorModality.RGB,
        frame_id=f"{name}_color_optical_frame",
        parent_frame=parent_frame,
        rate_hz=rgb_rate_hz,
        encoding="rgb8",
        intrinsics=_D435_RGB_INTRINSICS,
        fov_h_deg=69.4,
        fov_v_deg=42.5,
        ros2_topic=f"/{name}/color/image_raw",
        ros2_msg_type="sensor_msgs/Image",
        vendor="Intel",
        model="RealSense D435",
        driver_pkg="realsense2_camera",
        metadata=meta,
    )
    depth = SensorSpec(
        name=f"{name}_depth",
        modality=SensorModality.DEPTH,
        frame_id=f"{name}_depth_optical_frame",
        parent_frame=parent_frame,
        rate_hz=depth_rate_hz,
        encoding="16UC1",
        intrinsics=_D435_DEPTH_INTRINSICS,
        fov_h_deg=87.0,
        fov_v_deg=58.0,
        range_min_m=0.1,
        range_max_m=10.0,
        ros2_topic=f"/{name}/depth/image_rect_raw",
        ros2_msg_type="sensor_msgs/Image",
        vendor="Intel",
        model="RealSense D435",
        driver_pkg="realsense2_camera",
        metadata=meta,
    )
    imu = SensorSpec(
        name=f"{name}_imu",
        modality=SensorModality.IMU,
        frame_id=f"{name}_imu_optical_frame",
        parent_frame=parent_frame,
        rate_hz=imu_rate_hz,
        accel_noise_density=2.0e-3,
        gyro_noise_density=5.0e-3,
        ros2_topic=f"/{name}/imu",
        ros2_msg_type="sensor_msgs/Imu",
        vendor="Intel",
        model="RealSense D435",
        driver_pkg="realsense2_camera",
        metadata=meta,
    )
    return SensorBundle(
        bundle_name=name,
        sensors=[rgb, depth, imu],
        sync="hardware",
        sync_tolerance_ms=5.0,
    )

realsense_d435i_bundle(name: str = 'realsense', parent_frame: str = 'base_link', serial_no: str = '', rgb_rate_hz: float = 30.0, depth_rate_hz: float = 30.0, imu_rate_hz: float = 400.0) -> SensorBundle

Build a SensorBundle for an Intel RealSense D435i.

The D435i is a D435 with an integrated Bosch BMI085 IMU; the existing :func:realsense_d435_bundle already includes the IMU sensor, so this factory delegates to it and stamps model="RealSense D435i" on each constituent SensorSpec.

Example

bundle = realsense_d435i_bundle(name="wrist") bundle.sensors[0].model 'RealSense D435i'

Source code in python/sensors/src/openral_sensors/realsense.py

def realsense_d435i_bundle(
    name: str = "realsense",
    parent_frame: str = "base_link",
    serial_no: str = "",
    rgb_rate_hz: float = 30.0,
    depth_rate_hz: float = 30.0,
    imu_rate_hz: float = 400.0,
) -> SensorBundle:
    """Build a ``SensorBundle`` for an Intel RealSense D435i.

    The D435i is a D435 with an integrated Bosch BMI085 IMU; the existing
    :func:`realsense_d435_bundle` already includes the IMU sensor, so this
    factory delegates to it and stamps ``model="RealSense D435i"`` on each
    constituent ``SensorSpec``.

    Example:
        >>> bundle = realsense_d435i_bundle(name="wrist")
        >>> bundle.sensors[0].model
        'RealSense D435i'
    """
    bundle = realsense_d435_bundle(
        name=name,
        parent_frame=parent_frame,
        serial_no=serial_no,
        rgb_rate_hz=rgb_rate_hz,
        depth_rate_hz=depth_rate_hz,
        imu_rate_hz=imu_rate_hz,
    )
    return bundle.model_copy(
        update={
            "sensors": [s.model_copy(update={"model": "RealSense D435i"}) for s in bundle.sensors],
        }
    )

World State

openral World State — snapshot aggregator + persistent spatial memory public API.

Public surface: - WorldStateAggregator: tf2-aware, injectable aggregator that produces WorldState Pydantic snapshots at up to 30 Hz. - DEFAULT_RATE_HZ: Advertised default snapshot rate. - DEFAULT_STALENESS_S: Default staleness threshold in seconds. - SpatialMemory: ADR-0038 persistent object-centric scene-graph memory — accumulates WorldState.detected_objects and answers RecallObjectQuery / ResolvePlaceQuery. - compute_approach_viewpoint: camera-facing standoff-pose helper. - look_at_quat_wxyz / compute_gaze_pose: ADR-0044 shared gaze geometry (look-at rotations per camera convention; the rskill-moveit-look-at goal pose). - OccupancyGridIndex / refine_approach_pose: ADR-0044 occupancy-grid queries + approach-pose snapping (free cell + line-of-sight; planning-layer, not a safety surface). - OpenClipEmbedder / TextEmbedder: ADR-0038 §5 open-vocab text embedder (OpenCLIP ViT-B/32, MIT) — optional, uv sync --group clip. - emit_scene_objects_span / scene_objects_payload: publish the remembered object nodes as a world.scene_objects OTel span for the dashboard. - VoxelFrustumLifter / ObjectMemory / lift helpers (ADR-0035): lift 2D detections to 3D object centres and remember them with IoU gating.

DEFAULT_ASSOC_DISTANCE_M = 0.3 module-attribute

Default radius (m) for label-based instance association when track_id is absent.

DEFAULT_CAMERA_FRAME = 'gripper_camera' module-attribute

Default tf2 frame the approach viewpoint orients toward.

DEFAULT_CLIP_MODEL = 'ViT-B-32-quickgelu' module-attribute

OpenCLIP model name (ADR-0038 §5 default) — the quickgelu variant matches the openai pretrained weights' activation (avoids the QuickGELU mismatch warning).

DEFAULT_CLIP_PRETRAINED = 'openai' module-attribute

OpenCLIP pretrained tag — the original CLIP weights (MIT).

DEFAULT_MAP_FRAME = 'map' module-attribute

Default durable frame object poses are anchored in.

DEFAULT_MIN_TEXT_SIMILARITY = 0.85 module-attribute

Min CLIP cosine for an embedding-only object match (calibrated for ViT-B/32 openai).

OpenCLIP text-text cosines cluster high (~0.76-0.96), so this floor gates embedding-only hits; an exact/substring label match always qualifies regardless.

DEFAULT_STANDOFF_M = 0.6 module-attribute

Default standoff distance (m) from an object for the camera-facing approach viewpoint.

FREE_MAX = 25 module-attribute

Highest nav_msgs/OccupancyGrid value still treated as free (map_server's free_thresh is ~20/100; everything above, plus -1 unknown, blocks).

ObjectMemory(*, iou_threshold: float = 0.3, max_misses: int = 1)

Stateful, IoU-gated spatial memory of detected objects.

Parameters:

Name	Type	Description	Default
iou_threshold	float	Minimum 3D AABB IoU (same label) to associate a new detection with an existing track.	0.3
max_misses	int	Evict an in-view, unmatched track after this many consecutive misses. 1 ⇒ remove if not re-seen the next run.	1

Example

from openral_core.schemas import DetectedObject, Pose6D mem = ObjectMemory(iou_threshold=0.3, max_misses=1) cand = DetectedObject( ... label="cup", ... confidence=0.9, ... pose=Pose6D(xyz=(0.5, 0.5, 0.5), quat_xyzw=(0, 0, 0, 1), frame_id="map"), ... bbox_3d=(0.0, 0.0, 0.0, 1.0, 1.0, 1.0), ... ) out = mem.tick([cand], stamp_ns=1, in_fov=lambda o: True) out[0].track_id 0

Validate the IoU threshold and miss budget.

Source code in python/world_state/src/openral_world_state/object_memory.py

def __init__(self, *, iou_threshold: float = 0.3, max_misses: int = 1) -> None:
    """Validate the IoU threshold and miss budget."""
    if not 0.0 <= iou_threshold <= 1.0:
        raise ValueError(f"iou_threshold must be in [0,1]; got {iou_threshold}")
    if max_misses < 1:
        raise ValueError(f"max_misses must be >= 1; got {max_misses}")
    self._iou = iou_threshold
    self._max_misses = max_misses
    self._tracks: list[_Tracked] = []
    self._next_id = 0

tick(candidates: list[DetectedObject], *, stamp_ns: int, in_fov: Callable[[DetectedObject], bool]) -> list[DetectedObject]

Associate candidates, freeze matches, create new, evict in-view misses.

Parameters:

Name	Type	Description	Default
candidates	list[DetectedObject]	Newly lifted DetectedObjects for this run (map frame).	required
stamp_ns	int	Run timestamp in nanoseconds (stored as last-seen).	required
in_fov	Callable[[DetectedObject], bool]	Predicate — was this remembered object in the camera's view this run? Unmatched in-view objects accrue a miss; out-of-view objects are retained.	required

Returns:

Type	Description
list[DetectedObject]	The current remembered object set (stable track_ids).

Source code in python/world_state/src/openral_world_state/object_memory.py

def tick(
    self,
    candidates: list[DetectedObject],
    *,
    stamp_ns: int,
    in_fov: Callable[[DetectedObject], bool],
) -> list[DetectedObject]:
    """Associate candidates, freeze matches, create new, evict in-view misses.

    Args:
        candidates: Newly lifted ``DetectedObject``s for this run (map frame).
        stamp_ns: Run timestamp in nanoseconds (stored as last-seen).
        in_fov: Predicate — was this remembered object in the camera's view
            this run? Unmatched in-view objects accrue a miss; out-of-view
            objects are retained.

    Returns:
        The current remembered object set (stable ``track_id``s).
    """
    matched: set[int] = set()
    for cand in sorted(candidates, key=lambda d: d.confidence, reverse=True):
        best_i, best_iou = -1, self._iou
        for i, tr in enumerate(self._tracks):
            if tr.obj.label != cand.label:
                continue
            if cand.bbox_3d is None or tr.obj.bbox_3d is None:
                continue
            iou = aabb_iou_3d(cand.bbox_3d, tr.obj.bbox_3d)
            if iou >= best_iou:
                best_iou, best_i = iou, i
        if best_i >= 0:
            tr = self._tracks[best_i]
            tr.last_seen_ns = stamp_ns
            tr.miss_count = 0
            if cand.confidence > tr.obj.confidence:
                tr.obj = tr.obj.model_copy(update={"confidence": cand.confidence})
            matched.add(best_i)
        else:
            self._tracks.append(
                _Tracked(
                    obj=cand.model_copy(update={"track_id": self._next_id}),
                    last_seen_ns=stamp_ns,
                    miss_count=0,
                ),
            )
            matched.add(len(self._tracks) - 1)
            self._next_id += 1

    survivors: list[_Tracked] = []
    for i, tr in enumerate(self._tracks):
        if i in matched:
            survivors.append(tr)
            continue
        if in_fov(tr.obj):
            tr.miss_count += 1
            if tr.miss_count >= self._max_misses:
                continue
        survivors.append(tr)
    self._tracks = survivors
    return [tr.obj for tr in self._tracks]

OccupancyGridIndex(data: NDArray[np.int8], *, resolution_m: float, origin_xy: tuple[float, float], origin_yaw: float = 0.0)

Queryable view over one nav_msgs/OccupancyGrid snapshot.

Parameters:

Name	Type	Description	Default
data	NDArray[int8]	(height, width) int8 occupancy array (row-major, as on the wire: row 0 is the cell at the grid origin).	required
resolution_m	float	Cell edge length in metres.	required
origin_xy	tuple[float, float]	World x, y of the corner of cell (0, 0).	required
origin_yaw	float	Grid rotation about +Z (slam_toolbox maps are normally axis-aligned, but the origin pose may carry yaw).	0.0

Example

import numpy as np idx = OccupancyGridIndex( ... np.zeros((4, 4), dtype=np.int8), resolution_m=0.5, origin_xy=(0.0, 0.0) ... ) idx.is_free(1.0, 1.0) True

See the class docstring for argument semantics.

Source code in python/world_state/src/openral_world_state/grid.py

def __init__(
    self,
    data: NDArray[np.int8],
    *,
    resolution_m: float,
    origin_xy: tuple[float, float],
    origin_yaw: float = 0.0,
) -> None:
    """See the class docstring for argument semantics."""
    if data.ndim != _GRID_NDIM:
        raise ValueError(f"occupancy data must be 2-D (height, width); got {data.shape}")
    if resolution_m <= 0.0:
        raise ValueError(f"resolution_m must be > 0; got {resolution_m}")
    self._data = data
    self._res = float(resolution_m)
    self._ox, self._oy = float(origin_xy[0]), float(origin_xy[1])
    self._cos = math.cos(origin_yaw)
    self._sin = math.sin(origin_yaw)

resolution_m: float property

Cell edge length in metres.

from_msg(msg: Any) -> OccupancyGridIndex classmethod

Build from a (duck-typed) nav_msgs/OccupancyGrid message.

Source code in python/world_state/src/openral_world_state/grid.py

@classmethod
def from_msg(cls, msg: Any) -> OccupancyGridIndex:  # noqa: ANN401  # reason: duck-typed nav_msgs/OccupancyGrid keeps world_state ROS-free
    """Build from a (duck-typed) ``nav_msgs/OccupancyGrid`` message."""
    info = msg.info
    data = np.asarray(msg.data, dtype=np.int8).reshape(info.height, info.width)
    origin = info.origin
    yaw = _yaw_from_quat_xyzw(
        (
            origin.orientation.x,
            origin.orientation.y,
            origin.orientation.z,
            origin.orientation.w,
        )
    )
    return cls(
        data,
        resolution_m=float(info.resolution),
        origin_xy=(float(origin.position.x), float(origin.position.y)),
        origin_yaw=yaw,
    )

is_free(x: float, y: float, *, inflation_m: float = 0.0) -> bool

True iff the point — and every cell within inflation_m of it — is free.

Conservative: the disc is measured in world space from the query point to each cell's nearest face (not centre-cell to centre-cell, which under-inflates near cell edges). Off-grid points, and any part of the disc extending off-grid, count as blocked — the robot footprint must sit inside known-free space.

Source code in python/world_state/src/openral_world_state/grid.py

def is_free(self, x: float, y: float, *, inflation_m: float = 0.0) -> bool:
    """True iff the point — and every cell within ``inflation_m`` of it — is free.

    Conservative: the disc is measured in **world space** from the query
    point to each cell's nearest face (not centre-cell to centre-cell,
    which under-inflates near cell edges). Off-grid points, and any part
    of the disc extending off-grid, count as blocked — the robot footprint
    must sit inside known-free space.
    """
    centre = self.world_to_cell(x, y)
    if centre is None:
        return False
    row0, col0 = centre
    h, w = self._data.shape
    # Work in (rotated) grid coordinates so cell rectangles are axis-aligned.
    dx, dy = x - self._ox, y - self._oy
    gx = self._cos * dx + self._sin * dy
    gy = -self._sin * dx + self._cos * dy
    steps = max(0, math.ceil(inflation_m / self._res) + 1)
    for dr in range(-steps, steps + 1):
        for dc in range(-steps, steps + 1):
            r, c = row0 + dr, col0 + dc
            # Nearest point of cell (r, c) to the query point, grid coords.
            cx = min(max(gx, c * self._res), (c + 1) * self._res)
            cy = min(max(gy, r * self._res), (r + 1) * self._res)
            if math.hypot(cx - gx, cy - gy) > inflation_m + 1e-12:
                continue
            if not (0 <= r < h and 0 <= c < w):
                return False
            value = int(self._data[r, c])
            if value < 0 or value > FREE_MAX:
                return False
    return True

line_of_sight(a_xy: tuple[float, float], b_xy: tuple[float, float]) -> bool

True iff every cell strictly before b on the segment is free.

Bresenham over the grid; unknown cells block sight and either endpoint off-grid is no sight. The final cell is exempt: the target is typically an object whose own footprint is occupied in the 2-D grid (a mug shares its cell with the counter it sits on) — sight means an unobstructed ray up to it, not through it.

Source code in python/world_state/src/openral_world_state/grid.py

def line_of_sight(self, a_xy: tuple[float, float], b_xy: tuple[float, float]) -> bool:
    """True iff every cell strictly before ``b`` on the segment is free.

    Bresenham over the grid; unknown cells block sight and either endpoint
    off-grid is no sight. The **final cell is exempt**: the target is
    typically an object whose own footprint is occupied in the 2-D grid
    (a mug shares its cell with the counter it sits on) — sight means an
    unobstructed ray *up to* it, not through it.
    """
    a = self.world_to_cell(*a_xy)
    b = self.world_to_cell(*b_xy)
    if a is None or b is None:
        return False
    r0, c0 = a
    r1, c1 = b
    dr, dc = abs(r1 - r0), abs(c1 - c0)
    sr = 1 if r1 >= r0 else -1
    sc = 1 if c1 >= c0 else -1
    err = dc - dr
    r, c = r0, c0
    while True:
        if (r, c) == (r1, c1):
            return True
        value = int(self._data[r, c])
        if value < 0 or value > FREE_MAX:
            return False
        e2 = 2 * err
        if e2 > -dr:
            err -= dr
            c += sc
        if e2 < dc:
            err += dc
            r += sr

world_to_cell(x: float, y: float) -> tuple[int, int] | None

(row, col) containing the world point, or None off-grid.

Source code in python/world_state/src/openral_world_state/grid.py

def world_to_cell(self, x: float, y: float) -> tuple[int, int] | None:
    """``(row, col)`` containing the world point, or ``None`` off-grid."""
    dx, dy = x - self._ox, y - self._oy
    gx = self._cos * dx + self._sin * dy
    gy = -self._sin * dx + self._cos * dy
    col = math.floor(gx / self._res)
    row = math.floor(gy / self._res)
    h, w = self._data.shape
    if 0 <= row < h and 0 <= col < w:
        return (row, col)
    return None

OpenClipEmbedder(*, model_name: str = DEFAULT_CLIP_MODEL, pretrained: str = DEFAULT_CLIP_PRETRAINED, device: str | None = None)

OpenCLIP text embedder (ViT-B/32, MIT) for open-vocab matching (ADR-0038 §5).

Parameters:

Name	Type	Description	Default
model_name	str	OpenCLIP architecture (default "ViT-B-32").	DEFAULT_CLIP_MODEL
pretrained	str	OpenCLIP pretrained tag (default "openai" — MIT weights).	DEFAULT_CLIP_PRETRAINED
device	str | None	Torch device; defaults to "cuda" when available else "cpu".	None

Raises:

Type	Description
ROSConfigError	When open_clip / torch are not installed (uv sync --group clip) or the model cannot be created.

Create the embedder, loading the OpenCLIP model onto device.

Source code in python/world_state/src/openral_world_state/embedder.py

def __init__(
    self,
    *,
    model_name: str = DEFAULT_CLIP_MODEL,
    pretrained: str = DEFAULT_CLIP_PRETRAINED,
    device: str | None = None,
) -> None:
    """Create the embedder, loading the OpenCLIP model onto ``device``."""
    try:
        import open_clip  # noqa: PLC0415  # reason: optional dep; mypy via mypy.ini ignore_missing_imports
        import torch  # noqa: PLC0415  # reason: optional/compute-gated import (clip group)
    except ImportError as exc:  # pragma: no cover - exercised via the clip group
        raise ROSConfigError(
            "OpenClipEmbedder requires open-clip-torch + torch (`uv sync --group clip`)."
        ) from exc

    self._torch = torch
    self._device = device or ("cuda" if torch.cuda.is_available() else "cpu")
    try:
        model, _, _ = open_clip.create_model_and_transforms(
            model_name, pretrained=pretrained, device=self._device
        )
        self._tokenizer = open_clip.get_tokenizer(model_name)
    except Exception as exc:  # reason: open_clip raises bare exceptions on bad model/download
        raise ROSConfigError(
            f"OpenClipEmbedder failed to load {model_name!r}/{pretrained!r}: {exc}"
        ) from exc
    model.eval()
    self._model = model
    self._dim = int(model.text_projection.shape[-1])
    self.model_name = model_name
    self.pretrained = pretrained

dim: int property

Embedding dimensionality (512 for ViT-B/32).

embed_text(texts: Sequence[str]) -> NDArray[np.float32]

Embed texts into an (N, dim) L2-normalized float32 array.

Source code in python/world_state/src/openral_world_state/embedder.py

def embed_text(self, texts: Sequence[str]) -> NDArray[np.float32]:
    """Embed ``texts`` into an ``(N, dim)`` L2-normalized float32 array."""
    if not texts:
        return np.zeros((0, self._dim), dtype=np.float32)
    torch = self._torch
    tokens = self._tokenizer(list(texts)).to(self._device)
    with torch.no_grad():
        feats = self._model.encode_text(tokens)
        feats = feats / feats.norm(dim=-1, keepdim=True)
    return feats.cpu().numpy().astype(np.float32)  # type: ignore[no-any-return]  # reason: torch→numpy is untyped

SpatialMemory(*, assoc_distance_m: float = DEFAULT_ASSOC_DISTANCE_M, default_standoff_m: float = DEFAULT_STANDOFF_M, camera_frame_id: str = DEFAULT_CAMERA_FRAME, map_frame: str = DEFAULT_MAP_FRAME, embedder: TextEmbedder | None = None, min_text_similarity: float = DEFAULT_MIN_TEXT_SIMILARITY)

Accumulating, queryable scene-graph spatial memory (ADR-0038 Phase 2).

Parameters:

Name	Type	Description	Default
assoc_distance_m	float	Radius for label-based instance association when a detection carries no track_id.	DEFAULT_ASSOC_DISTANCE_M
default_standoff_m	float	Standoff used for approach viewpoints.	DEFAULT_STANDOFF_M
camera_frame_id	str	tf2 frame approach viewpoints orient toward.	DEFAULT_CAMERA_FRAME
map_frame	str	Durable frame object poses are anchored in.	DEFAULT_MAP_FRAME

Initialize an empty memory.

embedder (ADR-0038 §5, optional) enables open-vocabulary matching: object labels are embedded on creation and free-text queries match by CLIP cosine similarity (>= min_text_similarity) in addition to label substring. Without it, matching is label + pose + recency only.

Source code in python/world_state/src/openral_world_state/spatial_memory.py

def __init__(
    self,
    *,
    assoc_distance_m: float = DEFAULT_ASSOC_DISTANCE_M,
    default_standoff_m: float = DEFAULT_STANDOFF_M,
    camera_frame_id: str = DEFAULT_CAMERA_FRAME,
    map_frame: str = DEFAULT_MAP_FRAME,
    embedder: TextEmbedder | None = None,
    min_text_similarity: float = DEFAULT_MIN_TEXT_SIMILARITY,
) -> None:
    """Initialize an empty memory.

    ``embedder`` (ADR-0038 §5, optional) enables open-vocabulary matching:
    object labels are embedded on creation and free-text queries match by
    CLIP cosine similarity (>= ``min_text_similarity``) in addition to
    label substring. Without it, matching is label + pose + recency only.
    """
    self._assoc_distance_m = assoc_distance_m
    self._default_standoff_m = default_standoff_m
    self._camera_frame_id = camera_frame_id
    self._map_frame = map_frame
    self._embedder = embedder
    self._min_text_similarity = min_text_similarity
    self._nodes: dict[str, SpatialNode] = {}
    self._edges: dict[tuple[str, str, SpatialRelationKind], SpatialEdge] = {}
    self._embeddings: dict[str, NDArray[np.float32]] = {}
    self._auto_counter = 0

add_edge(src: str, dst: str, kind: SpatialRelationKind) -> None

Add a directed relation; src/dst must already exist (idempotent).

Source code in python/world_state/src/openral_world_state/spatial_memory.py

def add_edge(self, src: str, dst: str, kind: SpatialRelationKind) -> None:
    """Add a directed relation; src/dst must already exist (idempotent)."""
    if src not in self._nodes:
        raise KeyError(f"unknown src node: {src!r}")
    if dst not in self._nodes:
        raise KeyError(f"unknown dst node: {dst!r}")
    self._edges[(src, dst, kind)] = SpatialEdge(src=src, dst=dst, kind=kind)

from_scene_graph(graph: SceneGraph, *, embedder: TextEmbedder | None = None) -> SpatialMemory classmethod

Build a memory from a persisted :class:~openral_core.SceneGraph.

Embeddings are not serialized; when an embedder is supplied, object labels are re-embedded here (cheap, deterministic) so open-vocab queries work against a loaded graph.

Source code in python/world_state/src/openral_world_state/spatial_memory.py

@classmethod
def from_scene_graph(
    cls, graph: SceneGraph, *, embedder: TextEmbedder | None = None
) -> SpatialMemory:
    """Build a memory from a persisted :class:`~openral_core.SceneGraph`.

    Embeddings are not serialized; when an ``embedder`` is supplied, object
    labels are re-embedded here (cheap, deterministic) so open-vocab queries
    work against a loaded graph.
    """
    mem = cls(embedder=embedder)
    for node in graph.nodes:
        mem._nodes[node.node_id] = node
        mem._embed_node(node)
    for edge in graph.edges:
        mem._edges[(edge.src, edge.dst, edge.kind)] = edge
    return mem

ingest_detected_objects(objects: Sequence[DetectedObject], *, now_ns: int) -> list[str]

Fold a snapshot's detected objects into the graph (ADR-0038 §2).

Instance association: a detection is matched to an existing object node by track_id when present, else by identical label within assoc_distance_m. A match updates the pose, bumps last_seen_ns and observation_count and keeps the higher confidence; otherwise a new node is created. Object poses are expected in the map frame.

Parameters:

Name	Type	Description	Default
objects	Sequence[DetectedObject]	Detections from a :class:~openral_core.WorldState.	required
now_ns	int	Observation timestamp in nanoseconds.	required

Returns:

Type	Description
list[str]	The node ids touched (created or updated), in input order.

Source code in python/world_state/src/openral_world_state/spatial_memory.py

def ingest_detected_objects(
    self, objects: Sequence[DetectedObject], *, now_ns: int
) -> list[str]:
    """Fold a snapshot's detected objects into the graph (ADR-0038 §2).

    Instance association: a detection is matched to an existing ``object``
    node by ``track_id`` when present, else by identical label within
    ``assoc_distance_m``. A match updates the pose, bumps ``last_seen_ns``
    and ``observation_count`` and keeps the higher confidence; otherwise a
    new node is created. Object poses are expected in the map frame.

    Args:
        objects: Detections from a :class:`~openral_core.WorldState`.
        now_ns: Observation timestamp in nanoseconds.

    Returns:
        The node ids touched (created or updated), in input order.
    """
    touched: list[str] = []
    for obj in objects:
        match_id = self._associate(obj)
        if match_id is None:
            node_id = self._new_node_id(obj)
            self._nodes[node_id] = SpatialNode(
                node_id=node_id,
                kind=SpatialNodeKind.OBJECT,
                pose=obj.pose,
                label=obj.label,
                confidence=obj.confidence,
                bbox_3d=obj.bbox_3d,
                first_seen_ns=now_ns,
                last_seen_ns=now_ns,
                observation_count=1,
            )
            self._embed_node(self._nodes[node_id])
            touched.append(node_id)
        else:
            prev = self._nodes[match_id]
            self._nodes[match_id] = prev.model_copy(
                update={
                    "pose": obj.pose,
                    "bbox_3d": obj.bbox_3d if obj.bbox_3d is not None else prev.bbox_3d,
                    "confidence": max(prev.confidence, obj.confidence),
                    "last_seen_ns": max(prev.last_seen_ns, now_ns),
                    "observation_count": prev.observation_count + 1,
                }
            )
            touched.append(match_id)
    return touched

load(path: str | Path, *, embedder: TextEmbedder | None = None) -> SpatialMemory classmethod

Load a memory from a JSON scene graph written by :meth:save.

Source code in python/world_state/src/openral_world_state/spatial_memory.py

@classmethod
def load(cls, path: str | Path, *, embedder: TextEmbedder | None = None) -> SpatialMemory:
    """Load a memory from a JSON scene graph written by :meth:`save`."""
    graph = SceneGraph.model_validate_json(Path(path).read_text())
    return cls.from_scene_graph(graph, embedder=embedder)

recall_object(query: RecallObjectQuery, *, now_ns: int) -> RecallObjectResult

Recall remembered objects matching query (ADR-0038 §6).

Matches object nodes by exact (case-insensitive) or substring label against query.label / query.text, applies the optional recency filter, ranks by match quality x confidence (proximity to query.near as a tiebreaker), and attaches a camera-facing approach viewpoint and any occluding container. Returns an empty result when nothing matches — the caller decides whether to raise :class:~openral_core.exceptions.ROSObjectNotInMemory or search.

Source code in python/world_state/src/openral_world_state/spatial_memory.py

def recall_object(self, query: RecallObjectQuery, *, now_ns: int) -> RecallObjectResult:
    """Recall remembered objects matching ``query`` (ADR-0038 §6).

    Matches ``object`` nodes by exact (case-insensitive) or substring label
    against ``query.label`` / ``query.text``, applies the optional recency
    filter, ranks by match quality x confidence (proximity to
    ``query.near`` as a tiebreaker), and attaches a camera-facing approach
    viewpoint and any occluding container. Returns an empty result when
    nothing matches — the caller decides whether to raise
    :class:`~openral_core.exceptions.ROSObjectNotInMemory` or search.
    """
    term = (query.label or query.text).strip().lower()
    query_text = (query.text or query.label).strip()
    query_emb = (
        self._embedder.embed_text([query_text])[0]
        if self._embedder is not None and query_text
        else None
    )
    scored: list[tuple[float, float, SpatialNode]] = []
    for node in self._nodes.values():
        if node.kind is not SpatialNodeKind.OBJECT:
            continue
        if query.max_age_ns is not None and now_ns - node.last_seen_ns > query.max_age_ns:
            continue
        label_q = _label_match_quality(node.label, term)
        emb_sim = 0.0
        if query_emb is not None and node.node_id in self._embeddings:
            emb_sim = float(np.dot(query_emb, self._embeddings[node.node_id]))
        # Include on a label hit, or an embedding hit above the floor.
        if label_q <= 0.0 and emb_sim < self._min_text_similarity:
            continue
        score = max(label_q, emb_sim) * node.confidence
        proximity = _xyz_distance(node.pose, query.near) if query.near is not None else 0.0
        scored.append((score, proximity, node))

    # Rank: score desc, then nearer first, then most-recently seen.
    scored.sort(key=lambda t: (-t[0], t[1], -t[2].last_seen_ns))
    matches = [
        RecallObjectMatch(
            node_id=node.node_id,
            label=node.label,
            pose=node.pose,
            score=min(1.0, max(0.0, score)),
            last_seen_ns=node.last_seen_ns,
            approach=self._approach_for(node),
            inside_container_id=self._occluding_container_of(node.node_id),
        )
        for score, _prox, node in scored[: query.limit]
    ]
    return RecallObjectResult(matches=matches)

resolve_place(query: ResolvePlaceQuery, *, from_node_id: str | None = None) -> ResolvePlaceResult

Resolve a place/room/agent reference to a goal pose + path (ADR-0038 §6).

Raises:

Type	Description
ROSObjectNotInMemory	When the reference matches no node (the caller degrades to "unknown" / active search — it never fabricates a pose).

Source code in python/world_state/src/openral_world_state/spatial_memory.py

def resolve_place(
    self, query: ResolvePlaceQuery, *, from_node_id: str | None = None
) -> ResolvePlaceResult:
    """Resolve a place/room/agent reference to a goal pose + path (ADR-0038 §6).

    Raises:
        ROSObjectNotInMemory: When the reference matches no node (the caller
            degrades to "unknown" / active search — it never fabricates a pose).
    """
    target = self._resolve_reference(query.reference, query.kind)
    if target is None:
        raise ROSObjectNotInMemory(
            f"no scene-graph node resolves reference {query.reference!r}"
        )
    goal_node = target
    if target.kind is not SpatialNodeKind.PLACE:
        place_id = self._at_place_of(target.node_id)
        if place_id is not None:
            goal_node = self._nodes[place_id]
    path = (
        self._traversable_path(from_node_id, goal_node.node_id)
        if from_node_id is not None
        else []
    )
    return ResolvePlaceResult(
        node_id=goal_node.node_id, goal=goal_node.pose, path_node_ids=path
    )

save(path: str | Path) -> None

Persist the scene graph to path as JSON (the SceneGraph contract).

Source code in python/world_state/src/openral_world_state/spatial_memory.py

def save(self, path: str | Path) -> None:
    """Persist the scene graph to ``path`` as JSON (the SceneGraph contract)."""
    Path(path).write_text(self.to_scene_graph().model_dump_json(indent=2))

to_scene_graph() -> SceneGraph

Snapshot the memory as an immutable :class:~openral_core.SceneGraph.

Source code in python/world_state/src/openral_world_state/spatial_memory.py

def to_scene_graph(self) -> SceneGraph:
    """Snapshot the memory as an immutable :class:`~openral_core.SceneGraph`."""
    return SceneGraph(nodes=list(self._nodes.values()), edges=list(self._edges.values()))

upsert_node(node: SpatialNode) -> None

Insert or replace a node by node_id.

Source code in python/world_state/src/openral_world_state/spatial_memory.py

def upsert_node(self, node: SpatialNode) -> None:
    """Insert or replace a node by ``node_id``."""
    self._nodes[node.node_id] = node

TextEmbedder

Bases: Protocol

A text → unit-vector embedder for open-vocabulary matching.

dim: int property

Embedding dimensionality.

embed_text(texts: Sequence[str]) -> NDArray[np.float32]

Embed texts into an (N, dim) L2-normalized float32 array.

Source code in python/world_state/src/openral_world_state/embedder.py

def embed_text(self, texts: Sequence[str]) -> NDArray[np.float32]:
    """Embed ``texts`` into an ``(N, dim)`` L2-normalized float32 array."""
    ...

VoxelFrustumLifter(*, k_nearest: int = 25, min_voxels: int = 3)

Lift 2D detections to 3D object centres via voxel-frustum K-nearest.

For each detection: project the occupied voxels into the camera image, keep those inside the (resolution-scaled) box, take the k_nearest whose projection is closest to the box centre, and return their mean position + axis-aligned bbox in the map frame. Detections with fewer than min_voxels in-box voxels are skipped (insufficient 3D evidence).

Validate K and the min-voxel floor.

Source code in python/world_state/src/openral_world_state/object_lift.py

def __init__(self, *, k_nearest: int = 25, min_voxels: int = 3) -> None:
    """Validate K and the min-voxel floor."""
    if k_nearest < 1:
        raise ValueError(f"k_nearest must be >= 1; got {k_nearest}")
    if min_voxels < 1:
        raise ValueError(f"min_voxels must be >= 1; got {min_voxels}")
    self._k = k_nearest
    self._min_voxels = min_voxels

lift(*, detections: Sequence[ObjectDetection2D], occupied_centers_base: NDArray[np.float64], intrinsics: IntrinsicsPinhole, frame_size: tuple[int, int], t_cam_from_base: NDArray[np.float64], t_map_from_base: NDArray[np.float64], map_frame: str = 'map') -> list[DetectedObject]

Lift each 2D detection to a DetectedObject in the map frame.

Parameters:

Name	Type	Description	Default
detections	Sequence[ObjectDetection2D]	Per-camera 2D detections (pixel bbox_xyxy).	required
occupied_centers_base	NDArray[float64]	(N, 3) occupied voxel centres in the robot base frame.	required
intrinsics	IntrinsicsPinhole	Pinhole intrinsics of the detection camera.	required
frame_size	tuple[int, int]	(width, height) in pixels of the frame the detector ran on; the box is scaled to the intrinsics resolution.	required
t_cam_from_base	NDArray[float64]	(4, 4) transform mapping base-frame points into the camera optical frame.	required
t_map_from_base	NDArray[float64]	(4, 4) transform mapping base-frame points into the map frame.	required
map_frame	str	tf2 frame id stamped on the returned poses.	'map'

Returns:

Type	Description
list[DetectedObject]	One DetectedObject per detection with enough in-box voxels;
list[DetectedObject]	detections below min_voxels are omitted.

Source code in python/world_state/src/openral_world_state/object_lift.py

def lift(
    self,
    *,
    detections: Sequence[ObjectDetection2D],
    occupied_centers_base: NDArray[np.float64],
    intrinsics: IntrinsicsPinhole,
    frame_size: tuple[int, int],
    t_cam_from_base: NDArray[np.float64],
    t_map_from_base: NDArray[np.float64],
    map_frame: str = "map",
) -> list[DetectedObject]:
    """Lift each 2D detection to a ``DetectedObject`` in the map frame.

    Args:
        detections: Per-camera 2D detections (pixel ``bbox_xyxy``).
        occupied_centers_base: ``(N, 3)`` occupied voxel centres in the
            robot base frame.
        intrinsics: Pinhole intrinsics of the detection camera.
        frame_size: ``(width, height)`` in pixels of the frame the detector
            ran on; the box is scaled to the intrinsics resolution.
        t_cam_from_base: ``(4, 4)`` transform mapping base-frame points into
            the camera optical frame.
        t_map_from_base: ``(4, 4)`` transform mapping base-frame points into
            the map frame.
        map_frame: tf2 frame id stamped on the returned poses.

    Returns:
        One ``DetectedObject`` per detection with enough in-box voxels;
        detections below ``min_voxels`` are omitted.
    """
    out: list[DetectedObject] = []
    n = int(occupied_centers_base.shape[0])
    if n == 0 or not detections:
        return out

    homo = np.concatenate(
        [occupied_centers_base, np.ones((n, 1), dtype=np.float64)],
        axis=1,
    )  # (N,4)
    cam = (t_cam_from_base @ homo.T).T[:, :3]  # (N,3) camera-optical
    z = cam[:, 2]
    in_front = z > _DEPTH_EPS
    u = np.full(n, -1.0, dtype=np.float64)
    v = np.full(n, -1.0, dtype=np.float64)
    u[in_front] = intrinsics.fx * cam[in_front, 0] / z[in_front] + intrinsics.cx
    v[in_front] = intrinsics.fy * cam[in_front, 1] / z[in_front] + intrinsics.cy

    map_pts = (t_map_from_base @ homo.T).T[:, :3]  # (N,3) map frame

    fw, fh = frame_size
    scale_x = intrinsics.width / fw if fw else 1.0
    scale_y = intrinsics.height / fh if fh else 1.0

    for det in detections:
        x0, y0, x1, y1 = det.bbox_xyxy
        bx0, bx1 = x0 * scale_x, x1 * scale_x
        by0, by1 = y0 * scale_y, y1 * scale_y
        ucen, vcen = (bx0 + bx1) / 2.0, (by0 + by1) / 2.0
        inside = in_front & (u >= bx0) & (u <= bx1) & (v >= by0) & (v <= by1)
        idx = np.nonzero(inside)[0]
        if idx.size < self._min_voxels:
            continue
        d2 = (u[idx] - ucen) ** 2 + (v[idx] - vcen) ** 2
        nearest = idx[np.argpartition(d2, self._k)[: self._k]] if idx.size > self._k else idx
        pts = map_pts[nearest]
        center = pts.mean(axis=0)
        mn = pts.min(axis=0)
        mx = pts.max(axis=0)
        out.append(
            DetectedObject(
                label=det.label,
                confidence=det.confidence,
                pose=Pose6D(
                    xyz=(float(center[0]), float(center[1]), float(center[2])),
                    quat_xyzw=(0.0, 0.0, 0.0, 1.0),
                    frame_id=map_frame,
                ),
                bbox_3d=(
                    float(mn[0]),
                    float(mn[1]),
                    float(mn[2]),
                    float(mx[0]),
                    float(mx[1]),
                    float(mx[2]),
                ),
                track_id=None,
            ),
        )
    return out

WorldStateAggregator(description: RobotDescription, *, staleness_limit_s: float = DEFAULT_STALENESS_S, clock_fn: Callable[[], int] | None = None)

Aggregates sensor data and produces WorldState snapshots.

The aggregator holds no ROS 2 imports. All data arrives via typed update methods called from subscriber callbacks in the enclosing ROS 2 lifecycle node. :meth:snapshot can be called from any thread; internal state is protected by a reentrant lock.

Parameters:

Name	Type	Description	Default
description	RobotDescription	Normative RobotDescription for the target robot. Used to initialise diagnostic keys for all declared sensor bundles and end-effectors.	required
staleness_limit_s	float	Maximum age (seconds) for a component reading before it is classified as "stale". Default 0.1 s (3 frames at 30 Hz).	DEFAULT_STALENESS_S
clock_fn	Callable[[], int] | None	Callable returning the current time in nanoseconds. Defaults to time.time_ns. Override in tests to control time.	None

Example

from openral_core.schemas import JointState from openral_world_state.aggregator import WorldStateAggregator import time from openral_core import ( ... RobotDescription, ... EmbodimentKind, ... JointSpec, ... JointType, ... RobotCapabilities, ... SafetyEnvelope, ... ControlMode, ... ) desc = RobotDescription( ... name="t", ... embodiment_kind=EmbodimentKind.MANIPULATOR, ... joints=[ ... JointSpec( ... name="j0", ... joint_type=JointType.REVOLUTE, ... parent_link="base", ... child_link="l0", ... ) ... ], ... capabilities=RobotCapabilities( ... supported_control_modes=[ControlMode.JOINT_POSITION] ... ), ... safety=SafetyEnvelope(), ... ) agg = WorldStateAggregator(desc) agg.update_joint_state(JointState(name=["j0"], position=[0.5], stamp_ns=time.time_ns())) agg.snapshot().joint_state.position [0.5]

Initialise the aggregator; does not open any connection.

Source code in python/world_state/src/openral_world_state/aggregator.py

def __init__(
    self,
    description: RobotDescription,
    *,
    staleness_limit_s: float = DEFAULT_STALENESS_S,
    clock_fn: Callable[[], int] | None = None,
) -> None:
    """Initialise the aggregator; does not open any connection."""
    self.description = description
    self._staleness_limit_ns: int = int(staleness_limit_s * 1e9)
    self._clock_fn: Callable[[], int] = clock_fn or time.time_ns
    self._lock = threading.RLock()

    # ── Tracked state (all protected by _lock) ────────────────────────
    self._joint_state: JointState | None = None
    self._joint_state_stamp_ns: int = 0

    # sensor_name → (topic_ref, stamp_ns)
    self._images: dict[str, tuple[str, int]] = {}
    # sensor_name → (SensorFrame, stamp_ns) — actual bytes for the
    # consumer that wants pixels in-process (skill_runner →
    # rSkill). Populated by `update_image_frame`; surfaced through
    # `WorldState.image_frames` so a Skill can read pixels without
    # opening its own ROS subscription.
    self._image_frames: dict[str, tuple[SensorFrame, int]] = {}
    # ee_name → (Pose6D, stamp_ns)
    self._ee_poses: dict[str, tuple[Pose6D, int]] = {}
    # base pose + stamp
    self._base_pose: Pose6D | None = None
    self._base_pose_stamp_ns: int = 0
    self._base_twist: tuple[float, float, float, float, float, float] | None = None
    # battery
    self._battery_pct: float | None = None
    # ADR-0035 — latest object-memory snapshot (already deduped/evicted by
    # ObjectMemory in the enclosing node). Stored verbatim; no staleness
    # logic here (the memory owns lifecycle and refreshes every tick).
    self._detected_objects: list[DetectedObject] = []
    # latched diagnostics for explicitly set errors
    self._forced_errors: dict[str, DiagStatus] = {}
    # Stale components from the previous snapshot — used by snapshot() to
    # emit a ``staleness_latched`` span event only on the tick where a
    # component first goes stale (not every tick it stays stale).
    self._prev_stale_components: set[str] = set()
    # Same for latched errors so we don't re-emit on every snapshot.
    self._prev_latched_errors: set[str] = set()

    # Initialise diagnostic keys from description
    self._sensor_names: set[str] = {
        s.name for bundle in description.sensor_bundles for s in bundle.sensors
    }
    # End-effector poses are lazily registered (like cameras, see
    # update_ee_pose / update_image_frame): a declared EE only appears in
    # the diagnostics once it has received at least one pose. Pre-populating
    # from description.end_effectors meant every robot reported its
    # gripper(s) permanently STALE whenever no pose source was wired (the
    # tf2/forward-kinematics EE feed is not active on the sim deploy path),
    # which is pure noise. Track what the description *declares* separately
    # for reference/logging without forcing it into the stale ledger.
    self._declared_ee_names: frozenset[str] = (
        frozenset(ee.name for ee in description.end_effectors)
        if description.end_effectors
        else frozenset()
    )
    self._ee_names: set[str] = set()

    log.info(
        "world_state.aggregator.init",
        robot=description.name,
        staleness_limit_s=staleness_limit_s,
        sensor_count=len(self._sensor_names),
        ee_count=len(self._declared_ee_names),
    )

clear_error(component: str) -> None

Remove a forced diagnostic entry for a named component.

Parameters:

Name	Type	Description	Default
component	str	Diagnostic key to clear.	required

Source code in python/world_state/src/openral_world_state/aggregator.py

def clear_error(self, component: str) -> None:
    """Remove a forced diagnostic entry for a named component.

    Args:
        component: Diagnostic key to clear.
    """
    with self._lock:
        self._forced_errors.pop(component, None)

set_error(component: str, status: DiagStatus = 'error') -> None

Latch an explicit diagnostic status for a named component.

Use to surface hardware faults or driver errors that go beyond mere staleness. The forced status persists until :meth:clear_error is called.

Parameters:

Name	Type	Description	Default
component	str	Diagnostic key (e.g. "joint_state", sensor name).	required
status	DiagStatus	Diagnostic level to latch; typically "error" or "warn".	'error'

Source code in python/world_state/src/openral_world_state/aggregator.py

def set_error(self, component: str, status: DiagStatus = "error") -> None:
    """Latch an explicit diagnostic status for a named component.

    Use to surface hardware faults or driver errors that go beyond
    mere staleness.  The forced status persists until :meth:`clear_error`
    is called.

    Args:
        component: Diagnostic key (e.g. ``"joint_state"``, sensor name).
        status: Diagnostic level to latch; typically ``"error"`` or
            ``"warn"``.
    """
    with self._lock:
        self._forced_errors[component] = status
    log.warning("world_state.error.latched", component=component, status=status)

snapshot() -> WorldState

Produce a typed WorldState snapshot from current aggregated data.

Staleness is evaluated at call time. Components older than staleness_limit_s appear as "stale" in WorldState.diagnostics. Latched errors override staleness.

Emits a world_state.snapshot OTel span recording openral.world_state.components_stale and openral.world_state.has_latched_error. When a component first transitions to stale (or first acquires a latched error) the span carries a openral.event.staleness_latched (or ..._error_latched) event. Per-component staleness ages are recorded on the openral.world_state.staleness_ms histogram.

Returns:

Type	Description
WorldState	An immutable WorldState snapshot.

Raises:

Type	Description
RuntimeError	If no joint state has ever been received (None state would make the snapshot unusable).

Source code in python/world_state/src/openral_world_state/aggregator.py

def snapshot(self) -> WorldState:
    """Produce a typed ``WorldState`` snapshot from current aggregated data.

    Staleness is evaluated at call time.  Components older than
    ``staleness_limit_s`` appear as ``"stale"`` in
    ``WorldState.diagnostics``.  Latched errors override staleness.

    Emits a ``world_state.snapshot`` OTel span recording
    ``openral.world_state.components_stale`` and
    ``openral.world_state.has_latched_error``. When a component first
    transitions to stale (or first acquires a latched error) the span
    carries a ``openral.event.staleness_latched`` (or
    ``..._error_latched``) event. Per-component staleness ages are
    recorded on the ``openral.world_state.staleness_ms`` histogram.

    Returns:
        An immutable ``WorldState`` snapshot.

    Raises:
        RuntimeError: If no joint state has ever been received (``None``
            state would make the snapshot unusable).
    """
    with (
        self._lock,
        _tracer().start_as_current_span(semconv.SPAN_WORLD_STATE_SNAPSHOT) as span,
    ):
        now_ns = self._clock_fn()
        diag: dict[str, str] = {}
        ages_ms: dict[str, float] = {}

        # Joint state
        if self._joint_state is None:
            # Create a zeroed state so Skills always get a valid object;
            # mark diagnostic as stale.
            joint_count = len(self.description.joints)
            joint_names = [j.name for j in self.description.joints]
            js = JointState(
                name=joint_names,
                position=[0.0] * joint_count,
                velocity=[0.0] * joint_count,
                effort=[0.0] * joint_count,
                stamp_ns=now_ns,
            )
            diag["joint_state"] = "stale"
        else:
            js = self._joint_state
            age_ns = now_ns - self._joint_state_stamp_ns
            diag["joint_state"] = "ok" if age_ns <= self._staleness_limit_ns else "stale"
            ages_ms["joint_state"] = age_ns / 1e6

        # Images — topic refs from last received frames
        images: dict[str, str] = {}
        for sensor_name in self._sensor_names:
            if sensor_name in self._images:
                topic, stamp = self._images[sensor_name]
                age_ns = now_ns - stamp
                diag[sensor_name] = "ok" if age_ns <= self._staleness_limit_ns else "stale"
                images[sensor_name] = topic
                ages_ms[sensor_name] = age_ns / 1e6
            else:
                diag[sensor_name] = "stale"

        # EE poses
        ee_poses: dict[str, Pose6D] = {}
        for ee_name in self._ee_names:
            if ee_name in self._ee_poses:
                pose, stamp = self._ee_poses[ee_name]
                age_ns = now_ns - stamp
                diag[ee_name] = "ok" if age_ns <= self._staleness_limit_ns else "stale"
                ee_poses[ee_name] = pose
                ages_ms[ee_name] = age_ns / 1e6
            else:
                diag[ee_name] = "stale"

        # Forced errors override staleness classification
        diag.update(self._forced_errors)

        self._emit_snapshot_telemetry(span, diag, ages_ms)
        # Surface the embodiment view on the same span the dashboard
        # consumes — ee poses (named) + battery + joint stamp + the
        # per-component diagnostics list. Stale data still flows so
        # the dashboard can render a warn pill on the stale row.
        _producer.record_ee_poses(span, ee_poses)
        if self._battery_pct is not None:
            span.set_attribute("openral.world_state.battery_pct", float(self._battery_pct))
        span.set_attribute("openral.world_state.diagnostics_keys", sorted(diag.keys()))
        span.set_attribute(
            "openral.world_state.diagnostics_values",
            [diag[k] for k in sorted(diag.keys())],
        )
        span.set_attribute(semconv.HAL_JOINT_NAMES, list(js.name))
        span.set_attribute(
            semconv.HAL_JOINT_POSITIONS,
            [round(float(v), 3) for v in js.position],
        )

        # Inline pixel payloads: surface every camera that fed
        # `update_image_frame` since the last reset. The Pydantic
        # field is Optional; pass None when nothing has arrived so
        # the snapshot stays cheap when no cameras are wired.
        image_frames: dict[str, SensorFrame] | None = None
        if self._image_frames:
            image_frames = {n: f for n, (f, _) in self._image_frames.items()}

        return WorldState(
            stamp_ns=now_ns,
            joint_state=js,
            base_pose=self._base_pose,
            base_twist=self._base_twist,
            ee_poses=ee_poses,
            images=images,
            image_frames=image_frames,
            battery_pct=self._battery_pct,
            diagnostics=diag,
            detected_objects=list(self._detected_objects),
        )

update_base_pose(pose: Pose6D, twist: tuple[float, float, float, float, float, float] | None = None) -> None

Record a fresh base link pose (and optional twist) from tf2.

Parameters:

Name	Type	Description	Default
pose	Pose6D	Base link 6D pose in the world/map frame.	required
twist	tuple[float, float, float, float, float, float] | None	Optional (vx, vy, vz, wx, wy, wz) twist.	None

Source code in python/world_state/src/openral_world_state/aggregator.py

def update_base_pose(
    self,
    pose: Pose6D,
    twist: tuple[float, float, float, float, float, float] | None = None,
) -> None:
    """Record a fresh base link pose (and optional twist) from tf2.

    Args:
        pose: Base link 6D pose in the world/map frame.
        twist: Optional (vx, vy, vz, wx, wy, wz) twist.
    """
    with self._lock:
        self._base_pose = pose
        self._base_pose_stamp_ns = self._clock_fn()
        self._base_twist = twist
    log.debug("world_state.base_pose.updated")

update_battery(pct: float) -> None

Record the latest battery percentage.

Parameters:

Name	Type	Description	Default
pct	float	Battery percentage in [0, 100].	required

Source code in python/world_state/src/openral_world_state/aggregator.py

def update_battery(self, pct: float) -> None:
    """Record the latest battery percentage.

    Args:
        pct: Battery percentage in [0, 100].
    """
    with self._lock:
        self._battery_pct = pct

update_detected_objects(objects: list[DetectedObject]) -> None

Replace the remembered detected-object set (ADR-0035).

Called from the world-state node's memory tick with the current ObjectMemory output (already associated, frozen, and evicted). A copy is stored so later external mutation of objects cannot alter the snapshot.

Parameters:

Name	Type	Description	Default
objects	list[DetectedObject]	Current spatial-memory objects, anchored in the map frame.	required

Source code in python/world_state/src/openral_world_state/aggregator.py

def update_detected_objects(self, objects: list[DetectedObject]) -> None:
    """Replace the remembered detected-object set (ADR-0035).

    Called from the world-state node's memory tick with the current
    ``ObjectMemory`` output (already associated, frozen, and evicted). A
    copy is stored so later external mutation of ``objects`` cannot alter
    the snapshot.

    Args:
        objects: Current spatial-memory objects, anchored in the map frame.
    """
    with self._lock:
        self._detected_objects = list(objects)
    log.debug("world_state.detected_objects.updated", count=len(objects))

update_ee_pose(ee_name: str, pose: Pose6D) -> None

Record a fresh end-effector pose from a tf2 lookup.

Parameters:

Name	Type	Description	Default
ee_name	str	Matches an EndEffectorSpec.name in the description.	required
pose	Pose6D	Current 6D pose in the world frame.	required

Source code in python/world_state/src/openral_world_state/aggregator.py

def update_ee_pose(self, ee_name: str, pose: Pose6D) -> None:
    """Record a fresh end-effector pose from a tf2 lookup.

    Args:
        ee_name: Matches an ``EndEffectorSpec.name`` in the description.
        pose: Current 6D pose in the world frame.
    """
    with self._lock:
        self._ee_poses[ee_name] = (pose, self._clock_fn())
        # Lazy registration: only EEs that have actually been observed enter
        # the diagnostics ledger (see __init__ for why pre-population is noise).
        self._ee_names.add(ee_name)
    log.debug("world_state.ee_pose.updated", ee=ee_name)

update_image(sensor_name: str, topic: str, stamp_ns: int) -> None

Record a fresh image frame arrival for a named sensor.

Parameters:

Name	Type	Description	Default
sensor_name	str	Matches a SensorSpec.name in the description.	required
topic	str	ROS 2 topic the image was published on.	required
stamp_ns	int	Frame timestamp in nanoseconds.	required

Source code in python/world_state/src/openral_world_state/aggregator.py

def update_image(self, sensor_name: str, topic: str, stamp_ns: int) -> None:
    """Record a fresh image frame arrival for a named sensor.

    Args:
        sensor_name: Matches a ``SensorSpec.name`` in the description.
        topic: ROS 2 topic the image was published on.
        stamp_ns: Frame timestamp in nanoseconds.
    """
    with self._lock:
        self._images[sensor_name] = (topic, self._clock_fn())
    log.debug("world_state.image.updated", sensor=sensor_name)

update_image_frame(sensor_name: str, frame: SensorFrame) -> None

Record an inline pixel payload for a named sensor.

Unlike :meth:update_image (which just records "a frame arrived on topic X"), this method stores the actual :class:SensorFrame — usually with frame.data set — so the next :meth:snapshot carries pixels inline through :attr:WorldState.image_frames. This is the path a Skill uses when it needs RGB pixels without opening its own ROS subscription (CLAUDE.md §6.1 — Layer 1 (Sensors) writes through Layer 2 (World State) into Layer 3 (rSkill)).

Accepts arbitrary sensor_name values not declared in the :class:RobotDescription sensor_bundles list — synthetic digital-twin cameras live in the active MJCF, not the robot manifest, and the aggregator must aggregate whatever streams in.

Parameters:

Name	Type	Description	Default
sensor_name	str	Camera id; used as the key in :attr:WorldState.image_frames.	required
frame	SensorFrame	Validated :class:SensorFrame (data / topic / handle).	required

Source code in python/world_state/src/openral_world_state/aggregator.py

def update_image_frame(self, sensor_name: str, frame: SensorFrame) -> None:
    """Record an inline pixel payload for a named sensor.

    Unlike :meth:`update_image` (which just records "a frame arrived
    on topic X"), this method stores the actual :class:`SensorFrame`
    — usually with ``frame.data`` set — so the next
    :meth:`snapshot` carries pixels inline through
    :attr:`WorldState.image_frames`. This is the path a Skill uses
    when it needs RGB pixels without opening its own ROS
    subscription (CLAUDE.md §6.1 — Layer 1 (Sensors) writes through
    Layer 2 (World State) into Layer 3 (rSkill)).

    Accepts arbitrary ``sensor_name`` values not declared in the
    :class:`RobotDescription` ``sensor_bundles`` list — synthetic
    digital-twin cameras live in the active MJCF, not the robot
    manifest, and the aggregator must aggregate whatever streams
    in.

    Args:
        sensor_name: Camera id; used as the key in
            :attr:`WorldState.image_frames`.
        frame: Validated :class:`SensorFrame` (``data`` /
            ``topic`` / ``handle``).
    """
    stamp = self._clock_fn()
    with self._lock:
        self._image_frames[sensor_name] = (frame, stamp)
        # Mirror into `_images` so the diagnostics + the
        # `WorldState.images` topic map stay populated even for
        # sensors that are not in the robot.yaml. Use the frame's
        # `topic` when set, otherwise a placeholder.
        self._images[sensor_name] = (
            frame.topic if frame.topic is not None else f"<inline:{sensor_name}>",
            stamp,
        )
        self._sensor_names.add(sensor_name)
    log.debug(
        "world_state.image_frame.updated",
        sensor=sensor_name,
        encoding=str(frame.encoding),
        width=frame.width,
        height=frame.height,
    )

update_joint_state(state: JointState) -> None

Record a fresh joint state reading.

Parameters:

Name	Type	Description	Default
state	JointState	Latest JointState from the /joint_states topic.	required

Source code in python/world_state/src/openral_world_state/aggregator.py

def update_joint_state(self, state: JointState) -> None:
    """Record a fresh joint state reading.

    Args:
        state: Latest ``JointState`` from the ``/joint_states`` topic.
    """
    with self._lock:
        self._joint_state = state
        self._joint_state_stamp_ns = self._clock_fn()
    log.debug("world_state.joint_state.updated", robot=self.description.name)

aabb_iou_3d(a: tuple[float, float, float, float, float, float], b: tuple[float, float, float, float, float, float]) -> float

3D axis-aligned bbox IoU.

Parameters:

Name	Type	Description	Default
a	tuple[float, float, float, float, float, float]	First box as (xmin, ymin, zmin, xmax, ymax, zmax).	required
b	tuple[float, float, float, float, float, float]	Second box as (xmin, ymin, zmin, xmax, ymax, zmax).	required

Returns:

Type	Description
float	Intersection-over-union in [0, 1]; 0.0 for disjoint or
float	degenerate (zero-volume) boxes.

Source code in python/world_state/src/openral_world_state/object_lift.py

def aabb_iou_3d(
    a: tuple[float, float, float, float, float, float],
    b: tuple[float, float, float, float, float, float],
) -> float:
    """3D axis-aligned bbox IoU.

    Args:
        a: First box as ``(xmin, ymin, zmin, xmax, ymax, zmax)``.
        b: Second box as ``(xmin, ymin, zmin, xmax, ymax, zmax)``.

    Returns:
        Intersection-over-union in ``[0, 1]``; ``0.0`` for disjoint or
        degenerate (zero-volume) boxes.
    """
    ix = max(0.0, min(a[3], b[3]) - max(a[0], b[0]))
    iy = max(0.0, min(a[4], b[4]) - max(a[1], b[1]))
    iz = max(0.0, min(a[5], b[5]) - max(a[2], b[2]))
    inter = ix * iy * iz
    if inter <= 0.0:
        return 0.0
    vol_a = max(0.0, a[3] - a[0]) * max(0.0, a[4] - a[1]) * max(0.0, a[5] - a[2])
    vol_b = max(0.0, b[3] - b[0]) * max(0.0, b[4] - b[1]) * max(0.0, b[5] - b[2])
    union = vol_a + vol_b - inter
    return inter / union if union > 0.0 else 0.0

build_in_fov_predicate(*, intrinsics: IntrinsicsPinhole, t_cam_from_map: NDArray[np.float64]) -> Callable[[DetectedObject], bool]

Return a predicate: does a map-frame object project into the camera image?

Used by ObjectMemory to decide whether an unmatched object should have been re-seen (in view => a real miss) or is merely out of view (retain). Objects whose centre is behind the camera or outside the image bounds return False. Image bounds are inclusive of the far edge (0 <= u <= width), using continuous pixel coordinates.

Parameters:

Name	Type	Description	Default
intrinsics	IntrinsicsPinhole	Pinhole intrinsics of the camera; width/height bound the image rectangle.	required
t_cam_from_map	NDArray[float64]	(4, 4) transform mapping map-frame points into the camera optical frame.	required

Returns:

Type	Description
Callable[[DetectedObject], bool]	A callable in_fov(obj) -> bool.

Source code in python/world_state/src/openral_world_state/object_lift.py

def build_in_fov_predicate(
    *,
    intrinsics: IntrinsicsPinhole,
    t_cam_from_map: NDArray[np.float64],
) -> Callable[[DetectedObject], bool]:
    """Return a predicate: does a map-frame object project into the camera image?

    Used by ``ObjectMemory`` to decide whether an unmatched object *should* have
    been re-seen (in view => a real miss) or is merely out of view (retain).
    Objects whose centre is behind the camera or outside the image bounds return
    ``False``. Image bounds are inclusive of the far edge (``0 <= u <= width``),
    using continuous pixel coordinates.

    Args:
        intrinsics: Pinhole intrinsics of the camera; ``width``/``height`` bound
            the image rectangle.
        t_cam_from_map: ``(4, 4)`` transform mapping map-frame points into the
            camera optical frame.

    Returns:
        A callable ``in_fov(obj) -> bool``.
    """
    w, h = float(intrinsics.width), float(intrinsics.height)

    def in_fov(obj: DetectedObject) -> bool:
        x, y, z = obj.pose.xyz
        p = t_cam_from_map @ np.array([x, y, z, 1.0], dtype=np.float64)
        if p[2] <= _DEPTH_EPS:
            return False
        u = intrinsics.fx * p[0] / p[2] + intrinsics.cx
        v = intrinsics.fy * p[1] / p[2] + intrinsics.cy
        return bool(0.0 <= u <= w and 0.0 <= v <= h)

    return in_fov

compute_approach_viewpoint(target: Pose6D, *, standoff_m: float = DEFAULT_STANDOFF_M, camera_frame_id: str = DEFAULT_CAMERA_FRAME, approach_from: Pose6D | None = None) -> ApproachViewpoint

Compute a standoff pose whose camera faces target (ADR-0038 §6).

The viewpoint is placed standoff_m away from the target in the horizontal (x, y) plane and yawed to look at it. When approach_from is given (e.g. the place a robot would stand at), the viewpoint sits on that side of the target; otherwise it defaults to approaching from -X.

Parameters:

Name	Type	Description	Default
target	Pose6D	Object pose to view, in the map frame.	required
standoff_m	float	Standoff distance in metres (> 0).	DEFAULT_STANDOFF_M
camera_frame_id	str	tf2 frame of the camera the viewpoint orients toward.	DEFAULT_CAMERA_FRAME
approach_from	Pose6D | None	Optional pose giving the side to approach from.	None

Returns:

Name	Type	Description
An	ApproachViewpoint	class:~openral_core.ApproachViewpoint in the target's frame.

Source code in python/world_state/src/openral_world_state/spatial_memory.py

def compute_approach_viewpoint(
    target: Pose6D,
    *,
    standoff_m: float = DEFAULT_STANDOFF_M,
    camera_frame_id: str = DEFAULT_CAMERA_FRAME,
    approach_from: Pose6D | None = None,
) -> ApproachViewpoint:
    """Compute a standoff pose whose camera faces ``target`` (ADR-0038 §6).

    The viewpoint is placed ``standoff_m`` away from the target in the
    horizontal (x, y) plane and yawed to look at it. When ``approach_from`` is
    given (e.g. the place a robot would stand at), the viewpoint sits on that
    side of the target; otherwise it defaults to approaching from -X.

    Args:
        target: Object pose to view, in the map frame.
        standoff_m: Standoff distance in metres (``> 0``).
        camera_frame_id: tf2 frame of the camera the viewpoint orients toward.
        approach_from: Optional pose giving the side to approach from.

    Returns:
        An :class:`~openral_core.ApproachViewpoint` in the target's frame.
    """
    tx, ty, tz = target.xyz
    if approach_from is not None:
        ax, ay, _az = approach_from.xyz
        dx, dy = ax - tx, ay - ty
        norm = math.hypot(dx, dy)
        ux, uy = (dx / norm, dy / norm) if norm > _MIN_DIR_NORM else (-1.0, 0.0)
    else:
        ux, uy = -1.0, 0.0
    vp_x, vp_y = tx + ux * standoff_m, ty + uy * standoff_m
    # Yaw so the camera (looking down its +X) points from the viewpoint at the target.
    yaw = math.atan2(ty - vp_y, tx - vp_x)
    return ApproachViewpoint(
        pose=Pose6D(xyz=(vp_x, vp_y, tz), quat_xyzw=_xy_yaw_quat(yaw), frame_id=target.frame_id),
        standoff_m=standoff_m,
        camera_frame_id=camera_frame_id,
    )

compute_gaze_pose(camera_xyz: tuple[float, float, float], target_xyz: tuple[float, float, float], *, frame_id: str = 'map', up: tuple[float, float, float] = (0.0, 0.0, 1.0), view_axis: ViewAxis = '+z') -> Pose6D

Full 6-DOF camera pose at camera_xyz whose view axis hits target_xyz.

The pose the rskill-moveit-look-at rSkill (ADR-0044 Phase 3) plans the camera frame to: position fixed at camera_xyz, orientation from :func:look_at_quat_wxyz. Defaults to the ROS optical-frame convention ("+z" forward) since real camera frame_ids are optical frames.

Parameters:

Name	Type	Description	Default
camera_xyz	tuple[float, float, float]	Where the camera sits, in frame_id.	required
target_xyz	tuple[float, float, float]	The point to aim at, in frame_id.	required
frame_id	str	tf2 frame both points are expressed in.	'map'
up	tuple[float, float, float]	World up used to level the camera roll.	(0.0, 0.0, 1.0)
view_axis	ViewAxis	Camera forward-axis convention.	'+z'

Returns:

Name	Type	Description
A	Pose6D	class:~openral_core.Pose6D (quat_xyzw order).

Example

pose = compute_gaze_pose((0.0, 0.0, 1.0), (1.0, 0.0, 1.0)) pose.frame_id 'map' pose.xyz (0.0, 0.0, 1.0)

Source code in python/world_state/src/openral_world_state/geometry.py

def compute_gaze_pose(
    camera_xyz: tuple[float, float, float],
    target_xyz: tuple[float, float, float],
    *,
    frame_id: str = "map",
    up: tuple[float, float, float] = (0.0, 0.0, 1.0),
    view_axis: ViewAxis = "+z",
) -> Pose6D:
    """Full 6-DOF camera pose at ``camera_xyz`` whose view axis hits ``target_xyz``.

    The pose the ``rskill-moveit-look-at`` rSkill (ADR-0044 Phase 3) plans the
    camera frame to: position fixed at ``camera_xyz``, orientation from
    :func:`look_at_quat_wxyz`. Defaults to the ROS optical-frame convention
    (``"+z"`` forward) since real camera ``frame_id``s are optical frames.

    Args:
        camera_xyz: Where the camera sits, in ``frame_id``.
        target_xyz: The point to aim at, in ``frame_id``.
        frame_id: tf2 frame both points are expressed in.
        up: World up used to level the camera roll.
        view_axis: Camera forward-axis convention.

    Returns:
        A :class:`~openral_core.Pose6D` (``quat_xyzw`` order).

    Example:
        >>> pose = compute_gaze_pose((0.0, 0.0, 1.0), (1.0, 0.0, 1.0))
        >>> pose.frame_id
        'map'
        >>> pose.xyz
        (0.0, 0.0, 1.0)
    """
    w, x, y, z = look_at_quat_wxyz(camera_xyz, target_xyz, up=up, view_axis=view_axis)
    return Pose6D(xyz=camera_xyz, quat_xyzw=(x, y, z, w), frame_id=frame_id)

decode_occupied_centers(*, origin: tuple[float, float, float], resolution: float, size_xyz: tuple[int, int, int], occupancy: bytes) -> NDArray[np.float64]

Occupied-voxel centers (N, 3) in the grid frame.

Cell (ix, iy, iz) center is origin + (index + 0.5) * resolution. A cell is occupied when its byte is non-zero.

Parameters:

Name	Type	Description	Default
origin	tuple[float, float, float]	(x, y, z) grid origin (corner of cell (0, 0, 0)) in metres.	required
resolution	float	Edge length of a cubic voxel in metres.	required
size_xyz	tuple[int, int, int]	Grid dimensions (size_x, size_y, size_z) in voxels.	required
occupancy	bytes	Flat occupancy buffer, one byte per voxel, row-major with x fastest: idx = ix + size_x * (iy + size_y * iz). Non-zero byte means occupied.	required

Returns:

Type	Description
NDArray[float64]	A (N, 3) float64 array of occupied-voxel centers in the grid frame,
NDArray[float64]	or shape (0, 3) when none are occupied.

Raises:

Type	Description
ObjectsLiftError	If len(occupancy) does not equal size_x * size_y * size_z.

Source code in python/world_state/src/openral_world_state/object_lift.py

def decode_occupied_centers(
    *,
    origin: tuple[float, float, float],
    resolution: float,
    size_xyz: tuple[int, int, int],
    occupancy: bytes,
) -> NDArray[np.float64]:
    """Occupied-voxel centers ``(N, 3)`` in the grid frame.

    Cell ``(ix, iy, iz)`` center is ``origin + (index + 0.5) * resolution``. A
    cell is occupied when its byte is non-zero.

    Args:
        origin: ``(x, y, z)`` grid origin (corner of cell ``(0, 0, 0)``) in
            metres.
        resolution: Edge length of a cubic voxel in metres.
        size_xyz: Grid dimensions ``(size_x, size_y, size_z)`` in voxels.
        occupancy: Flat occupancy buffer, one byte per voxel, row-major with x
            fastest: ``idx = ix + size_x * (iy + size_y * iz)``. Non-zero byte
            means occupied.

    Returns:
        A ``(N, 3)`` float64 array of occupied-voxel centers in the grid frame,
        or shape ``(0, 3)`` when none are occupied.

    Raises:
        ObjectsLiftError: If ``len(occupancy)`` does not equal
            ``size_x * size_y * size_z``.
    """
    sx, sy, sz = size_xyz
    expected = sx * sy * sz
    arr = np.frombuffer(occupancy, dtype=np.uint8)
    if arr.size != expected:
        raise ObjectsLiftError(
            f"occupancy length {arr.size} != size_x*size_y*size_z {expected}",
        )
    occ = np.nonzero(arr)[0]
    if occ.size == 0:
        return np.empty((0, 3), dtype=np.float64)
    ix = occ % sx
    iy = (occ // sx) % sy
    iz = occ // (sx * sy)
    idx = np.stack([ix, iy, iz], axis=1).astype(np.float64)
    return np.asarray(origin, dtype=np.float64) + (idx + 0.5) * float(resolution)

depth_cloud_to_centers_base(points_cloud: NDArray[np.float64], t_base_from_cloud: NDArray[np.float64], *, max_points: int = 0) -> NDArray[np.float64]

Depth-cloud points → occupied centers (M, 3) in the base frame (#11).

The octomap-free fallback depth source for :class:VoxelFrustumLifter. Drops non-finite returns (depth holes), uniformly subsamples to max_points so a dense cloud can't stall the per-detection projection, and maps the cloud from its optical frame into the robot base frame. The result is interchangeable with :func:decode_occupied_centers output as the lifter's occupied_centers_base argument.

Parameters:

Name	Type	Description	Default
points_cloud	NDArray[float64]	(N, 3) raw cloud points in the camera optical frame.	required
t_base_from_cloud	NDArray[float64]	(4, 4) transform mapping cloud-frame points into the robot base frame.	required
max_points	int	Cap on returned points (uniform stride). 0 = no cap.	0

Returns:

Type	Description
NDArray[float64]	An (M, 3) float64 array of finite points in the base frame, or shape
NDArray[float64]	(0, 3) when the cloud has no finite points.

Source code in python/world_state/src/openral_world_state/object_lift.py

def depth_cloud_to_centers_base(
    points_cloud: NDArray[np.float64],
    t_base_from_cloud: NDArray[np.float64],
    *,
    max_points: int = 0,
) -> NDArray[np.float64]:
    """Depth-cloud points → occupied centers ``(M, 3)`` in the base frame (#11).

    The octomap-free fallback depth source for :class:`VoxelFrustumLifter`. Drops
    non-finite returns (depth holes), uniformly subsamples to ``max_points`` so
    a dense cloud can't stall the per-detection projection, and maps the cloud
    from its optical frame into the robot base frame. The result is
    interchangeable with :func:`decode_occupied_centers` output as the lifter's
    ``occupied_centers_base`` argument.

    Args:
        points_cloud: ``(N, 3)`` raw cloud points in the camera optical frame.
        t_base_from_cloud: ``(4, 4)`` transform mapping cloud-frame points into
            the robot base frame.
        max_points: Cap on returned points (uniform stride). ``0`` = no cap.

    Returns:
        An ``(M, 3)`` float64 array of finite points in the base frame, or shape
        ``(0, 3)`` when the cloud has no finite points.
    """
    pts: NDArray[np.float64] = np.asarray(points_cloud, dtype=np.float64).reshape(-1, 3)
    pts = pts[np.isfinite(pts).all(axis=1)]
    if pts.shape[0] == 0:
        return np.empty((0, 3), dtype=np.float64)
    if max_points and pts.shape[0] > max_points:
        stride = pts.shape[0] // max_points + 1
        pts = pts[::stride]
    homog = np.hstack([pts, np.ones((pts.shape[0], 1), dtype=np.float64)])
    return np.ascontiguousarray((homog @ np.asarray(t_base_from_cloud, dtype=np.float64).T)[:, :3])

emit_scene_objects_span(graph: SceneGraph, *, source_node: str) -> int

Emit one world.scene_objects span carrying graph's object nodes.

Parameters:

Name	Type	Description	Default
graph	SceneGraph	A scene-graph snapshot to publish.	required
source_node	str	Name of the node emitting the span (shown on the card so an operator knows which process produced the map).	required

Returns:

Type	Description
int	The number of object nodes published.

Example

import time from openral_core import DetectedObject, Pose6D from openral_world_state import SpatialMemory mem = SpatialMemory() mug = DetectedObject( ... label="mug", ... confidence=0.9, ... pose=Pose6D(xyz=(1.0, 2.0, 0.8), quat_xyzw=(0.0, 0.0, 0.0, 1.0), frame_id="map"), ... ) _ = mem.ingest_detected_objects([mug], now_ns=time.time_ns()) emit_scene_objects_span(mem.to_scene_graph(), source_node="demo") 1

Source code in python/world_state/src/openral_world_state/scene_objects_span.py

def emit_scene_objects_span(graph: SceneGraph, *, source_node: str) -> int:
    """Emit one ``world.scene_objects`` span carrying ``graph``'s object nodes.

    Args:
        graph: A scene-graph snapshot to publish.
        source_node: Name of the node emitting the span (shown on the card so
            an operator knows which process produced the map).

    Returns:
        The number of object nodes published.

    Example:
        >>> import time
        >>> from openral_core import DetectedObject, Pose6D
        >>> from openral_world_state import SpatialMemory
        >>> mem = SpatialMemory()
        >>> mug = DetectedObject(
        ...     label="mug",
        ...     confidence=0.9,
        ...     pose=Pose6D(xyz=(1.0, 2.0, 0.8), quat_xyzw=(0.0, 0.0, 0.0, 1.0), frame_id="map"),
        ... )
        >>> _ = mem.ingest_detected_objects([mug], now_ns=time.time_ns())
        >>> emit_scene_objects_span(mem.to_scene_graph(), source_node="demo")
        1
    """
    objects = scene_objects_payload(graph)
    frame = str(objects[0]["frame_id"]) if objects else _DEFAULT_FRAME
    tracer = trace.get_tracer("openral")
    with tracer.start_as_current_span(semconv.SPAN_WORLD_SCENE_OBJECTS) as span:
        span.set_attribute(semconv.WORLD_SCENE_OBJECTS_COUNT, len(objects))
        span.set_attribute(semconv.WORLD_SCENE_OBJECTS_SOURCE_NODE, source_node)
        span.set_attribute(semconv.WORLD_SCENE_OBJECTS_FRAME, frame)
        span.set_attribute(semconv.WORLD_SCENE_OBJECTS_LIST, json.dumps(objects))
    return len(objects)

homogeneous_from_quat_xyz(translation: tuple[float, float, float], quat_xyzw: tuple[float, float, float, float]) -> NDArray[np.float64]

Build a 4x4 homogeneous transform from a translation + xyzw quaternion.

Parameters:

Name	Type	Description	Default
translation	tuple[float, float, float]	(x, y, z) translation in metres.	required
quat_xyzw	tuple[float, float, float, float]	Rotation quaternion as (x, y, z, w); need not be unit length (it is normalised internally).	required

Returns:

Type	Description
NDArray[float64]	A (4, 4) float64 homogeneous transform matrix.

Raises:

Type	Description
ObjectsLiftError	If the quaternion norm is effectively zero.

Source code in python/world_state/src/openral_world_state/object_lift.py

def homogeneous_from_quat_xyz(
    translation: tuple[float, float, float],
    quat_xyzw: tuple[float, float, float, float],
) -> NDArray[np.float64]:
    """Build a 4x4 homogeneous transform from a translation + xyzw quaternion.

    Args:
        translation: ``(x, y, z)`` translation in metres.
        quat_xyzw: Rotation quaternion as ``(x, y, z, w)``; need not be unit
            length (it is normalised internally).

    Returns:
        A ``(4, 4)`` float64 homogeneous transform matrix.

    Raises:
        ObjectsLiftError: If the quaternion norm is effectively zero.
    """
    x, y, z, w = quat_xyzw
    n = x * x + y * y + z * z + w * w
    if n < _QUAT_NORM_EPS:
        raise ObjectsLiftError(f"degenerate quaternion {quat_xyzw!r}")
    s = 2.0 / n
    xx, yy, zz = x * x * s, y * y * s, z * z * s
    xy, xz, yz = x * y * s, x * z * s, y * z * s
    wx, wy, wz = w * x * s, w * y * s, w * z * s
    m = np.eye(4, dtype=np.float64)
    m[0, 0] = 1.0 - (yy + zz)
    m[0, 1] = xy - wz
    m[0, 2] = xz + wy
    m[1, 0] = xy + wz
    m[1, 1] = 1.0 - (xx + zz)
    m[1, 2] = yz - wx
    m[2, 0] = xz - wy
    m[2, 1] = yz + wx
    m[2, 2] = 1.0 - (xx + yy)
    m[0, 3], m[1, 3], m[2, 3] = translation
    return m

look_at_quat_wxyz(eye: tuple[float, float, float], target: tuple[float, float, float], *, up: tuple[float, float, float] = (0.0, 0.0, 1.0), view_axis: ViewAxis = '-z') -> tuple[float, float, float, float]

Quaternion (w, x, y, z) orienting a camera at eye to face target.

The rotated frame's view_axis points from eye toward target. Degenerate inputs fall back instead of raising: target == eye returns the MuJoCo straight-down quat for "-z" (preserving the sim composers' documented behaviour) and identity otherwise; a gaze nearly parallel to up swaps in a +Y up vector.

Parameters:

Name	Type	Description	Default
eye	tuple[float, float, float]	Camera position.	required
target	tuple[float, float, float]	Point to look at, same frame as eye.	required
up	tuple[float, float, float]	World up used to level the camera roll.	(0.0, 0.0, 1.0)
view_axis	ViewAxis	Which camera axis must point at the target (see module docstring for the conventions).	'-z'

Returns:

Type	Description
tuple[float, float, float, float]	Unit quaternion in (w, x, y, z) order (MuJoCo convention).

Example

look_at_quat_wxyz((0.0, 0.0, 0.0), (1.0, 0.0, 0.0), view_axis="+x") (1.0, 0.0, 0.0, 0.0)

Source code in python/world_state/src/openral_world_state/geometry.py

def look_at_quat_wxyz(
    eye: tuple[float, float, float],
    target: tuple[float, float, float],
    *,
    up: tuple[float, float, float] = (0.0, 0.0, 1.0),
    view_axis: ViewAxis = "-z",
) -> tuple[float, float, float, float]:
    """Quaternion (``w, x, y, z``) orienting a camera at ``eye`` to face ``target``.

    The rotated frame's ``view_axis`` points from ``eye`` toward ``target``.
    Degenerate inputs fall back instead of raising: ``target == eye`` returns
    the MuJoCo straight-down quat for ``"-z"`` (preserving the sim composers'
    documented behaviour) and identity otherwise; a gaze nearly parallel to
    ``up`` swaps in a +Y up vector.

    Args:
        eye: Camera position.
        target: Point to look at, same frame as ``eye``.
        up: World up used to level the camera roll.
        view_axis: Which camera axis must point at the target (see module
            docstring for the conventions).

    Returns:
        Unit quaternion in ``(w, x, y, z)`` order (MuJoCo convention).

    Example:
        >>> look_at_quat_wxyz((0.0, 0.0, 0.0), (1.0, 0.0, 0.0), view_axis="+x")
        (1.0, 0.0, 0.0, 0.0)
    """
    eye_v = np.asarray(eye, dtype=np.float64)
    target_v = np.asarray(target, dtype=np.float64)
    forward = target_v - eye_v
    norm = float(np.linalg.norm(forward))
    if norm < _ZERO_NORM:
        # Sim composers' documented fallback: 180° flip about X looks straight
        # down for a -Z camera. Other conventions get identity.
        return (0.0, 1.0, 0.0, 0.0) if view_axis == "-z" else (1.0, 0.0, 0.0, 0.0)
    forward /= norm
    up_v = np.asarray(up, dtype=np.float64)
    if abs(float(np.dot(up_v, forward))) > _PARALLEL:
        # Looking nearly straight up/down the up vector: pick an alternate up.
        up_v = np.asarray((0.0, 1.0, 0.0), dtype=np.float64)

    if view_axis == "-z":
        z_axis = -forward
        x_axis = np.cross(up_v, z_axis)
        x_axis /= float(np.linalg.norm(x_axis))
        y_axis = np.cross(z_axis, x_axis)
    elif view_axis == "+z":
        z_axis = forward
        x_axis = np.cross(forward, up_v)
        x_axis /= float(np.linalg.norm(x_axis))
        y_axis = np.cross(z_axis, x_axis)
    else:  # "+x"
        x_axis = forward
        y_axis = np.cross(up_v, forward)
        y_axis /= float(np.linalg.norm(y_axis))
        z_axis = np.cross(x_axis, y_axis)

    rot = np.column_stack([x_axis, y_axis, z_axis])
    return _basis_to_quat_wxyz(rot)

refine_approach_pose(grid: OccupancyGridIndex, viewpoint: ApproachViewpoint, target_xyz: tuple[float, float, float], *, inflation_m: float = 0.25, max_radius_m: float = 2.0, min_standoff_m: float | None = None, max_standoff_m: float | None = None) -> ApproachViewpoint | None

Snap an :class:~openral_core.ApproachViewpoint to the occupancy grid.

Returns the viewpoint unchanged when its pose already sits on a free cell (under inflation_m) with line-of-sight to the target. Otherwise ring- searches outward (at grid resolution) for the nearest point that is free, keeps the standoff within [min_standoff_m, max_standoff_m] (default 0.5x / 2.0x the ideal standoff), and still sees the target — and re-aims the yaw from there via :func:compute_approach_viewpoint. Returns None when nothing qualifies inside max_radius_m: the caller reports "no reachable viewpoint" rather than fabricating one (ADR-0038 posture).

Parameters:

Name	Type	Description	Default
grid	OccupancyGridIndex	Decoded occupancy snapshot.	required
viewpoint	ApproachViewpoint	The geometric (grid-blind) viewpoint to validate/refine.	required
target_xyz	tuple[float, float, float]	The object the camera must face, same frame as the grid.	required
inflation_m	float	Robot-footprint radius every candidate must clear.	0.25
max_radius_m	float	Search radius around the ideal viewpoint.	2.0
min_standoff_m	float | None	Closest admissible standoff (default 0.5 * viewpoint.standoff_m).	None
max_standoff_m	float | None	Farthest admissible standoff (default 2.0 * viewpoint.standoff_m).	None

Source code in python/world_state/src/openral_world_state/grid.py

def refine_approach_pose(
    grid: OccupancyGridIndex,
    viewpoint: ApproachViewpoint,
    target_xyz: tuple[float, float, float],
    *,
    inflation_m: float = 0.25,
    max_radius_m: float = 2.0,
    min_standoff_m: float | None = None,
    max_standoff_m: float | None = None,
) -> ApproachViewpoint | None:
    """Snap an :class:`~openral_core.ApproachViewpoint` to the occupancy grid.

    Returns the viewpoint unchanged when its pose already sits on a free cell
    (under ``inflation_m``) with line-of-sight to the target. Otherwise ring-
    searches outward (at grid resolution) for the nearest point that is free,
    keeps the standoff within ``[min_standoff_m, max_standoff_m]`` (default
    0.5x / 2.0x the ideal standoff), and still sees the target — and re-aims
    the yaw from there via :func:`compute_approach_viewpoint`. Returns ``None``
    when nothing qualifies inside ``max_radius_m``: the caller reports "no
    reachable viewpoint" rather than fabricating one (ADR-0038 posture).

    Args:
        grid: Decoded occupancy snapshot.
        viewpoint: The geometric (grid-blind) viewpoint to validate/refine.
        target_xyz: The object the camera must face, same frame as the grid.
        inflation_m: Robot-footprint radius every candidate must clear.
        max_radius_m: Search radius around the ideal viewpoint.
        min_standoff_m: Closest admissible standoff (default ``0.5 *
            viewpoint.standoff_m``).
        max_standoff_m: Farthest admissible standoff (default ``2.0 *
            viewpoint.standoff_m``).
    """
    lo = 0.5 * viewpoint.standoff_m if min_standoff_m is None else min_standoff_m
    hi = 2.0 * viewpoint.standoff_m if max_standoff_m is None else max_standoff_m
    ideal_x, ideal_y, _ideal_z = viewpoint.pose.xyz
    tx, ty = target_xyz[0], target_xyz[1]

    def _admissible(x: float, y: float) -> bool:
        standoff = math.hypot(x - tx, y - ty)
        return (
            lo <= standoff <= hi
            and grid.is_free(x, y, inflation_m=inflation_m)
            and grid.line_of_sight((x, y), (tx, ty))
        )

    if _admissible(ideal_x, ideal_y):
        return viewpoint

    target_pose = Pose6D(
        xyz=target_xyz, quat_xyzw=(0.0, 0.0, 0.0, 1.0), frame_id=viewpoint.pose.frame_id
    )
    step = grid.resolution_m
    radius = step
    while radius <= max_radius_m + 1e-9:
        # Enough samples that consecutive points on the ring are ~one cell apart.
        n_samples = max(8, math.ceil(2.0 * math.pi * radius / step))
        best: tuple[float, float, float] | None = None  # (standoff_error, x, y)
        for k in range(n_samples):
            theta = 2.0 * math.pi * k / n_samples
            x = ideal_x + radius * math.cos(theta)
            y = ideal_y + radius * math.sin(theta)
            if not _admissible(x, y):
                continue
            err = abs(math.hypot(x - tx, y - ty) - viewpoint.standoff_m)
            if best is None or err < best[0]:
                best = (err, x, y)
        if best is not None:
            _err, x, y = best
            candidate = Pose6D(
                xyz=(x, y, viewpoint.pose.xyz[2]),
                quat_xyzw=(0.0, 0.0, 0.0, 1.0),
                frame_id=viewpoint.pose.frame_id,
            )
            return compute_approach_viewpoint(
                target_pose,
                standoff_m=math.hypot(x - tx, y - ty),
                camera_frame_id=viewpoint.camera_frame_id,
                approach_from=candidate,
            )
        radius += step
    return None

rotation_to_quat_wxyz(rot: NDArray[np.float64]) -> tuple[float, float, float, float]

Public matrix→quaternion conversion (w, x, y, z), Shepperd's method.

Used by the rskill-moveit-look-at skill to re-express a camera gaze pose for the camera's mount link after a homogeneous-matrix composition.

Example

import numpy as np rotation_to_quat_wxyz(np.eye(3)) (1.0, 0.0, 0.0, 0.0)

Source code in python/world_state/src/openral_world_state/geometry.py

def rotation_to_quat_wxyz(rot: NDArray[np.float64]) -> tuple[float, float, float, float]:
    """Public matrix→quaternion conversion (``w, x, y, z``), Shepperd's method.

    Used by the ``rskill-moveit-look-at`` skill to re-express a camera gaze pose for
    the camera's mount link after a homogeneous-matrix composition.

    Example:
        >>> import numpy as np
        >>> rotation_to_quat_wxyz(np.eye(3))
        (1.0, 0.0, 0.0, 0.0)
    """
    return _basis_to_quat_wxyz(rot)

scene_objects_payload(graph: SceneGraph) -> list[dict[str, object]]

Project the object-kind nodes to JSON-friendly dicts.

Each dict carries the map-frame position, the semantic label, and the recency/confidence the dashboard renders. Non-object nodes (places, rooms, agents) are skipped — the dashboard surface is "what objects do we remember, and where".

Parameters:

Name	Type	Description	Default
graph	SceneGraph	A scene-graph snapshot (e.g. SpatialMemory.to_scene_graph()).	required

Returns:

Type	Description
list[dict[str, object]]	One dict per object node, in graph order.

Source code in python/world_state/src/openral_world_state/scene_objects_span.py

def scene_objects_payload(graph: SceneGraph) -> list[dict[str, object]]:
    """Project the ``object``-kind nodes to JSON-friendly dicts.

    Each dict carries the map-frame position, the semantic label, and the
    recency/confidence the dashboard renders. Non-object nodes (places, rooms,
    agents) are skipped — the dashboard surface is "what objects do we
    remember, and where".

    Args:
        graph: A scene-graph snapshot (e.g. ``SpatialMemory.to_scene_graph()``).

    Returns:
        One dict per object node, in graph order.
    """
    objects: list[dict[str, object]] = []
    for node in graph.nodes:
        if node.kind is not SpatialNodeKind.OBJECT:
            continue
        x, y, z = node.pose.xyz
        objects.append(
            {
                "id": node.node_id,
                "label": node.label,
                "x": x,
                "y": y,
                "z": z,
                "frame_id": node.pose.frame_id,
                "confidence": node.confidence,
                "last_seen_ns": node.last_seen_ns,
                "observation_count": node.observation_count,
                "is_container": node.is_container,
            }
        )
    return objects

rSkill (S1)

OpenRAL rSkill — lifecycle base class, runtime protocol, and rSkill loader.

Public surface

• rSkillBase: Abstract base class with the ROS 2 lifecycle state machine.
• Runtime: Structural protocol for inference backends.
• NullRuntime: No-op runtime for tests and development.
• QUANT_PRESETS: Named QuantizationConfig presets.
• auto_select_quant: Device-aware quantization preset selector.
• EngineCache: Filesystem cache for compiled engine files.
• DEFAULT_CACHE_DIR: Default cache directory path.
• rSkill: HF Hub rSkill loader (manifest + weights + license guard).
• InstalledRSkillEntry: Local registry entry schema.

Heavy-dependency adapters (SmolVLAAdapter, SO100SmolVLASkill) and backends (PyTorchRuntime, ONNXRuntime, TensorRTRuntime) are not imported here. Import them explicitly when their dependencies are installed:

from openral_rskill.smolvla import SmolVLAAdapter, SO100SmolVLASkill
from openral_rskill.runtime_pytorch import PyTorchRuntime
from openral_rskill.runtime_onnx import ONNXRuntime
from openral_rskill.runtime_tensorrt import TensorRTRuntime

DEFAULT_CACHE_DIR: Path = Path.home() / '.cache' / 'openral' / 'engines' module-attribute

Default filesystem path for compiled engine files.

Override by passing a different cache_dir to :class:EngineCache.

DEFAULT_REGISTRY_PATH: Path = _DATA_HOME / 'rskills.json' module-attribute

Default JSON registry file written by :meth:rSkill.from_pretrained.

EngineCache(cache_dir: Path = DEFAULT_CACHE_DIR)

Filesystem-backed cache for compiled skill engine files.

Each entry is a plain file stored as <cache_dir>/<key>.engine. Cache keys are 16-character SHA-256 truncations derived from the skill identifier, backend name, and serialised QuantizationConfig.

Parameters:

Name	Type	Description	Default
cache_dir	Path	Root directory for cache files. Created on first use.	DEFAULT_CACHE_DIR

Example

import tempfile, pathlib tmp = pathlib.Path(tempfile.mkdtemp()) cache = EngineCache(cache_dir=tmp) key = cache.cache_key("openral/rskill-pick-cube", "pytorch", QuantizationConfig()) cache.get(key) is None True

Initialize the cache, creating cache_dir if it does not exist.

Source code in python/rskill/src/openral_rskill/engine_cache.py

def __init__(self, cache_dir: Path = DEFAULT_CACHE_DIR) -> None:
    """Initialize the cache, creating *cache_dir* if it does not exist."""
    self._dir = cache_dir
    self._dir.mkdir(parents=True, exist_ok=True)

entry_count: int property

Number of engine files currently in the cache.

Example

import tempfile, pathlib cache = EngineCache(pathlib.Path(tempfile.mkdtemp())) cache.entry_count 0

size_bytes: int property

Total size in bytes of all cached engine files.

Example

import tempfile, pathlib cache = EngineCache(pathlib.Path(tempfile.mkdtemp())) cache.size_bytes 0

cache_key(rskill_id: str, backend: str, config: QuantizationConfig) -> str

Derive a stable cache key for the given skill + runtime + quant tuple.

The key is a 16-character hex prefix of the SHA-256 hash of the JSON- serialised payload, sorted by key for determinism.

Parameters:

Name	Type	Description	Default
rskill_id	str	HuggingFace Hub repo ID or local path (e.g. "openral/rskill-pick-cube-so100").	required
backend	str	Runtime backend name (e.g. "pytorch", "tensorrt").	required
config	QuantizationConfig	Quantization configuration.	required

Returns:

Type	Description
str	16-character lowercase hex string.

Example

import tempfile, pathlib cache = EngineCache(pathlib.Path(tempfile.mkdtemp())) k1 = cache.cache_key("my/skill", "onnx", QuantizationConfig()) k2 = cache.cache_key("my/skill", "onnx", QuantizationConfig()) k1 == k2 True

Source code in python/rskill/src/openral_rskill/engine_cache.py

def cache_key(
    self,
    rskill_id: str,
    backend: str,
    config: QuantizationConfig,
) -> str:
    """Derive a stable cache key for the given skill + runtime + quant tuple.

    The key is a 16-character hex prefix of the SHA-256 hash of the JSON-
    serialised payload, sorted by key for determinism.

    Args:
        rskill_id: HuggingFace Hub repo ID or local path
            (e.g. ``"openral/rskill-pick-cube-so100"``).
        backend: Runtime backend name (e.g. ``"pytorch"``, ``"tensorrt"``).
        config: Quantization configuration.

    Returns:
        16-character lowercase hex string.

    Example:
        >>> import tempfile, pathlib
        >>> cache = EngineCache(pathlib.Path(tempfile.mkdtemp()))
        >>> k1 = cache.cache_key("my/skill", "onnx", QuantizationConfig())
        >>> k2 = cache.cache_key("my/skill", "onnx", QuantizationConfig())
        >>> k1 == k2
        True
    """
    payload = json.dumps(
        {
            "rskill_id": rskill_id,
            "backend": backend,
            "config": config.model_dump(mode="json"),
        },
        sort_keys=True,
    )
    return hashlib.sha256(payload.encode()).hexdigest()[:16]

clear() -> None

Remove all engine files from the cache directory.

Source code in python/rskill/src/openral_rskill/engine_cache.py

def clear(self) -> None:
    """Remove all engine files from the cache directory."""
    for p in self._dir.glob("*.engine"):
        p.unlink()

get(key: str) -> Path | None

Return the cached engine path for key, or None on a miss.

Parameters:

Name	Type	Description	Default
key	str	16-character cache key from :meth:cache_key.	required

Returns:

Type	Description
Path | None	Path to the cached file if it exists, else None.

Example

import tempfile, pathlib cache = EngineCache(pathlib.Path(tempfile.mkdtemp())) cache.get("nonexistent") is None True

Source code in python/rskill/src/openral_rskill/engine_cache.py

def get(self, key: str) -> Path | None:
    """Return the cached engine path for *key*, or ``None`` on a miss.

    Args:
        key: 16-character cache key from :meth:`cache_key`.

    Returns:
        Path to the cached file if it exists, else ``None``.

    Example:
        >>> import tempfile, pathlib
        >>> cache = EngineCache(pathlib.Path(tempfile.mkdtemp()))
        >>> cache.get("nonexistent") is None
        True
    """
    p = self._key_path(key)
    return p if p.exists() else None

invalidate(key: str) -> None

Remove the cached entry for key (no-op on a miss).

Parameters:

Name	Type	Description	Default
key	str	16-character cache key from :meth:cache_key.	required

Source code in python/rskill/src/openral_rskill/engine_cache.py

def invalidate(self, key: str) -> None:
    """Remove the cached entry for *key* (no-op on a miss).

    Args:
        key: 16-character cache key from :meth:`cache_key`.
    """
    p = self._key_path(key)
    if p.exists():
        p.unlink()

put(key: str, engine_path: Path) -> Path

Copy engine_path into the cache under key.

Parameters:

Name	Type	Description	Default
key	str	16-character cache key from :meth:cache_key.	required
engine_path	Path	Source file to cache. Must exist.	required

Returns:

Type	Description
Path	Path of the newly cached file.

Raises:

Type	Description
FileNotFoundError	If engine_path does not exist.

Source code in python/rskill/src/openral_rskill/engine_cache.py

def put(self, key: str, engine_path: Path) -> Path:
    """Copy *engine_path* into the cache under *key*.

    Args:
        key: 16-character cache key from :meth:`cache_key`.
        engine_path: Source file to cache.  Must exist.

    Returns:
        Path of the newly cached file.

    Raises:
        FileNotFoundError: If *engine_path* does not exist.
    """
    if not engine_path.exists():
        raise FileNotFoundError(f"EngineCache.put: source file not found: '{engine_path}'")
    dest = self._key_path(key)
    shutil.copy2(engine_path, dest)
    return dest

InstalledRSkillEntry

Bases: BaseModel

One row in the local rSkill registry (~/.local/share/openral/rskills.json).

Attributes:

Name	Type	Description
repo_id	str	HF Hub repository identifier, e.g. "openral/rskill-pick-cube-so100".
version	str	SemVer version string from the manifest.
revision	str | None	HF Hub commit SHA (None when installed from a local path).
local_dir	str	Absolute path to the snapshot directory in the HF Hub cache.
manifest_path	str	Absolute path to the rskill.yaml file on disk.
license	str	License posture value string from :class:RSkillLicensePosture.
role	str	Skill slot — "s0", "s1", or "s2".
embodiment_tags	list[str]	Embodiment tags declared in the manifest.
installed_at	str	ISO 8601 timestamp of installation.

Example

entry = InstalledRSkillEntry( ... repo_id="example/skill", ... version="0.1.0", ... revision=None, ... local_dir="/tmp/skills/example", ... manifest_path="/tmp/skills/example/rskill.yaml", ... license="apache-2.0", ... role="s1", ... embodiment_tags=["so100_follower"], ... installed_at="2026-01-01T00:00:00+00:00", ... ) entry.repo_id 'example/skill'

NullRuntime(device: str = 'cpu')

No-op inference backend for testing and development.

Satisfies the Runtime Protocol without requiring any ML framework. infer() always returns an empty dict. All state transitions are tracked faithfully so lifecycle tests can assert on is_loaded.

Parameters:

Name	Type	Description	Default
device	str	Device string to report (default "cpu").	'cpu'

Example

rt = NullRuntime() rt.is_loaded False rt.load("any/path.pt") rt.is_loaded True rt.unload() rt.is_loaded False

Initialize with the given device string.

Parameters:

Name	Type	Description	Default
device	str	PyTorch-style device string (e.g. "cpu", "cuda:0").	'cpu'

Source code in python/rskill/src/openral_rskill/runtime.py

def __init__(self, device: str = "cpu") -> None:
    """Initialize with the given device string.

    Args:
        device: PyTorch-style device string (e.g. ``"cpu"``, ``"cuda:0"``).
    """
    self._device = device
    self._loaded = False

device: str property

PyTorch-style device string.

is_loaded: bool property

True after load() has been called and before unload().

infer(inputs: dict[str, Any]) -> dict[str, Any]

Return an empty dict without computation.

Parameters:

Name	Type	Description	Default
inputs	dict[str, Any]	Ignored; accepted for protocol compatibility.	required

Returns:

Type	Description
dict[str, Any]	Always {}.

Source code in python/rskill/src/openral_rskill/runtime.py

def infer(self, inputs: dict[str, Any]) -> dict[str, Any]:
    """Return an empty dict without computation.

    Args:
        inputs: Ignored; accepted for protocol compatibility.

    Returns:
        Always ``{}``.
    """
    return {}

load(path: Path | str) -> None

Mark weights as loaded (no I/O performed).

Parameters:

Name	Type	Description	Default
path	Path | str	Ignored; accepted for protocol compatibility.	required

Source code in python/rskill/src/openral_rskill/runtime.py

def load(self, path: Path | str) -> None:
    """Mark weights as loaded (no I/O performed).

    Args:
        path: Ignored; accepted for protocol compatibility.
    """
    self._loaded = True

quantize(config: QuantizationConfig) -> None

Accept any quantization config silently (no-op).

Parameters:

Name	Type	Description	Default
config	QuantizationConfig	Ignored; accepted for protocol compatibility.	required

Source code in python/rskill/src/openral_rskill/runtime.py

def quantize(self, config: QuantizationConfig) -> None:
    """Accept any quantization config silently (no-op).

    Args:
        config: Ignored; accepted for protocol compatibility.
    """

unload() -> None

Mark weights as unloaded (no memory freed).

Source code in python/rskill/src/openral_rskill/runtime.py

def unload(self) -> None:
    """Mark weights as unloaded (no memory freed)."""
    self._loaded = False

warmup(inputs: dict[str, Any]) -> None

No-op warmup.

Parameters:

Name	Type	Description	Default
inputs	dict[str, Any]	Ignored; accepted for protocol compatibility.	required

Source code in python/rskill/src/openral_rskill/runtime.py

def warmup(self, inputs: dict[str, Any]) -> None:
    """No-op warmup.

    Args:
        inputs: Ignored; accepted for protocol compatibility.
    """

Runtime

Bases: Protocol

Structural protocol for skill inference backends.

All methods are called from the Skill lifecycle hooks. Implementations must be thread-safe if step() is called concurrently (e.g. async executor pattern as in SmolVLA). Instances are created by backend-specific constructors (e.g. PyTorchRuntime(...), OnnxRuntime(...), NullRuntime()); the protocol itself is not instantiable.

Properties

is_loaded: True after :meth:load completes successfully. device: PyTorch-style device string, e.g. "cpu" or "cuda:0".

device: str property

PyTorch-style device string.

is_loaded: bool property

True after load() completes successfully.

infer(inputs: dict[str, Any]) -> dict[str, Any]

Run one forward pass and return named outputs.

Parameters:

Name	Type	Description	Default
inputs	dict[str, Any]	Named input tensors / arrays. Keys and shapes depend on the model contract declared in rskill.yaml.	required

Returns:

Type	Description
dict[str, Any]	Named output tensors / arrays.

Raises:

Type	Description
ROSRuntimeError	If no model is loaded or the forward pass fails.
ROSInferenceTimeout	If the backend exceeds its wall-clock budget.

Source code in python/rskill/src/openral_rskill/runtime.py

def infer(self, inputs: dict[str, Any]) -> dict[str, Any]:
    """Run one forward pass and return named outputs.

    Args:
        inputs: Named input tensors / arrays.  Keys and shapes depend on
            the model contract declared in ``rskill.yaml``.

    Returns:
        Named output tensors / arrays.

    Raises:
        ROSRuntimeError: If no model is loaded or the forward pass fails.
        ROSInferenceTimeout: If the backend exceeds its wall-clock budget.
    """
    ...

load(path: Path | str) -> None

Load model weights from path into memory.

Parameters:

Name	Type	Description	Default
path	Path | str	Local filesystem path or URL to weights file (*.safetensors, *.onnx, *.pt, etc.).	required

Raises:

Type	Description
ROSRuntimeError	If the file is missing, corrupt, or incompatible.

Source code in python/rskill/src/openral_rskill/runtime.py

def load(self, path: Path | str) -> None:
    """Load model weights from *path* into memory.

    Args:
        path: Local filesystem path or URL to weights file
            (``*.safetensors``, ``*.onnx``, ``*.pt``, etc.).

    Raises:
        ROSRuntimeError: If the file is missing, corrupt, or incompatible.
    """
    ...

quantize(config: QuantizationConfig) -> None

Apply post-load quantization.

Parameters:

Name	Type	Description	Default
config	QuantizationConfig	Target dtype, backend, and optional calibration dataset.	required

Raises:

Type	Description
ROSRuntimeError	If the requested dtype/backend combination is unsupported, or if no model has been loaded yet.

Source code in python/rskill/src/openral_rskill/runtime.py

def quantize(self, config: QuantizationConfig) -> None:
    """Apply post-load quantization.

    Args:
        config: Target dtype, backend, and optional calibration dataset.

    Raises:
        ROSRuntimeError: If the requested dtype/backend combination is
            unsupported, or if no model has been loaded yet.
    """
    ...

unload() -> None

Release model weights and free device memory.

Source code in python/rskill/src/openral_rskill/runtime.py

def unload(self) -> None:
    """Release model weights and free device memory."""
    ...

warmup(inputs: dict[str, Any]) -> None

Run a dummy forward pass to amortize JIT/kernel-launch overhead.

Parameters:

Name	Type	Description	Default
inputs	dict[str, Any]	Dummy inputs with the correct shapes (values are ignored).	required

Source code in python/rskill/src/openral_rskill/runtime.py

def warmup(self, inputs: dict[str, Any]) -> None:
    """Run a dummy forward pass to amortize JIT/kernel-launch overhead.

    Args:
        inputs: Dummy inputs with the correct shapes (values are ignored).
    """
    ...

rSkill(manifest: RSkillManifest, local_dir: Path)

Packaged, capability-tagged robot skill from the HF Hub.

.. warning:: Signature verification (sigstore) is not yet implemented (ADR-0006); the loader validates the manifest and license but does not verify the provenance of downloaded weights. See module docstring.

An rSkill represents the distribution artefact: the rskill.yaml manifest plus the associated weight files stored in the HF Hub cache (or locally). It is distinct from the runtime :class:~openral_rskill.Skill ABC, which is the in-process lifecycle node.

Typical workflow::

pkg = rSkill.from_pretrained("openral/rskill-pick-cube-so100")
# pkg.manifest  → RSkillManifest (license, embodiment_tags, latency_budget …)
# pkg.local_dir → Path to weights on disk

Attributes:

Name	Type	Description
manifest		Parsed and validated :class:~openral_core.schemas.RSkillManifest.
local_dir		Filesystem path to the directory containing all skill files.

Store the manifest and local directory path.

Parameters:

Name	Type	Description	Default
manifest	RSkillManifest	Parsed and validated rSkill manifest.	required
local_dir	Path	Filesystem path to the directory containing all skill files.	required

Source code in python/rskill/src/openral_rskill/loader.py

def __init__(self, manifest: RSkillManifest, local_dir: Path) -> None:
    """Store the manifest and local directory path.

    Args:
        manifest: Parsed and validated rSkill manifest.
        local_dir: Filesystem path to the directory containing all skill files.
    """
    self.manifest = manifest
    self.local_dir = local_dir

__repr__() -> str

Return a concise developer representation.

Source code in python/rskill/src/openral_rskill/loader.py

def __repr__(self) -> str:
    """Return a concise developer representation."""
    return (
        f"rSkill(name={self.manifest.name!r}, "
        f"version={self.manifest.version!r}, "
        f"license={self.manifest.license.value!r})"
    )

check_capabilities(manifest: RSkillManifest, robot_capabilities: RobotCapabilities) -> None staticmethod

Verify that the robot satisfies the rSkill's capability requirements.

Composition of :meth:check_embodiment_tags, :meth:check_capability_flags, :meth:check_runtime, and :meth:check_quantization_dtype — raises on the first mismatch. Callers that want a per-section verdict (e.g. openral rskill check <rskill_id>) should call the four narrower methods directly.

Parameters:

Name	Type	Description	Default
manifest	RSkillManifest	The rSkill manifest to check.	required
robot_capabilities	RobotCapabilities	The target robot's declared capabilities.	required

Raises:

Type	Description
ROSCapabilityMismatch	If any required capability is not satisfied or if embodiment tags do not intersect.

Example

from openral_core.schemas import RobotCapabilities caps = RobotCapabilities(embodiment_tags=["so100_follower"])

rSkill.check_capabilities(manifest, caps) # raises if mismatch

Source code in python/rskill/src/openral_rskill/loader.py

@staticmethod
def check_capabilities(
    manifest: RSkillManifest,
    robot_capabilities: RobotCapabilities,
) -> None:
    """Verify that the robot satisfies the rSkill's capability requirements.

    Composition of :meth:`check_embodiment_tags`,
    :meth:`check_capability_flags`, :meth:`check_runtime`, and
    :meth:`check_quantization_dtype` — raises on the first mismatch.
    Callers that want a per-section verdict (e.g. ``openral rskill check
    <rskill_id>``) should call the four narrower methods directly.

    Args:
        manifest: The rSkill manifest to check.
        robot_capabilities: The target robot's declared capabilities.

    Raises:
        ROSCapabilityMismatch: If any required capability is not satisfied
            or if embodiment tags do not intersect.

    Example:
        >>> from openral_core.schemas import RobotCapabilities
        >>> caps = RobotCapabilities(embodiment_tags=["so100_follower"])
        >>> # rSkill.check_capabilities(manifest, caps)  # raises if mismatch
    """
    rSkill.check_embodiment_tags(manifest, robot_capabilities)
    rSkill.check_capability_flags(manifest, robot_capabilities)
    rSkill.check_runtime(manifest, robot_capabilities)
    rSkill.check_quantization_dtype(manifest, robot_capabilities)

check_capability_flags(manifest: RSkillManifest, robot_capabilities: RobotCapabilities) -> None staticmethod

Verify every manifest.capabilities_required flag.

Raises:

Type	Description
ROSCapabilityMismatch	On the first unsatisfied flag or unknown field name.

Source code in python/rskill/src/openral_rskill/loader.py

@staticmethod
def check_capability_flags(
    manifest: RSkillManifest,
    robot_capabilities: RobotCapabilities,
) -> None:
    """Verify every ``manifest.capabilities_required`` flag.

    Raises:
        ROSCapabilityMismatch: On the first unsatisfied flag or
            unknown field name.
    """
    for flag, required_value in manifest.capabilities_required.items():
        robot_value = getattr(robot_capabilities, flag, None)
        if robot_value is None:
            raise ROSCapabilityMismatch(
                f"rSkill '{manifest.name}' requires capability flag '{flag}', "
                "which is not a known field of RobotCapabilities."
            )
        if isinstance(required_value, bool):
            if not robot_value:
                raise ROSCapabilityMismatch(
                    f"rSkill '{manifest.name}' requires '{flag}=True', "
                    f"but robot reports '{flag}={robot_value}'."
                )
        elif isinstance(required_value, (int, float)) and float(robot_value) < float(
            required_value
        ):
            raise ROSCapabilityMismatch(
                f"rSkill '{manifest.name}' requires '{flag}>={required_value}', "
                f"but robot reports '{flag}={robot_value}'."
            )

check_compatibility(manifest: RSkillManifest, robot: RobotDescription) -> None staticmethod

Run every rSkill ↔ robot compatibility check in one call.

This is the umbrella entry point — combines :meth:check_capabilities (embodiment tags + boolean / numeric capability flags) and :meth:check_sensors (sensor requirements against robot.sensors). Use this from runtime code that has the full :class:RobotDescription; the two narrower methods remain for cases where only one half is available.

Parameters:

Name	Type	Description	Default
manifest	RSkillManifest	The rSkill manifest to check.	required
robot	RobotDescription	The target robot's full description.	required

Raises:

Type	Description
ROSCapabilityMismatch	If embodiment tags mismatch, a capability flag is unsatisfied, or any sensor requirement fails.

Example

rSkill.check_compatibility(manifest, robot)

Source code in python/rskill/src/openral_rskill/loader.py

@staticmethod
def check_compatibility(
    manifest: RSkillManifest,
    robot: RobotDescription,
) -> None:
    """Run every rSkill ↔ robot compatibility check in one call.

    This is the umbrella entry point — combines
    :meth:`check_capabilities` (embodiment tags + boolean / numeric
    capability flags) and :meth:`check_sensors` (sensor requirements
    against ``robot.sensors``). Use this from runtime code that has the
    full :class:`RobotDescription`; the two narrower methods remain for
    cases where only one half is available.

    Args:
        manifest: The rSkill manifest to check.
        robot: The target robot's full description.

    Raises:
        ROSCapabilityMismatch: If embodiment tags mismatch, a capability
            flag is unsatisfied, or any sensor requirement fails.

    Example:
        >>> # rSkill.check_compatibility(manifest, robot)
    """
    rSkill.check_capabilities(manifest, robot.capabilities)
    rSkill.check_sensors(manifest, robot.sensors)

check_embodiment_tags(manifest: RSkillManifest, robot_capabilities: RobotCapabilities) -> None staticmethod

Verify the manifest's embodiment tags intersect the robot's.

Exempt for perception rSkills (kind in :data:_EMBODIMENT_AGNOSTIC_KINDS — detector / vlm): they are camera-in → detections/text-out producers with no action contract, so embodiment is not a meaningful axis; they run on any robot regardless of tags. Also skipped when a non-perception manifest declares no embodiment tags. Used by :meth:check_capabilities and by the per-section presenter in :func:openral_detect.check_single_rskill.

Raises:

Type	Description
ROSCapabilityMismatch	If the tag sets are disjoint.

Source code in python/rskill/src/openral_rskill/loader.py

@staticmethod
def check_embodiment_tags(
    manifest: RSkillManifest,
    robot_capabilities: RobotCapabilities,
) -> None:
    """Verify the manifest's embodiment tags intersect the robot's.

    Exempt for perception rSkills (``kind`` in
    :data:`_EMBODIMENT_AGNOSTIC_KINDS` — ``detector`` / ``vlm``): they are
    camera-in → detections/text-out producers with no action contract, so
    embodiment is not a meaningful axis; they run on any robot regardless of
    tags. Also skipped when a non-perception manifest declares no embodiment
    tags. Used by :meth:`check_capabilities` and by the per-section presenter
    in :func:`openral_detect.check_single_rskill`.

    Raises:
        ROSCapabilityMismatch: If the tag sets are disjoint.
    """
    if manifest.kind in _EMBODIMENT_AGNOSTIC_KINDS:
        return
    if not manifest.embodiment_tags:
        return
    robot_tags = set(robot_capabilities.embodiment_tags)
    skill_tags = set(manifest.embodiment_tags)
    if not skill_tags.intersection(robot_tags):
        raise ROSCapabilityMismatch(
            f"rSkill '{manifest.name}' requires embodiment tag(s) "
            f"{sorted(skill_tags)}, but robot only has {sorted(robot_tags)}."
        )

check_quantization_dtype(manifest: RSkillManifest, robot_capabilities: RobotCapabilities) -> None staticmethod

Verify the manifest's quantization dtype is in the robot's set.

Skipped when robot_capabilities.gpu_supported_dtypes is empty.

Raises:

Type	Description
ROSCapabilityMismatch	If the dtype is not supported.

Source code in python/rskill/src/openral_rskill/loader.py

@staticmethod
def check_quantization_dtype(
    manifest: RSkillManifest,
    robot_capabilities: RobotCapabilities,
) -> None:
    """Verify the manifest's quantization dtype is in the robot's set.

    Skipped when ``robot_capabilities.gpu_supported_dtypes`` is empty.

    Raises:
        ROSCapabilityMismatch: If the dtype is not supported.
    """
    if (
        robot_capabilities.gpu_supported_dtypes
        and manifest.quantization.dtype not in robot_capabilities.gpu_supported_dtypes
    ):
        raise ROSCapabilityMismatch(
            f"rSkill '{manifest.name}' requires quantization dtype "
            f"'{manifest.quantization.dtype.value}', but robot only supports "
            f"{[d.value for d in robot_capabilities.gpu_supported_dtypes]}."
        )

check_runtime(manifest: RSkillManifest, robot_capabilities: RobotCapabilities) -> None staticmethod

Verify the manifest's runtime is in the robot's supported set.

Skipped when robot_capabilities.gpu_supported_runtimes is empty (unknown — treat as "host has not been probed").

Raises:

Type	Description
ROSCapabilityMismatch	If the runtime is not supported.

Source code in python/rskill/src/openral_rskill/loader.py

@staticmethod
def check_runtime(
    manifest: RSkillManifest,
    robot_capabilities: RobotCapabilities,
) -> None:
    """Verify the manifest's runtime is in the robot's supported set.

    Skipped when ``robot_capabilities.gpu_supported_runtimes`` is empty
    (unknown — treat as "host has not been probed").

    Raises:
        ROSCapabilityMismatch: If the runtime is not supported.
    """
    if (
        robot_capabilities.gpu_supported_runtimes
        and manifest.runtime not in robot_capabilities.gpu_supported_runtimes
    ):
        raise ROSCapabilityMismatch(
            f"rSkill '{manifest.name}' requires runtime "
            f"'{manifest.runtime.value}', but robot only supports "
            f"{[r.value for r in robot_capabilities.gpu_supported_runtimes]}."
        )

check_sensors(manifest: RSkillManifest, robot_sensors: list[SensorSpec]) -> None staticmethod

Verify the robot exposes every sensor the rSkill requires.

Resolves each :class:~openral_core.SensorRequirement against robot_sensors per the rules documented on :class:~openral_core.SensorRequirement:

• If a requirement carries a vla_feature_key, exactly one robot sensor must expose that key, with matching modality, and meeting any specified min_width / min_height minimum.
• Otherwise, the robot must expose at least count sensors of the requested modality, each meeting any resolution minimum.

Parameters:

Name	Type	Description	Default
manifest	RSkillManifest	The rSkill manifest whose sensors_required to check.	required
robot_sensors	list[SensorSpec]	The robot's declared :class:SensorSpec list (typically RobotDescription.sensors).	required

Raises:

Type	Description
ROSCapabilityMismatch	If any requirement is not satisfied.

Example

from openral_core import SensorSpec, SensorModality robot = [ ... SensorSpec( ... name="cam", modality=SensorModality.RGB, frame_id="world", rate_hz=20.0 ... ) ... ]

rSkill.check_sensors(manifest, robot) # raises if mismatch

Source code in python/rskill/src/openral_rskill/loader.py

@staticmethod
def check_sensors(
    manifest: RSkillManifest,
    robot_sensors: list[SensorSpec],
) -> None:
    """Verify the robot exposes every sensor the rSkill requires.

    Resolves each :class:`~openral_core.SensorRequirement` against
    ``robot_sensors`` per the rules documented on
    :class:`~openral_core.SensorRequirement`:

    * If a requirement carries a ``vla_feature_key``, exactly one robot
      sensor must expose that key, with matching modality, and meeting
      any specified ``min_width`` / ``min_height`` minimum.
    * Otherwise, the robot must expose at least ``count`` sensors of
      the requested modality, each meeting any resolution minimum.

    Args:
        manifest: The rSkill manifest whose ``sensors_required`` to check.
        robot_sensors: The robot's declared :class:`SensorSpec` list
            (typically ``RobotDescription.sensors``).

    Raises:
        ROSCapabilityMismatch: If any requirement is not satisfied.

    Example:
        >>> from openral_core import SensorSpec, SensorModality
        >>> robot = [
        ...     SensorSpec(
        ...         name="cam", modality=SensorModality.RGB, frame_id="world", rate_hz=20.0
        ...     )
        ... ]
        >>> # rSkill.check_sensors(manifest, robot)  # raises if mismatch
    """
    for req in manifest.sensors_required:
        cls = rSkill  # for the raised-error name; statically rSkill here
        if req.vla_feature_key is not None:
            cls._match_keyed_sensor(manifest.name, req, robot_sensors)
        else:
            cls._match_modality_count(manifest.name, req, robot_sensors)

from_pretrained(repo_id: str, *, revision: str | None = None, cache_dir: Path | None = None, force_download: bool = False, commercial_use: bool = True, registry_path: Path | None = None) -> rSkill classmethod

Download an rSkill from the HF Hub, validate it, and register it locally.

Steps performed in order:

1. Download rskill.yaml via :func:huggingface_hub.hf_hub_download.
2. Parse and validate against :class:~openral_core.schemas.RSkillManifest.
3. Run the license guard (hard-block for NVIDIA non-commercial without env).
4. Run the provenance guard (warn that signatures are unverified; fail closed if OPENRAL_REQUIRE_SIGNED_SKILLS=1).
5. Download the full snapshot via :func:huggingface_hub.snapshot_download.
6. Register the entry in the local JSON registry.

.. warning:: No cryptographic signature verification is performed (ADR-0006). The weights are trusted on the basis of HF Hub transport security only. Pin revision to a commit SHA for reproducibility, and treat any *.pt weights as untrusted code (see :class:PyTorchRuntime).

Parameters:

Name	Type	Description	Default
repo_id	str	HF Hub repository, e.g. "openral/rskill-pick-cube-so100".	required
revision	str | None	Git commit SHA or branch to pin. None uses the latest commit on the default branch (not reproducible — pin in production).	None
cache_dir	Path | None	Override the default HF Hub cache directory.	None
force_download	bool	Re-download even if cached files exist.	False
commercial_use	bool	Set to False if the calling deployment is non-commercial research. Relaxes the NVIDIA non-commercial guard.	True
registry_path	Path | None	Override the default registry JSON path.	None

Returns:

Type	Description
rSkill	A validated :class:rSkill instance ready for use.

Raises:

Type	Description
ROSConfigError	If the manifest is invalid, the license blocks the deployment, or huggingface_hub is not installed.
ImportError	Propagated if huggingface_hub is not importable (caught and re-raised as :class:ROSConfigError).

Example

rSkill.from_pretrained("openral/rskill-pick-cube-so100")

rSkill.from_pretrained("openral/rskill-pick-cube-so100", revision="abc123")

Source code in python/rskill/src/openral_rskill/loader.py

@classmethod
def from_pretrained(
    cls,
    repo_id: str,
    *,
    revision: str | None = None,
    cache_dir: Path | None = None,
    force_download: bool = False,
    commercial_use: bool = True,
    registry_path: Path | None = None,
) -> rSkill:
    """Download an rSkill from the HF Hub, validate it, and register it locally.

    Steps performed in order:

    1. Download ``rskill.yaml`` via :func:`huggingface_hub.hf_hub_download`.
    2. Parse and validate against :class:`~openral_core.schemas.RSkillManifest`.
    3. Run the license guard (hard-block for NVIDIA non-commercial without env).
    4. Run the provenance guard (warn that signatures are unverified; fail
       closed if ``OPENRAL_REQUIRE_SIGNED_SKILLS=1``).
    5. Download the full snapshot via :func:`huggingface_hub.snapshot_download`.
    6. Register the entry in the local JSON registry.

    .. warning::
       No cryptographic signature verification is performed (ADR-0006).  The
       weights are trusted on the basis of HF Hub transport security only.
       Pin ``revision`` to a commit SHA for reproducibility, and treat any
       ``*.pt`` weights as untrusted code (see :class:`PyTorchRuntime`).

    Args:
        repo_id: HF Hub repository, e.g. ``"openral/rskill-pick-cube-so100"``.
        revision: Git commit SHA or branch to pin.  ``None`` uses the latest
            commit on the default branch (not reproducible — pin in production).
        cache_dir: Override the default HF Hub cache directory.
        force_download: Re-download even if cached files exist.
        commercial_use: Set to ``False`` if the calling deployment is
            non-commercial research.  Relaxes the NVIDIA non-commercial guard.
        registry_path: Override the default registry JSON path.

    Returns:
        A validated :class:`rSkill` instance ready for use.

    Raises:
        ROSConfigError: If the manifest is invalid, the license blocks the
            deployment, or ``huggingface_hub`` is not installed.
        ImportError: Propagated if ``huggingface_hub`` is not importable (caught
            and re-raised as :class:`ROSConfigError`).

    Example:
        >>> # rSkill.from_pretrained("openral/rskill-pick-cube-so100")
        >>> # rSkill.from_pretrained("openral/rskill-pick-cube-so100", revision="abc123")
    """
    try:
        from huggingface_hub import hf_hub_download, snapshot_download  # noqa: PLC0415
    except ImportError as exc:
        raise ROSConfigError(
            "rSkill.from_pretrained requires 'huggingface_hub'. "
            "Install it: uv add huggingface_hub --package openral-rskill"
        ) from exc

    _cache = str(cache_dir or _CACHE_HOME)

    # ── 1. Fetch manifest ──────────────────────────────────────────────────
    log.info("rskill.fetch_manifest", repo_id=repo_id, revision=revision)
    try:
        manifest_path_str = hf_hub_download(
            repo_id=repo_id,
            filename="rskill.yaml",
            revision=revision,
            cache_dir=_cache,
            force_download=force_download,
        )
    except Exception as exc:
        raise ROSConfigError(f"Failed to download rskill.yaml for '{repo_id}': {exc}") from exc

    # ── 2. Parse manifest ──────────────────────────────────────────────────
    manifest = RSkillManifest.from_yaml(manifest_path_str)

    # ── 3. License guard ───────────────────────────────────────────────────
    cls._check_license(manifest, commercial_use=commercial_use)

    # ── 4. Provenance guard (signatures not yet verified — ADR-0006) ────────
    cls._check_provenance(manifest, source=repo_id)

    # ── 5. Download weights ────────────────────────────────────────────────
    log.info("rskill.download_snapshot", repo_id=repo_id, revision=revision)
    try:
        local_dir_str = snapshot_download(
            repo_id=repo_id,
            revision=revision,
            cache_dir=_cache,
            force_download=force_download,
        )
    except Exception as exc:
        raise ROSConfigError(f"Failed to download snapshot for '{repo_id}': {exc}") from exc

    # ── 6. Register ────────────────────────────────────────────────────────
    entry = InstalledRSkillEntry(
        repo_id=repo_id,
        version=manifest.version,
        revision=revision,
        local_dir=local_dir_str,
        manifest_path=manifest_path_str,
        license=manifest.license.value,
        role=manifest.role,
        embodiment_tags=manifest.embodiment_tags,
        installed_at=datetime.now(timezone.utc).isoformat(),
    )
    cls._register(entry, registry_path or DEFAULT_REGISTRY_PATH)

    log.info(
        "rskill.installed",
        repo_id=repo_id,
        version=manifest.version,
        license=manifest.license.value,
    )
    return cls(manifest=manifest, local_dir=Path(local_dir_str))

from_yaml(path: str | Path, *, local_dir: Path | None = None) -> rSkill classmethod

Load an rSkill from a local rskill.yaml without network access.

Does NOT register in the local registry. Useful for offline development and testing.

Parameters:

Name	Type	Description	Default
path	str | Path	Filesystem path to rskill.yaml.	required
local_dir	Path | None	Directory containing the skill files (defaults to the directory that contains rskill.yaml).	None

Returns:

Type	Description
rSkill	A validated :class:rSkill instance.

Raises:

Type	Description
FileNotFoundError	If path does not exist.
ValidationError	If the YAML fails schema validation.
ROSConfigError	If the license blocks the deployment, or OPENRAL_REQUIRE_SIGNED_SKILLS=1 and verification is unavailable (see :meth:_check_provenance).

Example

rSkill.from_yaml("/path/to/my-skill/rskill.yaml")

Source code in python/rskill/src/openral_rskill/loader.py

@classmethod
def from_yaml(cls, path: str | Path, *, local_dir: Path | None = None) -> rSkill:
    """Load an rSkill from a local ``rskill.yaml`` without network access.

    Does NOT register in the local registry.  Useful for offline development
    and testing.

    Args:
        path: Filesystem path to ``rskill.yaml``.
        local_dir: Directory containing the skill files (defaults to the
            directory that contains ``rskill.yaml``).

    Returns:
        A validated :class:`rSkill` instance.

    Raises:
        FileNotFoundError: If ``path`` does not exist.
        pydantic.ValidationError: If the YAML fails schema validation.
        ROSConfigError: If the license blocks the deployment, or
            ``OPENRAL_REQUIRE_SIGNED_SKILLS=1`` and verification is
            unavailable (see :meth:`_check_provenance`).

    Example:
        >>> # rSkill.from_yaml("/path/to/my-skill/rskill.yaml")
    """
    manifest = RSkillManifest.from_yaml(str(path))
    resolved = Path(path).resolve()
    cls._check_license(manifest, commercial_use=True)
    cls._check_provenance(manifest, source=str(resolved))
    skill_dir = local_dir or resolved.parent
    cls._validate_eval_jsons(skill_dir)
    return cls(manifest=manifest, local_dir=skill_dir)

list_installed(registry_path: Path | None = None) -> list[InstalledRSkillEntry] staticmethod

Return all entries in the local skill registry.

Parameters:

Name	Type	Description	Default
registry_path	Path | None	Override the default registry JSON path.	None

Returns:

Type	Description
list[InstalledRSkillEntry]	List of :class:InstalledRSkillEntry, newest-first.

Raises:

Type	Description
ROSConfigError	If the registry file is corrupt (invalid JSON or schema).

Example

import tempfile, pathlib empty = pathlib.Path(tempfile.mkdtemp()) / "registry.json" result = rSkill.list_installed(registry_path=empty) isinstance(result, list) and len(result) == 0 True

Source code in python/rskill/src/openral_rskill/loader.py

@staticmethod
def list_installed(registry_path: Path | None = None) -> list[InstalledRSkillEntry]:
    """Return all entries in the local skill registry.

    Args:
        registry_path: Override the default registry JSON path.

    Returns:
        List of :class:`InstalledRSkillEntry`, newest-first.

    Raises:
        ROSConfigError: If the registry file is corrupt (invalid JSON or schema).

    Example:
        >>> import tempfile, pathlib
        >>> empty = pathlib.Path(tempfile.mkdtemp()) / "registry.json"
        >>> result = rSkill.list_installed(registry_path=empty)
        >>> isinstance(result, list) and len(result) == 0
        True
    """
    reg = registry_path or DEFAULT_REGISTRY_PATH
    if not reg.exists():
        return []
    try:
        raw: list[dict[str, Any]] = json.loads(reg.read_text(encoding="utf-8"))
    except json.JSONDecodeError as exc:
        raise ROSConfigError(f"Corrupt skill registry at {reg}: {exc}") from exc
    entries = [InstalledRSkillEntry.model_validate(r) for r in raw]
    # newest-first
    return sorted(entries, key=lambda e: e.installed_at, reverse=True)

uninstall(repo_id: str, registry_path: Path | None = None) -> bool staticmethod

Remove a skill from the local registry (does NOT delete cached weights).

Parameters:

Name	Type	Description	Default
repo_id	str	The HF Hub repository identifier to remove.	required
registry_path	Path | None	Override the default registry JSON path.	None

Returns:

Type	Description
bool	True if the entry was found and removed, False if not found.

Example

rSkill.uninstall("openral/rskill-pick-cube-so100") False

Source code in python/rskill/src/openral_rskill/loader.py

@staticmethod
def uninstall(repo_id: str, registry_path: Path | None = None) -> bool:
    """Remove a skill from the local registry (does NOT delete cached weights).

    Args:
        repo_id: The HF Hub repository identifier to remove.
        registry_path: Override the default registry JSON path.

    Returns:
        ``True`` if the entry was found and removed, ``False`` if not found.

    Example:
        >>> rSkill.uninstall("openral/rskill-pick-cube-so100")
        False
    """
    reg = registry_path or DEFAULT_REGISTRY_PATH
    if not reg.exists():
        return False
    try:
        raw: list[dict[str, Any]] = json.loads(reg.read_text(encoding="utf-8"))
    except json.JSONDecodeError:
        return False
    before = len(raw)
    raw = [r for r in raw if r.get("repo_id") != repo_id]
    if len(raw) == before:
        return False
    reg.write_text(json.dumps(raw, indent=2), encoding="utf-8")
    log.info("rskill.uninstalled", repo_id=repo_id)
    return True

rSkillBase(name: str, *, version: str = '0.1.0', role: str = 's1', embodiment_tags: list[str] | None = None, latency_budget_ms: float | None = None)

Bases: ABC

Abstract base class for all OpenRAL skills.

Subclasses must implement:

• :meth:_configure_impl — load config, parse manifest, etc.
• :meth:_activate_impl — final pre-execution setup (e.g. warm-up).
• :meth:_deactivate_impl — pause execution without unloading weights.
• :meth:_shutdown_impl — release all resources.
• :meth:_step_impl — one inference step; returns an Action.

Subclasses may optionally override:

• :meth:on_load_weights — called during configure to load model weights. Sets info.weights_loaded = True on return.
• :meth:on_quantize — called after weight loading to quantize. Sets info.quantized = True on return.
• :meth:on_warmup — called during activate to run a dummy inference. Sets info.warmed_up = True on return.

Parameters:

Name	Type	Description	Default
name	str	Skill name, used as the ROS 2 node name and in /skill/<name>/info.	required
version	str	SemVer string, forwarded to :class:RSkillInfo.	'0.1.0'
role	str	Skill slot — "s0", "s1", or "s2".	's1'
embodiment_tags	list[str] | None	Embodiment tags for capability matching.	None
latency_budget_ms	float | None	Maximum allowed inference latency. Exceeded latency is logged as a warning but does not raise.	None

Initialise the skill; does not configure or load weights.

Source code in python/rskill/src/openral_rskill/base.py

def __init__(
    self,
    name: str,
    *,
    version: str = "0.1.0",
    role: str = "s1",
    embodiment_tags: list[str] | None = None,
    latency_budget_ms: float | None = None,
) -> None:
    """Initialise the skill; does not configure or load weights."""
    self._info = RSkillInfo(
        name=name,
        version=version,
        state=RSkillState.UNCONFIGURED,
        role=role,
        embodiment_tags=embodiment_tags or [],
        latency_budget_ms=latency_budget_ms,
        stamp_ns=time.time_ns(),
    )
    log.info("skill.created", name=name, role=role, version=version)

info: RSkillInfo property

Current :class:~openral_core.schemas.RSkillInfo snapshot.

Returns a copy — mutating the returned object has no effect.

name: str property

Skill name.

state: RSkillState property

Current primary lifecycle state.

activate() -> None

Transition inactive → active.

Calls :meth:on_warmup, then :meth:_activate_impl.

Raises:

Type	Description
ROSRuntimeError	If the current state does not allow this transition.

Source code in python/rskill/src/openral_rskill/base.py

@final
def activate(self) -> None:
    """Transition ``inactive → active``.

    Calls :meth:`on_warmup`, then :meth:`_activate_impl`.

    Raises:
        ROSRuntimeError: If the current state does not allow this
            transition.
    """
    self._require_transition(RSkillState.ACTIVE)
    log.info("rskill.activate", name=self.name)
    try:
        with rskill_span("rskill.activate", rskill_id=self.name, role=self._info.role):
            self.on_warmup()
            self._update(warmed_up=True)
            self._activate_impl()
            self._transition(RSkillState.ACTIVE)
    except Exception as exc:
        self._enter_error(str(exc))
        raise

configure() -> None

Transition unconfigured → inactive.

Calls :meth:on_load_weights, then :meth:on_quantize, then :meth:_configure_impl in order.

Raises:

Type	Description
ROSRuntimeError	If the current state does not allow this transition.

Source code in python/rskill/src/openral_rskill/base.py

@final
def configure(self) -> None:
    """Transition ``unconfigured → inactive``.

    Calls :meth:`on_load_weights`, then :meth:`on_quantize`, then
    :meth:`_configure_impl` in order.

    Raises:
        ROSRuntimeError: If the current state does not allow this
            transition.
    """
    self._require_transition(RSkillState.INACTIVE)
    log.info("rskill.configure", name=self.name)
    try:
        with rskill_span("rskill.configure", rskill_id=self.name, role=self._info.role):
            self.on_load_weights()
            self._update(weights_loaded=True)
            self.on_quantize()
            self._update(quantized=True)
            self._configure_impl()
            self._transition(RSkillState.INACTIVE)
    except Exception as exc:
        self._enter_error(str(exc))
        raise

deactivate() -> None

Transition active → inactive.

Raises:

Type	Description
ROSRuntimeError	If the current state is not active.

Source code in python/rskill/src/openral_rskill/base.py

@final
def deactivate(self) -> None:
    """Transition ``active → inactive``.

    Raises:
        ROSRuntimeError: If the current state is not ``active``.
    """
    if self._info.state is not RSkillState.ACTIVE:
        raise ROSRuntimeError(
            f"Skill '{self.name}': deactivate() requires 'active' state, "
            f"current state is '{self._info.state.value}'."
        )
    log.info("rskill.deactivate", name=self.name)
    try:
        self._deactivate_impl()
        self._transition(RSkillState.INACTIVE)
    except Exception as exc:
        self._enter_error(str(exc))
        raise

on_load_weights() -> None

Load model weights into memory.

Called during :meth:configure before :meth:on_quantize. Default is a no-op; override to load safetensors / ONNX files.

Source code in python/rskill/src/openral_rskill/base.py

def on_load_weights(self) -> None:
    """Load model weights into memory.

    Called during :meth:`configure` before :meth:`on_quantize`.
    Default is a no-op; override to load ``safetensors`` / ONNX files.
    """

on_quantize() -> None

Apply quantization to loaded weights.

Called during :meth:configure after :meth:on_load_weights. Default is a no-op; override to apply INT8 / INT4 / NVFP4.

Source code in python/rskill/src/openral_rskill/base.py

def on_quantize(self) -> None:
    """Apply quantization to loaded weights.

    Called during :meth:`configure` after :meth:`on_load_weights`.
    Default is a no-op; override to apply INT8 / INT4 / NVFP4.
    """

on_unload_weights() -> None

Release model weights from memory (ADR-0050).

Symmetric with :meth:on_load_weights. Called by :meth:shutdown when weights_loaded is set (before :meth:_shutdown_impl), so the skill runner's single-resident eviction frees GPU VRAM before the next skill loads. Default is a no-op; override to drop model references and call torch.cuda.empty_cache() (or terminate an inference sidecar).

Source code in python/rskill/src/openral_rskill/base.py

def on_unload_weights(self) -> None:
    """Release model weights from memory (ADR-0050).

    Symmetric with :meth:`on_load_weights`. Called by :meth:`shutdown`
    when ``weights_loaded`` is set (before :meth:`_shutdown_impl`), so the
    skill runner's single-resident eviction frees GPU VRAM before the next
    skill loads. Default is a no-op; override to drop model references and
    call ``torch.cuda.empty_cache()`` (or terminate an inference sidecar).
    """

on_warmup() -> None

Run a dummy inference to warm up the model.

Called during :meth:activate before :meth:_activate_impl. Default is a no-op; override to call the model once with dummy input.

Source code in python/rskill/src/openral_rskill/base.py

def on_warmup(self) -> None:
    """Run a dummy inference to warm up the model.

    Called during :meth:`activate` before :meth:`_activate_impl`.
    Default is a no-op; override to call the model once with dummy input.
    """

shutdown() -> None

Transition any state → finalized.

Idempotent if already finalized.

Source code in python/rskill/src/openral_rskill/base.py

@final
def shutdown(self) -> None:
    """Transition any state → ``finalized``.

    Idempotent if already ``finalized``.
    """
    if self._info.state is RSkillState.FINALIZED:
        return
    log.info("rskill.shutdown", name=self.name)
    try:
        # ADR-0050 — release GPU weights on the way down so the skill
        # runner's single-resident eviction frees VRAM before the next
        # skill loads. Guarded by ``weights_loaded`` so a shutdown before
        # configure() never invokes a subclass hook that assumes loaded
        # state.
        if self._info.weights_loaded:
            self.on_unload_weights()
            self._update(weights_loaded=False)
        self._shutdown_impl()
    except Exception as exc:  # reason: shutdown must always reach finalized; log and swallow
        log.error("skill.shutdown.error", name=self.name, exc=str(exc))
    finally:
        self._transition(RSkillState.FINALIZED)

step(world_state: WorldState) -> Action | list[Action]

Execute one inference step and return one or more Action chunks.

Records wall-clock latency and logs a warning if it exceeds latency_budget_ms.

Parameters:

Name	Type	Description	Default
world_state	WorldState	Current :class:~openral_core.schemas.WorldState snapshot from the aggregator.	required

Returns:

Type	Description
Action | list[Action]	Either a single :class:~openral_core.schemas.Action chunk
Action | list[Action]	(the legacy single-control-surface path used by every skill
Action | list[Action]	shipped before ADR-0028b) OR a list of :class:Action
Action | list[Action]	chunks (ADR-0028b multi-surface dispatch — used by skills
Action | list[Action]	whose manifest declares an action_contract.slots block;
Action | list[Action]	each slot in the manifest becomes one :class:Action in
Action | list[Action]	the returned list, all routed by the HAL according to their
Action | list[Action]	individual control_mode).

Raises:

Type	Description
ROSRuntimeError	If the skill is not in the active state.

Source code in python/rskill/src/openral_rskill/base.py

@final
def step(self, world_state: WorldState) -> Action | list[Action]:
    """Execute one inference step and return one or more ``Action`` chunks.

    Records wall-clock latency and logs a warning if it exceeds
    ``latency_budget_ms``.

    Args:
        world_state: Current :class:`~openral_core.schemas.WorldState`
            snapshot from the aggregator.

    Returns:
        Either a single :class:`~openral_core.schemas.Action` chunk
        (the legacy single-control-surface path used by every skill
        shipped before ADR-0028b) OR a list of :class:`Action`
        chunks (ADR-0028b multi-surface dispatch — used by skills
        whose manifest declares an ``action_contract.slots`` block;
        each slot in the manifest becomes one :class:`Action` in
        the returned list, all routed by the HAL according to their
        individual ``control_mode``).

    Raises:
        ROSRuntimeError: If the skill is not in the ``active`` state.
    """
    if self._info.state is not RSkillState.ACTIVE:
        raise ROSRuntimeError(
            f"Skill '{self.name}' is in state '{self._info.state.value}'; "
            "must be 'active' to call step()."
        )
    t0 = time.perf_counter()
    with rskill_span("rskill.execute", rskill_id=self.name, role=self._info.role):
        action = self._step_impl(world_state)
    elapsed_ms = (time.perf_counter() - t0) * 1000.0
    self._update(last_inference_ms=elapsed_ms)
    budget = self._info.latency_budget_ms
    if budget is not None and elapsed_ms > budget:
        log.warning(
            "skill.step.latency_exceeded",
            name=self.name,
            elapsed_ms=round(elapsed_ms, 2),
            budget_ms=budget,
        )
    log.debug("skill.step", name=self.name, elapsed_ms=round(elapsed_ms, 2))
    return action

auto_select_quant(device_info: DeviceInfo) -> QuantizationConfig

Return the best QuantizationConfig for device_info.

Selection strategy (in priority order):

1. No GPU (gpu_memory_bytes == 0): int8_dynamic — CPU-friendly dynamic quantization, halves memory footprint with minimal accuracy loss.
2. Apple Silicon (arch == "apple_silicon"): bf16 — MLX / PyTorch MPS favour BF16; FP16 arithmetic is not natively accelerated.
3. GPU < 4 GiB: int4 — memory-constrained edge GPUs (Jetson Orin 8 GB shared, discrete with < 4 GiB VRAM).
4. GPU 4-8 GiB: fp16 — standard mid-range GPU sweet spot.
5. GPU > 8 GiB, CUDA CC ≥ 8.0 (Ampere+): bf16 — Ampere+ hardware accelerates BF16; better numerical stability than FP16.
6. GPU > 8 GiB, CUDA CC < 8.0 (Volta/Turing): fp16 — native FP16 tensor cores, BF16 not accelerated.

Parameters:

Name	Type	Description	Default
device_info	DeviceInfo	Host compute snapshot.	required

Returns:

Type	Description
QuantizationConfig	A QuantizationConfig from :data:QUANT_PRESETS.

Example

info = DeviceInfo(device_str="cpu", gpu_memory_bytes=0) cfg = auto_select_quant(info) cfg.dtype

info_big_gpu = DeviceInfo( ... device_str="cuda:0", ... gpu_memory_bytes=16 * (1 << 30), ... cuda_compute_capability=(8, 6), ... ) auto_select_quant(info_big_gpu).dtype

Source code in python/rskill/src/openral_rskill/quantization.py

def auto_select_quant(device_info: DeviceInfo) -> QuantizationConfig:
    """Return the best ``QuantizationConfig`` for *device_info*.

    Selection strategy (in priority order):

    1. **No GPU** (``gpu_memory_bytes == 0``): ``int8_dynamic`` — CPU-friendly
       dynamic quantization, halves memory footprint with minimal accuracy loss.
    2. **Apple Silicon** (``arch == "apple_silicon"``): ``bf16`` — MLX /
       PyTorch MPS favour BF16; FP16 arithmetic is not natively accelerated.
    3. **GPU < 4 GiB**: ``int4`` — memory-constrained edge GPUs (Jetson Orin 8 GB
       shared, discrete with < 4 GiB VRAM).
    4. **GPU 4-8 GiB**: ``fp16`` — standard mid-range GPU sweet spot.
    5. **GPU > 8 GiB, CUDA CC ≥ 8.0** (Ampere+): ``bf16`` — Ampere+
       hardware accelerates BF16; better numerical stability than FP16.
    6. **GPU > 8 GiB, CUDA CC < 8.0** (Volta/Turing): ``fp16`` — native FP16
       tensor cores, BF16 not accelerated.

    Args:
        device_info: Host compute snapshot.

    Returns:
        A ``QuantizationConfig`` from :data:`QUANT_PRESETS`.

    Example:
        >>> info = DeviceInfo(device_str="cpu", gpu_memory_bytes=0)
        >>> cfg = auto_select_quant(info)
        >>> cfg.dtype
        <QuantizationDtype.INT8: 'int8'>

        >>> info_big_gpu = DeviceInfo(
        ...     device_str="cuda:0",
        ...     gpu_memory_bytes=16 * (1 << 30),
        ...     cuda_compute_capability=(8, 6),
        ... )
        >>> auto_select_quant(info_big_gpu).dtype
        <QuantizationDtype.BF16: 'bf16'>
    """
    mem = device_info.gpu_memory_bytes

    # Apple Silicon always favours BF16 regardless of reported GPU memory
    if device_info.arch == "apple_silicon":
        return QUANT_PRESETS["bf16"]

    if mem == 0:
        return QUANT_PRESETS["int8_dynamic"]

    if mem < 4 * _GB:
        return QUANT_PRESETS["int4"]

    if mem <= 8 * _GB:
        return QUANT_PRESETS["fp16"]

    # > 8 GiB: check CUDA compute capability for BF16 acceleration
    cc = device_info.cuda_compute_capability
    if cc is not None and cc[0] >= _CC_BF16_MIN:  # Ampere (8.0) and newer
        return QUANT_PRESETS["bf16"]

    return QUANT_PRESETS["fp16"]

discover_intree_rskills() -> list[tuple[str, RSkillManifest]]

Walk the in-tree rskills/ directory and return (name, manifest) pairs.

Skipped on stderr-reported errors so a malformed local manifest does not break the listing. Sorted alphabetically by directory name. No HF Hub network call.

Returns:

Type	Description
list[tuple[str, RSkillManifest]]	List of (rskill_name, manifest) tuples, one per
list[tuple[str, RSkillManifest]]	rskills/<name>/rskill.yaml on disk.

Source code in python/rskill/src/openral_rskill/loader.py

def discover_intree_rskills() -> list[tuple[str, RSkillManifest]]:
    """Walk the in-tree ``rskills/`` directory and return ``(name, manifest)`` pairs.

    Skipped on stderr-reported errors so a malformed local manifest does
    not break the listing. Sorted alphabetically by directory name. No
    HF Hub network call.

    Returns:
        List of ``(rskill_name, manifest)`` tuples, one per
        ``rskills/<name>/rskill.yaml`` on disk.
    """
    repo_root = _find_repo_root_from(Path(__file__))
    if repo_root is None:
        return []
    rskills_dir = repo_root / "rskills"
    if not rskills_dir.is_dir():
        return []

    out: list[tuple[str, RSkillManifest]] = []
    for child in sorted(rskills_dir.iterdir()):
        if not child.is_dir() or not (child / "rskill.yaml").is_file():
            continue
        try:
            manifest = load_rskill_manifest(child.name)
        except (ROSError, ValueError) as exc:
            print(f"  {child.name}: {exc}", file=sys.stderr)
            continue
        out.append((child.name, manifest))
    return out

Eval (openral sim run)

openral sim runner — swappable (robot x scene x task x VLA) for rSkill validation.

Typical usage::

from openral_sim import SimRunner

# ``SimRunner`` consumes the runtime-composed ``SimEnvironment``
# (scene + task + VLA). The on-disk YAML is a ``SimScene`` (scene +
# task, no VLA); the CLI composes the ``SimEnvironment`` from a
# ``SimScene`` + an rSkill manifest. See ``openral sim run`` and
# :func:`openral_sim.cli._load_or_build_env` for the canonical
# compose path.
env_cfg: SimEnvironment = ...  # composed by the CLI
runner = SimRunner(env_cfg)
runner.activate()
runner.run(max_ticks=env_cfg.n_episodes * ((env_cfg.task.max_steps or 1000) + 1))
for episode in runner.episode_results:
    print(episode.success, episode.steps, episode.mean_step_latency_ms)
runner.deactivate()

CLI::

openral sim run --config scenes/benchmark/libero_spatial.yaml \
        --rskill smolvla-libero
openral sim run --robot franka_panda --scene libero_spatial \
        --task libero_spatial/0 \
        --rskill smolvla-libero
openral benchmark run --suite libero_spatial \
        --rskill smolvla-libero

The sim package itself depends only on openral-core, openral-runner, and openral-rskill. Physics backends (LIBERO, MetaWorld, ...) are imported lazily by the registered adapters so installing this package never pulls heavyweight ML deps.

EpisodeResult(success: bool = False, steps: int = 0, total_reward: float = 0.0, max_step_reward: float = 0.0, mean_step_latency_ms: float = 0.0, max_step_latency_ms: float = 0.0, latency_budget_ms: float | None = None, budget_violations: int = 0, frames: list[NDArray[np.uint8]] = list(), vla_input_frames: list[NDArray[np.uint8]] = list(), joint_positions: list[NDArray[np.float32]] = list(), actions: list[NDArray[np.float32]] = list(), num_input_cameras: int = 1, metadata: dict[str, Any] = dict()) dataclass

Outcome of one episode rollout.

Attributes:

Name	Type	Description
success	bool	Whether the task was completed (driven by task.success_key).
steps	int	Number of env.step() calls executed.
total_reward	float	Sum of per-step rewards.
max_step_reward	float	Maximum per-step reward observed during the episode. Used as the paper-faithful metric on scenes where the per-step reward is a "best-so-far" continuous signal (PushT reports coverage IoU between the T-block and its goal pose; the Diffusion Policy paper averages the per-rollout max across seeds).
mean_step_latency_ms	float	Mean policy step() latency in milliseconds.
max_step_latency_ms	float	Max policy step() latency in milliseconds.
latency_budget_ms	float | None	Latency budget from the rSkill manifest, if known.
budget_violations	int	Number of steps exceeding the latency budget.
frames	list[NDArray[uint8]]	Captured RGB frames (HWC uint8) when record_video is True; empty otherwise.
metadata	dict[str, Any]	Free-form per-run metadata propagated from :class:openral_core.SimEnvironment.metadata.

summary() -> str

Human-readable single-line summary.

Returns:

Type	Description
str	A compact summary string suitable for CLI output.

Source code in python/sim/src/openral_sim/rollout.py

def summary(self) -> str:
    """Human-readable single-line summary.

    Returns:
        A compact summary string suitable for CLI output.
    """
    return (
        f"success={self.success} steps={self.steps} reward={self.total_reward:.3f} "
        f"mean_lat={self.mean_step_latency_ms:.1f}ms"
        f" budget_viol={self.budget_violations}"
    )

PolicyAdapter

Bases: Protocol

Uniform VLA / policy interface used by the runner.

Attributes:

Name	Type	Description
spec	VLASpec	The :class:openral_core.VLASpec this adapter was built for.
device	str	Resolved torch / numpy device string ("cuda:0", "cpu").

close() -> None

Release any GPU memory / file handles.

Source code in python/sim/src/openral_sim/policy.py

def close(self) -> None:
    """Release any GPU memory / file handles."""

reset() -> None

Reset internal state (action queue, RNG) at the start of each episode.

Source code in python/sim/src/openral_sim/policy.py

def reset(self) -> None:
    """Reset internal state (action queue, RNG) at the start of each episode."""

step(observation: Observation, instruction: str) -> NDArray[np.float32]

Produce the next action vector for the given observation.

Parameters:

Name	Type	Description	Default
observation	Observation	Adapter-specific observation dict produced by the env.	required
instruction	str	Natural-language task string from :attr:openral_core.TaskSpec.instruction.	required

Returns:

Type	Description
NDArray[float32]	1-D float32 NumPy array of length action_dim. Adapters must
NDArray[float32]	return a flat per-step action even if their underlying VLA emits
NDArray[float32]	chunks — chunk caching belongs in the adapter.

Source code in python/sim/src/openral_sim/policy.py

def step(
    self,
    observation: Observation,
    instruction: str,
) -> NDArray[np.float32]:
    """Produce the next action vector for the given observation.

    Args:
        observation: Adapter-specific observation dict produced by the env.
        instruction: Natural-language task string from
            :attr:`openral_core.TaskSpec.instruction`.

    Returns:
        1-D float32 NumPy array of length ``action_dim``. Adapters must
        return a flat per-step action even if their underlying VLA emits
        chunks — chunk caching belongs in the adapter.
    """

SimRollout

Bases: Protocol

Minimal gym-style env contract used by the eval runner.

Every scene factory in :data:openral_sim.SCENES must return an object satisfying this protocol.

Attributes:

Name	Type	Description
scene	SceneSpec	Scene spec the rollout was built from.
task	TaskSpec	Task spec it is currently solving.

Optional duck-typed extensions (any of, mutually exclusive per adapter): mujoco_handles(self) -> tuple[mujoco.MjModel, mujoco.MjData] | None — MuJoCo-backed adapters expose this so openral sim run --view can open a passive mujoco.viewer window against the adapter's own MjModel / MjData.

``sim_time_ns(self) -> int | None``
    — ADR-0048 Phase 1. The backend's authoritative elapsed
    simulation time in nanoseconds — the seam a sim ``/clock``
    publisher reads so the deploy-sim ROS graph runs on simulation
    time rather than wall time. Contract:

    * MuJoCo-backed adapters return ``round(MjData.time * 1e9)`` (the
      physics clock advanced by ``model.opt.timestep`` per step) via
      :func:`sim_time_ns_from_mujoco_handles`.
    * The value is **monotonic non-decreasing within a single
      episode**. It is NOT guaranteed monotonic across ``reset`` —
      backends that rewind ``MjData.time`` to ``0`` on reset (e.g.
      robocasa) restart their clock, so a consumer that needs
      cross-reset monotonicity maintains its own offset (see
      :meth:`openral_hal.sim_attached.SimAttachedHAL.sim_time_ns`).
    * Returns ``None`` when the backend has **no sim clock** —
      clock-less backends (PushT) and out-of-process sidecars
      (Isaac Sim) whose time the adapter cannot read.

    Like the other extensions it is intentionally NOT part of the
    Protocol, so a clock-less adapter need not stub it. Callers MUST
    use ``getattr(env, "sim_time_ns", None)`` and treat both a missing
    attribute and a ``None`` return as "no sim clock" (fall back to
    wall time).

``viewer_render(self) -> None``
    — Adapters whose underlying engine owns the live viewer
    and needs per-step pumping (e.g. SAPIEN / ManiSkill3 envs
    constructed with ``render_mode='human'``). :class:`SimRunner`
    calls this once per applied step to pump the viewer; the
    adapter is responsible for promoting / opening the window
    on the first call and tearing it down in :meth:`close`.

``enable_intrinsic_viewer(self) -> None``
    — Adapters whose engine draws its own self-managed window
    and does not need per-step pumping (e.g. ``gym_pusht`` with
    ``render_mode="human"``; the engine updates the window inside
    its own ``step`` / ``reset``). :class:`SimRunner` calls this
    ONCE before the first ``reset()`` when ``--view`` is set, then
    skips both the MuJoCo viewer path AND the per-step
    ``viewer_render`` pump.

All three extensions are intentionally *not* part of the Protocol
so adapters do not need to stub a method they cannot honour;
callers MUST use ``getattr(env, "<name>", None)`` and treat both
a missing attribute and a ``None`` return as "viewer unsupported".

close() -> None

Release any underlying engine resources.

Source code in python/sim/src/openral_sim/rollout.py

def close(self) -> None:
    """Release any underlying engine resources."""

render() -> NDArray[np.uint8] | None

Return an HWC uint8 RGB frame, or None when rendering is unavailable.

Source code in python/sim/src/openral_sim/rollout.py

def render(self) -> NDArray[np.uint8] | None:
    """Return an HWC uint8 RGB frame, or ``None`` when rendering is unavailable."""

reset(seed: int | None = ...) -> Observation

Reset the simulator and return the initial observation.

Source code in python/sim/src/openral_sim/rollout.py

def reset(self, seed: int | None = ...) -> Observation:
    """Reset the simulator and return the initial observation."""

step(action: NDArray[np.float32]) -> StepResult

Apply action for one timestep and return the transition.

Source code in python/sim/src/openral_sim/rollout.py

def step(self, action: NDArray[np.float32]) -> StepResult:
    """Apply ``action`` for one timestep and return the transition."""

SimRunner(env_cfg: SimEnvironment, *, view: bool = False, strict_view: bool = False, instruction_override: str | None = None, deadline_overrun_policy: DeadlineOverrunPolicy = DeadlineOverrunPolicy.WARN, recorder: RolloutRecorder | None = None)

Bases: InferenceRunnerBase

One-tick = one-env-step :class:InferenceRunner for sim rollouts.

Drives a single (robot × scene × task × VLA) :class:SimEnvironment for env_cfg.n_episodes episodes. Each call to :meth:tick advances by one tick of one of two flavours:

• Reset tick — emitted at the start of each episode (and once more after a terminated / truncated step). Calls env.reset and policy.reset, finalises the previous :class:EpisodeResult if any, and returns a :class:TickResult with action_applied=False and inference_ms == 0.0.
• Step tick — runs policy.step + env.step. Records latency, reward, terminated / truncated, and (when env_cfg.record_video is set) the rendered frame, VLA input frame, joint state, and action. When the env signals termination the next tick will be a reset tick.

:meth:_should_terminate returns True once n_episodes EpisodeResults have been emitted, so callers can pass an upper-bound max_ticks (typically n_episodes * (max_steps + 1)) and rely on the hook for the real stop condition.

Parameters:

Name	Type	Description	Default
env_cfg	SimEnvironment	Validated :class:openral_core.SimEnvironment.	required
view	bool	Open a passive mujoco.viewer window during the rollout. Only valid for MuJoCo-backed scenes that expose mujoco_handles().	False
strict_view	bool	When view is True and the env adapter does not expose mujoco_handles(), raise :class:ROSConfigError instead of warning and continuing offscreen.	False
deadline_overrun_policy	DeadlineOverrunPolicy	Forwarded to the base class. Defaults to WARN since sim rollouts are not real-time.	WARN

Attributes:

Name	Type	Description
episode_results	list[EpisodeResult]	One :class:EpisodeResult per completed episode, populated as ticks complete.
manifest	RSkillManifest | None	The validated rSkill manifest, or None for mock policies that carry no weights.

Build the runner; defer env / policy construction to :meth:activate.

Parameters:

Name	Type	Description	Default
env_cfg	SimEnvironment	Validated :class:openral_core.SimEnvironment.	required
view	bool	Open a passive mujoco.viewer window during the rollout.	False
strict_view	bool	Raise on missing viewer handles instead of warning.	False
instruction_override	str | None	An explicit --instruction CLI value that must win over a scene's per-episode obs["task"] language (custom BDDL :language clause, RoboCasa sampled object). None when the user passed nothing — the env/YAML language then takes over. See :func:_resolve_step_instruction.	None
deadline_overrun_policy	DeadlineOverrunPolicy	Forwarded to the base class.	WARN
recorder	RolloutRecorder | None	ADR-0019 optional :class:openral_dataset.RolloutRecorder. When set, per-step state / images / action plus episode boundaries are fanned out to the recorder's sinks in addition to the existing :class:_EpisodeBuffer. The buffer drives the in-memory video / json / benchmark pipeline; the recorder drives the durable LeRobotDataset v3 path. The two are additive — the recorder is never a substitute for the buffer.	None

Source code in python/sim/src/openral_sim/sim_runner.py

def __init__(
    self,
    env_cfg: SimEnvironment,
    *,
    view: bool = False,
    strict_view: bool = False,
    instruction_override: str | None = None,
    deadline_overrun_policy: DeadlineOverrunPolicy = DeadlineOverrunPolicy.WARN,
    recorder: RolloutRecorder | None = None,
) -> None:
    """Build the runner; defer env / policy construction to :meth:`activate`.

    Args:
        env_cfg: Validated :class:`openral_core.SimEnvironment`.
        view: Open a passive ``mujoco.viewer`` window during the rollout.
        strict_view: Raise on missing viewer handles instead of warning.
        instruction_override: An explicit ``--instruction`` CLI value that
            must win over a scene's per-episode ``obs["task"]`` language
            (custom BDDL ``:language`` clause, RoboCasa sampled object).
            ``None`` when the user passed nothing — the env/YAML language
            then takes over. See :func:`_resolve_step_instruction`.
        deadline_overrun_policy: Forwarded to the base class.
        recorder: ADR-0019 optional :class:`openral_dataset.RolloutRecorder`.
            When set, per-step state / images / action plus episode
            boundaries are fanned out to the recorder's sinks in
            addition to the existing :class:`_EpisodeBuffer`. The
            buffer drives the in-memory video / json / benchmark
            pipeline; the recorder drives the durable LeRobotDataset
            v3 path. The two are additive — the recorder is never
            a substitute for the buffer.
    """
    # Sim rollouts run as fast as policy + env permit; rate_hz only
    # parameterises the base class' deadline machinery. 1000 Hz keeps
    # the period at 1 ms so a single env.step never naturally
    # over-runs the deadline (deadline policy is WARN by default
    # anyway).
    super().__init__(
        rate_hz=1000.0,
        deadline_overrun_policy=deadline_overrun_policy,
        runner_name=f"sim_runner:{env_cfg.scene.id}:{env_cfg.task.id}",
        save_dir=env_cfg.save_dir,
    )
    self._env_cfg = env_cfg
    self._view = view
    self._strict_view = strict_view
    self._instruction_override = instruction_override
    self._recorder = recorder

    # Lifecycle state. Initialised properly in activate() so the runner
    # can be re-used across runs.
    self.episode_results = []
    self.manifest = None
    self._env: SimRollout | None = None
    self._policy: PolicyAdapter | None = None
    self._obs: Observation = {}
    self._viewer: Any = None
    # True when the scene adapter draws its own window inside env.step /
    # env.reset (e.g. gym_pusht with render_mode="human"). Suppresses
    # the lazy mujoco-viewer-open path in _reset_tick.
    self._intrinsic_viewer: bool = False
    self._frame_dt_s: float | None = None
    self._episode_idx = 0
    self._step_idx = 0
    self._needs_reset = True
    self._buf = _EpisodeBuffer()
    self._budget_ms: float | None = None
    self._run_span_ctx: Any = None
    # Tracks whether the current episode is currently "open" on the
    # recorder. The recorder requires an explicit episode_start before
    # record_frame and episode_end at the boundary; we mirror the
    # _EpisodeBuffer.has_data state here.
    self._recorder_episode_open: bool = False

activate() -> None

Validate the manifest, build env + policy, and arm the first reset tick.

make_env (MuJoCo XML compile / dataset prefetch — ~30–60 s on LIBERO / RoboCasa) and make_policy (PaliGemma 3.4B graph allocation + NF4 quantization — ~100–150 s on π0.5) are independent: both only read the immutable :class:SimEnvironment and share no mutable state. By default they run concurrently on a 2-worker ThreadPoolExecutor (the GIL is released by MuJoCo C calls and the torch / safetensors load path), so the wall-clock for :meth:activate collapses to max(env_ms, policy_ms) instead of their sum. See GH-134.

Set OPENRAL_SIM_SEQUENTIAL_INIT=1 to force the legacy sequential path (debugging, profiling, or when interleaved logs would obscure a root cause).

Exceptions from either side propagate verbatim — no silent retry, no swallowed traceback. If the policy build raises while the env build is still running, the env future is left to finish on the worker thread (cancellation of a started future is a no-op in Python's executor model); :meth:deactivate is responsible for closing it if :meth:activate raised after a partial assignment.

Source code in python/sim/src/openral_sim/sim_runner.py

def activate(self) -> None:
    """Validate the manifest, build env + policy, and arm the first reset tick.

    ``make_env`` (MuJoCo XML compile / dataset prefetch — ~30–60 s on
    LIBERO / RoboCasa) and ``make_policy`` (PaliGemma 3.4B graph
    allocation + NF4 quantization — ~100–150 s on π0.5) are independent:
    both only read the immutable :class:`SimEnvironment` and share no
    mutable state. By default they run concurrently on a 2-worker
    ``ThreadPoolExecutor`` (the GIL is released by MuJoCo C calls and
    the torch / safetensors load path), so the wall-clock for
    :meth:`activate` collapses to ``max(env_ms, policy_ms)`` instead
    of their sum. See GH-134.

    Set ``OPENRAL_SIM_SEQUENTIAL_INIT=1`` to force the legacy
    sequential path (debugging, profiling, or when interleaved logs
    would obscure a root cause).

    Exceptions from either side propagate verbatim — no silent retry,
    no swallowed traceback. If the policy build raises while the env
    build is still running, the env future is left to finish on the
    worker thread (cancellation of a started future is a no-op in
    Python's executor model); :meth:`deactivate` is responsible for
    closing it if :meth:`activate` raised after a partial assignment.
    """
    # Validate before the (expensive) env / policy build so misconfigs
    # fail fast without burning GPU time.
    self.manifest, self._env_cfg = _check_rskill_compatibility(self._env_cfg)
    # SAPIEN-backed scenes (simpler-env / ManiSkill3 bridge envs)
    # need to know at gym.make() time whether to advertise
    # `render_mode='human'` — that toggle is what tells SAPIEN to
    # build a live viewer instead of an offscreen render target. The
    # SCENES factory only sees ``env_cfg``, so we publish the flag
    # through an env var scoped to the build window. MuJoCo-backed
    # adapters ignore the var (their viewer opens lazily from
    # ``mujoco_handles()`` on first reset, not at construct time).
    prev_view_env = os.environ.get(_VIEW_ENV)
    if self._view:
        os.environ[_VIEW_ENV] = "1"
    try:
        self._env, self._policy = _build_env_and_policy(self._env_cfg)
    finally:
        if prev_view_env is None:
            os.environ.pop(_VIEW_ENV, None)
        else:
            os.environ[_VIEW_ENV] = prev_view_env
    self._viewer = None
    self._intrinsic_viewer = False
    self._frame_dt_s = None

    # Scenes whose engine draws its own window (gym_pusht via
    # render_mode="human") expose enable_intrinsic_viewer(); for
    # those the mujoco-viewer path in _reset_tick is bypassed.
    if self._view:
        enable_fn = getattr(self._env, "enable_intrinsic_viewer", None)
        if callable(enable_fn):
            enable_fn()
            self._intrinsic_viewer = True
    self._episode_idx = 0
    self._step_idx = 0
    self._needs_reset = True
    self._buf = _EpisodeBuffer()
    self.episode_results = []
    self._budget_ms = (
        self.manifest.latency_budget.per_chunk_ms if self.manifest is not None else None
    )

    # Outer OTel span equivalent to the deleted run_evaluation's
    # ``eval.run_evaluation``. Held open until deactivate() so child
    # ``rskill.tick`` spans from the base class nest under it.
    self._run_span_ctx = _tracer().start_as_current_span(
        "sim.run",
        attributes={
            "robot.id": self._env_cfg.robot_id,
            "scene.id": self._env_cfg.scene.id,
            "task.id": self._env_cfg.task.id,
            "vla.id": self._env_cfg.vla.id,
            "vla.weights_uri": self._env_cfg.vla.weights_uri,
            "n_episodes": self._env_cfg.n_episodes,
            "seed": self._env_cfg.seed,
        },
    )
    self._run_span_ctx.__enter__()

    super().activate()

deactivate() -> None

Finalise any trailing episode, close env + policy, idempotently.

Source code in python/sim/src/openral_sim/sim_runner.py

def deactivate(self) -> None:
    """Finalise any trailing episode, close env + policy, idempotently."""
    # Flush partial episode (e.g. when max_ticks cut us short).
    if self._buf.has_data:
        self._finalize_episode()
    # ADR-0019: if an episode opened on the recorder never reached
    # _finalize_episode (e.g. activate() succeeded but no ticks ran),
    # close it as a failure before finalising the recorder.
    if self._recorder is not None:
        if self._recorder_episode_open:
            with contextlib.suppress(ValueError, RuntimeError):
                self._recorder.episode_end(success=False)
            self._recorder_episode_open = False
        try:
            self._recorder.finalize()
        except (ValueError, RuntimeError) as exc:
            _log.warning(
                "rollout_recorder_finalize_failed",
                error=str(exc),
                error_type=type(exc).__name__,
            )
    if self._viewer is not None:
        self._viewer.close()
        self._viewer = None
    if self._policy is not None:
        self._policy.close()
        self._policy = None
    if self._env is not None:
        self._env.close()
        self._env = None
    if self._run_span_ctx is not None:
        self._run_span_ctx.__exit__(None, None, None)
        self._run_span_ctx = None
    super().deactivate()

episode_end(*, success: bool) -> None

No-op on SimRunner: sim closes episodes inside _finalize_episode.

Argument intentionally unused — sim's success flag comes from step_result.info[task.success_key], not from the caller.

Source code in python/sim/src/openral_sim/sim_runner.py

def episode_end(self, *, success: bool) -> None:
    """No-op on SimRunner: sim closes episodes inside `_finalize_episode`.

    Argument intentionally unused — sim's success flag comes from
    ``step_result.info[task.success_key]``, not from the caller.
    """
    _ = success

episode_start(task_string: str) -> int

No-op on SimRunner: sim derives episodes from env signals.

Returns the current _episode_idx so callers expecting an integer don't crash. The recorder is opened by _reset_tick, not by this method.

Source code in python/sim/src/openral_sim/sim_runner.py

def episode_start(self, task_string: str) -> int:
    """No-op on SimRunner: sim derives episodes from env signals.

    Returns the current ``_episode_idx`` so callers expecting an
    integer don't crash. The recorder is opened by `_reset_tick`,
    not by this method.
    """
    return self._episode_idx

default_output_path(weights_uri: str, benchmark_id: str) -> str

Compute the canonical eval-JSON path from a bare rSkill reference.

A weights ref like rskills/smolvla-libero resolves to rskills/smolvla-libero/eval/<benchmark_id>.json. The caller is responsible for mkdir -p of the parent directory and for actually writing the JSON (typically via :meth:RSkillEvalResult.model_dump_json).

Parameters:

Name	Type	Description	Default
weights_uri	str	The bare rSkill reference from :attr:VLASpec.weights_uri.	required
benchmark_id	str	The benchmark suite id (the YAML filename stem) — used as the JSON filename stem so re-running the same suite overwrites the same eval JSON.	required

Returns:

Type	Description
str	A relative path string in the form <skill_dir>/eval/<id>.json.

Raises:

Type	Description
ValueError	If weights_uri starts with hf:// (cannot write to the HF Hub; only locally-resolvable rSkills are writable).

Source code in python/sim/src/openral_sim/benchmark.py

def default_output_path(weights_uri: str, benchmark_id: str) -> str:
    """Compute the canonical eval-JSON path from a bare rSkill reference.

    A weights ref like ``rskills/smolvla-libero`` resolves to
    ``rskills/smolvla-libero/eval/<benchmark_id>.json``. The caller is
    responsible for ``mkdir -p`` of the parent directory and for actually
    writing the JSON (typically via :meth:`RSkillEvalResult.model_dump_json`).

    Args:
        weights_uri: The bare rSkill reference from :attr:`VLASpec.weights_uri`.
        benchmark_id: The benchmark suite id (the YAML filename stem) — used
            as the JSON filename stem so re-running the same suite
            overwrites the same eval JSON.

    Returns:
        A relative path string in the form ``<skill_dir>/eval/<id>.json``.

    Raises:
        ValueError: If ``weights_uri`` starts with ``hf://`` (cannot write
            to the HF Hub; only locally-resolvable rSkills are writable).
    """
    if weights_uri.startswith("hf://"):
        raise ValueError(
            f"default_output_path expects a bare rSkill ref (e.g. rskills/<name>); "
            f"got {weights_uri!r}. The benchmark runner only writes JSONs for "
            "locally-resolvable rSkills."
        )
    return f"{weights_uri}/eval/{benchmark_id}.json"

make_env(env_cfg: SimEnvironment) -> SimRollout

Build the simulated environment described by env_cfg.

Parameters:

Name	Type	Description	Default
env_cfg	SimEnvironment	Validated :class:openral_core.SimEnvironment config.	required

Returns:

Name	Type	Description
A	SimRollout	class:openral_sim.rollout.SimRollout ready for
	SimRollout	meth:reset / :meth:step.

Raises:

Type	Description
ROSConfigError	If env_cfg.scene.id is not registered.

Source code in python/sim/src/openral_sim/factory.py

def make_env(env_cfg: SimEnvironment) -> SimRollout:
    """Build the simulated environment described by ``env_cfg``.

    Args:
        env_cfg: Validated :class:`openral_core.SimEnvironment` config.

    Returns:
        A :class:`openral_sim.rollout.SimRollout` ready for
        :meth:`reset` / :meth:`step`.

    Raises:
        openral_core.exceptions.ROSConfigError: If
            ``env_cfg.scene.id`` is not registered.
    """
    factory = SCENES.get(env_cfg.scene.id)
    return factory(env_cfg)

make_policy(env_cfg: SimEnvironment) -> PolicyAdapter

Build the VLA / policy adapter described by env_cfg.vla.

Parameters:

Name	Type	Description	Default
env_cfg	SimEnvironment	Validated :class:openral_core.SimEnvironment config.	required

Returns:

Name	Type	Description
A	PolicyAdapter	class:openral_sim.policy.PolicyAdapter ready for
	PolicyAdapter	meth:reset / :meth:step.

Raises:

Type	Description
ROSConfigError	If env_cfg.vla.id is not registered.

Source code in python/sim/src/openral_sim/factory.py

def make_policy(env_cfg: SimEnvironment) -> PolicyAdapter:
    """Build the VLA / policy adapter described by ``env_cfg.vla``.

    Args:
        env_cfg: Validated :class:`openral_core.SimEnvironment` config.

    Returns:
        A :class:`openral_sim.policy.PolicyAdapter` ready for
        :meth:`reset` / :meth:`step`.

    Raises:
        openral_core.exceptions.ROSConfigError: If
            ``env_cfg.vla.id`` is not registered.
    """
    factory = POLICIES.get(env_cfg.vla.id)
    return factory(env_cfg)

run_benchmark(scenes: list[BenchmarkScene], *, suite_id: str, vla: VLASpec, device: str | None = None, save_dir: str | None = None) -> tuple[RSkillEvalResult, list[EpisodeResult]]

Run a benchmark suite end-to-end against one rSkill.

ADR-0042: a benchmark suite is a bare list[BenchmarkScene] plus a suite_id (typically Path("benchmarks/<id>.yaml").stem). Callers that load from disk should use :func:openral_core.load_benchmark_suite and :func:openral_core.raise_on_invalid_suite before calling this function; the runner does not re-validate suite invariants.

Parameters:

Name	Type	Description	Default
scenes	list[BenchmarkScene]	The list of :class:BenchmarkScenes to evaluate. Each entry carries its own scene, task, robot, episode count, and seed offset. Suite-level invariants (uniformity of robot_id / n_episodes / seed / metadata, unique task ids, non-empty) are assumed pre-validated.	required
suite_id	str	Stable suite identifier (the YAML filename stem). Used in log lines and embedded into reproduction_cli.	required
vla	VLASpec	The single free axis — which rSkill to evaluate. Its weights_uri must be a bare rSkill reference (name, path, or HF repo ID); the strict runner rejects raw hf:// URIs.	required
device	str | None	Optional torch device override applied to every rollout ("cpu", "cuda:0", "mps", "auto"). None keeps the manifest's preferred device.	None
save_dir	str | None	Optional directory written to each :class:SimEnvironment for adapter-side artefacts (videos, traces). Unrelated to where the final :class:RSkillEvalResult JSON lives — that path is chosen by the caller (see :func:default_output_path).	None

Returns:

Type	Description
RSkillEvalResult	A pair (result, episodes) where result is a validated
list[EpisodeResult]	class:RSkillEvalResult ready to be written to
tuple[RSkillEvalResult, list[EpisodeResult]]	rskills/<vla>/eval/<suite_id>.json and episodes is the
tuple[RSkillEvalResult, list[EpisodeResult]]	flat per-(task, seed) list of :class:EpisodeResult objects for
tuple[RSkillEvalResult, list[EpisodeResult]]	callers that want fine-grained data (e.g. unit tests asserting on
tuple[RSkillEvalResult, list[EpisodeResult]]	latency).

Raises:

Type	Description
ROSConfigError	Any error propagated from :class:SimRunner — typically a missing rSkill manifest, an incompatible robot, or an unresolvable rSkill reference. The runner does not catch them; the whole suite fails so partial JSONs never reach disk.

Source code in python/sim/src/openral_sim/benchmark.py

def run_benchmark(
    scenes: list[BenchmarkScene],
    *,
    suite_id: str,
    vla: VLASpec,
    device: str | None = None,
    save_dir: str | None = None,
) -> tuple[RSkillEvalResult, list[EpisodeResult]]:
    """Run a benchmark suite end-to-end against one rSkill.

    ADR-0042: a benchmark suite is a bare ``list[BenchmarkScene]`` plus a
    ``suite_id`` (typically ``Path("benchmarks/<id>.yaml").stem``). Callers
    that load from disk should use :func:`openral_core.load_benchmark_suite`
    and :func:`openral_core.raise_on_invalid_suite` before calling this
    function; the runner does not re-validate suite invariants.

    Args:
        scenes: The list of :class:`BenchmarkScene`s to evaluate. Each
            entry carries its own scene, task, robot, episode count, and
            seed offset. Suite-level invariants (uniformity of robot_id /
            n_episodes / seed / metadata, unique task ids, non-empty) are
            assumed pre-validated.
        suite_id: Stable suite identifier (the YAML filename stem). Used
            in log lines and embedded into ``reproduction_cli``.
        vla: The single free axis — which rSkill to evaluate. Its
            ``weights_uri`` must be a bare rSkill reference (name, path, or
            HF repo ID); the strict runner rejects raw ``hf://`` URIs.
        device: Optional torch device override applied to every rollout
            (``"cpu"``, ``"cuda:0"``, ``"mps"``, ``"auto"``). ``None``
            keeps the manifest's preferred device.
        save_dir: Optional directory written to each :class:`SimEnvironment`
            for adapter-side artefacts (videos, traces). Unrelated to where
            the final :class:`RSkillEvalResult` JSON lives — that path is
            chosen by the caller (see :func:`default_output_path`).

    Returns:
        A pair ``(result, episodes)`` where ``result`` is a validated
        :class:`RSkillEvalResult` ready to be written to
        ``rskills/<vla>/eval/<suite_id>.json`` and ``episodes`` is the
        flat per-(task, seed) list of :class:`EpisodeResult` objects for
        callers that want fine-grained data (e.g. unit tests asserting on
        latency).

    Raises:
        openral_core.exceptions.ROSConfigError: Any error propagated
            from :class:`SimRunner` — typically a missing rSkill
            manifest, an incompatible robot, or an unresolvable rSkill
            reference. The runner does not catch them; the whole suite
            fails so partial JSONs never reach disk.
    """
    from openral_core import SimEnvironment

    from openral_sim.sim_runner import SimRunner

    # Suite invariants (see openral_core.raise_on_invalid_suite) guarantee
    # every BenchmarkScene shares robot_id / n_episodes / seed / metadata.
    # Read from scenes[0] for the seed sweep; pull per-scene robot_id /
    # task / max_steps from each scene inside the loop.
    first = scenes[0]
    seeds = list(range(first.seed, first.seed + first.n_episodes))

    per_task: dict[str, list[bool]] = {}
    all_episodes: list[EpisodeResult] = []

    for scene in scenes:
        task_id = scene.task.id
        # raise_on_invalid_suite asserts robot_id is not None for every
        # scene; BenchmarkScene._require_task_eval_fields asserts
        # task.success_key and task.max_steps are set. Re-narrow for mypy.
        robot_id = scene.robot_id
        max_steps = scene.task.max_steps
        assert robot_id is not None
        assert max_steps is not None
        per_task[task_id] = []
        for seed in seeds:
            vla_for_episode = vla.model_copy(update={"device": device}) if device else vla
            env_cfg = SimEnvironment(
                robot_id=robot_id,
                scene=scene.scene,
                task=scene.task,
                vla=vla_for_episode,
                base_pose=scene.base_pose,
                seed=seed,
                n_episodes=1,
                save_dir=save_dir,
            )
            runner = SimRunner(env_cfg, view=False, strict_view=False)
            try:
                runner.activate()
                runner.run(max_ticks=max_steps + 1)
                episodes = runner.episode_results
            finally:
                runner.deactivate()
            episode = episodes[0]
            per_task[task_id].append(episode.success)
            all_episodes.append(episode)
            _log.info(
                "benchmark_episode_done",
                benchmark=suite_id,
                task=task_id,
                seed=seed,
                success=episode.success,
                steps=episode.steps,
            )

    result = _aggregate_results(
        scenes,
        suite_id=suite_id,
        vla=vla,
        per_task=per_task,
        episodes=all_episodes,
    )
    return result, all_episodes

save_episode_mp4(result: EpisodeResult, path: Path, *, fps: int = 20, title: str | None = None) -> Path

Write a 3-panel debug MP4 for one :class:EpisodeResult.

Parameters:

Name	Type	Description	Default
result	EpisodeResult	An :class:EpisodeResult produced by :class:SimRunner with record_video=True. frames (rollout) populates the top-right panel; vla_input_frames populates the top-left; joint_positions populates the bottom plot.	required
path	Path	Destination .mp4 file. Parent directories are created.	required
fps	int	Playback frame rate.	20
title	str | None	Optional title text rendered above the joint plot.	None

Returns:

Type	Description
Path	The output path.

Raises:

Type	Description
ValueError	If both frames and vla_input_frames are empty (no video to render) — set record_video=True first; or if path does not have an .mp4 suffix (this helper is ffmpeg/x264/yuv420p only — see module docstring).

Source code in python/sim/src/openral_sim/_video.py

def save_episode_mp4(
    result: EpisodeResult,
    path: Path,
    *,
    fps: int = 20,
    title: str | None = None,
) -> Path:
    """Write a 3-panel debug MP4 for one :class:`EpisodeResult`.

    Args:
        result: An :class:`EpisodeResult` produced by :class:`SimRunner` with
            ``record_video=True``. ``frames`` (rollout) populates the
            top-right panel; ``vla_input_frames`` populates the top-left;
            ``joint_positions`` populates the bottom plot.
        path: Destination ``.mp4`` file. Parent directories are created.
        fps: Playback frame rate.
        title: Optional title text rendered above the joint plot.

    Returns:
        The output path.

    Raises:
        ValueError: If both ``frames`` and ``vla_input_frames`` are empty
            (no video to render) — set ``record_video=True`` first; or if
            ``path`` does not have an ``.mp4`` suffix (this helper is
            ffmpeg/x264/yuv420p only — see module docstring).
    """
    if not result.frames and not result.vla_input_frames:
        raise ValueError(
            "EpisodeResult has no frames; set SimEnvironment.record_video=True "
            "before driving SimRunner to capture rollout + VLA-input frames"
        )
    if path.suffix.lower() != ".mp4":
        raise ValueError(
            f"save_episode_mp4 only writes MP4 (libx264/yuv420p); got path={path!s} "
            f"with suffix {path.suffix!r}. Use a '.mp4' extension — the helper is "
            "intentionally MP4-only (see module docstring)."
        )

    # Length-align the per-step series. ``frames`` and ``vla_input_frames``
    # may be 1 longer than ``actions`` because the runner records the frame
    # *before* policy.step. We trim to the shortest non-empty length to keep
    # the joint cursor in lockstep with the videos.
    n_rollout = len(result.frames)
    n_vla = len(result.vla_input_frames)
    n_state = len(result.joint_positions)
    n = max(n_rollout, n_vla, n_state, 1)

    # Single-cam VLAs (PushT, ALOHA top-only, MetaWorld) feed the same
    # frame to both panels — the env's only render *is* the VLA input.
    # Collapse to one full-width tile in that case. Multi-cam VLAs
    # (LIBERO: wrist + agent-view) keep the two-panel layout.
    show_two = result.num_input_cameras > 1 and bool(result.vla_input_frames)

    states = _stack_padded_states(result.joint_positions, n)
    if show_two:
        top_w = _TILE_SIZE * 2
        rollout_seq = _resize_sequence(result.frames, _TILE_SIZE, _TILE_SIZE, target_len=n)
        vla_seq = _resize_sequence(result.vla_input_frames, _TILE_SIZE, _TILE_SIZE, target_len=n)
    else:
        top_w = _TILE_SIZE * 2
        # Pick whichever stream is non-empty; prefer the rollout view.
        primary = result.frames or result.vla_input_frames
        rollout_seq = _resize_sequence(primary, top_w, _TILE_SIZE, target_len=n)
        vla_seq = []  # unused

    plot_renderer = _JointPlotRenderer(
        states=states,
        width=_CANVAS_W,
        height=_PLOT_H,
        title=title,
    )

    path.parent.mkdir(parents=True, exist_ok=True)
    import imageio.v2 as iio

    writer = iio.get_writer(
        str(path),
        format="ffmpeg",  # type: ignore[arg-type]  # reason: imageio v2 stubs type format as Format enum but accept plugin strings at runtime
        fps=fps,
        codec="libx264",
        quality=8,
        macro_block_size=1,  # no resize; we build exact-size canvases
        pixelformat="yuv420p",
    )
    try:
        for i in range(n):
            if show_two:
                top = np.concatenate([vla_seq[i], rollout_seq[i]], axis=1)  # (H, 2W, 3)
            else:
                top = rollout_seq[i]  # already (H, 2W, 3) when single-panel
            plot = plot_renderer.render_at_step(i)  # (PLOT_H, CANVAS_W, 3)
            canvas = np.concatenate([top, plot], axis=0)  # (CANVAS_H, CANVAS_W, 3)
            writer.append_data(canvas)
    finally:
        writer.close()
    return path