GT-111/robotwin_v3_eef

RoboTwin v3 EEF (LeRobot v3)

This is a LeRobot v3 format conversion of the RoboTwin bimanual benchmark (source: robbyant/robotwin-clean-and-aug-lerobot), reorganized into a canonical bimanual 16D end-effector (EE) pose representation. 50 Hz, 50 tasks, two subsets (clean/ 50 demos/task and aug/ 500 demos/task).

Original Dataset

RoboTwin 2.0: A Scalable Data Generator and Benchmark with Strong Domain Randomization for Robust Bimanual Robotic Manipulation

Chen, T., et al. RoboTwin 2.0. arXiv:2506.18088, 2025.

• Original Source: https://huggingface.co/datasets/robbyant/robotwin-clean-and-aug-lerobot
• Project Website: https://robotwin-platform.github.io
• GitHub: https://github.com/RoboTwin-Platform/RoboTwin

RoboTwin is a bimanual manipulation benchmark on an Aloha-AgileX dual-arm robot. This conversion covers the 50 official tasks in both the clean demonstration split and the domain-randomized augmented split.

License

This dataset is released under CC BY-NC-SA 4.0, consistent with the original RoboTwin LeRobot release.

Conversion Details

What we changed

1. Canonical Bimanual EE Pose Representation: Proprioception and actions are a single 16D bimanual EE pose vector — per arm: position xyz (metres) + quaternion in scalar-first WXYZ order + gripper, left arm first then right arm.

2. Next-Observation-Absolute Actions: Row t stores the pose that observation row t+1 attains. Terminal rows have no next observation; they are marked transition_action_valid = false and must not be supervised.

3. Unified camera keys: Three 640×480 RGB views as observation.images.head (overhead), observation.images.wrist_left, and observation.images.wrist_right. Videos are lossless stream-copied from the source; they are not re-encoded.

4. LeRobot v3 Format: Per-task datasets with sharded MP4 video and Parquet frame data at 50 Hz.

What we preserved

• All 50 tasks in both clean/ and aug/
• Episode structure and task labels
• Frame-level timestamps
• Source numeric poses (WXYZ, next-observation-absolute)

Observation and action convention

• observation.state: 16D bimanual end-effector pose — per arm [x, y, z, qw, qx, qy, qz, gripper], left arm (dims 0–7) then right arm (dims 8–15). Gripper is continuous with 0 = closed and 1 = open.
• action: 16D absolute bimanual end-effector pose with the same layout; row t is the pose attained by observation row t+1.
• transition_action_valid is true exactly on rows that have a contiguous next observation in the same episode; the last row of each episode is false.
• Cameras: observation.images.head, observation.images.wrist_left, observation.images.wrist_right.

Layout

Each task is a self-contained LeRobot v3 dataset under clean/ or aug/:

robotwin_v3_eef/
├── clean/<task>/                 # 50 demos / task
│   ├── data/chunk-*/file-*.parquet
│   ├── videos/observation.images.{head,wrist_left,wrist_right}/chunk-*/file-*.mp4
│   └── meta/
│       ├── info.json
│       ├── tasks.parquet
│       ├── episodes/chunk-*/file-*.parquet
│       ├── stats.json
│       └── norm_stats.json       # post-pipeline normalization stats
├── aug/<task>/                   # 500 demos / task (same layout)
└── README.md

Usage

from lerobot.datasets import LeRobotDataset

dataset = LeRobotDataset(
    "GT-111/robotwin_v3_eef",
    root="clean/adjust_bottle-demo_clean_collect_200-50",
)

Per-task meta/norm_stats.json files are computed in the training transform space (canonical WXYZ → Rot6D → delta, before padding). The __norm_stats__ provenance block is kept only in local trees and is stripped from this Hub projection.

Citation

If you use this dataset, please cite the original RoboTwin paper:

@article{chen2025robotwin,
  title={RoboTwin 2.0: A Scalable Data Generator and Benchmark with Strong Domain Randomization for Robust Bimanual Robotic Manipulation},
  author={Chen, Tianxing and others},
  journal={arXiv preprint arXiv:2506.18088},
  year={2025}
}

Version History

• v3.0 (current): LeRobot v3 conversion with canonical 16D bimanual EE pose layout, next-observation-absolute actions, unified camera keys, and terminal-row validity mask

Canonical action contract v2

All child datasets use eef_absolute_next_observation_wxyz_v2. State/action rows are absolute metres + WXYZ quaternion + gripper openness; a valid action at t is the next persisted observation at t+1. Native timestamps and FPS are retained. Terminal, broken, demonstration, or dimension-masked targets contribute to no action, future, progress, or normalization loss. See each meta/action_contract.json and the root collection_manifest.json.