easyminnn/egotouch-lerobot

EgoTouch (LeRobot v2.0)

1,928 egocentric human-hand episodes across 213 everyday tasks with a two-hand 21×21 taxel glove, converted to LeRobot v2.0.

Part of a set of tactile manipulation datasets converted to a single common LeRobot v2.0 layout so they can be mixed in one training run.

At a glance

Episodes	1,928
Frames	2,183,142 (30 fps)
Tasks	213, namespaced by environment (Home/…, Office/…, Outdoor/…, Retail/…, Workbench/…)
Video	ego_view (observation.images.main) — 640×480 H.264
Tactile	array — [2, 21, 21] taxel grid (right, left), raw shape [2, 256]
Pose source	Rokoko positions + jq_pressure quaternions for wrist rotation
Tactile type	array, needs a taxel encoder — not an image tactile tokenizer

Notes

• Grid cells that fall outside the physical glove layout are NaN by design (grid_nan_means in info.json). Mask them rather than zero-filling.
• Hand order in the tactile tensor is ["right", "left"].
• 1,446 of 1,928 episodes carry actions; the rest are video + tactile only (has_action=false in episodes.jsonl). Filter on that flag when training an action head, or use the full set for representation learning.
• Frame rate is verified against the source: chest camera streams are 30.000 fps at 640×480 and frame counts match the tactile stream 1:1.

Format

GR00T-flavoured LeRobot v2.0: per-episode Parquet under data/chunk-{:03d}/ and per-episode MP4 under videos/chunk-{:03d}/{video_key}/, with meta/info.json, meta/episodes.jsonl, meta/tasks.jsonl and meta/modality.json.

from datasets import load_dataset          # metadata / parquet only
# or read directly:
import pyarrow.parquet as pq, json
info = json.load(open("meta/info.json"))
tbl  = pq.read_table("data/chunk-000/episode_000000.parquet")

meta/modality.json names the tactile stream and the state/action slot layout, so the dataset can be loaded by an Isaac-GR00T style loader without extra configuration.

Provenance & license

This is a format conversion of zhenyuxie-zhzh/EgoTouch_hdf5 (https://huggingface.co/datasets/zhenyuxie-zhzh/EgoTouch_hdf5), the HDF5 release of EgoTouch from the TouchAnything project, under the MIT license. All sensor data originates from the authors of that dataset; please cite their work. No frames were dropped or re-timed beyond what is listed under Conversion above.