⚠️ ADAS-TO-Critical — Difficult Hand-Backs

491 naturalistic Level-2 ADAS take-overs combining high intervention demand with unstable post-hand-back motion

A curated tail subset of ADAS-TO. Every clip keeps the full multimodal payload: a 20 s front-view video centered on the automation disengagement plus 13 synchronized CAN / perception / IMU signal files.

---

🎬 What a difficult hand-back looks like

Each clip spans t = 5 → 15 s; the red border marks t = 9–11 s, the moment control passes from the automation back to the driver.

Featured

Cut-in alongside a large vehicle · Rivian R1 · cut_in + dangerous_vehicle A vehicle enters the lane while a truck occupies the adjacent one — lateral clearance collapses from both sides.

Rapid closing on the lead vehicle Hyundai Sonata · rel_speed_high · severity 0.69

Highest-demand hand-back in the set RAM HD · U = 0.91, S = 0.06 · severity 0.95

Sudden cut-in Jeep Cherokee · severity 0.73	Abnormal lateral swaying RAM HD · swaying	Close-proximity large vehicle Rivian R1 · dangerous_vehicle
Sudden cut-in Honda Accord · cut_in	Sudden cut-in Ford Escape · cut_in	Sharp TTC drop Rivian R1 · cut-in / TTC drop

_{Examples are drawn from the ADAS-TO corpus to illustrate the scenario types; those with a severity score are members of the 491-clip difficult-hand-back subset.}

---

1. What this subset is for

Most ADAS take-overs are unremarkable. This subset isolates the ones that are simultaneously demanding to execute and poorly settled afterwards — the regime where hand-back quality, not hand-back timing, is the limiting factor. It is intended for studying post-transition control quality, warning/HMI design, and long-tail behavior modeling.

Important framing. The selection measures observed intervention demand and post-hand-back motion, not objective crash risk. These are not crashes or verified near-crashes.

---

2. Selection method

2.1 Eligible population

Starting from the ADAS-TO Maneuver-Filtered subset (cover — disengagements during ordinary lane-keeping/car-following, with observed turns, lane changes and traffic-control stops removed):

Filter	Effect
max_vego > 5 m/s	drop near-stationary events
exclude pure-gas take-overs	accelerator-only overrides are routine re-acceleration, not hand-backs
Eligible population	n = 4,971

2.2 Two deliberately separate constructs

Both are equal-weight rank averages (each feature → empirical-CDF percentile rank → averaged). Rank normalization makes the scores robust across vehicle platforms, avoiding raw-scale issues with steeringTorque and brake magnitude. PCA-1 was computed as a robustness check and agrees closely (ρ = +1.00 for urgency, +0.96 for stability).

A. Take-over urgency U_main — window [−1, +1] s around disengagement. Rank-average of three components:

Component	Definition
Brake	active × onset-proximity × duration, where proximity = `exp(−
Steering	`active × P95(
Multi-channel	brake and steering both used

Accelerator is excluded from the main score (gas often reflects routine re-acceleration); a gas-inclusive variant U_sens is reported as a sensitivity and agrees at ρ = +0.77.

B. Post-hand-back stability S = 1 − I — window [0, +2] s. Instability I is the rank-average of:

Component	Definition
P95 abs(a_x)	longitudinal acceleration
P95 abs(a_y)	curvature-implied lateral acceleration, vEgo² × signed curvature
P95 abs(j_x)	longitudinal jerk, differentiated from signed acceleration
P95 abs(j_y)	lateral jerk, differentiated from signed lateral acceleration
T_settle	first instant after hand-back where all four stay below their P60 for a continuous 1.0 s (capped at +10 s if never reached)

Jerk is taken from signed acceleration so that necessary, smooth hard braking is not penalized as "unstable".

2.3 The selection rule

difficult hand-back  ⇔  U_main > P75  AND  S < P20

→ 491 clips ≈ 9.9 % of the 4,971 eligible take-overs.

difficult_handback_manifest.csv ranks them by a combined severity score (rank 0 = most severe).

2.4 Why the conjunction matters

Urgency and stability are only moderately associated (Spearman ρ = −0.43): more urgent take-overs tend to be less stable, but far from deterministically. A median-split of the whole eligible population shows why neither axis alone is sufficient:

	stable	unstable
high urgency	24 % — decisive but controlled	26 %
low urgency	36 % — routine smooth	14 % — low-input unstable

Roughly a quarter of high-urgency take-overs are stabilized promptly, i.e. a strong intervention is often a controlled recovery. Requiring both high demand and poor settling isolates the genuinely difficult tail rather than merely the forceful one.

2.5 Construct validity

Both scores correlate with independent signals not used in their own definitions:

• Urgency ↔ curvature demand +0.35, closing speed +0.28, minimum TTC −0.28, minimum lane confidence −0.25
• Stability ↔ peak lane offset −0.32, minimum TLC +0.31, residual offset at +5 s −0.14, steering reversals −0.38

The difficult-hand-back group has the highest closing speed, curvature demand, peak lane offset and steering-reversal counts of the four quadrants.

---

3. Contents

ADAS-TO-Critical/
├── data/<CAR_MODEL>/<driver_NNN>/<route_MMM>/<clip_id>/
│   ├── takeover.mp4            20 s front-view video, takeover at t = 10 s
│   ├── meta.json               clip metadata & timing
│   ├── carState.csv            speed, accel, steering, pedals, cruise state
│   ├── carControl.csv          lateral/longitudinal commands
│   ├── carOutput.csv           actuator outputs
│   ├── controlsState.csv       ADAS controller state & alerts
│   ├── drivingModelData.csv    lane-line estimates, desired curvature
│   ├── longitudinalPlan.csv    planner targets, FCW
│   ├── radarState.csv          lead-vehicle radar tracks
│   ├── accelerometer.csv       IMU acceleration
│   ├── VehicleIMU.csv          body-frame IMU / yaw rate
│   ├── Gyroscope.csv           angular rates
│   ├── CameraOdometry.csv      visual odometry
│   ├── LiveCalibration.csv     device→vehicle frame calibration
│   └── LiveParameters.csv      online vehicle-parameter estimates
└── annotations/
    └── difficult_handback_manifest.csv

difficult_handback_manifest.csv columns:

column	meaning
rank	severity rank (0 = most severe)
clip_path	path to the clip inside this repo
video	path to that clip's takeover.mp4
U_main	take-over urgency (0–1, percentile-rank average)
S	post-hand-back stability (0–1; lower = less stable)
severity	combined ranking score used for rank

Timebase: t = 0 is the reconstructed ADAS disengagement, at the midpoint of every 20 s clip.

---

4. Caveats

• Not crash data. Selection reflects intervention demand and motion smoothness, not verified collisions or near-collisions.
• Associational. Reported correlations describe association, not causation.
• Dataset-relative thresholds. P75/P20 cuts are quantiles of this eligible population; they do not transfer unchanged to another corpus.
• a_y is geometric. Lateral acceleration is curvature-implied (vEgo² × curvature); a true IMU-lateral-axis version is a planned refinement.
• Signal validity varies. Lane-based quantities require a lane-confidence gate; radar quantities require a logged lead. Absence of a signal does not mean absence of the condition.
• Anonymized identifiers. driver_NNN / route_MMM replace the original device and route IDs, consistently with the parent ADAS-TO dataset.

---

5. Provenance & citation

Derived from ADAS-TO → https://huggingface.co/datasets/HenryYHW/ADAS-TO Sample subset → https://huggingface.co/datasets/HenryYHW/ADAS-TO-Sample

Please cite the ADAS-TO paper when using this subset. Access is granted manually; the data is released under CC BY-NC 4.0 for non-commercial research.