dataset card

README.md

@@ -0,0 +1,57 @@
+---
+license: mit
+task_categories:
+  - video-to-video
+tags:
+  - layer-decomposition
+  - video-object-removal
+  - harmonization
+  - latents
+pretty_name: LayerGen Eval-Set Latents (encoded)
+---
+
+# LayerGen — Eval-Set Latents (VAE latents + baked text embeddings)
+
+Pre-encoded **evaluation-set inputs** for the LayerGen layer-decomposition / harmonization models,
+so inference can run **anywhere (off-AIP)** without the raw video → VAE-encode → umT5-encode pipeline.
+
+Each `*.parquet` is **one clip** and is fully self-contained:
+
+| column group | contents |
+|---|---|
+| `{composite,mask,fg,bg}_latent_bytes` (+ `_shape`, `_dtype`) | 4-stream Wan-VAE latents, 81f/21 latent-T, fp16, `[16,21,60,104]` |
+| `text_embedding_bytes` (+ `_shape`, `_dtype`) | **baked umT5 edit-instruction embedding** (Wan text encoder, 512-token) |
+| `caption`, `source`, `id`, `width`, `height`, `num_frames`, `fps` | metadata |
+
+Because the umT5 embedding is baked in, **no text encoder is needed at inference** — read with
+`data_format: latents` and the per-clip `text_embedding` comes straight from the parquet.
+
+## Pools (latest **editcap** captioning scheme)
+
+| pool | clips | task | notes |
+|---|---|---|---|
+| `_val_latents_81f_editcap_dec_combined/all` | 678 | decompose | full eval: RORD50 / PROVE-M/H / VOR-Eval/Wild / ROSE-Bench + synthetic |
+| `_val_latents_81f_editcap_har_combined/all` | 458 | harmonize | full harmonize eval |
+| `_val_latents_81f_editcap_dec_pvix/all` | 76 | decompose | PVIX (no-GT) |
+| `_val_latents_xcompv2_editcap_har/all` | 75 | harmonize | cross-composition (harmonize-hard-v1) |
+| `prompt_fix_pool/all` | 306 | decompose | CFG expt — 34 clips × 3 prompt-fixes × 3 seeds (no-CFG) |
+| `prompt_fix_pool_cfg/all` | 171 | decompose | CFG expt — CFG-2 subset |
+| `prompt_fix_pool_har/all` | 180 | harmonize | CFG expt — 20 xcomp clips × 3 × 3 (CFG + no-CFG) |
+
+Crop to 41f/61f at load via `num_latent_t` (11 / 16 / 21). Type-4 harmonize checkpoints require
+`FASTVIDEO_FG_CORRUPT_TYPE_ID=4`; combined checkpoints need `COMBINED_INFER=1`.
+
+## Off-AIP inference (recipe)
+
+```python
+from huggingface_hub import snapshot_download
+# pull one pool locally
+snapshot_download("cs-mshah/layergen-eval-latents", repo_type="dataset",
+                  allow_patterns="_val_latents_81f_editcap_dec_combined/**",
+                  local_dir="evals")
+# then point the inference config's validation.data_path at
+#   evals/_val_latents_81f_editcap_dec_combined/all   (data_format: latents)
+```
+
+DiT checkpoints are public under [`cs-mshah`](https://huggingface.co/cs-mshah); decode uses the public
+Wan2.2-14B VAE. Companion output-viewer datasets: `cs-mshah/layergen-evals` (+ `-baselines`).