---
license: apache-2.0
tags:
  - sparse-autoencoder
  - sae
  - interpretability
  - reproducibility
pipeline_tag: feature-extraction
base_model: Qwen/Qwen2.5-0.5B-Instruct
---

# Null-step control for SAE feature drift

Artifact for *A Null-Step Control for Sparse-Autoencoder Feature Drift Across
Fine-Tuning Checkpoints*. Anonymous for double-blind review.

An SAE fitted at every fine-tuning checkpoint, each warm-started from the last,
is a standard way to watch features move. Between two adjacent checkpoint SAEs
the model changed, but the SAE was also **retrained**, and retraining alone moves
the basis. The **null step** isolates that: retrain the base SAE under the
identical recipe on activations from the *frozen* base model, so any drift it
produces is retraining.

## Layout

```
tables/            every number in the paper, as CSV, plus the code that makes them
run_metadata/      per-run hyperparameters and training logs
weights/
  chains/          112 checkpoint SAEs, 5 fine-tuning chains x 4 layers
  null_step/       the control: 1 canonical + 12 seeds per layer
  null_trajectories/  7-step null chains, 4 layers x 3 seeds
  controlled_rerun/   base SAEs for the one-split TopK / JumpReLU rerun
  real_arm_L18/       per-step decoders, both arms, 4 seeds each
  tuned_lens/         per-layer tuned lenses for the token-overlap metric
```

## What backs what

| Paper element | Path |
|---|---|
| §3.1 fidelity envelope (84 evals at layers 6-18; CSV holds all 118) | `tables/sae_checkpoint_metrics.csv` |
| §3.1 and Table 1, inherited-index alignment | `tables/drift_out/drift_by_stage.csv` |
| **Table 2**, controlled rerun | `tables/controlled_rerun/`, `tables/real_arm/` |
| **Figure 1a**, null trajectory | `tables/drift_out/null_trajectory_l18_seeds.csv` |
| **Figure 1b** and §3.4, direction test | `tables/real_arm/L18_pairwise_cosines.csv` |
| §3.2 and Appendix D, artifact null step | `tables/drift_out/null_replicates.csv` |
| §3.3 and Table 3, checkpoint selection | `tables/drift_out/checkpoint_selection.csv` |
| §3.3, shuffled reward | `weights/chains/sae_shuffled/`, `tables/task_performance.csv` |
| §3.5, architecture arms | `tables/controlled_rerun/jumprelu_*.csv` |
| **Appendix G**, heterogeneity and the selection rule | `tables/drift_out/drift_per_feature.csv` |
| §3.1 dead-latent fractions of the rerun (27-31% L18, 59-63% L12) | `tables/real_arm/L*.csv`, column `dead_frac` |

## Reproducing the numbers

```bash
export SAE_RL_ROOT=/path/to/this/repo
python3 tables/scripts/paper_numbers.py --json > fresh.json
diff fresh.json tables/expected/paper_numbers_full.json    # expect no output
```

`tables/scripts/make_numbers.py` emits the LaTeX macros the manuscript quotes, so
no number in the paper is typed by hand. `--skip-weights` drops the five
weight-loading sections and finishes in about a minute.

Two rounding conventions are deliberate and worth knowing before you diff:
printed bounds round **outward** (`floor`/`ceil`), so a measured minimum of 4.69
prints as `4`, not `5`; and the fidelity envelope is scoped to the artifact's 84
evaluations at layers 6-18, not to the controlled rerun, whose dictionaries are
far deader because it uses no dead-latent resampling.

## Notes on the weights

- `chains/` holds 112 files: SFT (8 checkpoints), PPO flexible (8), PPO strict
  (6), PPO high-KL (3), PPO shuffled-label (3), each at layers 6, 12, 18, 23.
  `k=64` throughout except where layer 23 uses `k=256`.
- `real_arm_L18/` stores bare decoder matrices (`[7168, 896]`), not full state
  dicts. These are what the pairwise direction test consumes.
- `null_trajectories/` excludes the `DONE` marker files.
- **Per-step weights exist only for the layer-18 TopK real-arm replication.** The
  controlled rerun saved base SAEs alone, for both architectures, so
  `controlled_rerun/` holds 9 base checkpoints (4 TopK, 5 JumpReLU) and no
  intermediate chain weights. Nothing is lost for verification: every per-step
  `dec_cos`, `dead_frac`, `mean_l0` and `kept_epoch` is recorded in
  `tables/controlled_rerun/*.csv`, which is enough to reproduce both the raw and
  the dead-latent-adjusted shares. For example, at layer 18 the JumpReLU arm gives
  a raw share of 95.1% and 91.3% after dividing out frozen latents, both
  recomputable from that CSV alone.
- Activation tensors are not included; they are regenerable from the public base
  model and GSM8K.

## Caveats we would rather state than have you find

- Layer 23 appears throughout the tables and is **not read** in the paper: its
  residual stream is mean-dominated and no SAE we trained reconstructs it
  acceptably.
- Seed replication covers the SAE fit, not the fine-tuning run. Every real chain
  descends from the same sequence of checkpoints.
- An earlier BatchTopK architecture grid is deliberately excluded: its threshold
  received no gradient, so it is not evidence and the paper does not use it.
- The strict chain's layer-23 `k=256` robustness variant is not here. Layer 23 is
  not read in the paper and that variant backs no claim in it.
- The epoch-budget sweep weights are not here either; Appendix E.1's numbers are
  in `tables/drift_out/`.

## Contents at a glance

| Path | Files |
|---|---|
| `weights/chains/` | 112 |
| `weights/null_step/` | 48 (4 canonical + 44 seeds) |
| `weights/null_trajectories/` | 96 (4 layers x 3 seeds x 8 steps) |
| `weights/real_arm_L18/` | 57 (base + 4 real x 7 + 4 null x 7) |
| `weights/controlled_rerun/` | 9 |
| `weights/tuned_lens/` | 4 |
| `tables/` | 38 |
| `run_metadata/` | 26 |