Positive control 2 — the healthy SimSiam baseline (P0.2.1)¶
The third Phase-0 sub-study (P0.2.1; see the Phase 0 overview). It continues the collapse-instrument calibration begun in P0.2: same vehicle (predictorless / stop-gradient-off SimSiam as the forced-collapse arm), same goal (prove the instruments catch a real collapse), but it fixes the flaw P0.2 exposed — the intact "healthy" arm was not actually a healthy baseline, so the gate had no trustworthy upper reference. Here we look for this baseline, then ask how far that baseline reaches: a 2×2 regime matrix — {IID, class-blocked} × {40 epochs, single pass} (+ ablations on LR and horizon length) — testing whether the calibration survives toward the project's target regime (correlated ordering, single-pass streaming). It does not: only the clean, long-horizon corner certifies.
Why this exists (what P0.2 left open)¶
P0.2 ran both arms from-scratch, single-pass (~25 steps), on the class-blocked stream — the P0.1 toy horizon. At that horizon the intact arm's RankMe decayed to ~30–40% of its init, indistinguishable from the forced collapse; only the loss floor separated the two. That left the central question unanswerable: was the intact arm's RankMe decay (a) SimSiam's genuine single-pass dynamics — in which case the baseline is fundamentally compromised — or (b) a small-experiment artifact that disappears with a fair training budget? P0.2.1 answers it: (b). With a fair training regime the intact model's RankMe re-expands and holds, cleanly above the collapse floor, and the gate can finally separate the arms on RankMe itself.
The finding: a fair training regime, not a bigger model¶
The load-bearing change is the training regime, not capacity. The backbone — the representation actually under study
— is the same tiny ViT as P0.1/P0.2 (embed_dim=96, depth=4, img_size=32). What changed, and why each matters
(established by the exploration sweep, scratchpad-only):
| Change (vs P0.2) | Why it is load-bearing |
|---|---|
| IID (shuffled) ordering | Isolates the anti-collapse ablation as the only cause of collapse. On the class-blocked correlated stream the same regime does not separate the arms (intact ~2.7–3.5 vs collapse ~2.0) — see the 2×2 regime matrix below; that is a Phase-1 phenomenon, not a calibration baseline. |
| Full STL-10 train split (~400 train/class, was capped at 120→80) | The dominant lever. With P0.2's small data the intact arm's RankMe stalls at ~3.5; with the full split it re-expands to ~5.5. |
| 40 epochs (was a single ~25-step pass) | SimSiam needs a real horizon to reach its stable non-collapsed regime; the recovery happens in the later epochs. (60 epochs adds nothing: ~5.8 → plateaued.) |
| Warmup + cosine LR (was flat) | The healthy RankMe re-expands as the LR anneals — the U-shape. A flat LR does not produce the clean recovery. |
Predictor pred_hidden 64 → 128 |
SimSiam's predictor is its anti-collapse mechanism; the repo default (64 = proj_dim/2) is too narrow at this scale and left the intact arm marginal (~4.0, gate-borderline). Widening only the predictor lifts it to ~5.5. The projector (proj_hidden=256) is unchanged. |
Not load-bearing (checked, held fixed for other reasons): batch size — 32 vs 128 give the same RankMe separation, so 128 is kept only because it is ~2× faster in wall-clock; and model capacity / image resolution — never scaled.
The gate, redesigned (two changes from P0.2, both driven by the runs)¶
P0.2's gate is recalibrated in cafl4ds/configs/positive_control.yaml and evaluated in scripts/positive_control.py
(prints PASS/FAIL, writes comparison.json, exits non-zero on failure). It passes iff all hold:
- PC collapses on RankMe (relative). PC final RankMe ≤ 35% of its own random-init RankMe (measured ~18%).
Relative, because the projector
BatchNorm(affine=False)keeps deep collapse off literal rank-1 (floor ~1.9), so an absolute low floor is unreachable at this scale. - PC "right reason" (point-collapse fingerprint). PC loss floor (min over steps) ≤ −0.9 — it rides to its −1 constant-solution floor (measured ~−1.00). If rank fell but the loss did not approach −1, the wiring would be suspect.
- The healthy control holds (absolute). Intact final RankMe ≥ 4.0 (measured ~5.4–5.7; floor ~1.9).
- RankMe separation.
healthy_final / PC_final≥ 2.0× (measured ~2.7–2.8×). This is the discriminator P0.2 lacked.
Two deliberate departures from P0.2¶
- The discriminator flipped from loss to RankMe. In this fair-training regime both arms reach a low SSL loss (~−0.9 — a low cosine loss is entirely compatible with a healthy SimSiam), so the loss-floor gap is ~0.03 and no longer separates the arms. RankMe now does. (P0.2's toy horizon was the opposite: loss separated, RankMe did not.) Loss is kept only as the PC's "right reason" fingerprint.
- Dropped the "healthy ≥ 60% of its own random-init RankMe" criterion. Random init is diffuse/inflated (RankMe ~10.8); a healthy SSL model legitimately sheds that before re-expanding to ~5.5 (≈53% of init), so "hold near init" is the wrong reference. The healthy arm is gated on an absolute floor well above the collapse floor instead.
The healthy U-shape (the signature)¶
At random init the embeddings are diffuse and high-rank (RankMe ~10.8 — an inflated value, not a healthy reference). Early training organizes the representation and RankMe dips (to ~3.5 by epoch ~6). Then, as the LR anneals over the remaining epochs, the intact model's effective rank re-expands and holds at ~5.5 — the healthy U-shape. The forced-collapse arm, by contrast, sinks monotonically to the ~1.9 projector-BatchNorm floor and stays pinned. That U-shape vs. monotone-to-floor is the separation the gate reads.
The calibration corner — IID × 40 epochs (CPU)¶
This is the corner the calibration is derived in, and the reference the
2×2 matrix below is measured against. One session per seed, both
arms, IID STL-10, full train split, batch_size=128, 40 epochs, warmup+cosine LR, tiny ViT (same backbone as P0.1).
pc_* = collapse ablation, hc_* = intact SimSiam. Both seeds PASS (later corroborated at 4 seeds by the
horizon sweep, worst-of-four
2.47×).
seed PC RankMe (init -> final, % of init) healthy RankMe (final) ratio PC loss floor verdict
0 10.79 -> 2.02 (18.7%) 5.69 2.82x -1.000 PASS
1 10.88 -> 1.98 (18.2%) 5.38 2.72x -0.999 PASS
Trajectory (seed 0, one row per epoch abridged) — the healthy U-shape vs. the PC's monotone sink:
step era pc_loss pc_rankme hc_loss hc_rankme
------------ -------- ------------ ------------ ------------ ------------
0.0000 0.0 -0.2739 10.7941 0.0011 10.7893
64.0000 0.0 -0.7389 4.8885 -0.6084 5.6245 <- both shed the inflated init
160.0000 0.0 -0.9539 2.5240 -0.8261 3.7188 <- healthy dips ...
320.0000 0.0 -0.9890 1.9363 -0.9333 4.8068 <- ... then re-expands as LR anneals
640.0000 0.0 -0.9695 2.1126 -0.8412 4.1533
960.0000 0.0 -0.9495 2.0268 -0.8107 5.4744
1279.0000 0.0 -0.9835 2.0164 -0.9344 5.6912 <- healthy holds ~5.7; PC pinned ~2.0
Device portability (Gaudi 2 HPU)¶
Per the P0.1/P0.2 milestone convention, the identical run (device=hpu, inside the container) reproduces on a Gaudi 2
HPU: the loop is device-portable, every logged value is finite, and the gate verdict is unchanged. The healthy arm
re-expands to RankMe 5.18 (CPU 5.69) and the PC sinks to the ~1.9 floor while its loss rides to −1.0 — the same
qualitative contrast; the modest endpoint delta is the expected CPU-vs-HPU floating-point divergence over 1280 steps
(from-scratch init is bit-identical, built on CPU before the move to hpu).
(HPU, seed 0) PC RankMe 10.88 -> 1.89 (17.4%) healthy RankMe -> 5.18 ratio 2.74x PC loss floor -1.00 verdict PASS
Does the baseline generalize? The 2×2 regime matrix¶
The calibration above is derived in one specific corner — IID ordering, 40 epochs — chosen to isolate the anti-collapse ablation as the only cause of collapse. But the regime the rest of the project runs in differs on two axes:
- Ordering. Phase 1+ streams are temporally correlated (class-blocked and worse), not shuffled IID.
- Horizon. The target loop is single-pass streaming — each sample seen roughly once — not a 40-epoch offline budget.
That raises a question with a potential immediate win: does the healthy baseline (and the gate built on it)
survive toward the target regime? If a single-pass and/or correlated baseline certified just as cleanly, we would not
need to carry a separate 40-epoch IID calibration run — we could calibrate in the regime we actually deploy in. So we
ran the full 2×2 — {IID, class-blocked} × {40 epochs, single pass} — holding everything else in the P0.2.1 fair regime
fixed (same tiny ViT, full STL-10 train split, warmup+cosine LR, pred_hidden=128), and asked which corners the gate
passes.
Only the IID × 40-epoch corner passes. (healthy = intact SimSiam final RankMe; PC = collapse-ablation final
RankMe; ratio = healthy / PC; the gate needs healthy ≥ 4.0 and ratio ≥ 2.0×. Ranges span seeds 0/1.)
| ordering ↓ / horizon → | 40 epochs (calibration) | single pass (target streaming) |
|---|---|---|
| IID (clean) | ✅ PASS — healthy 5.4–5.7, PC ~2.0, 2.7–2.8× | ❌ FAIL — healthy 4.2–4.9, PC 3.0–3.8, 1.3–1.4× |
| class-blocked (corr.) | ❌ FAIL — healthy 2.7–3.5, PC ~2.0–2.2, 1.3–1.7× | ❌ FAIL — healthy 4.2–4.6, PC 3.2–3.5, ~1.3× |
Per-seed numbers:
ordering horizon seed healthy PC (init -> final, %) ratio PC loss floor verdict
IID 40 epochs 0 5.69 10.79 -> 2.02 (18.7%) 2.82x -1.000 PASS
IID 40 epochs 1 5.38 10.88 -> 1.98 (18.2%) 2.72x -0.999 PASS
class_blocked 40 epochs 0 3.52 10.79 -> 2.06 (19.1%) 1.71x -1.000 FAIL
class_blocked 40 epochs 1 2.73 10.88 -> 2.17 (19.9%) 1.26x -0.999 FAIL
IID single pass 0 4.92 10.69 -> 3.85 (36.0%) 1.28x -0.933 FAIL
IID single pass 1 4.20 10.66 -> 3.03 (28.4%) 1.39x -0.989 FAIL
class_blocked single pass 0 4.58 10.64 -> 3.50 (32.9%) 1.31x -0.967 FAIL
class_blocked single pass 1 4.18 10.49 -> 3.25 (31.0%) 1.29x -0.995 FAIL
The three failing corners fail for two distinct reasons, one per axis — and reading them apart is the whole point.
Horizon axis (→ single pass): the collapse control never finishes collapsing. Over a single pass (32 IID / 40 class-blocked steps) the PC only descends to ~3.0–3.8 RankMe — 28–36% of its init, still visibly mid-collapse, nowhere near the ~1.9 projector-BatchNorm floor it reaches given the full horizon. The healthy arm, by contrast, clears the 4.0 floor in every single-pass run (4.2–4.9). So the gate fails not because the baseline is unhealthy but because the collapse reference is under-developed: both arms sit in the ~3–5 band and never separate by 2×. (In the IID single-pass seed-0 run the PC even misses the "≤ 35% of init" relative-collapse check, at 36% — it simply has not had the steps.)
Ordering axis (→ correlated, at the long horizon): the healthy arm is suppressed. At 40 epochs the PC fully collapses (~2.0) but the class-blocked ordering drags the intact arm down to ~2.7–3.5, toward the collapse floor, so again there is no 2× gap — the mirror image of the single-pass failure (the contrast closes from above, not from below). Holding budget, schedule, model and data volume fixed, merely presenting classes one-block-at-a-time suppresses the intact model's effective rank by ~40%; even 40 repeated passes do not undo it. This is exactly the Phase-1 degradation phenomenon the project is about, in miniature.
And the axes do not simply add: at the single-pass horizon, ordering barely matters. IID and class-blocked single-pass are nearly identical (healthy ~4.2–4.9 either way; PC ~3.0–3.8 either way) — the ~40% correlation penalty that halves the healthy RankMe at 40 epochs does not bite in one pass. Seeing each class-block once is, for these transient dynamics, close to seeing the shuffled stream once; correlation needs repeated multi-epoch exposure to express its suppression.
Trajectories make the two mechanisms visible:
Class-blocked × 40 epochs — the healthy arm dips as under IID but its recovery is suppressed, plateauing at ~3.5 instead of re-expanding to ~5.7:
step era pc_loss pc_rankme hc_loss hc_rankme
------------ -------- ------------ ------------ ------------ ------------
0.0000 0.0 -0.2863 10.7913 -0.0078 10.7947
40.0000 0.0 -0.8135 7.2419 -0.3738 8.5563 <- both shed the inflated init
80.0000 0.0 -0.8500 4.5287 -0.7630 5.2474
320.0000 0.0 -0.9705 2.2875 -0.9223 2.4850 <- healthy dips to ~2.5 (as under IID)
800.0000 0.0 -0.8632 2.1625 -0.7759 3.2064 <- ... but only weakly recovers ...
1599.0000 9.0 -0.8319 2.0595 -0.9234 3.5223 <- ... and plateaus at ~3.5, not ~5.7
(Health is read once per epoch; since one epoch = one full class cycle, every reading lands at the same phase — just
after class 9 — so the era column shows 0 except at the final step. The suppression is a property of the trained
representation, not of the sampling phase.)
IID × single pass (seed 0) — neither arm reaches its asymptote in 32 steps: the PC is still descending (~3.8, not ~2.0) and the healthy arm settles at ~4.9 without the late re-expansion the 40-epoch horizon produces:
step era pc_loss pc_rankme hc_loss hc_rankme
0 0 -0.2739 10.6917 0.0011 10.5544
4 0 -0.7887 7.6111 -0.2115 8.0888 <- both shed the inflated init
8 0 -0.7755 5.5657 -0.3780 6.2151
16 0 -0.9320 4.0589 -0.7102 5.2130
24 0 -0.8417 3.8717 -0.6920 4.9657 <- PC still ~3.9 (not collapsed), healthy ~5.0
31 0 -0.7207 3.8446 -0.5585 4.9177 <- gap only ~1.3x: the contrast never opens
(Class-blocked × single pass is nearly identical in shape — healthy ~4.6, PC ~3.5.)
Ablation: Was the default LR the culprit? An LR sweep on the single-pass IID cell¶
The single-pass runs above use the default AdamW LR (1e-3), chosen for no principled reason. Before concluding the
single-pass contrast is unrecoverable, we must rule out the obvious lever: is there a single LR at which — over the
same single pass — the PC floors while the healthy arm stays up? "Floors" means the genuine point-collapse fingerprint
(RankMe → the ~1.9 BatchNorm floor and loss → −1), so we are not fooled by rank merely wandering; "stays up" means the
healthy arm holds the ≥ 4.0 gate floor. If such an LR exists, the single-pass contrast is recoverable and LR was the
missing lever. So we swept the optimizer's base LR across three orders of magnitude, both arms, both seeds, everything
else fixed at the single-pass IID cell (probes off — only RankMe and the loss floor are read; the 1e-3 point
reproduces the cell's documented 4.92 / 3.85 exactly, since the probe flags change only what is measured, not the
trajectory).
healthy PC (collapse ablation)
LR hc_final loss_floor pc_final % of init loss_floor ratio PC floored? healthy up (>=4.0)?
1e-4 10.39 -0.16 10.11 93.5% -0.82 1.03x no yes
3e-4 8.63 -0.35 7.79 72.0% -0.89 1.11x no yes
1e-3 * 4.92 -0.71 3.85 36.0% -0.93 1.28x no yes <- default; best contrast
2e-3 3.33 -0.85 2.61 25.7% -0.94 1.27x no no <- crossover: NEITHER holds
3e-3 2.11 -0.86 2.17 22.7% -0.94 0.97x YES no
5e-3 2.03 -0.87 1.85 21.6% -0.95 1.10x YES no
1e-2 1.53 -0.89 1.50 21.2% -0.94 1.02x YES no
3e-2 1.29 -0.91 1.44 24.2% -0.95 0.89x YES no
1e-1 1.08 -0.89 1.15 24.7% -0.94 0.94x YES no
(* = default LR, reproduces the single-pass IID cell. Seed 1 is nearly identical: healthy holds ≥ 4.0 only for LR ≤
1e-3 [4.20 at 1e-3], the PC floors only for LR ≥ 3e-3 [1.57, loss −0.99] where the healthy arm has already sunk to 1.97
— the same disjoint boundary. Max ratio across the whole grid, either seed, is ~1.4×.)
The two regions are disjoint — no such LR exists. "Healthy up" holds only at LR ≤ 1e-3; "PC floored" only at LR ≥
3e-3; they never overlap. At the crossover (2e-3) neither condition holds — the healthy arm has already fallen below
4.0 (3.33) while the PC has not yet reached its floor (2.61). And the separation ratio never clears ~1.4× anywhere on
the grid, nowhere near the 2× bar.
Why they can't be separated: at single-pass horizon, LR is a second collapse cause coupled to the first. A high LR
does not just accelerate training — it is itself a collapse driver, so cranking it up is no longer a clean anti-collapse
ablation. The one knob (LR) sets how far both arms travel within the fixed 32-step budget. Turn it down and neither
arm reaches its endpoint — the PC has not collapsed (it is still ~3.8–10, loss > −0.9). Turn it up enough to drive the
PC to its floor within 32 steps and the same aggressive updates crush the healthy arm to the same floor — but that
healthy collapse is now LR-driven, not a verdict on the representation, so it cannot be read as "healthy failed."
Flooring the PC by raising LR therefore does not rescue the contrast; it only adds a confound. The anti-collapse
machinery (predictor + stop-gradient) needs horizon — steps over which to build the non-collapsed structure and let
RankMe re-expand as the LR anneals (the 40-epoch U-shape) — not merely a better LR. A single pass denies it those steps
at every LR. Note too that the best single-pass contrast on the entire grid (~1.3×) sits at the default 1e-3, so the
matrix's single-pass FAIL was not an artifact of a poorly-chosen LR — the default was already near-optimal for the
(insufficient) single-pass separation.
Was it the schedule, not the LR? A flat-LR control. One objection remains: the sweep above ran the default
warmup+cosine schedule, which over a 32-step pass spends ~3 steps warming up and the remaining ~29 annealing the LR to
zero. So a nominally high LR is only briefly at peak and is near-frozen exactly at the endpoint where we read RankMe —
maybe the anneal starves the PC's collapse, and the "no band" verdict is a schedule artifact. We tested this directly:
re-ran the single-pass IID sweep with a flat LR (schedule.warmup_frac=0.0 schedule.min_lr_frac=1.0 holds the
multiplier at 1.0 for all 32 steps), both arms, seed 0.
flat LR hc_final pc_final % of init pc_loss ratio PC floored? healthy up (>=4.0)?
1e-4 10.01 9.43 87.1% -0.85 1.06x no yes
3e-4 6.76 5.65 52.6% -0.90 1.20x no yes
1e-3 3.13 3.06 31.9% -0.95 1.02x YES no
3e-3 1.77 1.73 23.8% -0.95 1.03x YES no
1e-2 1.32 1.27 21.3% -0.97 1.04x YES no
3e-2 1.17 1.22 25.3% -0.94 0.97x YES no
The objection is mechanically right about the PC, yet the verdict is unchanged — indeed reinforced. Without the anneal
the PC floors at ~3× lower LR (flat 1e-3 vs annealed 3e-3): the cosine decay was holding the PC back. But that
same anneal is the only thing that lifts the healthy arm's RankMe back up (the U-shape is anneal-driven — see
cafl4ds/schedule.py); remove it and at flat 1e-3 the healthy arm is already at 3.13, below the 4.0 floor. Both
thresholds slide down in lockstep — the anneal is the best case for separation, and it still is not enough.
A dense sweep of the flip zone (3e-4 → 1e-3, both seeds, flat LR) confirms there is no band hiding between the coarse points, and shows why — the crossover is not a plateau where a healthy arm sits above a floored PC, it is the point where both arms free-fall through the same RankMe values together, the healthy one a step behind:
seed 0 (flat LR) seed 1 (flat LR)
LR hc_fin pc_fin(%init) ratio PC/up hc_fin pc_fin(%init) ratio PC/up
3e-4 6.76 5.65 (53%) 1.20x no / yes 5.49 4.66 (44%) 1.18x no / yes
4e-4 5.67 4.77 (45%) 1.19x no / yes 4.57 3.96 (37%) 1.16x no / yes
5e-4 5.08 4.20 (40%) 1.21x no / yes 4.34 3.41 (33%) 1.27x YES / yes <- both flags (seed 1 only)
6e-4 4.56 3.81 (37%) 1.20x no / yes 3.95 3.00 (29%) 1.32x YES / no
7e-4 3.92 3.55 (35%) 1.10x no / no <- gap (seed 0) 3.61 2.75 (27%) 1.31x YES / no
8e-4 3.45 3.35 (34%) 1.03x YES/ no 3.00 2.51 (25%) 1.19x YES / no
1e-3 3.22 3.03 (32%) 1.06x YES/ no 2.76 2.26 (24%) 1.22x YES / no
The single LR where both binary gate flags co-trip (seed 1, 5e-4) is neither robust — on seed 0 the same LR leaves
the PC uncollapsed, with an empty gap at 7e-4 where neither holds — nor meaningful: healthy 4.34 vs PC 3.41
is only 1.27×, and "PC floored" fires there only on the relative 35%-of-init rule, with the PC still at 3.41,
nowhere near its ~1.9 floor. The decisive number: the best RankMe ratio anywhere on the entire fine grid is ~1.3×,
against a 2.0× gate. The disjointness is therefore a property of the single-pass horizon, not of the schedule or the LR
granularity.
Ablation: How cheap can the reference be? A horizon sweep (is 40 the minimum, and is it a plateau?)¶
Single-pass is dead as a shortcut, but the horizon axis is continuous — so where between one pass and 40 epochs does the
≥2× contrast first become trustworthy, and is 40 itself a stable plateau rather than a lucky point? We swept the
horizon on the canonical config (IID, default 1e-3, warmup+cosine, pred_hidden=128; probes off; each horizon gets
its own fully-annealed schedule since total_steps = epochs × batches), reading the gate ratio healthy / PC. The
robustness bar is the minimum ratio across seeds, not the mean — a reference you re-run casually must pass on a bad
draw, not just on average. The crossover region (25/30) and 40 were run at 4 seeds; the rest at 2.
epochs n | healthy(mean) PC(mean) | ratios per seed | MIN mean | robust (min>=2x)?
5 2 | 2.67 2.79 | 0.98 0.93 | 0.93 0.96 | no (healthy still IN the dip)
10 2 | 3.53 2.64 | 1.28 1.38 | 1.28 1.33 | no
20 2 | 4.24 2.40 | 1.76 1.78 | 1.76 1.77 | no (healthy clears 4.0; sep short)
25 4 | 4.70 2.14 | 2.14 2.08 2.55 2.05 | 2.05 2.20 | yes, but THIN (min 2.05)
30 4 | 4.69 2.04 | 2.20 1.89 2.59 2.57 | 1.89 2.31 | NO (one seed fails at 1.89)
40 4 | 5.53 1.94 | 2.82 2.72 3.48 2.47 | 2.47 2.87 | yes (first comfortable margin)
50 2 | 5.68 1.95 | 2.74 3.10 | 2.74 2.92 | yes
60 2 | 5.61 1.94 | 2.83 2.96 | 2.83 2.89 | yes
80 2 | 5.79 1.77 | 3.21 3.33 | 3.21 3.27 | yes
(40-epoch seeds 0/1 are the canonical corner numbers; seeds 2/3 were added here to level it to 4 seeds against the crossover.)
Three results:
- 40 is not a local optimum. The mean ratio climbs monotonically and keeps climbing past 40, out to 80 (2× the horizon): 0.96 → 1.33 → 1.77 → 2.20 → 2.31 → 2.87 → 2.92 → 2.89 → 3.27. No peak-then-decline; the region above 40 is at least as good. 40 sits on a stable, still-improving plateau, not a bump.
- The 2× crossover is noisy, and the first robust margin is 40, not 25. 25 epochs passes all four seeds but with min 2.05 — inside the seed spread — and its neighbour 30 epochs fails a seed at 1.89. So 25–30 straddle the 2× line within the run-to-run noise; a single casual run there cannot be trusted. 40 is the cheapest horizon whose worst-of-four-seeds (2.47) clears 2× comfortably, and 50/60/80 stay comfortable. The hoped-for 4–8× cheaper reference (5–10 epochs) is dead: those are 0.9–1.4×.
- Mechanism, resolved on the horizon axis. The healthy re-expansion saturates by ~40 epochs (5.53, then flat 5.6–5.8 through 80). Above 40 the ratio still creeps up almost entirely because the PC keeps slowly sinking (1.94 → 1.77 at 80, below the nominal ~1.9 BN floor). "Healthy" is a horizon-saturating quantity; "collapsed" is a slowly-deepening one; and the dip bottom itself drifts later with horizon (epoch ~3 at 5ep → ~8–9 at 20ep) because the recovery is anneal-phased. SimSiam at this toy scale needs ~40 epochs for the healthy signal to saturate, and the contrast becomes trustworthy only once it does.
Verdict: keep 40 epochs as the calibration reference — it is already the minimum robust-≥2× horizon (now on 4 seeds, min 2.47×) and is confirmed to be a plateau, not a lucky point. The horizon sweep's payoff is mechanistic (how much horizon SimSiam needs to become healthy here), not a cheaper reference.
Conclusion: no immediate win — carry the 40-epoch IID reference¶
There is no single-pass corner we can certify as a calibration baseline, and the reason is structural (the single-pass and LR mechanisms above), not a threshold or hyperparameter that needs nudging. Re-tuning the gate cannot manufacture a 2× separation the dynamics do not produce at this horizon. So the IID × 40-epoch corner remains the calibration reference we must carry; Phase-1 single-pass RankMe is read against that fixed upper reference, not against a single-pass "healthy" arm (a mid-transient, non-asymptotic value that would move under us).
The silver lining is real but narrower than the hoped-for win: the single-pass healthy representation is not broken — it holds above 4.0, well clear of the eventual collapse floor — and it is ordering-insensitive at this horizon. A single-pass intact backbone is thus a sane Phase-1 starting point, but not a substitute for the calibration corner, which needs the long horizon to make collapse legible.
Insights¶
Main finding:
The instrument was never broken — the baseline was. P0.2's ambiguous RankMe decay on the intact arm was a training-budget artifact, not a property of SimSiam or of RankMe. Given a fair regime, RankMe cleanly separates a healthy representation from a collapsed one at this toy scale — which is exactly what licenses reading Phase-1 RankMe drops as degradation.
Also:
- The discriminator is regime-dependent. Loss separates the arms at a short toy horizon; RankMe separates them once trained properly. A calibration gate must be tied to the regime it will be read in — which is why P0.2.1 both finds the baseline and re-derives the gate in that regime, rather than porting P0.2's loss-based thresholds.
- The healthy signature is a U-shape, not monotonicity. The early RankMe dip is healthy reorganization, not incipient collapse; the tell is whether it re-expands. Phase 1 readouts should expect this shape from intact backbones and not misread the dip.
- Predictor width is part of "healthy SimSiam." The anti-collapse predictor must be wide enough to do its job; an over-narrow bottleneck weakens the very mechanism under study.
- The calibration is a long-horizon, IID phenomenon — no target-regime shortcut. The 2×2 matrix fails for two distinct reasons: correlated ordering suppresses the healthy arm (contrast closes from above), a single pass leaves the collapse control under-developed (from below). Only IID × 40 epochs separates cleanly, so Phase 1 reads against that fixed reference — the baseline does not transfer to the single-pass or correlated corners.
- Correlation is a multi-epoch effect at this scale. IID and class-blocked are nearly indistinguishable at single-pass; the ~40% suppression from correlated ordering emerges only under repeated exposure — so Phase-1 degradation from ordering alone may be milder than the 40-epoch ablation suggests.
- LR is not the missing lever. At single-pass horizon a high LR is itself a collapse cause, coupled to the ablation through one knob, so no LR (annealed or flat, either seed) floors the PC while holding the healthy arm up. The contrast's failure is the short horizon, not a bad hyperparameter.
How to run¶
# CPU (canonical calibration corner: IID x 40 epochs, deterministic init; ~3 min/arm):
uv run python scripts/positive_control.py # seed=0; add seed=1 for the robustness check
# The other three corners of the 2x2 matrix (all expected to FAIL — see the matrix section):
uv run python scripts/positive_control.py stream=class_blocked # correlated x 40 epochs
uv run python scripts/positive_control.py epochs=1 eval_every_epochs=0.125 # IID x single pass
uv run python scripts/positive_control.py stream=class_blocked epochs=1 eval_every_epochs=0.125 # correlated x single pass
# (eval_every_epochs<1 gives a legible per-few-step trajectory over the short single-pass horizon.)
# LR sweep on the single-pass IID cell (is there an LR where PC floors AND healthy holds? there is not).
# Probes off => only RankMe + loss floor are read, so each run is cheap; sweep optim.lr, both seeds:
for lr in 1e-4 3e-4 1e-3 2e-3 3e-3 5e-3 1e-2 3e-2 1e-1; do
uv run python scripts/positive_control.py epochs=1 eval_every_epochs=0.25 optim.lr=$lr \
monitor.run_knn=false monitor.run_linear=false
done
# Flat-LR control (rules out the warmup+cosine schedule as the cause): hold the LR constant for
# all 32 steps. Confirms the disjointness is the horizon, not the anneal (regions still disjoint).
for lr in 1e-4 3e-4 1e-3 3e-3 1e-2 3e-2; do
uv run python scripts/positive_control.py epochs=1 eval_every_epochs=0.25 optim.lr=$lr \
schedule.warmup_frac=0.0 schedule.min_lr_frac=1.0 monitor.run_knn=false monitor.run_linear=false
done
# Horizon sweep (what is the cheapest robust-2x reference, and is 40 a plateau?). Canonical config,
# vary only epochs; read min-ratio across seeds. 40 is the minimum robust corner; 25-30 straddle 2x.
for ep in 5 10 20 25 30 40 50 60 80; do for seed in 0 1 2 3; do
uv run python scripts/positive_control.py epochs=$ep seed=$seed \
monitor.run_knn=false monitor.run_linear=false
done; done
# Fast network-free smoke (synthetic has 100 imgs/class, so shrink the probe reservations):
uv run python scripts/positive_control.py data=synthetic img_size=16 epochs=8 \
stream.support_per_class=8 stream.query_per_class=8 stream.era_eval_per_class=5
# Gaudi HPU (inside the container; see docs/developing.md). torch is from the Habana base
# image, so use the container's system python, and bind STL-10:
./scripts/run_gaudi_dev.sh -m /mnt/stl10 gaudi-env-cafl4ds:latest 0 \
python scripts/positive_control.py device=hpu
Each run writes both arms' run_log.jsonl and a comparison.json (per-checkpoint series + the gate result) to the
Hydra run dir, and prints the side-by-side table, the RankMe/loss sparklines, and the PASS/FAIL verdict. Regime and
gate thresholds live in cafl4ds/configs/positive_control.yaml; the multi-epoch + LR-schedule machinery is in
cafl4ds/loop.py and cafl4ds/schedule.py (unit-tested in tests/unit/test_loop.py and
tests/unit/test_schedule.py).