Similar charts tell us more about range than direction

Reading (2026-09-04 12:30Z, written after the numbers; the bar was fixed before them). PASS. 478 subjects. Against k random members of the SAME situation (identity bucket, same as-of cutoff, same 2-per-symbol cap), the k nearest by shape have narrower 5 d outcome dispersion at every k, and the ratio falls monotonically as the neighborhood tightens: R(std) = 0.916 at k=200, 0.881 at 50, 0.865 at 20, 0.791 at 5; R(200) − R(5) = +0.125, CI90 [0.084, 0.193]; R(5)'s CI90 [0.721, 0.826] is clear of 1. Same picture at 1 d (0.921 → 0.826) and 10 d (0.918 → 0.811). So the state definition carries real information: tighter neighborhoods are genuinely more similar in what followed.

Where the narrowing lives: in the tails. The 80 % width narrows (R 0.975 → 0.935 from k=200 to 20) and the std narrows, but the interquartile range does NOT narrow at k ≥ 20 (R IQR 1.001 / 0.997 / 0.989) — only at k=5 (0.781). Shape similarity within a situation removes extreme outcomes among the nearest analogs more than it compresses the central half.

The construction confound was large: the RAW real std falls 37 % from k=200 to k=5 (4.73 → 2.96 pp), but the random-k control falls 30 % over the same range (5.44 → 3.81) from small-sample bias alone. Without the control the whole 37 % would have been credited to similarity; the honest number is the ratio.

The floor: the ratio is still falling at the tightest threshold tested (k=5, mean distance 0.307), so the irreducible-noise floor is NOT reached within k ∈ [5, 200]. The practical bound at this resolution: at the tightest servable neighborhood about 79 % of the within-situation outcome dispersion remains; at the served k (200–300) about 92 %. The situation itself (identity) is the larger lever (studies 90/91 measured the analog SET vs the base rate); the shape ranking adds ~8 % at served k and ~21 % at k=5.

Not licensed by this: any direction claim (none was measured). Candidate follow-up (its own registration): distance-weighted bands at k=200 (weight nearer analogs more) vs the unweighted band — measure band width at held coverage; the 21 % at k=5 is the ceiling of what such weighting could recover. Runtime 3.5 min at low priority with the clustered rank table.

Run 2026-09-04T12:21:15.896410+00:00 (smoke=False). Subjects 478 kept of 500 sampled ({'sampled': 500, 'no_embedding': 0, 'retrieve_not_ok': 0, 'short_set': 17, 'short_bucket': 5, 'kept': 478}). Pre-registration: research/specs/study100_similarity_dispersion_prereg_2026_09_04.md (ledger 100).

1d

primary (std): {'R5': 0.826, 'R200': 0.9207, 'diff_R200_minus_R5': 0.0947, 'diff_ci90': [0.0332, 0.147], 'n': 478} · floor: {'k': 5, 'R_at_floor': 0.826, 'R_by_k': {5: 0.826, 20: 0.8819, 50: 0.8778, 200: 0.9207}, 'monotone_ratio': False, 'monotone_raw_std': True}

5d

primary (std): {'R5': 0.7912, 'R200': 0.9163, 'diff_R200_minus_R5': 0.1251, 'diff_ci90': [0.0844, 0.1929], 'n': 478} · floor: {'k': 5, 'R_at_floor': 0.7912, 'R_by_k': {5: 0.7912, 20: 0.8645, 50: 0.8805, 200: 0.9163}, 'monotone_ratio': True, 'monotone_raw_std': True}

10d

primary (std): {'R5': 0.8114, 'R200': 0.9182, 'diff_R200_minus_R5': 0.1068, 'diff_ci90': [0.0596, 0.1433], 'n': 478} · floor: {'k': 5, 'R_at_floor': 0.8114, 'R_by_k': {5: 0.8114, 20: 0.8764, 50: 0.9093, 200: 0.9182}, 'monotone_ratio': True, 'monotone_raw_std': True}

Registered 2026-09-04 ~12:20 UTC, before any number was pulled. Graham: "Test whether cohort similarity actually narrows forward outcome dispersion. For a set of subject symbol-date pairs, pull comps at several similarity thresholds — top five, top twenty, top fifty, top two hundred nearest analogs. At each threshold compute the dispersion of forward returns across cohort members: standard deviation, interquartile range, the width of the eighty percent interval. Then plot dispersion against cohort tightness. The hypothesis is that dispersion falls monotonically as similarity increases. If it does, the state definition carries real information and tighter neighborhoods mean genuinely more similar states. If dispersion flattens out past a certain tightness, that floor is the irreducible noise in the state representation, and it tells us the practical limit of forward predictability."

This is a measurement of the memory's RESOLUTION (how much, never which way). It ships nothing and trades nothing.

The confound, and the control that removes it

The sample dispersion of five members is smaller and noisier than that of two hundred by construction, so "dispersion falls with k" is true even for random members. The question is whether the top-k by SHAPE within the same SITUATION are tighter than k random members of that same situation. Control: for each subject and each k, k random rows from the subject's own identity bucket (situation_context_1d, same happening_ident, dates ≤ the same as-of cutoff, the same 2-per-symbol cap, the subject excluded), three disjoint draws averaged. The statistic is the per-subject ratio real / random.

Frozen design

• Subjects: 500 random liquid (dollar_vol ≥ 5e7, close ≥ 5) situation_context rows with a happening identity and a 1d V5 embedding, dates 2022-01-01 … 2026-08-15 (so 10 d forward exists), TABLESAMPLE draw, seed 100.
• Comps: the production happening-then-shape retrieve (retrieve_happening_shape, scale 1d, k = 200, as-of cutoff = date − 15 d, 2-per-symbol cap), members sorted by distance; thresholds k ∈ {5, 20, 50, 200} are prefixes. Subjects whose retrieve returns < 200 members are skipped and counted.
• Outcomes: date-matched excess forward return at 1 / 5 / 10 sessions (forward_returns_cache minus liquid_date_medians), in percentage points. Primary horizon 5 d.
• Dispersion per subject per k: sample standard deviation, interquartile range, 80 % width (p90 − p10; reported only for k ≥ 20). Tightness = mean embedding distance of the prefix.
• Aggregation: median across subjects of each measure (real and random) per k; per-subject ratio real / random; median ratio with a 2,000-resample bootstrap CI90 over subjects.

PRIMARY (one bar, pre-stated)

R(k) = median per-subject ratio of 5 d standard deviations, real / random. PASS ("similarity narrows outcomes") iff R(5) < R(200) with the bootstrap CI90 of the difference R(200) − R(5) clear of zero AND R(5) < 1 with CI90 clear of 1. FAIL otherwise: within a situation, the shape ranking does not narrow width; the situation IS the state.

Reported either way:

• the monotonicity of R(k) across 200 → 50 → 20 → 5 (and of the raw real dispersion, which is expected to fall regardless — the ratio is the claim);
• the floor: the largest k at which R(k) is within 5 % (relative) of R(5) — beyond that tightness, no further narrowing; R at the floor is the share of dispersion the representation cannot remove (irreducible noise);
• the same for IQR and, for k ≥ 20, the 80 % width; and for 1 d and 10 d.

Pre-stated readings:

• PASS with R(5) well below 1 (e.g. ≤ 0.85): the embedding's distance is informative about width within a situation; the calibration layer could use tightness (it already uses cohort tightness — this measures how much).
• PASS but R(5) near 1 (≥ 0.95): statistically real, practically nil — the situation carries the information.
• FAIL: the shape ranking is not where width lives; retrieval can be situation-only for width purposes.

No constant is tuned on this sample. Honest prior: studies 90/91 found width information in the analog set as a whole (IC 0.34–0.42 vs base rate); whether the WITHIN-situation shape ranking adds to that is genuinely open.

Machinery

scripts/research/study100_similarity_dispersion.py run [--smoke] → research/results/study100_*.{md,json,png}. study_ledger row 100. Runs in the day at low priority (retrievals hit the clustered rank table, ~0.2 s each).