Testing software backlog data as a comparison signal

Disposition: NO_PILOT_SIGNAL — keep RPO out of similarity ranking

Registered protocol: research/specs/software_rpo_chart_pilot_prereg_v1_2026_08_19.md

Pre-outcome implementation commit: 09b957d3a155f51abb7d248270d65f851e6adcb6

GitHub Actions run: 32300771544 — full tests, frontend build, deployment, and production smoke passed.

Execution order and firewall

The exact committed runner first executed with --structural-only. It read no future path and froze:

• 23 source RPO derivations;
• 22 eligible events after one automatic semantic quarantine;
• 18 structurally powered walk-forward queries across 17 dates; and
• structural manifest 16fd389f5ace0b90d0e606070ce945d84b270fb9f16f271d40fa0adda481e0aa.

Only after that receipt matched the preregistered 23/22/18/17 census did the fixed runner resolve paths. It reproduced the same structural hash before its first outcome query. No candidate was replaced after outcomes.

Coverage and immutable receipts

Transition batch SHA-256: 5237e9af418429bd03fb9d1246f3c53ad6328bb2415839915e41e8dacb1e52d6

Evaluation SHA-256: 083a6bd10d51f53b9b2c466fdec927594d1f9adb1d94823af0bf8f57a840df24

First run SHA-256: 8d9d36a6ac645f863bb30dd23687ecc7cc20c8b7c59d224cba03624d70d7d78a

Replay run SHA-256: 39ff525d33a6d6d6a6d484272b37878b8d4bf163b84f2723f7d0dfd6a64c6113

The full run hash includes the completion clock and therefore differs by design. The structural, transition, and evaluation hashes matched exactly on replay, as did the verdict.

Frozen primary result

• Mean paired delta: +0.011441431 (positive favors chart+RPO)
• Relative improvement: +1.341633%
• 95% query-date block bootstrap interval for mean delta: [-0.067291711, +0.101333695]
• Verdict: NO_PILOT_SIGNAL

Registered gates

The combined ranker was slightly better on the point estimate, but the effect was much smaller than registered and statistically compatible with both harm and benefit. The pilot therefore rejects promoting the 50/50 chart+RPO weight.

Decision

• Keep RPO as inspectable point-in-time state, not a live or research-default ranking component.
• Do not tune the RPO weight, K, horizon, clipping, quarantine threshold, or event subset on these opened outcomes.
• Do not claim that RPO similarity narrows subsequent price paths.
• More immutable RPO history or another fundamental KPI may justify a new independent preregistration; this exact 22-event sample is spent.
• No API, scheduler, notification, paper/live strategy, or production ranking changed.

Status: frozen before any production future-price/path query for this experiment

As of this registration, only structural RPO receipts, public-availability clocks, the research-session calendar, and V5 embedding coverage/distances have been queried. No daily_bars path, forward return, excursion, barrier, or outcome record has been opened for this pilot.

Question

Inside one setup—an auditable software RPO update—does adding point-in-time RPO-growth similarity to chart similarity produce a more coherent distribution of next-five-session price paths than chart similarity alone?

This is a bounded representation pilot. It is not a directional forecast, trading backtest, universal fundamental weight, or production-ranking test.

Frozen population

• Evidence lane: imported software/v1 RPO derivations only.
• Setup: software_rpo_update_completed_close/v1.
• Scale: V5 daily (1d).
• Decision anchor: the earliest audited research-session 16:00 ET close at or after the derivation's exact SEC public-availability clock.
• Population order: decision date, symbol, derivation ID.
• Known-through clock for eventual path resolution: 2026-08-19T20:00:00Z.
• Maximum horizon: five global research sessions after the anchor close.

Every event must have a V5 embedding, a non-gap decision session, and a calendar-defined fifth future session. The outcome-blind census found all 23 derivations had V5 coverage.

Automatic semantic quarantine

An RPO-growth magnitude above 500% in either direction is excluded before outcomes. This deliberately loose plausibility bound quarantines one derivation: 0fb23678202b67656fe914465cc7e545f36ff88b32bf829632c815e7abbdd37f (AI, 2024-04-30, reported as +4,786.08%). Its inputs were $244.304m and $5m; the older primitive is inconsistent with the company's surrounding RPO scale and is treated as a semantic extraction failure. The immutable source record is not rewritten.

The frozen structural expectation is therefore 22 eligible events. Any different count stops the run before outcomes.

Walk-forward candidate eligibility

For each query event, a candidate must:

1. be another eligible RPO event with a strictly earlier decision date;
2. have its fifth-session calendar resolution strictly before the query observation clock, so its eventual path was knowable when the query was graded;
3. not be the same symbol within 180 calendar days of the query; and
4. retain its exact event-specific RPO derivation rather than a current company value.

No future event, contemporaneous same-date event, or outcome-dependent replacement is allowed. A query is structurally powered only with at least three eligible candidates. The outcome-blind census found 18 powered queries across 17 decision dates.

Frozen rankings

Both policies use the same eligible candidate pool and select exactly K=3 before outcomes:

1. software-rpo-chart-only-pilot/v1: V5 L2 shape distance only.
2. software-rpo-chart-plus-rpo-pilot/v1: 50% query-relative chart percentile and 50% event-specific RPO-growth similarity.

Chart percentile is endpoint-scaled within that query's eligible pool: nearest is 1, farthest is 0, and ties use midrank. RPO similarity is the existing software RPO adapter's signed-log distance score, converted from 0–100 to the common 0–1 component contract. Both components are required with complete coverage. Ties break by raw V5 distance and then stable event key. These weights are engineering priors.

The structural module must not import the transition/evaluation layer. It freezes every anchor, candidate grade hash, policy selection, and a complete manifest hash before the runner may request paths.

Frozen outcome and primary score

Only the complete next-five-session adjusted-close path is evaluated. Each cumulative close return is converted with log1p, divided by 0.10, and clipped to [-3, 3] so one extreme move cannot dominate the pilot.

For the three selected analog paths x_j and query path y, the primary per-query multivariate energy score is:

mean_j ||x_j - y||_2 - 0.5 * mean_jk ||x_j - x_k||_2

Lower is better. The paired delta is:

chart-only energy score - chart+RPO energy score

Positive favors adding RPO. If the query or any selected candidate path is unresolved, that query-policy is unusable; the evaluator may not scan farther down either ranking.

Pilot gates

Uncertainty uses 5,000 bootstrap resamples of whole query-date blocks, seed 20260819. Same-date queries remain together. chart+RPO earns PILOT_SIGNAL only if all hold:

1. at least 15 paired usable queries;
2. at least 12 distinct paired query dates;
3. paired usable coverage is at least 80% of structurally powered queries;
4. mean relative energy-score improvement is at least 5%;
5. the 95% date-block bootstrap lower bound for mean paired delta is above zero; and
6. chart+RPO loses no more than one usable query versus chart-only.

Otherwise the verdict is NO_PILOT_SIGNAL when powered or UNDERPOWERED when gates 1–3 fail. Underpowered is not null.

What the verdict licenses

• PILOT_SIGNAL licenses only expanding the immutable RPO history and registering an independent confirmation with at least 50 powered queries.
• NO_PILOT_SIGNAL leaves RPO descriptive and out of ranking.
• UNDERPOWERED licenses no outcome claim and prioritizes more source history.

No verdict licenses API exposure, scheduling, notifications, paper/live trading, directional claims, or a production similarity weight. Individual symbols, returns, and attractive examples cannot change the disposition.

Fixed execution

The committed runner has only structural-only and output-path controls:

python -m scripts.research.eval_software_rpo_chart_pilot \
  --out /tmp/software_rpo_chart_pilot_v1.json

Execution order is population load → complete structural freeze → structural power gate → exact transition resolution → fixed evaluation. The runner and tests must be committed and deployed before the first non-structural production execution.