96 trials, none invalid, analysed exactly as pre-registered. passed: gate 25/48
against off 18/48, 48 pairs, discordant 15 to 8, exact McNemar p = 0.210. At the
registered alpha of 0.05 the hypothesis is not confirmed, and the bootstrap
interval agrees this time at -15% with a range of -33% to +4%, so there is no
disagreement between the tests to adjudicate. Secondary measures point the same
way and reach nothing: asked 29/48 against 22/48 at p = 0.230, deciding fact
27/48 against 21/48 at p = 0.327.
The effect halved on replication, from +29 points at p = 0.065 to +15 at
p = 0.210, which is the ordinary fate of a near-threshold finding. The runs are
not pooled: the quota leak was closed between them, and combining them after
seeing both would be the post-hoc choice the pre-registration exists to stop.
Two things survive every run. Obtaining the deciding fact is necessary and
nearly sufficient, with no trial in either arm passing without it across 96
trials. And the gate costs 37 to 45% more turns and spend, which is the most
stable number the evaluation has produced.
montjeffrey
d545071699