mirror of
https://github.com/affaan-m/ECC.git
synced 2026-10-16 21:25:42 +02:00
montjeffrey
acf4980442
The hidden-intent report printed bare counts. run.js reported Wilson intervals, a bootstrap paired difference and an exact McNemar test, and that machinery was lost in the new runner, so four trials rendered as a table that reads like a result. On this run it is not one: gate 1/2 against off 0/2 gives intervals of 9%-91% and 0%-66%, a paired difference of -50% with a 95% interval spanning -100% to 0%, and McNemar p = 1.000. Reuse lib.js's pairedComparison and ai-eval-lib's wilson rather than adding a second statistics path, and state the shortfall in words whenever a comparison has fewer than twenty pairs: about 20 trials per arm are needed for 80% power on a 40-point pass-rate difference at a two-sided 5% level, about 36 for 30 points, and about 160 for the 15-point threshold the pilot pre-registered. Mean turns and cost sit in the same table, because the only efficiency figure this harness has produced points the other way: the gate roughly doubles turns and cost per trial.