The previous fix stopped a throwing hook from aborting the cold replay by
recording it as an allow, matching the warm path. But coldSummary only counted
disagreements, so a failure on a step the worker also allowed was no
disagreement and left no trace: the report showed no failure, a run with no
measured step reported a cold latency of 0 ms because percentile of an empty
list is 0, and the exit status passed. Trading a crash for a clean-looking
success hid exactly the failure the crash had exposed.
coldSummary now counts errors and samples, and reports no latency when nothing
was measured rather than a 0 that reads as a fast hook. The report prints "no
valid sample" in place of a latency and adds a line for steps that failed in a
fresh process, beside the existing disagreement line. The exit check moves into
gateFails, which fails on cold errors as it already did on warm errors and cold
disagreements, and is exported with coldSummary so both can be tested.
The render test builds its report from summarize([]), the real summary shape,
so it keeps working as metrics are added.
measureColdLatency ran each step in a fresh process through execFileSync,
inside try/finally with no catch. A hook that threw made the child exit
non-zero, execFileSync threw, and the error left measureColdLatency and then
evaluate, so neither the report nor the requested SARIF was written for a run
that had hit a single bad step.
The probe now catches a throwing require or run itself and records it the way
workerMain does: the raw payload stands in for the result, so the step reads as
an allow, and the error is kept. Matching the warm path matters because
coldSummary counts every cold decision that differs from the warm one as a
disagreement; any other choice would have reported disagreements that do not
exist. A child that dies without writing parseable output is caught outside the
probe and recorded the same way with no latency, and coldSummary leaves
unmeasured steps out of the percentiles while still counting them.
The new tests run a hook that always throws through both passes. They fail
against the previous code, 2 of 17, and pass against this one.
Across 240 evaluation trials every question the gate raised was about code
structure - importers, callers, public API, data schemas - while every deciding
fact in the hidden-intent corpus was an external contract: a consumer's CSV
dialect, a reporting timezone, retry safety against a non-idempotent endpoint,
collation rules, a quota policy. Obtaining the deciding fact predicted the
outcome almost perfectly in both arms, and the gate did not reliably cause it.
Add an external-contract question for code targets whose change profile touches
data, beside data-schema rather than replacing it: data-schema asks what the
change touches, external-contract asks who decides what it has to be. It reuses
the existing touchesData signal, so it stays as selective as the question it
sits with and adds no new detection, and it therefore does not reach a change
with no data involvement - pure retry semantics, for instance - which is a known
limit to be measured rather than guessed at.
Code edits that touch data now ask four questions instead of three, so the
condensed hint for an unknown profile changes too. That string was pinned by a
test as legacy-stable; the pin is updated deliberately, because a hint that
names fewer questions than the gate asks is worse than a hint that changed.
Suites: target-class 31, fact-force 1019, scenario-gate 25, eval 15, and the two
effectiveness suites 29 and 26, all passing. The 23 relevantQuestions
annotations that listed data-schema now list external-contract as well.
haelyra