mirror of
https://github.com/affaan-m/ECC.git
synced 2026-10-14 12:15:29 +02:00
montjeffrey
b7c1fe3eeb
Three of six scenarios sat at floor in the 48-trial run, and the run data says
the cause was the scenarios, not the agents.
csv-export-delimiter was asked about in 5 of 8 trials yet disclosedDecisive
never fired: the simulated user disclosed delimiter five times and lineending
twice, and quoting never. disclosedDecisive requires every decisive fact, and a
user who answers only what is asked and never volunteers cannot deliver three of
them to one question. username-collation failed the same way, with only case
ever disclosed.
soft-limit-overage failed differently and worse: the facts reached the agent in
4 of 8 trials and every trial still scored 0, because the grader demanded an
{ allowed, overage } shape that nothing disclosed. It was scoring whether the
agent guessed field names, not whether it learned the policy.
So each scenario now carries exactly one decisive fact, which is one hidden
decision a single well-aimed question can obtain, and anything a grader checks
about shape is stated in the prompt. The three working scenarios are changed the
same way, so disclosedDecisive means the same thing across the corpus: their
second and third decisive facts move into the prompt or become non-decisive
colour.