Files
ECC/examples/eval-harness
f8640355e4 Consolidate recovered eval framework and operator workflows (#3040)
* feat: consolidate offline eval and operator workflows

Compose the retained framework, operator skill, roadmap and cleanup ranges on current main. Preserve current release dependencies and keep candidate execution disabled pending OS containment. Repair draft/DOCX behavior, obligation uniqueness, trusted send and audience guidance, runner provenance and eval diagnostics.

Source-PR: 2930 0abe3727d2b500c6e4830bdeb47ed67cae3f4785
Source-PR: 2931 992b49c44ed872def49675b791168b8fcd091df6
Source-PR: 2932 4a193dd13041cb7a6bebf4d2e910a0cd32bcc797
Source-PR: 2933 59cdfe500a91949ba1415f1edd7279620f21e804
Source-Base: ca185ef5f7

* fix: repair foundation CI and update js-yaml

* fix: reconcile pending-delete capsule locks after close

---------

Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-10 13:20:52 +01:00
..

Eval Harness Example

node scripts/eval-harness.js example
# Keep the temporary artifacts for inspection:
node examples/eval-harness/run-example.js --keep

The example verifies that candidate execution is unavailable, inspects source without loading it, records and replays a locally declared fixture function, and builds an offline capsule receipt. It changes a journal value in a copy and checks that verification detects the changed entry. All five capsule lineages describe these observations; none represent a scored candidate run.

Supported candidate execution backends: none. gate run, runGate, runVariant, gate-child.js, and the retired effect-fence.js preload refuse with gate.isolation_required. No trusted_local, --trusted-local, or caller-supplied isolation claim enables execution. The example emits no gate receipt, score, or promotion verdict.

With --keep, inspect capsule/journal.ndjson, capsule/projection.json, fixtures/, and bundle/receipt.json in the printed work directory.

Path Purpose
taskset.json Twelve slugify tasks for static inspection, three marked held out
gate.config.json Preserved gate input example; gate run currently refuses it
variants/baseline Known-weak source fixture; never executed by this example
variants/candidate Honest source fixture; never executed by this example
variants/reward-hack Source fixture with visible syntactic warnings

See docs/architecture/eval-harness-frameworks.md for the OS containment requirements and the limits of static inspection and receipt verification.