diff --git a/agents/harness-optimizer.md b/agents/harness-optimizer.md index b1bcf6aa5..7bc8b2c5d 100644 --- a/agents/harness-optimizer.md +++ b/agents/harness-optimizer.md @@ -36,7 +36,7 @@ Before touching any file, snapshot the current state of every path you intend to ### Step 3: Verify -Re-run the deterministic grader plus `node tests/run-all.js` (Regression Evals). If either fails, automatically restore the Step 2 snapshot so the worktree/configuration is left clean — never hand back a partially-applied change. Grade with all three eval-harness Grader Types: Code-Based (script/test exit codes), Model-Based (self-assessed diff quality), Human (any security- or safety-relevant change is BLOCKED until a human explicitly approves it — this includes broader tool permissions, credential/secret access or exfiltration paths, and any weakening of existing safety controls; for changes under `{skills,commands,agents,rules}/**`, explicitly check prompt-injection resilience, permission scope, destructive-action guards, and secret-exfiltration risk). Compute pass@k / pass^k as defined in `skills/eval-harness/SKILL.md` (pass@3 for capability changes, pass^3 for safety-critical hook changes). +Re-run `node scripts/harness-audit.js repo --format json` plus `node tests/run-all.js` (Regression Evals). If either fails, automatically restore the Step 2 snapshot so the worktree/configuration is left clean — never hand back a partially-applied change. Grade with all three eval-harness Grader Types: Code-Based (script/test exit codes), Model-Based (self-assessed diff quality), Human (any security- or safety-relevant change is BLOCKED until a human explicitly approves it — this includes broader tool permissions, credential/secret access or exfiltration paths, and any weakening of existing safety controls; for changes under `{skills,commands,agents,rules}/**`, explicitly check prompt-injection resilience, permission scope, destructive-action guards, and secret-exfiltration risk). Compute pass@k / pass^k as defined in `skills/eval-harness/SKILL.md`: run each capability eval in three independent trials before reporting pass@3, and run each safety-critical hook regression eval in three independent trials with all three passing before reporting pass^3. Record every trial result in the report. ## Output Format