mirror of
https://github.com/affaan-m/ECC.git
synced 2026-10-18 05:57:40 +02:00
test(skills): assert COMMAND-REGISTRY.json is fresh in plan-orchestrate fixtures
- Problem: the registry fixtures validated SKILL.md against the committed docs/COMMAND-REGISTRY.json but nothing in the file asserted the registry itself was fresh, so a standalone run passed against a stale registry (npm test does run command-registry:check first, but this file is also run on its own). - Fix: rebuild the registry in memory via the exported generateRegistry() and assert deep equality with the committed JSON so drift fails as a regenerate hint with a field diff instead of passing silently; add one sentence to SKILL.md stating that a command retired from commands/ fails closed with the BLOCKED marker like a no-argument command. - Verification: node tests/skills/plan-orchestrate-registry.test.js passes with the new check 8 (8/8); sabotage with a mutated registry description fails check 8 with the regenerate hint and a field diff.
This commit is contained in:
@@ -52,7 +52,7 @@ BLOCKED — /<command> cannot carry plan scope; run manually if wanted: <exact d
|
||||
|
||||
Blocked steps appear in the overview and per-step output and are excluded from the Batch execution block. This is deliberate: a bare `/update-docs` pasted inside a five-step batch has no machine-readable link to the step it was planned for. The user can still run the manual command themselves; the router just refuses to present an unscoped command as if the step were orchestrated.
|
||||
|
||||
This table is a validated snapshot of the command files, not an independent source of truth. `tests/skills/plan-orchestrate-registry.test.js` validates every command named here against `commands/<name>.md` and `docs/COMMAND-REGISTRY.json` and fails when they disagree. If a command file and this skill ever disagree at runtime, the command file wins.
|
||||
This table is a validated snapshot of the command files, not an independent source of truth. `tests/skills/plan-orchestrate-registry.test.js` validates every command named here against `commands/<name>.md` and `docs/COMMAND-REGISTRY.json` and fails when they disagree. If a command file and this skill ever disagree at runtime, the command file wins. A command that has since been retired from `commands/` fails closed like a no-argument command — the step is reported `BLOCKED` and the router never emits a name the command surface no longer has.
|
||||
|
||||
## Task description (scope-carrying commands only)
|
||||
|
||||
|
||||
@@ -17,7 +17,11 @@
|
||||
* argument (re-classify the skill if a command grows one);
|
||||
* 5. the fail-closed rule and the BLOCKED marker are actually stated;
|
||||
* 6. the agent-chain machinery stays gone — commands own their execution;
|
||||
* 7. emitted examples carry the [Plan: ...#step-N] scope marker.
|
||||
* 7. emitted examples carry the [Plan: ...#step-N] scope marker;
|
||||
* 8. the committed docs/COMMAND-REGISTRY.json is fresh — it equals a
|
||||
* regeneration from commands/, so a contributor who adds a command and
|
||||
* forgets to regenerate fails here with a regenerate hint instead of the
|
||||
* router silently validating against a stale surface.
|
||||
*/
|
||||
|
||||
'use strict';
|
||||
@@ -219,6 +223,22 @@ function run() {
|
||||
assert.ok(checked >= 3, `expected at least 3 command examples, checked ${checked}`);
|
||||
console.log(` \u2713 emitted examples carry the scope marker (${checked} checked)`);
|
||||
|
||||
// --- 8. The committed registry itself is fresh -------------------------
|
||||
|
||||
// The checks above validate the skill against the committed JSON, which
|
||||
// makes a stale registry silently pass. Rebuild it in memory from the
|
||||
// command files and require equality, so drift fails as "regenerate".
|
||||
// npm test also runs `command-registry:check` before the suites; this keeps
|
||||
// the file honest when it is run standalone.
|
||||
const { generateRegistry } = require(path.join(REPO_ROOT, 'scripts', 'ci', 'generate-command-registry.js'));
|
||||
const committedRegistry = JSON.parse(fs.readFileSync(REGISTRY_PATH, 'utf8'));
|
||||
assert.deepStrictEqual(
|
||||
committedRegistry,
|
||||
generateRegistry(),
|
||||
'docs/COMMAND-REGISTRY.json is stale — regenerate it with `npm run command-registry:write`'
|
||||
);
|
||||
console.log(' \u2713 docs/COMMAND-REGISTRY.json matches a fresh generation from commands/');
|
||||
|
||||
console.log(' plan-orchestrate registry fixtures passed');
|
||||
}
|
||||
|
||||
|
||||
Reference in New Issue
Block a user