fix(continuous-learning): /evolve never produces skill or agent candidates (#2664)

* fix(continuous-learning): cluster instincts by keyword overlap in /evolve

`cmd_evolve` grouped instincts by exact string equality of the whole
normalized trigger sentence. Triggers are free-form sentences, so every
instinct landed in its own bucket and `skill_candidates` was always empty.
`agent_candidates` is derived from `skill_candidates`, so agents never
generated either — `/evolve --generate` could only ever emit commands.

Measured on a 42-instinct project: 42 instincts produced 42 unique cluster
keys, largest cluster size 1.

Group on keyword overlap instead. Jaccard is the wrong metric here — trigger
keyword sets average ~7 words, so even clearly related pairs top out around
0.33 — so this uses the overlap coefficient (shared / smaller set) at 0.5,
plus a floor of 2 shared keywords so one incidental word cannot pull
unrelated instincts together. The same 42 instincts now yield 4 clusters.

Also unify the command/agent slug used by the preview and the writer. The
preview called `.replace('a ', '')`, which strips "a " anywhere in the
string, mangling "extracting data from Reddit" into
`/extracting-datfrom-R` while `--generate` wrote `extracting-data-from.md`.
Both paths now share `_evolved_command_name()` / `_evolved_agent_name()`.

Adds tests/scripts/instinct-cli-evolve.test.js, which fails on the previous
implementation (0 clusters instead of 1; preview name `extracting-datfrom-R`)
and covers the negative cases so unrelated triggers still stay apart.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* docs(continuous-learning): correct clustering metric name in docstring

The docstring said "Jaccard" while the implementation uses the overlap
coefficient, which is the point of the change.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(continuous-learning): generate every evolve candidate and cut slugs on word boundaries

_generate_evolved() wrote only skill_candidates[:5], workflow_instincts[:5]
and agent_candidates[:3]. On a project with 36 command candidates that meant
5 files and no warning, so the output read as complete while 86% of the
candidates were dropped.

Generation is now unbounded by default and takes a --limit N flag for callers
that want a cap. A cap that truncates says so:

  Note: writing 3 of 36 command candidates (--limit 3); 33 skipped.

The analysis preview keeps showing five per kind but now names the remainder
("... and 31 more command candidates not shown") instead of presenting a
sample as the whole set.

Slugs were also cut with a hard slice, which split words mid-token and
produced /investigating-comple, /learning-about-compl and
/researching-mechanis. _truncate_slug() retreats to the last separator that
fits, and keeps the full head when the cut already lands on one, so
"analyzing large text files" stays /analyzing-large-text rather than losing
a word. A first word longer than the limit still falls back to a hard cut
because no boundary is available.

Shorter slugs collide more easily, and a collision used to mean one file
silently overwriting another. _assign_unique_slugs() suffixes duplicates
(-2, -3) and is called by both the preview and the writer over the same
ordered list, so advertised names and written names cannot drift apart.

Skill directory naming moved to _evolved_skill_name(); it previously used its
own inline slug expression, so it was the one truncation the shared helper
did not cover.

Adds tests/scripts/instinct-cli-evolve-generate.test.js: 7 cases covering
word-boundary cuts, the separator-aligned cut, unbounded generation, --limit
reporting, collision dedup, preview remainder and preview/writer agreement.
Six of the seven fail against the previous implementation.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: haelyra <49814733+haelyra@users.noreply.github.com>
This commit is contained in:
romanclaudersoai1
2026-08-04 16:57:24 -04:00
committed by GitHub
co-authored by Claude Opus 5 haelyra
parent b2bc8dcd14
commit 8a97868b5b
3 changed files with 658 additions and 32 deletions
@@ -1145,6 +1145,163 @@ def cmd_export(args) -> int:
# Evolve Command
# ─────────────────────────────────────────────
# Words carrying no topical signal in a trigger sentence.
TRIGGER_STOP_WORDS = {
'when', 'while', 'the', 'and', 'or', 'to', 'of', 'in', 'on', 'for', 'with',
'that', 'this', 'from', 'into', 'at', 'by', 'as', 'is', 'are', 'be', 'it',
'its', 'they', 'them', 'their', 'you', 'your', 'new', 'any', 'all', 'about',
'after', 'before', 'over', 'via', 'use', 'using', 'need', 'needs', 'not',
}
# Overlap coefficient (shared / smaller set) two triggers need to cluster.
# Jaccard is the wrong metric here: trigger keyword sets average ~7 words, so
# even clearly-related pairs top out near 0.33 and nothing ever groups.
TRIGGER_SIMILARITY_THRESHOLD = 0.5
# Guard against one incidental shared word pulling unrelated instincts together.
TRIGGER_MIN_SHARED_KEYWORDS = 2
# Evolved artefact slugs are trimmed to keep file names short. The cut has to
# land on a word boundary: a hard slice produced names like
# "investigating-comple" and "learning-about-compl", which read as typos.
EVOLVED_SKILL_SLUG_LENGTH = 30
EVOLVED_COMMAND_SLUG_LENGTH = 20
EVOLVED_AGENT_SLUG_LENGTH = 20
def _truncate_slug(slug: str, max_length: int) -> str:
"""Trim a slug to max_length without splitting a word.
Falls back to a hard cut only when the first word is already longer than
the limit, because then there is no boundary left to retreat to.
"""
if len(slug) <= max_length:
return slug
head = slug[:max_length]
# The cut can already land on a separator, in which case head is a whole
# sequence of words and dropping one more would lose a word for nothing.
if slug[max_length] == '-':
return head.rstrip('-')
boundary = head.rfind('-')
if boundary > 0:
return head[:boundary]
return head.strip('-')
def _evolved_skill_name(trigger: str) -> str:
"""Slug used for a generated skill directory. Shared by preview and writer."""
return _truncate_slug(
re.sub(r'[^a-z0-9]+', '-', str(trigger or '').lower()).strip('-'),
EVOLVED_SKILL_SLUG_LENGTH,
)
def _evolved_command_name(trigger: str) -> str:
"""Slug used for a generated command file. Shared by preview and writer."""
stripped = str(trigger or 'unknown').lower().replace('when ', '').replace('implementing ', '')
return _truncate_slug(
re.sub(r'[^a-z0-9]+', '-', stripped).strip('-'),
EVOLVED_COMMAND_SLUG_LENGTH,
)
def _evolved_agent_name(trigger: str) -> str:
"""Slug used for a generated agent file. Shared by preview and writer."""
return _truncate_slug(
re.sub(r'[^a-z0-9]+', '-', str(trigger or '').lower()).strip('-'),
EVOLVED_AGENT_SLUG_LENGTH,
)
# How many candidates of each kind the analysis prints before summarising the
# rest. The preview is a sample, never the whole set, so it always says so.
PREVIEW_LIMIT = 5
def _print_preview_remainder(total: int, shown: int, noun: str) -> None:
"""State how many candidates the preview left out.
Without this the truncated list reads as the complete set.
"""
if total > shown:
print(f" ... and {total - shown} more {noun} not shown\n")
def _assign_unique_slugs(items: list, slug_fn) -> list:
"""Pair every item with a collision-free slug, preserving input order.
Word-boundary trimming makes collisions more likely because two triggers
can now share a whole prefix, and a collision previously meant one
generated file silently overwriting another. Preview and writer both call
this over the same ordered list, so the names shown and the names written
stay identical.
"""
used = set()
assigned = []
for item in items:
base = slug_fn(item)
if not base:
assigned.append((item, ''))
continue
name = base
suffix = 2
while name in used:
name = f"{base}-{suffix}"
suffix += 1
used.add(name)
assigned.append((item, name))
return assigned
def _trigger_keywords(trigger: str) -> set:
"""Reduce a trigger sentence to the words that carry its topic."""
words = re.findall(r'[a-z0-9]+', str(trigger or '').lower())
return {w for w in words if len(w) > 2 and w not in TRIGGER_STOP_WORDS}
def _cluster_by_keyword_overlap(instincts: list) -> dict:
"""Group instincts whose triggers share enough keywords.
Triggers are free-form sentences, so grouping on the whole normalized
string puts every instinct in its own bucket and no skill or agent
candidate is ever produced. Greedy clustering on keyword overlap groups
the near-duplicate instincts that accumulate in a project.
"""
clusters = [] # [(shared_keywords, [instincts])]
for inst in instincts:
keywords = _trigger_keywords(inst.get('trigger', ''))
if not keywords:
continue
best_index, best_score, best_shared = -1, 0.0, 0
for index, (cluster_keywords, _members) in enumerate(clusters):
shared = len(keywords & cluster_keywords)
smaller = min(len(keywords), len(cluster_keywords))
score = shared / smaller if smaller else 0.0
if score > best_score:
best_index, best_score, best_shared = index, score, shared
if (best_index >= 0
and best_score >= TRIGGER_SIMILARITY_THRESHOLD
and best_shared >= TRIGGER_MIN_SHARED_KEYWORDS):
cluster_keywords, members = clusters[best_index]
members.append(inst)
# Keep the shared core so a cluster stays on one topic.
clusters[best_index] = (cluster_keywords & keywords, members)
else:
clusters.append((keywords, [inst]))
grouped = {}
for cluster_keywords, members in clusters:
label = ' '.join(sorted(cluster_keywords)[:4]) or 'general'
while label in grouped:
label += ' +'
grouped[label] = members
return grouped
def cmd_evolve(args) -> int:
"""Analyze instincts and suggest evolutions to skills/commands/agents."""
project = detect_project()
@@ -1175,14 +1332,7 @@ def cmd_evolve(args) -> int:
print(f"High confidence instincts (>=80%): {len(high_conf)}")
# Find clusters (instincts with similar triggers)
trigger_clusters = defaultdict(list)
for inst in instincts:
trigger = inst.get('trigger', '')
# Normalize trigger
trigger_key = trigger.lower()
for keyword in ['when', 'creating', 'writing', 'adding', 'implementing', 'testing']:
trigger_key = trigger_key.replace(keyword, '').strip()
trigger_clusters[trigger_key].append(inst)
trigger_clusters = _cluster_by_keyword_overlap(instincts)
# Find clusters with 2+ instincts (good skill candidates)
skill_candidates = []
@@ -1203,8 +1353,8 @@ def cmd_evolve(args) -> int:
print(f"\nPotential skill clusters found: {len(skill_candidates)}")
if skill_candidates:
print(f"\n## SKILL CANDIDATES\n")
for i, cand in enumerate(skill_candidates[:5], 1):
print(f"\n## SKILL CANDIDATES ({len(skill_candidates)})\n")
for i, cand in enumerate(skill_candidates[:PREVIEW_LIMIT], 1):
scope_info = ', '.join(cand['scopes'])
print(f"{i}. Cluster: \"{cand['trigger']}\"")
print(f" Instincts: {len(cand['instincts'])}")
@@ -1215,37 +1365,51 @@ def cmd_evolve(args) -> int:
for inst in cand['instincts'][:3]:
print(f" - {inst.get('id')} [{inst.get('scope', '?')}]")
print()
_print_preview_remainder(len(skill_candidates), PREVIEW_LIMIT, 'skill clusters')
# Command candidates (workflow instincts with high confidence)
workflow_instincts = [i for i in instincts if i.get('domain') == 'workflow' and i.get('confidence', 0) >= 0.7]
if workflow_instincts:
print(f"\n## COMMAND CANDIDATES ({len(workflow_instincts)})\n")
for inst in workflow_instincts[:5]:
trigger = inst.get('trigger', 'unknown')
cmd_name = trigger.replace('when ', '').replace('implementing ', '').replace('a ', '')
cmd_name = cmd_name.replace(' ', '-')[:20]
# Slugs come from the same helper the writer uses, over the same ordered
# list, or the preview advertises names that differ from the files
# --generate actually writes.
for inst, cmd_name in _assign_unique_slugs(
workflow_instincts,
lambda i: _evolved_command_name(i.get('trigger', 'unknown')),
)[:PREVIEW_LIMIT]:
print(f" /{cmd_name}")
print(f" From: {inst.get('id')} [{inst.get('scope', '?')}]")
print(f" Confidence: {inst.get('confidence', 0.5):.0%}")
print()
_print_preview_remainder(len(workflow_instincts), PREVIEW_LIMIT, 'command candidates')
# Agent candidates (complex multi-step patterns)
agent_candidates = [c for c in skill_candidates if len(c['instincts']) >= 3 and c['avg_confidence'] >= 0.75]
if agent_candidates:
print(f"\n## AGENT CANDIDATES ({len(agent_candidates)})\n")
for cand in agent_candidates[:3]:
agent_name = cand['trigger'].replace(' ', '-')[:20] + '-agent'
for cand, agent_name in _assign_unique_slugs(
agent_candidates,
lambda c: _evolved_agent_name(str(c.get('trigger', '')).strip()),
)[:PREVIEW_LIMIT]:
print(f" {agent_name}")
print(f" Covers {len(cand['instincts'])} instincts")
print(f" Avg confidence: {cand['avg_confidence']:.0%}")
print()
_print_preview_remainder(len(agent_candidates), PREVIEW_LIMIT, 'agent candidates')
# Promotion candidates (project instincts that could be global)
_show_promotion_candidates(project)
if args.generate:
evolved_dir = project["evolved_dir"] if project["id"] != "global" else GLOBAL_EVOLVED_DIR
generated = _generate_evolved(skill_candidates, workflow_instincts, agent_candidates, evolved_dir)
generated = _generate_evolved(
skill_candidates,
workflow_instincts,
agent_candidates,
evolved_dir,
limit=max(0, getattr(args, 'limit', 0) or 0),
)
if generated:
print(f"\nGenerated {len(generated)} evolved structures:")
for path in generated:
@@ -1770,17 +1934,33 @@ def _cmd_projects_merge(args) -> int:
# Generate Evolved Structures
# ─────────────────────────────────────────────
def _generate_evolved(skill_candidates: list, workflow_instincts: list, agent_candidates: list, evolved_dir: Path) -> list[str]:
"""Generate skill/command/agent files from analyzed instinct clusters."""
def _generate_evolved(skill_candidates: list, workflow_instincts: list, agent_candidates: list, evolved_dir: Path, limit: int = 0) -> list[str]:
"""Generate skill/command/agent files from analyzed instinct clusters.
``limit`` caps how many candidates of each kind are written; 0 writes them
all. Anything a cap leaves out is reported, because the previous fixed
caps (5 skills, 5 commands, 3 agents) discarded most candidates without
saying a word — 35 command candidates produced 5 files and no warning.
"""
generated = []
# Generate skills from top candidates
for cand in skill_candidates[:5]:
def bounded(assigned: list, kind: str) -> list:
if limit and len(assigned) > limit:
print(f"\nNote: writing {limit} of {len(assigned)} {kind} candidates "
f"(--limit {limit}); {len(assigned) - limit} skipped.")
return assigned[:limit]
return assigned
# Generate skills from candidate clusters
for cand, name in bounded(
_assign_unique_slugs(
skill_candidates,
lambda c: _evolved_skill_name(str(c.get('trigger', '')).strip()),
),
'skill',
):
trigger = cand['trigger'].strip()
if not trigger:
continue
name = re.sub(r'[^a-z0-9]+', '-', trigger.lower()).strip('-')[:30]
if not name:
if not trigger or not name:
continue
skill_dir = evolved_dir / "skills" / name
@@ -1802,10 +1982,13 @@ def _generate_evolved(skill_candidates: list, workflow_instincts: list, agent_ca
generated.append(str(skill_dir / "SKILL.md"))
# Generate commands from workflow instincts
for inst in workflow_instincts[:5]:
trigger = inst.get('trigger', 'unknown')
cmd_name = re.sub(r'[^a-z0-9]+', '-', trigger.lower().replace('when ', '').replace('implementing ', ''))
cmd_name = cmd_name.strip('-')[:20]
for inst, cmd_name in bounded(
_assign_unique_slugs(
workflow_instincts,
lambda i: _evolved_command_name(i.get('trigger', 'unknown')),
),
'command',
):
if not cmd_name:
continue
@@ -1819,9 +2002,13 @@ def _generate_evolved(skill_candidates: list, workflow_instincts: list, agent_ca
generated.append(str(cmd_file))
# Generate agents from complex clusters
for cand in agent_candidates[:3]:
trigger = cand['trigger'].strip()
agent_name = re.sub(r'[^a-z0-9]+', '-', trigger.lower()).strip('-')[:20]
for cand, agent_name in bounded(
_assign_unique_slugs(
agent_candidates,
lambda c: _evolved_agent_name(str(c.get('trigger', '')).strip()),
),
'agent',
):
if not agent_name:
continue
@@ -2018,6 +2205,8 @@ def main() -> int:
# Evolve
evolve_parser = subparsers.add_parser('evolve', help='Analyze and evolve instincts')
evolve_parser.add_argument('--generate', action='store_true', help='Generate evolved structures')
evolve_parser.add_argument('--limit', type=int, default=0, metavar='N',
help='Max candidates of each kind to generate (default: 0 = all)')
# Promote (new in v2.1)
promote_parser = subparsers.add_parser('promote', help='Promote project instincts to global scope')
@@ -0,0 +1,249 @@
const assert = require('assert');
const fs = require('fs');
const os = require('os');
const path = require('path');
const { spawnSync } = require('child_process');
let passed = 0;
let failed = 0;
const repoRoot = path.resolve(__dirname, '..', '..');
const cliPath = path.join(
repoRoot,
'skills',
'continuous-learning-v2',
'scripts',
'instinct-cli.py'
);
function detectPython3() {
for (const bin of ['python3', 'python']) {
const r = spawnSync(bin, ['--version'], { encoding: 'utf8' });
if (r.status === 0 && /Python 3/.test(r.stdout + r.stderr)) return bin;
}
return null;
}
const PYTHON3 = detectPython3();
if (!PYTHON3) {
console.log('\n=== Testing instinct-cli.py evolve generation ===\n');
console.log(' - skipped: Python 3 not found in PATH');
console.log('\nPassed: 0');
console.log('Failed: 0');
process.exit(0);
}
function test(name, fn) {
try {
fn();
console.log(`${name}`);
passed += 1;
} catch (error) {
console.log(`${name}`);
console.log(` Error: ${error.message}`);
failed += 1;
}
}
function createTempDir() {
return fs.mkdtempSync(path.join(os.tmpdir(), 'ecc-instinct-cli-evolve-'));
}
function cleanupDir(dir) {
fs.rmSync(dir, { recursive: true, force: true });
}
function writeInstinct(root, id, trigger, confidence = 0.8, domain = 'workflow') {
const dir = path.join(root, 'instincts', 'personal');
fs.mkdirSync(dir, { recursive: true });
fs.writeFileSync(
path.join(dir, `${id}.yaml`),
[
'---',
`id: ${id}`,
`trigger: "${trigger}"`,
`confidence: ${confidence}`,
`domain: ${domain}`,
'---',
'',
`## Action`,
'',
`Action for ${id}.`,
'',
].join('\n')
);
}
// CLV2_NO_PROJECT pins the run to global scope, so seeded instincts live in
// <root>/instincts/personal and generated files land in <root>/evolved.
function runCli(root, args) {
return spawnSync(PYTHON3, [cliPath, ...args], {
cwd: repoRoot,
encoding: 'utf8',
env: {
...process.env,
CLV2_HOMUNCULUS_DIR: root,
CLV2_NO_PROJECT: '1',
HOME: path.join(root, 'home'),
USERPROFILE: path.join(root, 'home'),
CLAUDE_PROJECT_DIR: '',
},
});
}
function generatedCommands(root) {
const dir = path.join(root, 'evolved', 'commands');
if (!fs.existsSync(dir)) return [];
return fs.readdirSync(dir).sort();
}
// Eight unrelated workflow triggers: no two share enough keywords to cluster,
// so each one is its own command candidate.
const EIGHT_TRIGGERS = [
['run-tests', 'when running tests'],
['build-images', 'when building images'],
['deploy-services', 'when deploying services'],
['profile-memory', 'when profiling memory'],
['rotate-secrets', 'when rotating secrets'],
['tag-releases', 'when tagging releases'],
['prune-caches', 'when pruning caches'],
['review-requests', 'when reviewing pull requests'],
];
function seedEight(root) {
for (const [id, trigger] of EIGHT_TRIGGERS) {
writeInstinct(root, id, trigger);
}
}
console.log('\n=== Testing instinct-cli.py evolve generation ===\n');
test('generated command names are cut on a word boundary', () => {
const root = createTempDir();
try {
writeInstinct(root, 'archaeology', 'when investigating complex systems');
writeInstinct(root, 'codebases', 'when learning about complex codebases');
writeInstinct(root, 'large-text', 'when analyzing large text files');
const result = runCli(root, ['evolve', '--generate']);
assert.strictEqual(result.status, 0, result.stderr);
const names = generatedCommands(root);
// A hard slice used to yield investigating-comple.md and
// learning-about-compl.md, which read as typos.
assert.deepStrictEqual(names, [
'analyzing-large-text.md',
'investigating.md',
'learning-about.md',
]);
} finally {
cleanupDir(root);
}
});
test('a cut landing on a separator keeps the whole word', () => {
const root = createTempDir();
try {
writeInstinct(root, 'a', 'when analyzing large text files');
writeInstinct(root, 'b', 'when running tests');
writeInstinct(root, 'c', 'when building images');
assert.strictEqual(runCli(root, ['evolve', '--generate']).status, 0);
// "analyzing-large-text" is exactly the slug limit and ends on a word, so
// nothing further may be dropped.
assert.ok(generatedCommands(root).includes('analyzing-large-text.md'));
} finally {
cleanupDir(root);
}
});
test('every command candidate is generated, not just the first five', () => {
const root = createTempDir();
try {
seedEight(root);
const result = runCli(root, ['evolve', '--generate']);
assert.strictEqual(result.status, 0, result.stderr);
assert.strictEqual(
generatedCommands(root).length,
EIGHT_TRIGGERS.length,
'a fixed cap silently dropped candidates'
);
} finally {
cleanupDir(root);
}
});
test('--limit caps generation and reports what it skipped', () => {
const root = createTempDir();
try {
seedEight(root);
const result = runCli(root, ['evolve', '--generate', '--limit', '3']);
assert.strictEqual(result.status, 0, result.stderr);
assert.strictEqual(generatedCommands(root).length, 3);
assert.match(result.stdout, /writing 3 of 8 command candidates/);
assert.match(result.stdout, /5 skipped/);
} finally {
cleanupDir(root);
}
});
test('colliding slugs produce distinct files instead of overwriting', () => {
const root = createTempDir();
try {
// Both triggers trim to "investigating".
writeInstinct(root, 'first', 'when investigating complex systems');
writeInstinct(root, 'second', 'when investigating extraordinarily convoluted pipelines');
writeInstinct(root, 'third', 'when running tests');
assert.strictEqual(runCli(root, ['evolve', '--generate']).status, 0);
const names = generatedCommands(root);
assert.ok(names.includes('investigating.md'), `missing base name in ${names}`);
assert.ok(names.includes('investigating-2.md'), `missing deduped name in ${names}`);
assert.strictEqual(new Set(names).size, names.length);
} finally {
cleanupDir(root);
}
});
test('preview states how many candidates it left out', () => {
const root = createTempDir();
try {
seedEight(root);
const result = runCli(root, ['evolve']);
assert.strictEqual(result.status, 0, result.stderr);
assert.match(result.stdout, /COMMAND CANDIDATES \(8\)/);
assert.match(result.stdout, /and 3 more command candidates not shown/);
} finally {
cleanupDir(root);
}
});
test('preview names match the files --generate writes', () => {
const root = createTempDir();
try {
writeInstinct(root, 'first', 'when investigating complex systems');
writeInstinct(root, 'second', 'when investigating extraordinarily convoluted pipelines');
writeInstinct(root, 'third', 'when running tests');
const preview = runCli(root, ['evolve']);
assert.strictEqual(preview.status, 0, preview.stderr);
assert.match(preview.stdout, /\/investigating\b/);
assert.match(preview.stdout, /\/investigating-2\b/);
assert.strictEqual(runCli(root, ['evolve', '--generate']).status, 0);
const names = generatedCommands(root);
assert.ok(names.includes('investigating.md'));
assert.ok(names.includes('investigating-2.md'));
} finally {
cleanupDir(root);
}
});
console.log(`\nPassed: ${passed}`);
console.log(`Failed: ${failed}`);
process.exit(failed > 0 ? 1 : 0);
+188
View File
@@ -0,0 +1,188 @@
const assert = require('assert');
const fs = require('fs');
const os = require('os');
const path = require('path');
const { spawnSync } = require('child_process');
let passed = 0;
let failed = 0;
const repoRoot = path.resolve(__dirname, '..', '..');
const cliPath = path.join(
repoRoot,
'skills',
'continuous-learning-v2',
'scripts',
'instinct-cli.py'
);
function detectPython3() {
for (const bin of ['python3', 'python']) {
const r = spawnSync(bin, ['--version'], { encoding: 'utf8' });
if (r.status === 0 && /Python 3/.test(r.stdout + r.stderr)) return bin;
}
return null;
}
const PYTHON3 = detectPython3();
if (!PYTHON3) {
console.log('\n=== Testing instinct-cli.py evolve clustering ===\n');
console.log(' - skipped: Python 3 not found in PATH');
console.log('\nPassed: 0');
console.log('Failed: 0');
process.exit(0);
}
function test(name, fn) {
try {
fn();
console.log(`${name}`);
passed += 1;
} catch (error) {
console.log(`${name}`);
console.log(` Error: ${error.message}`);
failed += 1;
}
}
function createTempDir() {
return fs.mkdtempSync(path.join(os.tmpdir(), 'ecc-instinct-cli-evolve-'));
}
function cleanupDir(dir) {
fs.rmSync(dir, { recursive: true, force: true });
}
function writeInstinct(root, id, trigger, confidence = 0.8, domain = 'workflow') {
const filePath = path.join(root, 'instincts', 'personal', `${id}.yaml`);
fs.mkdirSync(path.dirname(filePath), { recursive: true });
fs.writeFileSync(
filePath,
[
'---',
`id: ${id}`,
`trigger: ${trigger}`,
`confidence: ${confidence}`,
`domain: ${domain}`,
'scope: global',
'---',
'',
'## Action',
'',
`Action for ${id}.`,
'',
].join('\n')
);
}
// cwd is the temp dir (not a git repo) so project detection falls back to
// global scope and the fixture instincts are the only ones loaded.
function runEvolve(root, args = []) {
return spawnSync(PYTHON3, [cliPath, 'evolve', ...args], {
cwd: root,
encoding: 'utf8',
env: {
...process.env,
CLV2_HOMUNCULUS_DIR: root,
HOME: path.join(root, 'home'),
USERPROFILE: path.join(root, 'home'),
CLAUDE_PROJECT_DIR: '',
},
});
}
function clusterCount(stdout) {
const match = stdout.match(/Potential skill clusters found:\s*(\d+)/);
assert.ok(match, `cluster count missing from output:\n${stdout}`);
return Number(match[1]);
}
console.log('\n=== Testing instinct-cli.py evolve clustering ===\n');
// evolve refuses to analyze fewer than 3 instincts, so each fixture adds a
// filler whose trigger shares no keywords with the pair under test.
const FILLER = ['filler-unrelated', 'when rotating expired signing certificates'];
test('instincts with overlapping trigger keywords form one cluster', () => {
const root = createTempDir();
try {
writeInstinct(root, 'l10n-merge', 'when adding a language to the localization dictionary');
writeInstinct(root, 'l10n-verify', 'when verifying the localization dictionary for a language');
writeInstinct(root, ...FILLER);
const result = runEvolve(root);
assert.strictEqual(result.status, 0, result.stderr);
assert.strictEqual(clusterCount(result.stdout), 1);
} finally {
cleanupDir(root);
}
});
test('unrelated triggers do not cluster together', () => {
const root = createTempDir();
try {
writeInstinct(root, 'docker-build', 'when building container images for deployment');
writeInstinct(root, 'sql-index', 'when tuning slow database queries');
writeInstinct(root, ...FILLER);
const result = runEvolve(root);
assert.strictEqual(result.status, 0, result.stderr);
assert.strictEqual(clusterCount(result.stdout), 0);
} finally {
cleanupDir(root);
}
});
test('a single shared keyword is not enough to cluster', () => {
const root = createTempDir();
try {
writeInstinct(root, 'bash-archives', 'when sampling compressed archives with bash pipelines');
writeInstinct(root, 'bash-signing', 'when inspecting bash exit codes after failures');
writeInstinct(root, ...FILLER);
const result = runEvolve(root);
assert.strictEqual(result.status, 0, result.stderr);
assert.strictEqual(clusterCount(result.stdout), 0);
} finally {
cleanupDir(root);
}
});
test('preview command name matches the file --generate writes', () => {
const root = createTempDir();
try {
writeInstinct(root, 'reddit-scrape', 'when extracting data from Reddit pages');
writeInstinct(root, 'filler-one', 'when rotating expired signing certificates', 0.8, 'testing');
writeInstinct(root, 'filler-two', 'when pruning stale feature branches', 0.8, 'testing');
const preview = runEvolve(root);
assert.strictEqual(preview.status, 0, preview.stderr);
const match = preview.stdout.match(/^\s+\/(\S+)$/m);
assert.ok(match, `no command candidate in output:\n${preview.stdout}`);
const previewName = match[1];
// The old preview stripped every "a " occurrence, mangling "data from"
// into "datfrom" and advertising a name --generate never wrote.
assert.ok(
!previewName.includes('datfrom'),
`preview mangled the trigger: ${previewName}`
);
const generated = runEvolve(root, ['--generate']);
assert.strictEqual(generated.status, 0, generated.stderr);
const commandFile = path.join(root, 'evolved', 'commands', `${previewName}.md`);
assert.ok(
fs.existsSync(commandFile),
`expected ${commandFile}, got: ${fs.readdirSync(path.join(root, 'evolved', 'commands')).join(', ')}`
);
} finally {
cleanupDir(root);
}
});
console.log(`\nPassed: ${passed}`);
console.log(`Failed: ${failed}`);
process.exit(failed > 0 ? 1 : 0);