WEBDIENSTE/ECC
1
0
Fork 0
mirror of https://github.com/affaan-m/ECC.git synced 2026-09-12 12:47:57 +02:00
Code Issues Packages Projects Releases Wiki Activity

fix(continuous-learning): /evolve never produces skill or agent candidates (#2664)

Browse Source 
* fix(continuous-learning): cluster instincts by keyword overlap in /evolve

`cmd_evolve` grouped instincts by exact string equality of the whole
normalized trigger sentence. Triggers are free-form sentences, so every
instinct landed in its own bucket and `skill_candidates` was always empty.
`agent_candidates` is derived from `skill_candidates`, so agents never
generated either — `/evolve --generate` could only ever emit commands.

Measured on a 42-instinct project: 42 instincts produced 42 unique cluster
keys, largest cluster size 1.

Group on keyword overlap instead. Jaccard is the wrong metric here — trigger
keyword sets average ~7 words, so even clearly related pairs top out around
0.33 — so this uses the overlap coefficient (shared / smaller set) at 0.5,
plus a floor of 2 shared keywords so one incidental word cannot pull
unrelated instincts together. The same 42 instincts now yield 4 clusters.

Also unify the command/agent slug used by the preview and the writer. The
preview called `.replace('a ', '')`, which strips "a " anywhere in the
string, mangling "extracting data from Reddit" into
`/extracting-datfrom-R` while `--generate` wrote `extracting-data-from.md`.
Both paths now share `_evolved_command_name()` / `_evolved_agent_name()`.

Adds tests/scripts/instinct-cli-evolve.test.js, which fails on the previous
implementation (0 clusters instead of 1; preview name `extracting-datfrom-R`)
and covers the negative cases so unrelated triggers still stay apart.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* docs(continuous-learning): correct clustering metric name in docstring

The docstring said "Jaccard" while the implementation uses the overlap
coefficient, which is the point of the change.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

* fix(continuous-learning): generate every evolve candidate and cut slugs on word boundaries

_generate_evolved() wrote only skill_candidates[:5], workflow_instincts[:5]
and agent_candidates[:3]. On a project with 36 command candidates that meant
5 files and no warning, so the output read as complete while 86% of the
candidates were dropped.

Generation is now unbounded by default and takes a --limit N flag for callers
that want a cap. A cap that truncates says so:

  Note: writing 3 of 36 command candidates (--limit 3); 33 skipped.

The analysis preview keeps showing five per kind but now names the remainder
("... and 31 more command candidates not shown") instead of presenting a
sample as the whole set.

Slugs were also cut with a hard slice, which split words mid-token and
produced /investigating-comple, /learning-about-compl and
/researching-mechanis. _truncate_slug() retreats to the last separator that
fits, and keeps the full head when the cut already lands on one, so
"analyzing large text files" stays /analyzing-large-text rather than losing
a word. A first word longer than the limit still falls back to a hard cut
because no boundary is available.

Shorter slugs collide more easily, and a collision used to mean one file
silently overwriting another. _assign_unique_slugs() suffixes duplicates
(-2, -3) and is called by both the preview and the writer over the same
ordered list, so advertised names and written names cannot drift apart.

Skill directory naming moved to _evolved_skill_name(); it previously used its
own inline slug expression, so it was the one truncation the shared helper
did not cover.

Adds tests/scripts/instinct-cli-evolve-generate.test.js: 7 cases covering
word-boundary cuts, the separator-aligned cut, unbounded generation, --limit
reporting, collision dedup, preview remainder and preview/writer agreement.
Six of the seven fail against the previous implementation.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: haelyra <49814733+haelyra@users.noreply.github.com>
This commit is contained in:
romanclaudersoai1
2026-08-04 16:57:24 -04:00
committed by GitHub
co-authored by Claude Opus 5 haelyra
parent b2bc8dcd14
commit 8a97868b5b
3 changed files with 658 additions and 32 deletions
Download Patch File Download Diff File Expand all files Collapse all files
skills/continuous-learning-v2/scripts/instinct-cli.py
+221 -32
View File
@@ -1145,6 +1145,163 @@ def cmd_export(args) -> int:
# Evolve Command
# ─────────────────────────────────────────────
# Words carrying no topical signal in a trigger sentence.
TRIGGER_STOP_WORDS = {
'when', 'while', 'the', 'and', 'or', 'to', 'of', 'in', 'on', 'for', 'with',
'that', 'this', 'from', 'into', 'at', 'by', 'as', 'is', 'are', 'be', 'it',
'its', 'they', 'them', 'their', 'you', 'your', 'new', 'any', 'all', 'about',
'after', 'before', 'over', 'via', 'use', 'using', 'need', 'needs', 'not',
}
# Overlap coefficient (shared / smaller set) two triggers need to cluster.
# Jaccard is the wrong metric here: trigger keyword sets average ~7 words, so
# even clearly-related pairs top out near 0.33 and nothing ever groups.
TRIGGER_SIMILARITY_THRESHOLD = 0.5
# Guard against one incidental shared word pulling unrelated instincts together.
TRIGGER_MIN_SHARED_KEYWORDS = 2
# Evolved artefact slugs are trimmed to keep file names short. The cut has to
# land on a word boundary: a hard slice produced names like
# "investigating-comple" and "learning-about-compl", which read as typos.
EVOLVED_SKILL_SLUG_LENGTH = 30
EVOLVED_COMMAND_SLUG_LENGTH = 20
EVOLVED_AGENT_SLUG_LENGTH = 20
def _truncate_slug(slug: str, max_length: int) -> str:
"""Trim a slug to max_length without splitting a word.
Falls back to a hard cut only when the first word is already longer than
the limit, because then there is no boundary left to retreat to.
"""
if len(slug) <= max_length:
return slug
head = slug[:max_length]
# The cut can already land on a separator, in which case head is a whole
# sequence of words and dropping one more would lose a word for nothing.
if slug[max_length] == '-':
return head.rstrip('-')
boundary = head.rfind('-')
if boundary > 0:
return head[:boundary]
return head.strip('-')
def _evolved_skill_name(trigger: str) -> str:
"""Slug used for a generated skill directory. Shared by preview and writer."""
return _truncate_slug(
re.sub(r'[^a-z0-9]+', '-', str(trigger or '').lower()).strip('-'),
EVOLVED_SKILL_SLUG_LENGTH,
)
def _evolved_command_name(trigger: str) -> str:
"""Slug used for a generated command file. Shared by preview and writer."""
stripped = str(trigger or 'unknown').lower().replace('when ', '').replace('implementing ', '')
return _truncate_slug(
re.sub(r'[^a-z0-9]+', '-', stripped).strip('-'),
EVOLVED_COMMAND_SLUG_LENGTH,
)
def _evolved_agent_name(trigger: str) -> str:
"""Slug used for a generated agent file. Shared by preview and writer."""
return _truncate_slug(
re.sub(r'[^a-z0-9]+', '-', str(trigger or '').lower()).strip('-'),
EVOLVED_AGENT_SLUG_LENGTH,
)
# How many candidates of each kind the analysis prints before summarising the
# rest. The preview is a sample, never the whole set, so it always says so.
PREVIEW_LIMIT = 5
def _print_preview_remainder(total: int, shown: int, noun: str) -> None:
"""State how many candidates the preview left out.
Without this the truncated list reads as the complete set.
"""
if total > shown:
print(f" ... and {total - shown} more {noun} not shown\n")
def _assign_unique_slugs(items: list, slug_fn) -> list:
"""Pair every item with a collision-free slug, preserving input order.
Word-boundary trimming makes collisions more likely because two triggers
can now share a whole prefix, and a collision previously meant one
generated file silently overwriting another. Preview and writer both call
this over the same ordered list, so the names shown and the names written
stay identical.
"""
used = set()
assigned = []
for item in items:
base = slug_fn(item)
if not base:
assigned.append((item, ''))
continue
name = base
suffix = 2
while name in used:
name = f"{base}-{suffix}"
suffix += 1
used.add(name)
assigned.append((item, name))
return assigned
def _trigger_keywords(trigger: str) -> set:
"""Reduce a trigger sentence to the words that carry its topic."""
words = re.findall(r'[a-z0-9]+', str(trigger or '').lower())
return {w for w in words if len(w) > 2 and w not in TRIGGER_STOP_WORDS}
def _cluster_by_keyword_overlap(instincts: list) -> dict:
"""Group instincts whose triggers share enough keywords.
Triggers are free-form sentences, so grouping on the whole normalized
string puts every instinct in its own bucket and no skill or agent
candidate is ever produced. Greedy clustering on keyword overlap groups
the near-duplicate instincts that accumulate in a project.
"""
clusters = [] # [(shared_keywords, [instincts])]
for inst in instincts:
keywords = _trigger_keywords(inst.get('trigger', ''))
if not keywords:
continue
best_index, best_score, best_shared = -1, 0.0, 0
for index, (cluster_keywords, _members) in enumerate(clusters):
shared = len(keywords & cluster_keywords)
smaller = min(len(keywords), len(cluster_keywords))
score = shared / smaller if smaller else 0.0
if score > best_score:
best_index, best_score, best_shared = index, score, shared
if (best_index >= 0
and best_score >= TRIGGER_SIMILARITY_THRESHOLD
and best_shared >= TRIGGER_MIN_SHARED_KEYWORDS):
cluster_keywords, members = clusters[best_index]
members.append(inst)
# Keep the shared core so a cluster stays on one topic.
clusters[best_index] = (cluster_keywords & keywords, members)
else:
clusters.append((keywords, [inst]))
grouped = {}
for cluster_keywords, members in clusters:
label = ' '.join(sorted(cluster_keywords)[:4]) or 'general'
while label in grouped:
label += ' +'
grouped[label] = members
return grouped
def cmd_evolve(args) -> int:
"""Analyze instincts and suggest evolutions to skills/commands/agents."""
project = detect_project()
@@ -1175,14 +1332,7 @@ def cmd_evolve(args) -> int:
print(f"High confidence instincts (>=80%): {len(high_conf)}")
# Find clusters (instincts with similar triggers)
trigger_clusters = defaultdict(list)
for inst in instincts:
trigger = inst.get('trigger', '')
# Normalize trigger
trigger_key = trigger.lower()
for keyword in ['when', 'creating', 'writing', 'adding', 'implementing', 'testing']:
trigger_key = trigger_key.replace(keyword, '').strip()
trigger_clusters[trigger_key].append(inst)
trigger_clusters = _cluster_by_keyword_overlap(instincts)
# Find clusters with 2+ instincts (good skill candidates)
skill_candidates = []
@@ -1203,8 +1353,8 @@ def cmd_evolve(args) -> int:
print(f"\nPotential skill clusters found: {len(skill_candidates)}")
if skill_candidates:
print(f"\n## SKILL CANDIDATES\n")
for i, cand in enumerate(skill_candidates[:5], 1):
print(f"\n## SKILL CANDIDATES ({len(skill_candidates)})\n")
for i, cand in enumerate(skill_candidates[:PREVIEW_LIMIT], 1):
scope_info = ', '.join(cand['scopes'])
print(f"{i}. Cluster: \"{cand['trigger']}\"")
print(f" Instincts: {len(cand['instincts'])}")
@@ -1215,37 +1365,51 @@ def cmd_evolve(args) -> int:
for inst in cand['instincts'][:3]:
print(f" - {inst.get('id')} [{inst.get('scope', '?')}]")
print()
_print_preview_remainder(len(skill_candidates), PREVIEW_LIMIT, 'skill clusters')
# Command candidates (workflow instincts with high confidence)
workflow_instincts = [i for i in instincts if i.get('domain') == 'workflow' and i.get('confidence', 0) >= 0.7]
if workflow_instincts:
print(f"\n## COMMAND CANDIDATES ({len(workflow_instincts)})\n")
for inst in workflow_instincts[:5]:
trigger = inst.get('trigger', 'unknown')
cmd_name = trigger.replace('when ', '').replace('implementing ', '').replace('a ', '')
cmd_name = cmd_name.replace(' ', '-')[:20]
# Slugs come from the same helper the writer uses, over the same ordered
# list, or the preview advertises names that differ from the files
# --generate actually writes.
for inst, cmd_name in _assign_unique_slugs(
workflow_instincts,
lambda i: _evolved_command_name(i.get('trigger', 'unknown')),
)[:PREVIEW_LIMIT]:
print(f" /{cmd_name}")
print(f" From: {inst.get('id')} [{inst.get('scope', '?')}]")
print(f" Confidence: {inst.get('confidence', 0.5):.0%}")
print()
_print_preview_remainder(len(workflow_instincts), PREVIEW_LIMIT, 'command candidates')
# Agent candidates (complex multi-step patterns)
agent_candidates = [c for c in skill_candidates if len(c['instincts']) >= 3 and c['avg_confidence'] >= 0.75]
if agent_candidates:
print(f"\n## AGENT CANDIDATES ({len(agent_candidates)})\n")
for cand in agent_candidates[:3]:
agent_name = cand['trigger'].replace(' ', '-')[:20] + '-agent'
for cand, agent_name in _assign_unique_slugs(
agent_candidates,
lambda c: _evolved_agent_name(str(c.get('trigger', '')).strip()),
)[:PREVIEW_LIMIT]:
print(f" {agent_name}")
print(f" Covers {len(cand['instincts'])} instincts")
print(f" Avg confidence: {cand['avg_confidence']:.0%}")
print()
_print_preview_remainder(len(agent_candidates), PREVIEW_LIMIT, 'agent candidates')
# Promotion candidates (project instincts that could be global)
_show_promotion_candidates(project)
if args.generate:
evolved_dir = project["evolved_dir"] if project["id"] != "global" else GLOBAL_EVOLVED_DIR
generated = _generate_evolved(skill_candidates, workflow_instincts, agent_candidates, evolved_dir)
generated = _generate_evolved(
skill_candidates,
workflow_instincts,
agent_candidates,
evolved_dir,
limit=max(0, getattr(args, 'limit', 0) or 0),
)
if generated:
print(f"\nGenerated {len(generated)} evolved structures:")
for path in generated:
@@ -1770,17 +1934,33 @@ def _cmd_projects_merge(args) -> int:
# Generate Evolved Structures
# ─────────────────────────────────────────────
def _generate_evolved(skill_candidates: list, workflow_instincts: list, agent_candidates: list, evolved_dir: Path) -> list[str]:
"""Generate skill/command/agent files from analyzed instinct clusters."""
def _generate_evolved(skill_candidates: list, workflow_instincts: list, agent_candidates: list, evolved_dir: Path, limit: int = 0) -> list[str]:
"""Generate skill/command/agent files from analyzed instinct clusters.
``limit`` caps how many candidates of each kind are written; 0 writes them
all. Anything a cap leaves out is reported, because the previous fixed
caps (5 skills, 5 commands, 3 agents) discarded most candidates without
saying a word — 35 command candidates produced 5 files and no warning.
"""
generated = []
# Generate skills from top candidates
for cand in skill_candidates[:5]:
def bounded(assigned: list, kind: str) -> list:
if limit and len(assigned) > limit:
print(f"\nNote: writing {limit} of {len(assigned)} {kind} candidates "
f"(--limit {limit}); {len(assigned) - limit} skipped.")
return assigned[:limit]
return assigned
# Generate skills from candidate clusters
for cand, name in bounded(
_assign_unique_slugs(
skill_candidates,
lambda c: _evolved_skill_name(str(c.get('trigger', '')).strip()),
),
'skill',
):
trigger = cand['trigger'].strip()
if not trigger:
continue
name = re.sub(r'[^a-z0-9]+', '-', trigger.lower()).strip('-')[:30]
if not name:
if not trigger or not name:
continue
skill_dir = evolved_dir / "skills" / name
@@ -1802,10 +1982,13 @@ def _generate_evolved(skill_candidates: list, workflow_instincts: list, agent_ca
generated.append(str(skill_dir / "SKILL.md"))
# Generate commands from workflow instincts
for inst in workflow_instincts[:5]:
trigger = inst.get('trigger', 'unknown')
cmd_name = re.sub(r'[^a-z0-9]+', '-', trigger.lower().replace('when ', '').replace('implementing ', ''))
cmd_name = cmd_name.strip('-')[:20]
for inst, cmd_name in bounded(
_assign_unique_slugs(
workflow_instincts,
lambda i: _evolved_command_name(i.get('trigger', 'unknown')),
),
'command',
):
if not cmd_name:
continue
@@ -1819,9 +2002,13 @@ def _generate_evolved(skill_candidates: list, workflow_instincts: list, agent_ca
generated.append(str(cmd_file))
# Generate agents from complex clusters
for cand in agent_candidates[:3]:
trigger = cand['trigger'].strip()
agent_name = re.sub(r'[^a-z0-9]+', '-', trigger.lower()).strip('-')[:20]
for cand, agent_name in bounded(
_assign_unique_slugs(
agent_candidates,
lambda c: _evolved_agent_name(str(c.get('trigger', '')).strip()),
),
'agent',
):
if not agent_name:
continue
@@ -2018,6 +2205,8 @@ def main() -> int:
# Evolve
evolve_parser = subparsers.add_parser('evolve', help='Analyze and evolve instincts')
evolve_parser.add_argument('--generate', action='store_true', help='Generate evolved structures')
evolve_parser.add_argument('--limit', type=int, default=0, metavar='N',
help='Max candidates of each kind to generate (default: 0 = all)')
# Promote (new in v2.1)
promote_parser = subparsers.add_parser('promote', help='Promote project instincts to global scope')