feat(session-start): rank injected instincts by project/stack relevance (#2466)

* feat(session-start): rank injected instincts by project/stack relevance

Instinct selection at SessionStart ranked purely by confidence, so a
high-confidence instinct about an unrelated stack could take an injection
slot from a lower-confidence instinct that is actually relevant to the
current project.

Rank by confidence + location/stack relevance instead: project-scoped
instincts, and instincts whose domain/trigger matches the detected stack
(languages/frameworks via detectProjectType, plus terraform/dbt markers),
get a small additive boost. The confidence>=threshold floor and the
injection cap are unchanged, and ranking degrades to confidence-only when
nothing matches or when ECC_INSTINCT_RELEVANCE_RANKING is set to off.

The ranking helpers live in scripts/lib/instinct-relevance.js with unit
coverage in tests/lib/, plus an end-to-end ordering test in tests/hooks/.

Completes part (b) of #2371; part (a) (configurable count + threshold)
shipped in #2413.

Fixes #2371

* refactor(session-start): drop redundant confidence tiebreaker in instinct sort

Greptile flagged that the secondary `right.confidence` comparison in
summarizeActiveInstincts' sort was dead code when relevance ranking is
disabled and, when enabled, was reached only on a floating-point tie of the
combined score — where it skipped the intended scope-label tiebreaker.

Remove it: the primary combined-score comparison already reduces to
confidence-only ordering when relevance is off, so behavior there is
unchanged; a genuine combined-score tie now falls through to the documented
scope-first, then id, order.

* test: isolate instinct relevance environment

---------

Co-authored-by: haelyra <49814733+haelyra@users.noreply.github.com>
This commit is contained in:
Gaurav Dubey
2026-08-10 21:46:24 -04:00
committed by GitHub
co-authored by haelyra
parent 3d4ef3184b
commit 5987bd4dc6
5 changed files with 496 additions and 1 deletions
+27 -1
View File
@@ -24,6 +24,11 @@ const { resolveProjectContext, writeSessionLease, resolveSessionId, getHomunculu
const { getPackageManager, getSelectionPrompt } = require('../lib/package-manager');
const { listAliases } = require('../lib/session-aliases');
const { detectProjectType } = require('../lib/project-detect');
const {
isRelevanceRankingEnabled,
detectStackKeywords,
computeRelevanceBoost,
} = require('../lib/instinct-relevance');
const path = require('path');
const fs = require('fs');
@@ -422,6 +427,20 @@ function summarizeActiveInstincts(observerContext) {
const confidenceThreshold = getInstinctConfidenceThreshold();
const maxInjected = getMaxInjectedInstincts();
// Relevance ranking (issue #2371 part b): at SessionStart there is no user
// task yet, so relevance is location/stack based. Project-scoped and
// stack-matching instincts get a small additive boost over their confidence.
// Gated by ECC_INSTINCT_RELEVANCE_RANKING (default on); when off, or when no
// stack is detected and nothing is project-scoped, every boost is 0 and the
// ranking collapses to confidence-only (unchanged behaviour).
// Detect the stack from the real project source tree (projectRoot), not the
// homunculus state dir (projectDir). In a global session projectRoot is empty,
// so detectStackKeywords falls back to process.cwd().
const relevanceEnabled = isRelevanceRankingEnabled();
const stackKeywords = relevanceEnabled
? detectStackKeywords(observerContext.projectRoot || undefined)
: new Set();
const deduped = new Map();
for (const instinct of scopedInstincts) {
if (!instinct.id || instinct.confidence < confidenceThreshold) continue;
@@ -435,10 +454,17 @@ function summarizeActiveInstincts(observerContext) {
.map(instinct => ({
...instinct,
action: extractInstinctAction(instinct.content),
_relevance: relevanceEnabled ? computeRelevanceBoost(instinct, stackKeywords) : 0,
}))
.filter(instinct => instinct.action)
.sort((left, right) => {
if (right.confidence !== left.confidence) return right.confidence - left.confidence;
// Primary: combined confidence + relevance. When relevance is off every
// _relevance is 0, so this reduces to the prior confidence-only ordering.
// Tie-breaks on a genuinely equal combined score: project scope first,
// then id (deterministic).
const leftScore = left.confidence + left._relevance;
const rightScore = right.confidence + right._relevance;
if (rightScore !== leftScore) return rightScore - leftScore;
if (left._scopeLabel !== right._scopeLabel) return left._scopeLabel === 'project' ? -1 : 1;
return String(left.id).localeCompare(String(right.id));
})