mirror of
https://github.com/affaan-m/ECC.git
synced 2026-08-20 14:32:24 +02:00
Gaurav Dubey
9d1ecb0754
The `.opencode/commands/*.md` frontmatter referenced agents with the Claude Code plugin namespace (`agent: everything-claude-code:<name>`), but ECC's opencode integration registers its agents unscoped in `opencode.json`'s `agent` map (`code-reviewer`, `planner`, ...), and that file's own `command` section already references them unscoped. The `everything-claude-code:` scope resolves under no opencode config (the opencode plugin package is `ecc-universal`, and inline-config agents are bare), so subtask commands like `/code-review` hard-fail with `Agent not found: everything-claude-code:code-reviewer`. Non-subtask commands fall back to the default agent and appear to work — which is why only some commands failed. Strip the `everything-claude-code:` prefix from all 30 command frontmatter agent ids so they match the registered agents, fix the MIGRATION.md example, and replace the test that enforced the broken scoped invariant with one that asserts each command agent id is a registered opencode agent (fails on the old scoped ids, passes on the fix). Fixes #2477
89 lines
1.6 KiB
Markdown
89 lines
1.6 KiB
Markdown
---
|
||
description: Run evaluation against acceptance criteria
|
||
agent: build
|
||
---
|
||
|
||
# Eval Command
|
||
|
||
Evaluate implementation against acceptance criteria: $ARGUMENTS
|
||
|
||
## Your Task
|
||
|
||
Run structured evaluation to verify the implementation meets requirements.
|
||
|
||
## Evaluation Framework
|
||
|
||
### Grader Types
|
||
|
||
1. **Binary Grader** - Pass/Fail
|
||
- Does it work? Yes/No
|
||
- Good for: feature completion, bug fixes
|
||
|
||
2. **Scalar Grader** - Score 0-100
|
||
- How well does it work?
|
||
- Good for: performance, quality metrics
|
||
|
||
3. **Rubric Grader** - Category scores
|
||
- Multiple dimensions evaluated
|
||
- Good for: comprehensive review
|
||
|
||
## Evaluation Process
|
||
|
||
### Step 1: Define Criteria
|
||
|
||
```
|
||
Acceptance Criteria:
|
||
1. [Criterion 1] - [weight]
|
||
2. [Criterion 2] - [weight]
|
||
3. [Criterion 3] - [weight]
|
||
```
|
||
|
||
### Step 2: Run Tests
|
||
|
||
For each criterion:
|
||
- Execute relevant test
|
||
- Collect evidence
|
||
- Score result
|
||
|
||
### Step 3: Calculate Score
|
||
|
||
```
|
||
Final Score = Σ (criterion_score × weight) / total_weight
|
||
```
|
||
|
||
### Step 4: Report
|
||
|
||
## Evaluation Report
|
||
|
||
### Overall: [PASS/FAIL] (Score: X/100)
|
||
|
||
### Criterion Breakdown
|
||
|
||
| Criterion | Score | Weight | Weighted |
|
||
|-----------|-------|--------|----------|
|
||
| [Criterion 1] | X/10 | 30% | X |
|
||
| [Criterion 2] | X/10 | 40% | X |
|
||
| [Criterion 3] | X/10 | 30% | X |
|
||
|
||
### Evidence
|
||
|
||
**Criterion 1: [Name]**
|
||
- Test: [what was tested]
|
||
- Result: [outcome]
|
||
- Evidence: [screenshot, log, output]
|
||
|
||
### Recommendations
|
||
|
||
[If not passing, what needs to change]
|
||
|
||
## Pass@K Metrics
|
||
|
||
For non-deterministic evaluations:
|
||
- Run K times
|
||
- Calculate pass rate
|
||
- Report: "Pass@K = X/K"
|
||
|
||
---
|
||
|
||
**TIP**: Use eval for acceptance testing before marking features complete.
|