From 9aac8585ab887d9c51252730240b25d9cca180da Mon Sep 17 00:00:00 2001 From: haelyra <49814733+haelyra@users.noreply.github.com> Date: Thu, 6 Aug 2026 17:42:22 -0400 Subject: [PATCH] fix(skills): default GAN harness models to sonnet (#2442) (#2695) Completes the model re-tiering from #2442: the gan-planner, gan-generator, and gan-evaluator agents were already re-pinned to sonnet, but the gan-style-harness script and docs still defaulted GAN_PLANNER_MODEL, GAN_GENERATOR_MODEL, and GAN_EVALUATOR_MODEL to opus. Align the script defaults, skill docs (en/ja/zh), and example commands with the landed agent tiers. Opus remains available via the existing env overrides. Co-authored-by: Claude Fable 5 --- docs/ja-JP/skills/gan-style-harness/SKILL.md | 26 ++++++++++---------- docs/zh-CN/skills/gan-style-harness/SKILL.md | 26 ++++++++++---------- examples/gan-harness/README.md | 8 +++--- scripts/gan-harness.sh | 12 ++++----- skills/gan-style-harness/SKILL.md | 26 ++++++++++---------- 5 files changed, 49 insertions(+), 49 deletions(-) diff --git a/docs/ja-JP/skills/gan-style-harness/SKILL.md b/docs/ja-JP/skills/gan-style-harness/SKILL.md index 410dbba6b..a2f88c4cc 100644 --- a/docs/ja-JP/skills/gan-style-harness/SKILL.md +++ b/docs/ja-JP/skills/gan-style-harness/SKILL.md @@ -37,7 +37,7 @@ This is the same dynamic as GANs (Generative Adversarial Networks): the Generato ``` ┌─────────────┐ │ PLANNER │ - │ (Opus 4.6) │ + │ (Sonnet) │ └──────┬──────┘ │ Product Spec │ (features, sprints, design direction) @@ -49,14 +49,14 @@ This is the same dynamic as GANs (Generative Adversarial Networks): the Generato │ │ │ ┌──────────┐ │ │ │GENERATOR │--build-->│──┐ - │ │(Opus 4.6)│ │ │ + │ │ (Sonnet) │ │ │ │ └────▲─────┘ │ │ │ │ │ │ live app │ feedback │ │ │ │ │ │ │ ┌────┴─────┐ │ │ │ │EVALUATOR │<-test----│──┘ - │ │(Opus 4.6)│ │ + │ │ (Sonnet) │ │ │ │+Playwright│ │ │ └──────────┘ │ │ │ @@ -76,7 +76,7 @@ This is the same dynamic as GANs (Generative Adversarial Networks): the Generato - Is deliberately **ambitious** — conservative planning leads to underwhelming results - Produces evaluation criteria that the Evaluator will use later -**Model:** Opus 4.6 (needs deep reasoning for spec expansion) +**Model:** Sonnet by default; raise via `GAN_PLANNER_MODEL=opus` for deeper spec expansion ### 2. Generator Agent @@ -89,7 +89,7 @@ This is the same dynamic as GANs (Generative Adversarial Networks): the Generato - Manages git for version control between iterations - Reads Evaluator feedback and incorporates it in next iteration -**Model:** Opus 4.6 (needs strong coding capability) +**Model:** Sonnet by default; raise via `GAN_GENERATOR_MODEL=opus` for maximum coding capability ### 3. Evaluator Agent @@ -106,7 +106,7 @@ This is the same dynamic as GANs (Generative Adversarial Networks): the Generato - Returns structured feedback with scores and specific issues - Is engineered to be **ruthlessly strict** — never praises mediocre work -**Model:** Opus 4.6 (needs strong judgment + tool use) +**Model:** Sonnet by default; raise via `GAN_EVALUATOR_MODEL=opus` for stronger judgment + tool use ## Evaluation Criteria @@ -178,16 +178,16 @@ GAN_EVAL_CRITERIA="functionality,performance,security" \ ```bash # Step 1: Plan -claude -p --model opus "You are a Product Planner. Read PLANNER_PROMPT.md. Expand this brief into a full product spec: 'Build a Kanban board app'. Write spec to spec.md" +claude -p --model sonnet "You are a Product Planner. Read PLANNER_PROMPT.md. Expand this brief into a full product spec: 'Build a Kanban board app'. Write spec to spec.md" # Step 2: Generate (iteration 1) -claude -p --model opus "You are a Generator. Read spec.md. Implement Sprint 1. Start the dev server on port 3000." +claude -p --model sonnet "You are a Generator. Read spec.md. Implement Sprint 1. Start the dev server on port 3000." # Step 3: Evaluate (iteration 1) -claude -p --model opus --allowedTools "Read,Bash,mcp__playwright__*" "You are an Evaluator. Read EVALUATOR_PROMPT.md. Test the live app at http://localhost:3000. Score against the rubric. Write feedback to feedback-001.md" +claude -p --model sonnet --allowedTools "Read,Bash,mcp__playwright__*" "You are an Evaluator. Read EVALUATOR_PROMPT.md. Test the live app at http://localhost:3000. Score against the rubric. Write feedback to feedback-001.md" # Step 4: Generate (iteration 2 — reads feedback) -claude -p --model opus "You are a Generator. Read spec.md and feedback-001.md. Address all issues. Improve the scores." +claude -p --model sonnet "You are a Generator. Read spec.md and feedback-001.md. Address all issues. Improve the scores." # Repeat steps 3-4 until pass threshold met ``` @@ -224,9 +224,9 @@ The harness should simplify as models improve. Following Anthropic's evolution: |----------|---------|-------------| | `GAN_MAX_ITERATIONS` | `15` | Maximum generator-evaluator cycles | | `GAN_PASS_THRESHOLD` | `7.0` | Weighted score to pass (1-10) | -| `GAN_PLANNER_MODEL` | `opus` | Model for planning agent | -| `GAN_GENERATOR_MODEL` | `opus` | Model for generator agent | -| `GAN_EVALUATOR_MODEL` | `opus` | Model for evaluator agent | +| `GAN_PLANNER_MODEL` | `sonnet` | Model for planning agent | +| `GAN_GENERATOR_MODEL` | `sonnet` | Model for generator agent | +| `GAN_EVALUATOR_MODEL` | `sonnet` | Model for evaluator agent | | `GAN_EVAL_CRITERIA` | `design,originality,craft,functionality` | Comma-separated criteria | | `GAN_DEV_SERVER_PORT` | `3000` | Port for the live app | | `GAN_DEV_SERVER_CMD` | `npm run dev` | Command to start dev server | diff --git a/docs/zh-CN/skills/gan-style-harness/SKILL.md b/docs/zh-CN/skills/gan-style-harness/SKILL.md index 303c0d7d0..c67eebda4 100644 --- a/docs/zh-CN/skills/gan-style-harness/SKILL.md +++ b/docs/zh-CN/skills/gan-style-harness/SKILL.md @@ -37,7 +37,7 @@ tools: Read, Write, Edit, Bash, Grep, Glob, Task ``` ┌─────────────┐ │ 规划器 │ - │ (Opus 4.6) │ + │ (Sonnet) │ └──────┬──────┘ │ 产品规格 │ (功能、冲刺、设计方向) @@ -49,14 +49,14 @@ tools: Read, Write, Edit, Bash, Grep, Glob, Task │ │ │ ┌──────────┐ │ │ │ 生成器 │--构建-->│──┐ - │ │(Opus 4.6)│ │ │ + │ │ (Sonnet) │ │ │ │ └────▲─────┘ │ │ │ │ │ │ 实时应用 │ 反馈 │ │ │ │ │ │ │ ┌────┴─────┐ │ │ │ │ 评估器 │<-测试---│──┘ - │ │(Opus 4.6)│ │ + │ │ (Sonnet) │ │ │ │+Playwright│ │ │ └──────────┘ │ │ │ @@ -77,7 +77,7 @@ tools: Read, Write, Edit, Bash, Grep, Glob, Task * 故意**雄心勃勃**——保守规划会导致结果平庸 * 生成评估器后续使用的评估标准 -**模型：** Opus 4.6（需要深度推理进行规格扩展） +**模型：** 默认 Sonnet；可通过 `GAN_PLANNER_MODEL=opus` 提升以获得更深入的规格扩展 ### 2. 生成器智能体 @@ -91,7 +91,7 @@ tools: Read, Write, Edit, Bash, Grep, Glob, Task * 管理 git 进行迭代间的版本控制 * 读取评估器反馈并在下一轮迭代中采纳 -**模型：** Opus 4.6（需要强大的编码能力） +**模型：** 默认 Sonnet；可通过 `GAN_GENERATOR_MODEL=opus` 提升以获得最强编码能力 ### 3. 评估器智能体 @@ -109,7 +109,7 @@ tools: Read, Write, Edit, Bash, Grep, Glob, Task * 返回结构化反馈，包含分数和具体问题 * 设计为**极度严格**——从不赞美平庸的工作 -**模型：** Opus 4.6（需要强大的判断力 + 工具使用能力） +**模型：** 默认 Sonnet；可通过 `GAN_EVALUATOR_MODEL=opus` 提升以获得更强的判断力 + 工具使用能力 ## 评估标准 @@ -181,16 +181,16 @@ GAN_EVAL_CRITERIA="functionality,performance,security" \ ```bash # Step 1: Plan -claude -p --model opus "You are a Product Planner. Read PLANNER_PROMPT.md. Expand this brief into a full product spec: 'Build a Kanban board app'. Write spec to spec.md" +claude -p --model sonnet "You are a Product Planner. Read PLANNER_PROMPT.md. Expand this brief into a full product spec: 'Build a Kanban board app'. Write spec to spec.md" # Step 2: Generate (iteration 1) -claude -p --model opus "You are a Generator. Read spec.md. Implement Sprint 1. Start the dev server on port 3000." +claude -p --model sonnet "You are a Generator. Read spec.md. Implement Sprint 1. Start the dev server on port 3000." # Step 3: Evaluate (iteration 1) -claude -p --model opus --allowedTools "Read,Bash,mcp__playwright__*" "You are an Evaluator. Read EVALUATOR_PROMPT.md. Test the live app at http://localhost:3000. Score against the rubric. Write feedback to feedback-001.md" +claude -p --model sonnet --allowedTools "Read,Bash,mcp__playwright__*" "You are an Evaluator. Read EVALUATOR_PROMPT.md. Test the live app at http://localhost:3000. Score against the rubric. Write feedback to feedback-001.md" # Step 4: Generate (iteration 2 — reads feedback) -claude -p --model opus "You are a Generator. Read spec.md and feedback-001.md. Address all issues. Improve the scores." +claude -p --model sonnet "You are a Generator. Read spec.md and feedback-001.md. Address all issues. Improve the scores." # Repeat steps 3-4 until pass threshold met ``` @@ -230,9 +230,9 @@ claude -p --model opus "You are a Generator. Read spec.md and feedback-001.md. A |----------|---------|-------------| | `GAN_MAX_ITERATIONS` | `15` | 最大生成器-评估器循环次数 | | `GAN_PASS_THRESHOLD` | `7.0` | 通过所需的加权分数（1-10） | -| `GAN_PLANNER_MODEL` | `opus` | 规划智能体的模型 | -| `GAN_GENERATOR_MODEL` | `opus` | 生成器智能体的模型 | -| `GAN_EVALUATOR_MODEL` | `opus` | 评估器智能体的模型 | +| `GAN_PLANNER_MODEL` | `sonnet` | 规划智能体的模型 | +| `GAN_GENERATOR_MODEL` | `sonnet` | 生成器智能体的模型 | +| `GAN_EVALUATOR_MODEL` | `sonnet` | 评估器智能体的模型 | | `GAN_EVAL_CRITERIA` | `design,originality,craft,functionality` | 逗号分隔的标准 | | `GAN_DEV_SERVER_PORT` | `3000` | 实时应用的端口 | | `GAN_DEV_SERVER_CMD` | `npm run dev` | 启动开发服务器的命令 | diff --git a/examples/gan-harness/README.md b/examples/gan-harness/README.md index cb0627cb0..bd32b8bd0 100644 --- a/examples/gan-harness/README.md +++ b/examples/gan-harness/README.md @@ -34,27 +34,27 @@ For maximum control, run each agent separately: ```bash # Step 1: Plan (produces spec.md) -claude -p --model opus "$(cat agents/gan-planner.md) +claude -p --model sonnet "$(cat agents/gan-planner.md) Your brief: 'Build a retro game maker with sprite editor and level designer' Write the full spec to gan-harness/spec.md and eval rubric to gan-harness/eval-rubric.md." # Step 2: Generate (iteration 1) -claude -p --model opus "$(cat agents/gan-generator.md) +claude -p --model sonnet "$(cat agents/gan-generator.md) Iteration 1. Read gan-harness/spec.md. Build the initial application. Start dev server on port 3000. Commit as iteration-001." # Step 3: Evaluate (iteration 1) -claude -p --model opus "$(cat agents/gan-evaluator.md) +claude -p --model sonnet "$(cat agents/gan-evaluator.md) Iteration 1. Read gan-harness/eval-rubric.md. Test http://localhost:3000. Write feedback to gan-harness/feedback/feedback-001.md. Be ruthlessly strict." # Step 4: Generate (iteration 2 — reads feedback) -claude -p --model opus "$(cat agents/gan-generator.md) +claude -p --model sonnet "$(cat agents/gan-generator.md) Iteration 2. Read gan-harness/feedback/feedback-001.md FIRST. Address every issue. Then read gan-harness/spec.md for remaining features. diff --git a/scripts/gan-harness.sh b/scripts/gan-harness.sh index f720135e2..9aa4289ca 100755 --- a/scripts/gan-harness.sh +++ b/scripts/gan-harness.sh @@ -11,9 +11,9 @@ # Environment Variables: # GAN_MAX_ITERATIONS — Max generator-evaluator cycles (default: 15) # GAN_PASS_THRESHOLD — Weighted score to pass, 1-10 (default: 7.0) -# GAN_PLANNER_MODEL — Model for planner (default: opus) -# GAN_GENERATOR_MODEL — Model for generator (default: opus) -# GAN_EVALUATOR_MODEL — Model for evaluator (default: opus) +# GAN_PLANNER_MODEL — Model for planner (default: sonnet) +# GAN_GENERATOR_MODEL — Model for generator (default: sonnet) +# GAN_EVALUATOR_MODEL — Model for evaluator (default: sonnet) # GAN_DEV_SERVER_PORT — Port for live app (default: 3000) # GAN_DEV_SERVER_CMD — Command to start dev server (default: "npm run dev") # GAN_PROJECT_DIR — Working directory (default: current dir) @@ -27,9 +27,9 @@ set -euo pipefail BRIEF="${1:?Usage: ./scripts/gan-harness.sh \"description of what to build\"}" MAX_ITERATIONS="${GAN_MAX_ITERATIONS:-15}" PASS_THRESHOLD="${GAN_PASS_THRESHOLD:-7.0}" -PLANNER_MODEL="${GAN_PLANNER_MODEL:-opus}" -GENERATOR_MODEL="${GAN_GENERATOR_MODEL:-opus}" -EVALUATOR_MODEL="${GAN_EVALUATOR_MODEL:-opus}" +PLANNER_MODEL="${GAN_PLANNER_MODEL:-sonnet}" +GENERATOR_MODEL="${GAN_GENERATOR_MODEL:-sonnet}" +EVALUATOR_MODEL="${GAN_EVALUATOR_MODEL:-sonnet}" DEV_PORT="${GAN_DEV_SERVER_PORT:-3000}" DEV_CMD="${GAN_DEV_SERVER_CMD:-npm run dev}" PROJECT_DIR="${GAN_PROJECT_DIR:-.}" diff --git a/skills/gan-style-harness/SKILL.md b/skills/gan-style-harness/SKILL.md index c920a2e06..febb48414 100644 --- a/skills/gan-style-harness/SKILL.md +++ b/skills/gan-style-harness/SKILL.md @@ -38,7 +38,7 @@ This is the same dynamic as GANs (Generative Adversarial Networks): the Generato ``` ┌─────────────┐ │ PLANNER │ - │ (Opus 4.6) │ + │ (Sonnet) │ └──────┬──────┘ │ Product Spec │ (features, sprints, design direction) @@ -50,14 +50,14 @@ This is the same dynamic as GANs (Generative Adversarial Networks): the Generato │ │ │ ┌──────────┐ │ │ │GENERATOR │--build-->│──┐ - │ │(Opus 4.6)│ │ │ + │ │ (Sonnet) │ │ │ │ └────▲─────┘ │ │ │ │ │ │ live app │ feedback │ │ │ │ │ │ │ ┌────┴─────┐ │ │ │ │EVALUATOR │<-test----│──┘ - │ │(Opus 4.6)│ │ + │ │ (Sonnet) │ │ │ │+Playwright│ │ │ └──────────┘ │ │ │ @@ -77,7 +77,7 @@ This is the same dynamic as GANs (Generative Adversarial Networks): the Generato - Is deliberately **ambitious** — conservative planning leads to underwhelming results - Produces evaluation criteria that the Evaluator will use later -**Model:** Opus 4.6 (needs deep reasoning for spec expansion) +**Model:** Sonnet by default; raise via `GAN_PLANNER_MODEL=opus` for deeper spec expansion ### 2. Generator Agent @@ -90,7 +90,7 @@ This is the same dynamic as GANs (Generative Adversarial Networks): the Generato - Manages git for version control between iterations - Reads Evaluator feedback and incorporates it in next iteration -**Model:** Opus 4.6 (needs strong coding capability) +**Model:** Sonnet by default; raise via `GAN_GENERATOR_MODEL=opus` for maximum coding capability ### 3. Evaluator Agent @@ -107,7 +107,7 @@ This is the same dynamic as GANs (Generative Adversarial Networks): the Generato - Returns structured feedback with scores and specific issues - Is engineered to be **ruthlessly strict** — never praises mediocre work -**Model:** Opus 4.6 (needs strong judgment + tool use) +**Model:** Sonnet by default; raise via `GAN_EVALUATOR_MODEL=opus` for stronger judgment + tool use ## Evaluation Criteria @@ -179,16 +179,16 @@ GAN_EVAL_CRITERIA="functionality,performance,security" \ ```bash # Step 1: Plan -claude -p --model opus "You are a Product Planner. Read PLANNER_PROMPT.md. Expand this brief into a full product spec: 'Build a Kanban board app'. Write spec to spec.md" +claude -p --model sonnet "You are a Product Planner. Read PLANNER_PROMPT.md. Expand this brief into a full product spec: 'Build a Kanban board app'. Write spec to spec.md" # Step 2: Generate (iteration 1) -claude -p --model opus "You are a Generator. Read spec.md. Implement Sprint 1. Start the dev server on port 3000." +claude -p --model sonnet "You are a Generator. Read spec.md. Implement Sprint 1. Start the dev server on port 3000." # Step 3: Evaluate (iteration 1) -claude -p --model opus --allowedTools "Read,Bash,mcp__playwright__*" "You are an Evaluator. Read EVALUATOR_PROMPT.md. Test the live app at http://localhost:3000. Score against the rubric. Write feedback to feedback-001.md" +claude -p --model sonnet --allowedTools "Read,Bash,mcp__playwright__*" "You are an Evaluator. Read EVALUATOR_PROMPT.md. Test the live app at http://localhost:3000. Score against the rubric. Write feedback to feedback-001.md" # Step 4: Generate (iteration 2 — reads feedback) -claude -p --model opus "You are a Generator. Read spec.md and feedback-001.md. Address all issues. Improve the scores." +claude -p --model sonnet "You are a Generator. Read spec.md and feedback-001.md. Address all issues. Improve the scores." # Repeat steps 3-4 until pass threshold met ``` @@ -225,9 +225,9 @@ The harness should simplify as models improve. Following Anthropic's evolution: |----------|---------|-------------| | `GAN_MAX_ITERATIONS` | `15` | Maximum generator-evaluator cycles | | `GAN_PASS_THRESHOLD` | `7.0` | Weighted score to pass (1-10) | -| `GAN_PLANNER_MODEL` | `opus` | Model for planning agent | -| `GAN_GENERATOR_MODEL` | `opus` | Model for generator agent | -| `GAN_EVALUATOR_MODEL` | `opus` | Model for evaluator agent | +| `GAN_PLANNER_MODEL` | `sonnet` | Model for planning agent | +| `GAN_GENERATOR_MODEL` | `sonnet` | Model for generator agent | +| `GAN_EVALUATOR_MODEL` | `sonnet` | Model for evaluator agent | | `GAN_EVAL_CRITERIA` | `design,originality,craft,functionality` | Comma-separated criteria | | `GAN_DEV_SERVER_PORT` | `3000` | Port for the live app | | `GAN_DEV_SERVER_CMD` | `npm run dev` | Command to start dev server |