Files
ECC/skills/benchmark-optimization-loop/SKILL.md
T
754b8dd76c fix: make the installer runtime pass strict supply-chain vetting (#2503)
* fix: make the installer runtime pass strict supply-chain vetting

Remediate the four enterprise supply-chain vetting blockers from
affaan-m/ECC#2502 so the installer runtime (package.json + manifests +
scripts/lib/**) passes strict exact-pin evidence policy:

1. Remove the package.json `postinstall` lifecycle script (it only echoed a
   post-install banner) and move that banner to an explicit opt-in
   `npm run welcome` command. No install-time lifecycle script remains.
2. Exact-pin every dependency in package.json (dependencies + devDependencies)
   to the versions already resolved in package-lock.json; no ^/~ ranges.
3. Replace non-ASCII characters on the installer runtime script/config surface:
   em-dashes (U+2014) in scripts/lib/{path-safety,install-executor,
   install/link-rewrite}.js comments and the two "Itô" (U+00F4) occurrences in
   manifests/{install-components,install-modules}.json descriptions become
   ASCII, so strict-surface Unicode scanners are clean.
4. Drop the bare `require("ajv")` from scripts/lib/install-state.js; the file
   already carries a complete hand-rolled validator enforcing the same
   schemas/install-state.schema.json (ecc.install.v1) constraints, so the
   installer closure is dependency-free (zero non-builtin bare requires).

Refs affaan-m/ECC#2502

* fix: avoid unpinned welcome invocations

Signed-off-by: Samar Tomar <samar_tomar@hotmail.com>

* fix: validate translated skill frontmatter

Signed-off-by: Samar Tomar <samar_tomar@hotmail.com>

* fix: repair skill frontmatter YAML

Signed-off-by: Samar Tomar <samar_tomar@hotmail.com>

* fix: add MIT license to core skill manifests; pin verification-loop tsc invocation

* fix: preserve tsc/pyright exit status in verification-loop type-check (set -o pipefail)

* chore(deps): sync lockfiles with exact-pinned package.json

Regenerate package-lock.json and yarn.lock so the pinned dependency
specs are reflected in both lockfiles. npm ci and Yarn's --immutable
install now pass the sync check. The resolution tree is unchanged
(231 yarn resolutions, byte-identical set; zero npm transitive drift);
only the root descriptor strings move from ranges to the versions
already resolved in the committed lockfiles.

Addresses the Codex P1 on #2503.

---------

Signed-off-by: Samar Tomar <samar_tomar@hotmail.com>
Co-authored-by: Samarjeet Singh Tomar <samartomar@gmail.com>
2026-07-17 17:13:49 -04:00

2.4 KiB

name, description, license, metadata, tools
name description license metadata tools
benchmark-optimization-loop Use when the user asks to make something faster, try many variants, run recursive optimization, benchmark latency/throughput/cost, or choose the best implementation by repeated measured tests. MIT
origin
ECC
Read, Write, Edit, Bash, Grep, Glob

Benchmark Optimization Loop

Use this skill to convert "make it 20x faster" or "try 50 recursive optimizations" into a bounded measured loop that can actually improve a system.

Required Baseline

Do not optimize until these exist:

  • the operation being optimized;
  • the correctness gate that must stay green;
  • the metric: wall time, p95 latency, rows/sec, cost/run, memory, error rate;
  • the current baseline;
  • the search budget: max variants, max time, max spend, max data impact.

If the user asks for an unrealistic target, keep the ambition but make the loop bounded and measurable.

Loop

  1. Measure the baseline.
  2. Identify bottlenecks from evidence.
  3. Generate variants that test one hypothesis each.
  4. Run variants with the same input shape.
  5. Reject variants that fail correctness, safety, or reproducibility.
  6. Promote the fastest safe variant.
  7. Codify the winning path in a script, command, test, config, or doc.
  8. Rerun the baseline and winner to confirm the delta.

Variant Table

Track variants like this:

Variant | Hypothesis | Command | Time | Correct? | Notes
baseline | current path | npm run job | 120s | yes | stable
batch-500 | fewer round trips | npm run job -- --batch 500 | 42s | yes | winner
parallel-8 | more workers | npm run job -- --workers 8 | 31s | no | rate limited

For recursive or hyperparameter work:

  • persist every run to a ledger;
  • compare against the prior accepted winner, not only the previous run;
  • keep a holdout or replay check;
  • stop when improvement is within noise, correctness fails, cost exceeds the budget, or the search starts changing more variables than it can explain.

Use phrases like "best measured safe variant" instead of "global optimum" unless the search space was actually exhaustive.

Promotion Gate

A variant cannot become the new default until:

  • correctness tests pass;
  • the performance delta is repeated or explained;
  • rollback is obvious;
  • the change is encoded in source control or a durable runbook;
  • the final summary includes exact commands and measurements.