mirror of
https://github.com/affaan-m/ECC.git
synced 2026-08-26 09:22:24 +02:00
fix(ecc2): preserve harness evidence and legacy IDs
This commit is contained in:
@@ -509,9 +509,9 @@ Kimi Code discovers the installed `.kimi/AGENTS.md` instructions and `.kimi/skil
|
||||
|
||||
### Itô compute CLI bridge
|
||||
|
||||
`ecc ito` delegates to the separately installed canonical Itô client; ECC does not maintain a second API client. `ecc ito auth` delegates to the canonical CLI's device authorization, opens the Itô verification page by default, and persists a device token in macOS Keychain; pass `--no-browser` to suppress the page handoff. ECC itself does no browser automation. The available operations are `ecc ito auth`, `ecc ito find`, `ecc ito status`, and the separately gated `ecc ito evals`. The matching MCP tools remain `ito_auth`, `ito_find`, and `ito_status`; node qualification is CLI-only.
|
||||
`ecc ito` delegates to the separately installed canonical Itô client; ECC does not maintain a second API client. `ecc ito login [--no-browser]` performs device authorization, opens the Itô verification page by default, and persists a device token in macOS Keychain; `--no-browser` suppresses the page handoff. ECC itself does no browser automation. `ecc ito auth` is validation-only and rejects `--no-browser`. The available operations are `ecc ito login`, `ecc ito auth`, `ecc ito find`, `ecc ito status`, and the separately gated `ecc ito evals`. The matching MCP tools remain `ito_auth`, `ito_find`, and `ito_status`; `ito_auth` validates existing credentials and node qualification is CLI-only.
|
||||
|
||||
The `ito-compute-cli` package is currently unpublished. Build it locally from the Itô runtime repo (private while the desk hardens; design partners get access) under `cli/ito-compute-cli`, run `npm ci` and `npm run check`, then set `ECC_ITO_CLI_EXECUTABLE` to that build's absolute `dist/bin/ito.js` path. Device authorization is the default; keep legacy `ITO_API_KEY` only with explicit `ITO_AUTH_MODE=legacy`. ECC does not discover this credential-bearing client through `PATH`. See the [`ito-compute` skill](skills/ito-compute/SKILL.md) for the full RFQ authority and MCP setup contract.
|
||||
The `ito-compute-cli` package is currently unpublished. Build it locally from the Itô runtime repo (private while the desk hardens; design partners get access) under `cli/ito-compute-cli`, run `npm ci` and `npm run check`, then set `ECC_ITO_CLI_EXECUTABLE` to that build's absolute `dist/bin/ito.js` path. Login never inherits `ITO_API_KEY`; auth, find, and status forward `ITO_API_KEY` directly when configured, and `ITO_AUTH_MODE=legacy` is not required. Device tokens use macOS Keychain by default; explicit file fallback must retain owner-only directory/file permissions. ECC does not discover this credential-bearing client through `PATH`. See the [`ito-compute` skill](skills/ito-compute/SKILL.md) for the full RFQ authority and MCP setup contract.
|
||||
|
||||
`find` submits a live authenticated RFQ. It does not reserve capacity. `evals` requires both `ITO_ENABLE_SIXTYTWO_LIVE=1` and `--live-sixtytwo`, a separately installed `sixtytwo-cli==0.3.33`, an explicit node list, and an existing absolute configuration directory. It cannot rent, launch, recover, repair, or purchase. ECC exposes no quote lock, purchase, workload, or inference path, and it never replaces a missing client or failed live call with a local result.
|
||||
</details>
|
||||
|
||||
@@ -24,9 +24,10 @@ ECC delegates to the canonical Itô package in
|
||||
`Ito-Markets/ito-cloud-runtime/cli/ito-compute-cli`. ECC does not maintain a
|
||||
second API client or response schema.
|
||||
|
||||
The wrapper exposes only the canonical CLI's `auth`, `find`, `status`, and `evals`
|
||||
The wrapper exposes only the canonical CLI's `login`, `auth`, `find`, `status`, and `evals`
|
||||
operations:
|
||||
|
||||
ecc ito login [--no-browser]
|
||||
ecc ito auth
|
||||
ecc ito find <all required RFQ constraints>
|
||||
ecc ito status
|
||||
@@ -36,11 +37,12 @@ The canonical MCP server exposes only `ito_auth`, `ito_find`, and `ito_status`.
|
||||
ECC includes an opt-in configuration template pointing to the local built MCP
|
||||
entry. It does not enable the server by default.
|
||||
|
||||
The former browser/manual-copy command is retired. `ecc ito auth` delegates to
|
||||
The former browser/manual-copy command is retired. `ecc ito login` delegates to
|
||||
the canonical CLI's device authorization, which opens the Itô verification page
|
||||
by default and persists a device token in macOS Keychain. `--no-browser`
|
||||
suppresses that page handoff. ECC itself performs no browser automation and
|
||||
stores no economic state.
|
||||
stores no economic state. `ecc ito auth` is validation-only, never starts
|
||||
device login, and rejects `--no-browser`.
|
||||
|
||||
## Local install
|
||||
|
||||
@@ -63,15 +65,18 @@ For MCP, configure `node` with:
|
||||
|
||||
/absolute/path/to/ito-cloud-runtime/cli/ito-compute-cli/dist/bin/ito-mcp.js
|
||||
|
||||
Device authorization is the default. ECC forwards only the required device
|
||||
authorization settings, optional Itô endpoint overrides, and the minimum
|
||||
process environment. Legacy `ITO_API_KEY` is forwarded only with explicit
|
||||
`ITO_AUTH_MODE=legacy`; ECC does not inspect or log it.
|
||||
Device login forwards only required authorization settings, optional Itô
|
||||
endpoint overrides, and the minimum process environment; it never inherits
|
||||
`ITO_API_KEY`. The `auth`, `find`, and `status` commands forward `ITO_API_KEY`
|
||||
directly when configured; `ITO_AUTH_MODE=legacy` is not required. Device tokens
|
||||
use macOS Keychain by default. Explicit file fallback retains owner-only 0700
|
||||
directory and 0600 token-file permissions. ECC does not inspect or log secrets.
|
||||
|
||||
## Authority and economics
|
||||
|
||||
- `auth` starts canonical device authorization, with `--no-browser` available
|
||||
- `login` starts canonical device authorization, with `--no-browser` available
|
||||
when the operator does not want the CLI to open the verification page.
|
||||
- `auth` validates existing credentials only.
|
||||
- `find` reads live inventory and submits a live authenticated RFQ. An operator
|
||||
or agent must gather every hard topology/economic constraint and obtain
|
||||
explicit buyer authority before invoking it.
|
||||
|
||||
@@ -1,14 +1,14 @@
|
||||
# ECC × Itô Real CLI Bridge — TDD Evidence
|
||||
|
||||
Date: 2026-07-23
|
||||
Date: 2026-08-05
|
||||
|
||||
Source plan: requirements were derived from the approved implementation
|
||||
handoff. No external plan file was executed.
|
||||
|
||||
## User journeys
|
||||
|
||||
1. As an ECC operator, I can invoke the canonical local Itô `auth`, `find`, and
|
||||
`status` operations without a duplicate client or browser workflow.
|
||||
1. As an ECC operator, I can explicitly invoke streaming device `login`, then
|
||||
use validation-only `auth`, `find`, and `status` without a duplicate client.
|
||||
2. As a security reviewer, I can prove unsupported operations, missing local
|
||||
installs, and ECC dry-run requests fail before any child process or network
|
||||
operation.
|
||||
@@ -21,62 +21,46 @@ Before production changes:
|
||||
|
||||
```text
|
||||
node tests/scripts/ito-cli-bridge.test.js
|
||||
Passed: 0
|
||||
Failed: 9
|
||||
Passed: 13
|
||||
Failed: 8
|
||||
|
||||
node tests/ci/ito-compute-skill.test.js
|
||||
Passed: 0
|
||||
Failed: 4
|
||||
Passed: 2
|
||||
Failed: 3
|
||||
```
|
||||
|
||||
The failures were caused by the old browser-only `rent` command and the missing
|
||||
real skill/install/MCP surfaces.
|
||||
The failures captured the old combined auth/login surface, legacy-mode API-key
|
||||
gate, buffered login output, and stale help, skill, MCP, and integration wording.
|
||||
|
||||
## GREEN evidence
|
||||
|
||||
```text
|
||||
node tests/scripts/ito-cli-bridge.test.js
|
||||
Passed: 9
|
||||
Passed: 21
|
||||
Failed: 0
|
||||
|
||||
node tests/ci/ito-compute-skill.test.js
|
||||
Passed: 4
|
||||
Passed: 5
|
||||
Failed: 0
|
||||
|
||||
NODE_PATH=<existing-ecc-checkout>/node_modules \
|
||||
node scripts/ci/validate-install-manifests.js
|
||||
Validated 33 install modules, 80 install components, and 7 profiles
|
||||
|
||||
npm test
|
||||
Total Tests: 3159
|
||||
Passed: 3159
|
||||
Failed: 0
|
||||
|
||||
npm run coverage
|
||||
Statements: 89.21%
|
||||
Branches: 79.71%
|
||||
Functions: 93.96%
|
||||
Lines: 89.21%
|
||||
|
||||
npm run security:ioc-scan
|
||||
Supply-chain IOC scan passed
|
||||
node scripts/ci/validate-skills.js
|
||||
Validated 281 skill directories
|
||||
```
|
||||
|
||||
The isolated worktree temporarily reused the canonical ECC checkout's existing
|
||||
`node_modules` through an untracked local symlink. The symlink was removed
|
||||
after validation; no dependency installation or source change was made in the
|
||||
canonical checkout.
|
||||
ESLint and Markdown lint also pass for every changed source file. The complete
|
||||
package dry-run contains the wrapper, environment boundary, skill, and MCP
|
||||
configuration.
|
||||
`node tests/scripts/ito-compute-sponsor.test.js` reached 11 passes and 2 failures;
|
||||
both failures are setup failures because the current worktree lacks `ajv`.
|
||||
`node scripts/ci/validate-install-manifests.js` is blocked by the same missing
|
||||
module. No dependency installation was performed.
|
||||
|
||||
## Test specification
|
||||
|
||||
| Guarantee | Test | Type | Result |
|
||||
|---|---|---|---|
|
||||
| Only `auth`, `find`, and `status` spawn | `tests/scripts/ito-cli-bridge.test.js` | end-to-end process contract | PASS |
|
||||
| `login`, `auth`, `find`, and `status` forward only their reviewed surfaces | `tests/scripts/ito-cli-bridge.test.js` | end-to-end process contract | PASS |
|
||||
| Login output streams before completion and its exit status propagates | `tests/scripts/ito-cli-bridge.test.js` | async process contract | PASS |
|
||||
| `auth --no-browser` fails before spawn | `tests/scripts/ito-cli-bridge.test.js` | negative process contract | PASS |
|
||||
| Full RFQ arguments cross unchanged | `tests/scripts/ito-cli-bridge.test.js` | integration | PASS |
|
||||
| Only required Itô settings cross the child boundary | `tests/scripts/ito-cli-bridge.test.js` | security integration | PASS |
|
||||
| Login scrubs the API key; auth/find/status forward it directly; evals stays isolated | `tests/scripts/ito-cli-bridge.test.js` | security integration | PASS |
|
||||
| Unsupported and dry-run operations fail before spawn | `tests/scripts/ito-cli-bridge.test.js` | negative end-to-end | PASS |
|
||||
| Missing/relative executables fail with exact local guidance | `tests/scripts/ito-cli-bridge.test.js` | negative end-to-end | PASS |
|
||||
| Child output and exit code are preserved | `tests/scripts/ito-cli-bridge.test.js` | end-to-end process contract | PASS |
|
||||
|
||||
+191
-14
@@ -5,22 +5,93 @@ mod tests {
|
||||
use std::collections::BTreeMap;
|
||||
|
||||
#[test]
|
||||
fn candidate_id_is_content_addressed_over_canonical_json() {
|
||||
fn candidate_id_addresses_canonical_config_and_normalized_references() {
|
||||
let first = CandidateSpec::new(
|
||||
json!({"model": "fixed", "limits": {"steps": 3, "tools": ["read"]}}),
|
||||
vec!["trace://one".into()],
|
||||
vec!["evidence://one".into()],
|
||||
vec![" trace://two ".into(), "trace://one".into()],
|
||||
vec!["evidence://two".into(), " evidence://one ".into()],
|
||||
)
|
||||
.unwrap();
|
||||
let second = CandidateSpec::new(
|
||||
json!({"limits": {"tools": ["read"], "steps": 3}, "model": "fixed"}),
|
||||
vec!["trace://two".into()],
|
||||
vec!["evidence://two".into()],
|
||||
vec!["trace://one".into(), "trace://two".into()],
|
||||
vec!["evidence://one".into(), "evidence://two".into()],
|
||||
)
|
||||
.unwrap();
|
||||
|
||||
assert_eq!(first.id, second.id);
|
||||
assert_eq!(first.canonical_config, second.canonical_config);
|
||||
assert_eq!(first.trace_refs, vec!["trace://one", "trace://two"]);
|
||||
assert_eq!(
|
||||
first.evidence_refs,
|
||||
vec!["evidence://one", "evidence://two"]
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn candidate_id_changes_when_any_immutable_reference_changes() {
|
||||
let original = CandidateSpec::new(
|
||||
json!({"model": "fixed"}),
|
||||
vec!["trace://one".into()],
|
||||
vec!["evidence://one".into()],
|
||||
)
|
||||
.unwrap();
|
||||
let changed_trace = CandidateSpec::new(
|
||||
json!({"model": "fixed"}),
|
||||
vec!["trace://two".into()],
|
||||
vec!["evidence://one".into()],
|
||||
)
|
||||
.unwrap();
|
||||
let changed_evidence = CandidateSpec::new(
|
||||
json!({"model": "fixed"}),
|
||||
vec!["trace://one".into()],
|
||||
vec!["evidence://two".into()],
|
||||
)
|
||||
.unwrap();
|
||||
|
||||
assert_ne!(original.id, changed_trace.id);
|
||||
assert_ne!(original.id, changed_evidence.id);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn candidate_integrity_rejects_reference_tampering() {
|
||||
let mut candidate = CandidateSpec::new(
|
||||
json!({"model": "fixed"}),
|
||||
vec!["trace://one".into()],
|
||||
vec!["evidence://one".into()],
|
||||
)
|
||||
.unwrap();
|
||||
candidate.trace_refs = vec!["trace://tampered".into()];
|
||||
|
||||
assert!(candidate.verify_integrity().is_err());
|
||||
|
||||
let mut noncanonical = CandidateSpec::new(
|
||||
json!({"model": "fixed"}),
|
||||
vec!["trace://one".into(), "trace://two".into()],
|
||||
vec!["evidence://one".into()],
|
||||
)
|
||||
.unwrap();
|
||||
noncanonical.trace_refs.reverse();
|
||||
assert!(noncanonical.verify_integrity().is_err());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn persisted_candidate_integrity_accepts_only_exact_v1_or_v2_ids() {
|
||||
let candidate = CandidateSpec::new(
|
||||
json!({"model": "fixed", "limits": {"steps": 3}}),
|
||||
vec!["trace://one".into()],
|
||||
vec!["evidence://one".into()],
|
||||
)
|
||||
.unwrap();
|
||||
let legacy_id = candidate.legacy_id();
|
||||
|
||||
candidate.verify_persisted_id(&candidate.id).unwrap();
|
||||
candidate.verify_persisted_id(&legacy_id).unwrap();
|
||||
assert!(candidate
|
||||
.verify_persisted_id(&"a".repeat(64))
|
||||
.unwrap_err()
|
||||
.to_string()
|
||||
.contains("content address"));
|
||||
}
|
||||
|
||||
#[test]
|
||||
@@ -120,14 +191,18 @@ pub struct CandidateSpec {
|
||||
|
||||
impl CandidateSpec {
|
||||
pub fn new(config: Value, trace_refs: Vec<String>, evidence_refs: Vec<String>) -> Result<Self> {
|
||||
validate_refs("trace", &trace_refs)?;
|
||||
validate_refs("evidence", &evidence_refs)?;
|
||||
let trace_refs = normalize_refs("trace", trace_refs)?;
|
||||
let evidence_refs = normalize_refs("evidence", evidence_refs)?;
|
||||
let canonical_config = serde_json::to_string(&canonicalize(config))?;
|
||||
if canonical_config.len() > 1024 * 1024 {
|
||||
bail!("candidate configuration exceeds 1 MiB");
|
||||
}
|
||||
let digest = Sha256::digest(canonical_config.as_bytes());
|
||||
let id = digest.iter().map(|byte| format!("{byte:02x}")).collect();
|
||||
let artifact = serde_json::to_string(&CanonicalCandidateArtifact {
|
||||
config: serde_json::from_str(&canonical_config)?,
|
||||
trace_refs: &trace_refs,
|
||||
evidence_refs: &evidence_refs,
|
||||
})?;
|
||||
let id = sha256_hex(artifact.as_bytes());
|
||||
Ok(Self {
|
||||
id,
|
||||
canonical_config,
|
||||
@@ -137,23 +212,71 @@ impl CandidateSpec {
|
||||
}
|
||||
|
||||
pub fn verify_integrity(&self) -> Result<()> {
|
||||
let value: Value = serde_json::from_str(&self.canonical_config)?;
|
||||
let rebuilt = Self::new(value, self.trace_refs.clone(), self.evidence_refs.clone())?;
|
||||
if rebuilt.id != self.id || rebuilt.canonical_config != self.canonical_config {
|
||||
self.verify_persisted_id(&self.id)?;
|
||||
if self.id != self.id_for_v2()? {
|
||||
bail!("candidate content address or canonical configuration is invalid");
|
||||
}
|
||||
Ok(())
|
||||
}
|
||||
|
||||
pub fn legacy_id(&self) -> String {
|
||||
sha256_hex(self.canonical_config.as_bytes())
|
||||
}
|
||||
|
||||
pub fn verify_persisted_id(&self, persisted_id: &str) -> Result<()> {
|
||||
let value: Value = serde_json::from_str(&self.canonical_config)?;
|
||||
let rebuilt = Self::new(value, self.trace_refs.clone(), self.evidence_refs.clone())?;
|
||||
let is_v1 = persisted_id == self.legacy_id();
|
||||
let is_v2 = persisted_id == rebuilt.id;
|
||||
if rebuilt.canonical_config != self.canonical_config
|
||||
|| (!is_v1 && !is_v2)
|
||||
|| (is_v2
|
||||
&& (rebuilt.trace_refs != self.trace_refs
|
||||
|| rebuilt.evidence_refs != self.evidence_refs))
|
||||
{
|
||||
bail!("candidate content address or canonical configuration is invalid");
|
||||
}
|
||||
Ok(())
|
||||
}
|
||||
|
||||
pub(crate) fn id_for_v2(&self) -> Result<String> {
|
||||
Ok(Self::new(
|
||||
serde_json::from_str(&self.canonical_config)?,
|
||||
self.trace_refs.clone(),
|
||||
self.evidence_refs.clone(),
|
||||
)?
|
||||
.id)
|
||||
}
|
||||
}
|
||||
|
||||
fn validate_refs(kind: &str, refs: &[String]) -> Result<()> {
|
||||
#[derive(Serialize)]
|
||||
struct CanonicalCandidateArtifact<'a> {
|
||||
config: Value,
|
||||
trace_refs: &'a [String],
|
||||
evidence_refs: &'a [String],
|
||||
}
|
||||
|
||||
fn normalize_refs(kind: &str, refs: Vec<String>) -> Result<Vec<String>> {
|
||||
if refs.is_empty() || refs.iter().any(|reference| reference.trim().is_empty()) {
|
||||
bail!("at least one non-empty {kind} reference is required");
|
||||
}
|
||||
if refs.len() > 100 || refs.iter().any(|reference| reference.len() > 4096) {
|
||||
bail!("{kind} references exceed bounded limits");
|
||||
}
|
||||
Ok(())
|
||||
let mut normalized = refs
|
||||
.into_iter()
|
||||
.map(|reference| reference.trim().to_string())
|
||||
.collect::<Vec<_>>();
|
||||
normalized.sort();
|
||||
normalized.dedup();
|
||||
Ok(normalized)
|
||||
}
|
||||
|
||||
fn sha256_hex(bytes: &[u8]) -> String {
|
||||
Sha256::digest(bytes)
|
||||
.iter()
|
||||
.map(|byte| format!("{byte:02x}"))
|
||||
.collect()
|
||||
}
|
||||
|
||||
fn canonicalize(value: Value) -> Value {
|
||||
@@ -184,6 +307,50 @@ pub struct RecordedEvidence {
|
||||
pub health: BTreeMap<String, bool>,
|
||||
}
|
||||
|
||||
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
|
||||
pub struct HealthEvidenceSnapshot {
|
||||
pub schema_version: u8,
|
||||
pub evaluator: String,
|
||||
pub candidate_id: String,
|
||||
pub asserted_healthy: bool,
|
||||
}
|
||||
|
||||
impl HealthEvidenceSnapshot {
|
||||
pub fn new(evaluator: &str, candidate_id: &str, asserted_healthy: bool) -> Result<Self> {
|
||||
let snapshot = Self {
|
||||
schema_version: 1,
|
||||
evaluator: evaluator.to_string(),
|
||||
candidate_id: candidate_id.to_string(),
|
||||
asserted_healthy,
|
||||
};
|
||||
snapshot.verify()?;
|
||||
Ok(snapshot)
|
||||
}
|
||||
|
||||
pub fn canonical_json(&self) -> Result<String> {
|
||||
self.verify()?;
|
||||
Ok(serde_json::to_string(self)?)
|
||||
}
|
||||
|
||||
pub fn digest(&self) -> Result<String> {
|
||||
Ok(sha256_hex(self.canonical_json()?.as_bytes()))
|
||||
}
|
||||
|
||||
pub fn verify(&self) -> Result<()> {
|
||||
if self.schema_version != 1
|
||||
|| self.evaluator != "recorded-v1"
|
||||
|| self.candidate_id.len() != 64
|
||||
|| !self
|
||||
.candidate_id
|
||||
.bytes()
|
||||
.all(|byte| byte.is_ascii_digit() || (b'a'..=b'f').contains(&byte))
|
||||
{
|
||||
bail!("invalid canonical health evidence snapshot");
|
||||
}
|
||||
Ok(())
|
||||
}
|
||||
}
|
||||
|
||||
pub struct RecordedEvaluator {
|
||||
name: String,
|
||||
scores: BTreeMap<(String, u64), f64>,
|
||||
@@ -238,6 +405,16 @@ impl RecordedEvaluator {
|
||||
})
|
||||
}
|
||||
|
||||
pub fn health_evidence_snapshot(&self) -> Result<HealthEvidenceSnapshot> {
|
||||
HealthEvidenceSnapshot::new(
|
||||
&self.name,
|
||||
self.health_candidate
|
||||
.as_deref()
|
||||
.context("candidate-keyed health evidence is required")?,
|
||||
self.health_ok,
|
||||
)
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
pub fn calls(&self) -> &[(String, u64)] {
|
||||
&self.calls
|
||||
|
||||
+116
-29
@@ -447,25 +447,37 @@ enum Commands {
|
||||
enum HarnessEvalCommands {
|
||||
/// Record an immutable content-addressed candidate from a local JSON file
|
||||
Record {
|
||||
#[arg(long)] config: PathBuf,
|
||||
#[arg(long = "trace-ref", required = true)] trace_refs: Vec<String>,
|
||||
#[arg(long = "evidence-ref", required = true)] evidence_refs: Vec<String>,
|
||||
#[arg(long)]
|
||||
config: PathBuf,
|
||||
#[arg(long = "trace-ref", required = true)]
|
||||
trace_refs: Vec<String>,
|
||||
#[arg(long = "evidence-ref", required = true)]
|
||||
evidence_refs: Vec<String>,
|
||||
},
|
||||
/// Set the first baseline; subsequent changes require evaluation
|
||||
ActivateInitial {
|
||||
candidate_id: String,
|
||||
#[arg(long)] evidence_ref: String,
|
||||
#[arg(long)]
|
||||
evidence_ref: String,
|
||||
},
|
||||
/// Evaluate paired scores and conditionally promote with a health gate
|
||||
Run {
|
||||
#[arg(long)] candidate: String,
|
||||
#[arg(long)] baseline: String,
|
||||
#[arg(long = "seed", required = true)] seeds: Vec<u64>,
|
||||
#[arg(long)] measurements: PathBuf,
|
||||
#[arg(long)] evidence_ref: String,
|
||||
#[arg(long)] min_samples: usize,
|
||||
#[arg(long)] min_mean_delta: f64,
|
||||
#[arg(long)] min_win_rate: f64,
|
||||
#[arg(long)]
|
||||
candidate: String,
|
||||
#[arg(long)]
|
||||
baseline: String,
|
||||
#[arg(long = "seed", required = true)]
|
||||
seeds: Vec<u64>,
|
||||
#[arg(long)]
|
||||
measurements: PathBuf,
|
||||
#[arg(long)]
|
||||
evidence_ref: String,
|
||||
#[arg(long)]
|
||||
min_samples: usize,
|
||||
#[arg(long)]
|
||||
min_mean_delta: f64,
|
||||
#[arg(long)]
|
||||
min_win_rate: f64,
|
||||
},
|
||||
/// Show append-only promotion audit entries
|
||||
Audit,
|
||||
@@ -1423,30 +1435,72 @@ async fn main() -> Result<()> {
|
||||
|
||||
match cli.command {
|
||||
Some(Commands::HarnessEval { command }) => match command {
|
||||
HarnessEvalCommands::Record { config, trace_refs, evidence_refs } => {
|
||||
let value: serde_json::Value = serde_json::from_slice(&read_bounded_file(&config, 1_048_576, "candidate configuration")?)
|
||||
.with_context(|| format!("Invalid JSON in {}", config.display()))?;
|
||||
HarnessEvalCommands::Record {
|
||||
config,
|
||||
trace_refs,
|
||||
evidence_refs,
|
||||
} => {
|
||||
let value: serde_json::Value = serde_json::from_slice(&read_bounded_file(
|
||||
&config,
|
||||
1_048_576,
|
||||
"candidate configuration",
|
||||
)?)
|
||||
.with_context(|| format!("Invalid JSON in {}", config.display()))?;
|
||||
let candidate = harness_eval::CandidateSpec::new(value, trace_refs, evidence_refs)?;
|
||||
db.record_harness_candidate(&candidate)?;
|
||||
println!("{}", candidate.id);
|
||||
}
|
||||
HarnessEvalCommands::ActivateInitial { candidate_id, evidence_ref } => {
|
||||
HarnessEvalCommands::ActivateInitial {
|
||||
candidate_id,
|
||||
evidence_ref,
|
||||
} => {
|
||||
db.activate_initial_harness(&candidate_id, &evidence_ref)?;
|
||||
println!("Activated initial baseline: {candidate_id}");
|
||||
}
|
||||
HarnessEvalCommands::Run { candidate, baseline, seeds, measurements, evidence_ref, min_samples, min_mean_delta, min_win_rate } => {
|
||||
HarnessEvalCommands::Run {
|
||||
candidate,
|
||||
baseline,
|
||||
seeds,
|
||||
measurements,
|
||||
evidence_ref,
|
||||
min_samples,
|
||||
min_mean_delta,
|
||||
min_win_rate,
|
||||
} => {
|
||||
use harness_eval::Evaluator;
|
||||
let evidence: harness_eval::RecordedEvidence = serde_json::from_slice(&read_bounded_file(&measurements, 8_388_608, "recorded measurements")?)
|
||||
.with_context(|| format!("Invalid recorded evidence in {}", measurements.display()))?;
|
||||
let evidence: harness_eval::RecordedEvidence = serde_json::from_slice(
|
||||
&read_bounded_file(&measurements, 8_388_608, "recorded measurements")?,
|
||||
)
|
||||
.with_context(|| {
|
||||
format!("Invalid recorded evidence in {}", measurements.display())
|
||||
})?;
|
||||
let mut evaluator = harness_eval::RecordedEvaluator::from_evidence(evidence)?;
|
||||
let evaluator_name = evaluator.name().to_string();
|
||||
let samples = harness_eval::evaluate_paired(&mut evaluator, &candidate, &baseline, &seeds)?;
|
||||
let policy = harness_eval::PromotionPolicy { min_samples, min_mean_delta, min_win_rate };
|
||||
let outcome = db.evaluate_promote_and_health_check(&candidate, &baseline, &evaluator_name, &samples, policy, &evidence_ref, |id| evaluator.health_check(id))?;
|
||||
let health_evidence = evaluator.health_evidence_snapshot()?;
|
||||
let samples =
|
||||
harness_eval::evaluate_paired(&mut evaluator, &candidate, &baseline, &seeds)?;
|
||||
let policy = harness_eval::PromotionPolicy {
|
||||
min_samples,
|
||||
min_mean_delta,
|
||||
min_win_rate,
|
||||
};
|
||||
let outcome = db.evaluate_promote_and_health_check(
|
||||
&candidate,
|
||||
&baseline,
|
||||
&evaluator_name,
|
||||
&samples,
|
||||
policy,
|
||||
&evidence_ref,
|
||||
&health_evidence,
|
||||
|id| evaluator.health_check(id),
|
||||
)?;
|
||||
println!("{}", serde_json::to_string_pretty(&outcome)?);
|
||||
}
|
||||
HarnessEvalCommands::Audit => {
|
||||
println!("{}", serde_json::to_string_pretty(&db.harness_audit_entries()?)?);
|
||||
println!(
|
||||
"{}",
|
||||
serde_json::to_string_pretty(&db.harness_audit_entries()?)?
|
||||
);
|
||||
}
|
||||
},
|
||||
Some(Commands::Dashboard) | None => {
|
||||
@@ -8628,18 +8682,51 @@ mod tests {
|
||||
#[test]
|
||||
fn harness_eval_cli_requires_explicit_bounded_inputs() {
|
||||
let cli = Cli::try_parse_from([
|
||||
"ecc", "harness-eval", "run", "--candidate", "candidate", "--baseline", "baseline",
|
||||
"--seed", "1", "--seed", "2", "--measurements", "scores.json", "--evidence-ref",
|
||||
"evidence://run", "--min-samples", "2", "--min-mean-delta", "0.1", "--min-win-rate", "0.5",
|
||||
]).expect("valid harness evaluation command");
|
||||
"ecc",
|
||||
"harness-eval",
|
||||
"run",
|
||||
"--candidate",
|
||||
"candidate",
|
||||
"--baseline",
|
||||
"baseline",
|
||||
"--seed",
|
||||
"1",
|
||||
"--seed",
|
||||
"2",
|
||||
"--measurements",
|
||||
"scores.json",
|
||||
"--evidence-ref",
|
||||
"evidence://run",
|
||||
"--min-samples",
|
||||
"2",
|
||||
"--min-mean-delta",
|
||||
"0.1",
|
||||
"--min-win-rate",
|
||||
"0.5",
|
||||
])
|
||||
.expect("valid harness evaluation command");
|
||||
match cli.command {
|
||||
Some(Commands::HarnessEval { command: HarnessEvalCommands::Run { seeds, min_samples, .. } }) => {
|
||||
Some(Commands::HarnessEval {
|
||||
command:
|
||||
HarnessEvalCommands::Run {
|
||||
seeds, min_samples, ..
|
||||
},
|
||||
}) => {
|
||||
assert_eq!(seeds, vec![1, 2]);
|
||||
assert_eq!(min_samples, 2);
|
||||
}
|
||||
other => panic!("unexpected command: {other:?}"),
|
||||
}
|
||||
assert!(Cli::try_parse_from(["ecc", "harness-eval", "run", "--candidate", "c", "--baseline", "b"]).is_err());
|
||||
assert!(Cli::try_parse_from([
|
||||
"ecc",
|
||||
"harness-eval",
|
||||
"run",
|
||||
"--candidate",
|
||||
"c",
|
||||
"--baseline",
|
||||
"b"
|
||||
])
|
||||
.is_err());
|
||||
}
|
||||
|
||||
#[test]
|
||||
|
||||
+1058
-150
File diff suppressed because it is too large
Load Diff
@@ -8,7 +8,7 @@
|
||||
"ito-compute": {
|
||||
"command": "node",
|
||||
"args": ["/absolute/path/to/ito-cloud-runtime/cli/ito-compute-cli/dist/bin/ito-mcp.js"],
|
||||
"description": "Opt-in local Itô compute MCP. The canonical package is unpublished and must be built from Ito-Markets/ito-cloud-runtime/cli/ito-compute-cli. Exposes only ito_auth, ito_find, and ito_status. Authentication delegates to canonical CLI device authorization, which opens the Itô verification page by default and persists a device token in macOS Keychain; ecc ito auth --no-browser suppresses the CLI page handoff. ECC itself performs no browser automation. Legacy ITO_API_KEY requires explicit ITO_AUTH_MODE=legacy."
|
||||
"description": "Opt-in local Itô compute MCP. The canonical package is unpublished and must be built from Ito-Markets/ito-cloud-runtime/cli/ito-compute-cli. Exposes only ito_auth, ito_find, and ito_status. ito_auth validates existing credentials; it does not start device login. Use ecc ito login [--no-browser] for device authorization, which stores tokens in macOS Keychain by default; explicit file fallback must retain owner-only settings. ECC itself performs no browser automation. ITO_API_KEY is forwarded directly to auth, find, and status when configured; ITO_AUTH_MODE=legacy is not required."
|
||||
},
|
||||
"jira": {
|
||||
"command": "uvx",
|
||||
|
||||
+6
-2
@@ -4,7 +4,7 @@ const { spawnSync } = require('child_process');
|
||||
const path = require('path');
|
||||
const { listAvailableLanguages } = require('./lib/install-executor');
|
||||
const { getComputeSponsorCopy } = require('./lib/compute-sponsor');
|
||||
const { createSafeItoInvocationEnvironment } = require('./lib/ito-environment');
|
||||
const { createSafeItoInvocationEnvironment, getInvocationCommand } = require('./lib/ito-environment');
|
||||
|
||||
const COMMANDS = {
|
||||
install: {
|
||||
@@ -148,6 +148,7 @@ Examples:
|
||||
ecc catalog show framework:nextjs
|
||||
ecc consult "security reviews"
|
||||
ecc control-pane --port 8765
|
||||
ecc ito login [--no-browser]
|
||||
ecc ito auth
|
||||
ecc ito find --gpu h200 --count 8 --nodes 1 --gpus-per-node 8 --days 30 --storage-tb 1 --start-window 2099-08-15 --max-rate 3.00 --form-factor bare_metal --contract-type reservation --fabric infiniband --region us-east-1
|
||||
ecc ito status --json
|
||||
@@ -241,6 +242,7 @@ function runCommand(commandName, args) {
|
||||
if (!command) {
|
||||
throw new Error(`Unknown command: ${commandName}`);
|
||||
}
|
||||
const isItoLogin = commandName === 'ito' && getInvocationCommand(args) === 'login';
|
||||
const result = spawnSync(
|
||||
process.execPath,
|
||||
[path.join(__dirname, command.script), ...args],
|
||||
@@ -253,7 +255,9 @@ function runCommand(commandName, args) {
|
||||
}),
|
||||
}
|
||||
: process.env,
|
||||
stdio: commandName === 'memory'
|
||||
stdio: isItoLogin
|
||||
? 'inherit'
|
||||
: commandName === 'memory'
|
||||
? ['inherit', 'pipe', 'pipe']
|
||||
: ['pipe', 'pipe', 'pipe'],
|
||||
encoding: 'utf8',
|
||||
|
||||
+17
-8
@@ -10,7 +10,7 @@ const {
|
||||
getInvocationCommand,
|
||||
} = require("./lib/ito-environment");
|
||||
|
||||
const SUPPORTED_COMMANDS = Object.freeze(["auth", "find", "status", "evals"]);
|
||||
const SUPPORTED_COMMANDS = Object.freeze(["login", "auth", "find", "status", "evals"]);
|
||||
const CANONICAL_REPOSITORY = "https://github.com/Ito-Markets/ito-cloud-runtime.git";
|
||||
const CANONICAL_PACKAGE_PATH = "cli/ito-compute-cli";
|
||||
const CANONICAL_ENTRY_SEGMENTS = Object.freeze([
|
||||
@@ -28,18 +28,20 @@ function showHelp() {
|
||||
ECC × Itô local CLI bridge
|
||||
|
||||
Usage:
|
||||
ecc ito auth [--no-browser]
|
||||
ecc ito login [--no-browser]
|
||||
ecc ito auth
|
||||
ecc ito find <all required RFQ options>
|
||||
ecc ito status
|
||||
ecc ito evals --cluster <id> --live-sixtytwo --nodes <list> --config-dir <dir>
|
||||
ecc ito <auth|find|status|evals> --json
|
||||
ecc ito <login|auth|find|status|evals> --json
|
||||
|
||||
The bridge invokes the separately installed canonical Itô CLI and returns its
|
||||
real stdout, stderr, and exit code unchanged. "ecc ito auth" delegates to the
|
||||
real stdout, stderr, and exit code unchanged. "ecc ito login" delegates to the
|
||||
canonical CLI's device authorization. It opens the Itô verification page by default
|
||||
and persists its device token in macOS Keychain. Pass --no-browser to
|
||||
suppress that handoff. ECC itself performs no browser automation and adds no
|
||||
lock, workload, inference, or purchase path.
|
||||
"ecc ito auth" is validation-only and never starts device login.
|
||||
|
||||
Important:
|
||||
- "find" reads live inventory and submits an authenticated RFQ.
|
||||
@@ -70,9 +72,11 @@ The same package's MCP server exposes only:
|
||||
Configure the MCP command as "node" with this absolute argument:
|
||||
/absolute/path/to/ito-cloud-runtime/${CANONICAL_PACKAGE_PATH}/dist/bin/ito-mcp.js
|
||||
|
||||
Device authorization is the default. Legacy ITO_API_KEY authentication is
|
||||
forwarded only with explicit ITO_AUTH_MODE=legacy. Never put a key or token in
|
||||
arguments, tracked files, or chat.
|
||||
Device login never inherits ITO_API_KEY. The auth, find, and status commands
|
||||
forward ITO_API_KEY directly when configured; ITO_AUTH_MODE=legacy is not
|
||||
required. The canonical client stores device credentials in macOS Keychain by
|
||||
default; file-token fallback remains explicit and must use restrictive settings.
|
||||
Never put a key or token in arguments, tracked files, or chat.
|
||||
|
||||
Live node qualification requires ITO_ENABLE_SIXTYTWO_LIVE=1,
|
||||
--live-sixtytwo, an explicit node list, and an existing absolute config
|
||||
@@ -157,9 +161,12 @@ function parseArgs(argv, environment = process.env) {
|
||||
const command = withoutJson.shift();
|
||||
if (!SUPPORTED_COMMANDS.includes(command)) {
|
||||
throw new Error(
|
||||
`Unsupported Itô command "${command || "(missing)"}"; ECC permits only auth, find, status, and evals.`
|
||||
`Unsupported Itô command "${command || "(missing)"}"; ECC permits only login, auth, find, status, and evals.`
|
||||
);
|
||||
}
|
||||
if (command === "auth" && withoutJson.includes("--no-browser")) {
|
||||
throw new Error("--no-browser is valid only for ecc ito login; auth is validation-only.");
|
||||
}
|
||||
if (command === "evals") {
|
||||
validateNodeQualificationArgs(withoutJson, environment);
|
||||
}
|
||||
@@ -258,12 +265,14 @@ function invokeIto(executable, args, environment = process.env) {
|
||||
const invocation = buildInvocation(executable, args);
|
||||
const command = getInvocationCommand(args);
|
||||
const isNodeQualification = command === "evals";
|
||||
const isDeviceLogin = command === "login";
|
||||
const result = spawnSync(invocation.executable, invocation.args, {
|
||||
cwd: process.cwd(),
|
||||
encoding: "utf8",
|
||||
// Keep policy helpers immutable for callers, but give child-process
|
||||
// instrumentation its own mutable copy (for example NODE_V8_COVERAGE).
|
||||
env: { ...createSafeItoInvocationEnvironment(environment, args) },
|
||||
stdio: isDeviceLogin ? "inherit" : ["pipe", "pipe", "pipe"],
|
||||
maxBuffer: MAX_OUTPUT_BYTES,
|
||||
timeout: isNodeQualification ? NODE_QUALIFICATION_TIMEOUT_MS : undefined,
|
||||
shell: false,
|
||||
|
||||
@@ -45,7 +45,7 @@ const ECC_ITO_CONTROL_KEYS = Object.freeze([
|
||||
"ECC_ITO_CLI_EXECUTABLE",
|
||||
"NODE_ENV",
|
||||
]);
|
||||
const ITO_RUNTIME_COMMANDS = new Set(["auth", "find", "status"]);
|
||||
const ITO_RUNTIME_COMMANDS = new Set(["login", "auth", "find", "status"]);
|
||||
|
||||
function copyDefined(source, target, key) {
|
||||
if (typeof source[key] === "string") {
|
||||
@@ -64,7 +64,7 @@ function createSafeItoEnvironment(source = process.env, options = {}) {
|
||||
|
||||
if (options.includeItoRuntime) {
|
||||
for (const key of ITO_RUNTIME_ENVIRONMENT_KEYS) {
|
||||
if (key === "ITO_API_KEY" && source.ITO_AUTH_MODE !== "legacy") continue;
|
||||
if (key === "ITO_API_KEY" && options.includeItoApiKey !== true) continue;
|
||||
copyDefined(source, safe, key);
|
||||
}
|
||||
}
|
||||
@@ -97,6 +97,7 @@ function createSafeItoInvocationEnvironment(
|
||||
return createSafeItoEnvironment(source, {
|
||||
includeControls: options.includeControls === true,
|
||||
includeItoRuntime: ITO_RUNTIME_COMMANDS.has(command),
|
||||
includeItoApiKey: ["auth", "find", "status"].includes(command),
|
||||
includeItoEvals: command === "evals",
|
||||
});
|
||||
}
|
||||
|
||||
@@ -30,25 +30,30 @@ Set `ECC_ITO_CLI_EXECUTABLE` to the explicit absolute built entry:
|
||||
```
|
||||
|
||||
ECC never discovers this credential-bearing client through `PATH`.
|
||||
Device authorization is the default. Keep legacy `ITO_API_KEY` only with
|
||||
explicit `ITO_AUTH_MODE=legacy`; never put it in arguments, tracked files, MCP
|
||||
results, logs, or chat.
|
||||
`ecc ito login` performs device authorization and never inherits `ITO_API_KEY`.
|
||||
The validation-only `auth`, plus `find` and `status`, forward `ITO_API_KEY`
|
||||
directly when configured; `ITO_AUTH_MODE=legacy` is not required. Never put a
|
||||
key or token in arguments, tracked files, MCP results, logs, or chat.
|
||||
|
||||
## CLI workflow
|
||||
|
||||
1. Run `ecc ito auth` before the first operation. ECC delegates this to the
|
||||
1. Run `ecc ito login` before the first operation. ECC delegates this to the
|
||||
canonical CLI's device authorization, which opens the Itô verification page
|
||||
by default and persists a device token in macOS Keychain. Use
|
||||
`ecc ito auth --no-browser` to suppress the page handoff. ECC itself does no
|
||||
`ecc ito login --no-browser` to suppress the page handoff. ECC itself does no
|
||||
browser automation.
|
||||
2. Before `ecc ito find`, obtain explicit buyer authority to submit an RFQ.
|
||||
Device tokens use macOS Keychain by default. File-token fallback is explicit
|
||||
and its directory and token file must remain owner-only (0700 and 0600).
|
||||
2. Run `ecc ito auth` to validate existing credentials; it never starts login
|
||||
and rejects `--no-browser`.
|
||||
3. Before `ecc ito find`, obtain explicit buyer authority to submit an RFQ.
|
||||
- Require `gpu`, `count`, whole `days`, `max-rate`, `nodes`,
|
||||
`gpus-per-node`, `storage-tb`, `start-window`, `form-factor`,
|
||||
`contract-type`, `fabric`, `region`, and the split-fill decision.
|
||||
- Require `count == nodes * gpus-per-node`; never derive topology.
|
||||
- Use `any` only when the buyer explicitly accepts any fabric or region.
|
||||
- Omitted `--allow-split` means false.
|
||||
3. Run the live RFQ command:
|
||||
4. Run the live RFQ command:
|
||||
|
||||
```sh
|
||||
ecc ito find \
|
||||
@@ -66,7 +71,7 @@ results, logs, or chat.
|
||||
--region us-east-1
|
||||
```
|
||||
|
||||
4. Run `ecc ito status` to inspect RFQs and procurement orders.
|
||||
5. Run `ecc ito status` to inspect RFQs and procurement orders.
|
||||
After an ambiguous transport failure, check status before repeating `find`.
|
||||
|
||||
Inventory prices are indicative. An RFQ is not reserved capacity. Treat a rate
|
||||
@@ -122,7 +127,8 @@ The server exposes only:
|
||||
- `ito_find`
|
||||
- `ito_status`
|
||||
|
||||
Use `ito_auth`, gather explicit buyer authority and every hard constraint, call
|
||||
`ito_auth` validates existing credentials; it does not start device login. Use
|
||||
`ito_auth`, gather explicit buyer authority and every hard constraint, call
|
||||
`ito_find`, then poll with `ito_status` when needed.
|
||||
|
||||
## Unsupported operations
|
||||
|
||||
@@ -35,6 +35,7 @@ function main() {
|
||||
["documents only the real CLI commands and MCP tools", () => {
|
||||
const skill = read("skills/ito-compute/SKILL.md");
|
||||
for (const command of [
|
||||
"ecc ito login",
|
||||
"ecc ito auth",
|
||||
"ecc ito find",
|
||||
"ecc ito status",
|
||||
@@ -57,10 +58,12 @@ function main() {
|
||||
assert.match(skill, /ECC_ITO_CLI_EXECUTABLE/);
|
||||
assert.match(skill, /explicit absolute built entry/);
|
||||
assert.match(skill, /never discovers[^\n]*through `PATH`/);
|
||||
assert.match(skill, /opens the Itô verification page\s+by default/i);
|
||||
assert.match(skill, /ecc ito login --no-browser/);
|
||||
assert.match(skill, /auth.*validat/i);
|
||||
assert.match(skill, /--no-browser/);
|
||||
assert.match(skill, /macOS Keychain/i);
|
||||
assert.match(skill, /ITO_AUTH_MODE=legacy/);
|
||||
assert.match(skill, /(?:auth|find|status).*ITO_API_KEY/i);
|
||||
assert.match(skill, /ITO_AUTH_MODE=legacy[^.]*not required/i);
|
||||
assert.match(skill, /ECC (?:itself )?(?:does|performs) no browser automation/i);
|
||||
assert.match(skill, /ITO_ENABLE_SIXTYTWO_LIVE/);
|
||||
assert.match(skill, /sixtytwo-cli==0\.3\.33/);
|
||||
@@ -68,6 +71,21 @@ function main() {
|
||||
assert.match(skill, /cannot (?:rent|launch|recover|repair)/i);
|
||||
assert.doesNotMatch(skill, /npm link/);
|
||||
}],
|
||||
["keeps README and integration docs aligned with the separated auth contract", () => {
|
||||
for (const relativePath of [
|
||||
"README.md",
|
||||
"docs/design/ecc-ito-compute-integration.md",
|
||||
]) {
|
||||
const source = read(relativePath);
|
||||
assert.match(source, /ecc ito login \[?--no-browser\]?/i, relativePath);
|
||||
assert.match(source, /ecc ito auth/i, relativePath);
|
||||
assert.match(source, /auth.*validat/i, relativePath);
|
||||
assert.match(source, /login.*(?:Keychain|device authorization)/is, relativePath);
|
||||
assert.doesNotMatch(source, /ecc ito auth --no-browser/i, relativePath);
|
||||
assert.match(source, /ITO_API_KEY.*(?:auth|find|status)/is, relativePath);
|
||||
assert.match(source, /ITO_AUTH_MODE=legacy[^.]*not required/i, relativePath);
|
||||
}
|
||||
}],
|
||||
["registers one opt-in install module and capability", () => {
|
||||
const modules = readJson("manifests/install-modules.json").modules;
|
||||
const module = modules.find((candidate) => candidate.id === "ito-compute");
|
||||
@@ -111,7 +129,7 @@ function main() {
|
||||
assert.doesNotMatch(JSON.stringify(server), /npx|ito_lock|ito_run|paper|simulat/i);
|
||||
assert.match(server.description, /ito_auth, ito_find, and ito_status/);
|
||||
assert.match(server.description, /unpublished/i);
|
||||
assert.match(server.description, /device authorization/i);
|
||||
assert.match(server.description, /ito_auth.*validat/i);
|
||||
assert.match(server.description, /macOS Keychain/i);
|
||||
assert.match(server.description, /no browser automation/i);
|
||||
}],
|
||||
|
||||
@@ -9,7 +9,7 @@ const assert = require("assert");
|
||||
const fs = require("fs");
|
||||
const os = require("os");
|
||||
const path = require("path");
|
||||
const { spawnSync } = require("child_process");
|
||||
const { spawn, spawnSync } = require("child_process");
|
||||
|
||||
const REPO_ROOT = path.join(__dirname, "..", "..");
|
||||
const ECC_SCRIPT = path.join(REPO_ROOT, "scripts", "ecc.js");
|
||||
@@ -36,6 +36,34 @@ function runCli(args, environment = {}) {
|
||||
});
|
||||
}
|
||||
|
||||
function runCliAndObserveFirstOutput(args, environment = {}) {
|
||||
return new Promise((resolve, reject) => {
|
||||
const child = spawn(process.execPath, [ECC_SCRIPT, ...args], {
|
||||
cwd: REPO_ROOT,
|
||||
env: { ...process.env, NODE_ENV: "test", ...environment },
|
||||
stdio: ["ignore", "pipe", "pipe"],
|
||||
});
|
||||
let stdout = "";
|
||||
let stderr = "";
|
||||
let firstOutputAt;
|
||||
const startedAt = Date.now();
|
||||
child.stdout.on("data", (chunk) => {
|
||||
if (firstOutputAt === undefined) firstOutputAt = Date.now();
|
||||
stdout += chunk;
|
||||
});
|
||||
child.stderr.on("data", (chunk) => { stderr += chunk; });
|
||||
child.once("error", reject);
|
||||
child.once("close", (status) => resolve({
|
||||
status,
|
||||
stdout,
|
||||
stderr,
|
||||
startedAt,
|
||||
firstOutputAt,
|
||||
closedAt: Date.now(),
|
||||
}));
|
||||
});
|
||||
}
|
||||
|
||||
function makeItoProbe(exitCode = 0) {
|
||||
const directory = fs.mkdtempSync(path.join(os.tmpdir(), "ecc-ito-cli-"));
|
||||
const log = path.join(directory, "invocation.json");
|
||||
@@ -73,9 +101,9 @@ function readInvocation(probe) {
|
||||
return JSON.parse(fs.readFileSync(probe.log, "utf8"));
|
||||
}
|
||||
|
||||
function runTest(name, fn) {
|
||||
async function runTest(name, fn) {
|
||||
try {
|
||||
fn();
|
||||
await fn();
|
||||
console.log(` ✓ ${name}`);
|
||||
return true;
|
||||
} catch (error) {
|
||||
@@ -85,12 +113,12 @@ function runTest(name, fn) {
|
||||
}
|
||||
}
|
||||
|
||||
function main() {
|
||||
async function main() {
|
||||
console.log("\n=== Testing ECC × Itô real CLI bridge ===\n");
|
||||
|
||||
const tests = [
|
||||
["forwards only the reviewed RFQ CLI surface to an explicit local executable", () => {
|
||||
for (const command of ["auth", "find", "status"]) {
|
||||
for (const command of ["login", "auth", "find", "status"]) {
|
||||
const probe = makeItoProbe();
|
||||
try {
|
||||
const result = runCli(["ito", command], {
|
||||
@@ -104,14 +132,27 @@ function main() {
|
||||
}
|
||||
}
|
||||
}],
|
||||
["forwards the canonical auth browser opt-out without performing browser automation", () => {
|
||||
["forwards the canonical login browser opt-out without performing browser automation", () => {
|
||||
const probe = makeItoProbe();
|
||||
try {
|
||||
const result = runCli(["ito", "login", "--no-browser"], {
|
||||
ECC_ITO_CLI_EXECUTABLE: probe.executable,
|
||||
});
|
||||
assert.strictEqual(result.status, 0, result.stderr);
|
||||
assert.deepStrictEqual(readInvocation(probe).argv, ["login", "--no-browser"]);
|
||||
} finally {
|
||||
fs.rmSync(probe.directory, { recursive: true, force: true });
|
||||
}
|
||||
}],
|
||||
["rejects --no-browser on validation-only auth before spawning", () => {
|
||||
const probe = makeItoProbe();
|
||||
try {
|
||||
const result = runCli(["ito", "auth", "--no-browser"], {
|
||||
ECC_ITO_CLI_EXECUTABLE: probe.executable,
|
||||
});
|
||||
assert.strictEqual(result.status, 0, result.stderr);
|
||||
assert.deepStrictEqual(readInvocation(probe).argv, ["auth", "--no-browser"]);
|
||||
assert.notStrictEqual(result.status, 0);
|
||||
assert.match(result.stderr, /--no-browser.*only.*login/i);
|
||||
assert.ok(!fs.existsSync(probe.log));
|
||||
} finally {
|
||||
fs.rmSync(probe.directory, { recursive: true, force: true });
|
||||
}
|
||||
@@ -148,10 +189,10 @@ function main() {
|
||||
fs.rmSync(probe.directory, { recursive: true, force: true });
|
||||
}
|
||||
}],
|
||||
["passes only the required device-auth runtime settings across the process boundary", () => {
|
||||
["login never inherits ITO_API_KEY but preserves secure token settings", () => {
|
||||
const probe = makeItoProbe();
|
||||
try {
|
||||
const result = runCli(["ito", "auth"], {
|
||||
const result = runCli(["ito", "login"], {
|
||||
ECC_ITO_CLI_EXECUTABLE: probe.executable,
|
||||
ITO_API_KEY: "must-not-cross-without-legacy-mode",
|
||||
ITO_AUTH_MODE: "device",
|
||||
@@ -179,26 +220,46 @@ function main() {
|
||||
fs.rmSync(probe.directory, { recursive: true, force: true });
|
||||
}
|
||||
}],
|
||||
["forwards the legacy API key only with explicit legacy auth mode", () => {
|
||||
for (const [mode, expectedKey] of [
|
||||
[undefined, undefined],
|
||||
["device", undefined],
|
||||
["legacy", "ito_test_key"],
|
||||
]) {
|
||||
["forwards ITO_API_KEY directly to auth, find, and status without legacy mode", () => {
|
||||
for (const command of ["auth", "find", "status"]) {
|
||||
const probe = makeItoProbe();
|
||||
try {
|
||||
const result = runCli(["ito", "status"], {
|
||||
const result = runCli(["ito", command], {
|
||||
ECC_ITO_CLI_EXECUTABLE: probe.executable,
|
||||
ITO_API_KEY: "ito_test_key",
|
||||
...(mode ? { ITO_AUTH_MODE: mode } : {}),
|
||||
});
|
||||
assert.strictEqual(result.status, 0, result.stderr);
|
||||
assert.strictEqual(readInvocation(probe).env.ITO_API_KEY, expectedKey);
|
||||
assert.strictEqual(readInvocation(probe).env.ITO_API_KEY, "ito_test_key");
|
||||
} finally {
|
||||
fs.rmSync(probe.directory, { recursive: true, force: true });
|
||||
}
|
||||
}
|
||||
}],
|
||||
["streams device login output before completion and propagates its exit status", async () => {
|
||||
const probe = makeItoProbe(7);
|
||||
try {
|
||||
fs.writeFileSync(
|
||||
probe.executable,
|
||||
[
|
||||
'"use strict";',
|
||||
'process.stdout.write("device-code-now\\n");',
|
||||
'setTimeout(() => process.exit(7), 500);',
|
||||
"",
|
||||
].join("\n")
|
||||
);
|
||||
const result = await runCliAndObserveFirstOutput(["ito", "login"], {
|
||||
ECC_ITO_CLI_EXECUTABLE: probe.executable,
|
||||
});
|
||||
assert.strictEqual(result.status, 7, result.stderr);
|
||||
assert.match(result.stdout, /device-code-now/);
|
||||
assert.ok(
|
||||
result.closedAt - result.firstOutputAt >= 350,
|
||||
"login output was buffered until process completion",
|
||||
);
|
||||
} finally {
|
||||
fs.rmSync(probe.directory, { recursive: true, force: true });
|
||||
}
|
||||
}],
|
||||
["isolates live node qualification from Itô and unrelated credentials", () => {
|
||||
const probe = makeItoProbe();
|
||||
try {
|
||||
@@ -398,7 +459,7 @@ function main() {
|
||||
ECC_ITO_CLI_EXECUTABLE: probe.executable,
|
||||
});
|
||||
assert.notStrictEqual(result.status, 0, command);
|
||||
assert.match(result.stderr, /only auth, find, status, and evals/i);
|
||||
assert.match(result.stderr, /only login, auth, find, status, and evals/i);
|
||||
assert.ok(!fs.existsSync(probe.log), `${command} must not spawn the Itô CLI`);
|
||||
} finally {
|
||||
fs.rmSync(probe.directory, { recursive: true, force: true });
|
||||
@@ -563,14 +624,15 @@ function main() {
|
||||
fs.rmSync(probe.directory, { recursive: true, force: true });
|
||||
}
|
||||
}],
|
||||
["help exposes canonical device auth without claiming ECC browser automation", () => {
|
||||
["help separates device login from auth validation", () => {
|
||||
const probe = makeItoProbe();
|
||||
try {
|
||||
const result = runCli(["ito", "--help"], {
|
||||
ECC_ITO_CLI_EXECUTABLE: probe.executable,
|
||||
});
|
||||
assert.strictEqual(result.status, 0, result.stderr);
|
||||
assert.match(result.stdout, /ecc ito auth \[--no-browser\]/);
|
||||
assert.match(result.stdout, /ecc ito login \[--no-browser\]/);
|
||||
assert.match(result.stdout, /ecc ito auth/);
|
||||
assert.match(result.stdout, /ecc ito find/);
|
||||
assert.match(result.stdout, /ecc ito status/);
|
||||
assert.match(result.stdout, /ecc ito evals/);
|
||||
@@ -585,7 +647,8 @@ function main() {
|
||||
assert.match(result.stdout, /opens the Itô verification page by default/i);
|
||||
assert.match(result.stdout, /macOS Keychain/i);
|
||||
assert.match(result.stdout, /ECC itself performs no browser automation/i);
|
||||
assert.match(result.stdout, /ITO_AUTH_MODE=legacy/);
|
||||
assert.match(result.stdout, /auth.*validat/i);
|
||||
assert.match(result.stdout, /ITO_AUTH_MODE=legacy is not\s+required/i);
|
||||
assert.doesNotMatch(
|
||||
result.stdout,
|
||||
/manual copy|ito_lock|ito_run|npm link|paper|simulat/i
|
||||
@@ -600,7 +663,7 @@ function main() {
|
||||
let passed = 0;
|
||||
let failed = 0;
|
||||
for (const [name, fn] of tests) {
|
||||
if (runTest(name, fn)) passed += 1;
|
||||
if (await runTest(name, fn)) passed += 1;
|
||||
else failed += 1;
|
||||
}
|
||||
|
||||
|
||||
Reference in New Issue
Block a user