mirror of
https://github.com/affaan-m/ECC.git
synced 2026-09-13 05:07:53 +02:00
Validate unified memory evidence across local CLI and MCP handoffs (#3027)
* test(memory): add cross-harness conformance example * fix: update js-yaml to patched 4.3.2 * feat(memory): verify recalled evidence against scoped source catalog * fix: preserve control-character validation under lint
This commit is contained in:
@@ -0,0 +1,115 @@
|
||||
# Cross-harness memory conformance example
|
||||
|
||||
Run the existing ECC CLI and local stdio MCP server against one disposable
|
||||
synthetic vault. The example checks that the same scoped query returns the same
|
||||
ordered records, scores, excerpts, and provenance for each configured identity.
|
||||
|
||||
From an ECC checkout with its runtime dependencies already available:
|
||||
|
||||
```sh
|
||||
node examples/unified-memory/conformance.cjs
|
||||
```
|
||||
|
||||
No model, network, Graphiti service, package installation, or native harness
|
||||
application is required. The example uses the existing Ajv dependency. It
|
||||
creates temporary synthetic project, team, and user records, starts bounded
|
||||
Node subprocesses, and removes the temporary vaults when finished. Existing
|
||||
vault locations and ambient credential variables are not passed to children.
|
||||
|
||||
## What runs
|
||||
|
||||
The CLI creates a shared project record, team context, a Codex-targeted record,
|
||||
a user record, and another project's record. Separate MCP processes configured
|
||||
as `codex`, `claude`, and `hermes` each perform the same requests. These names
|
||||
are host configuration in the example, not authenticated sessions in those
|
||||
applications.
|
||||
|
||||
The 24 checks cover:
|
||||
|
||||
- Ordered CLI/MCP search parity and reproducibility after process restart.
|
||||
- Stable IDs, scope, source attribution, timestamps, body, and unreviewed trust.
|
||||
- Targeted read visibility and separate project roots.
|
||||
- Rejection of client identity overrides, target-filter overrides, trust
|
||||
promotion, and user access without host opt-in.
|
||||
- Server-stamped Hermes handoff attribution, preserved memory links, and evidence
|
||||
verification in both CLI-to-MCP and MCP-to-CLI directions.
|
||||
- Source-content matching against a separate synthetic source catalog, with
|
||||
tampered content/digest, missing-source and foreign-context rejection.
|
||||
- Synthetic private-key marker rejection through CLI and MCP without changing
|
||||
the recalled dataset.
|
||||
- Explicit user-scope recall after operator opt-in.
|
||||
- Failed startup when the host provides no identity.
|
||||
- Source files and Git HEAD unchanged after execution.
|
||||
|
||||
Success prints a JSON receipt with individual checks, timestamps, Node version,
|
||||
source hashes, and the example's digest. Failure returns a nonzero exit status
|
||||
without printing raw subprocess output or memory content. The source hashes
|
||||
identify the executed files; Git HEAD alone does not prove that a checkout is
|
||||
clean. Installed dependencies are reused and are not digest-pinned by this
|
||||
example. This is focused conformance verification, not a full-suite result or
|
||||
a deployment receipt. The source receipt includes the example verifier digest;
|
||||
dependency identity and native-harness integration remain separate checks.
|
||||
|
||||
## Contract and auth boundary
|
||||
|
||||
The example reuses `ecc.memory.v1` without adding fields. Project and team are
|
||||
the default scopes; user recall requires an explicit request and MCP host
|
||||
opt-in. The host pins `ECC_MEMORY_HARNESS`; clients cannot supply their own
|
||||
source identity or target filter through tool arguments. All writes remain
|
||||
`unreviewed` context subordinate to current instructions.
|
||||
|
||||
The fixture body uses `ecc.memory.example-evidence.v1`, an **example-local**
|
||||
JSON envelope inside the existing Markdown body. No fields are added to
|
||||
`ecc.memory.v1`. `evidence.cjs` checks a source reference, content digest,
|
||||
observation time, session ID and checkpoint ID against an independent,
|
||||
host-owned in-memory catalog. The envelope text must equal the catalog's exact
|
||||
source bytes. There is no summary/derivation validation in this example.
|
||||
|
||||
The verifier requires an exact workspace and scope match. Context is supplied
|
||||
by the example host using the selected vault and returned memory scope; it is
|
||||
not accepted from claims in the envelope. Only bounded `fixture:` identifiers
|
||||
are supported, with no path/URL lookup, filesystem read, network fallback or
|
||||
ambient source discovery. Missing evidence fails explicitly. Success returns
|
||||
`source-content-match`, never a trust promotion. The original observation time
|
||||
is compared to the catalog, not treated as proof of current factual validity.
|
||||
|
||||
This verifies integrity relative to the host's catalog, not signed authorship,
|
||||
identity authentication, an immutable journal or statement truth. An operator
|
||||
who rewrites both catalog and memory can create another matching pair. The
|
||||
catalog is synthetic, process-local and not a durable archive; references do
|
||||
not promise continued source availability. The verifier does not execute
|
||||
memory text or make it authoritative. All vault records remain `unreviewed`.
|
||||
|
||||
Run the pure in-memory negative and boundary checks separately:
|
||||
|
||||
```sh
|
||||
node examples/unified-memory/evidence.test.cjs
|
||||
```
|
||||
|
||||
These checks cover changed text, recomputed/altered digests, altered timestamps,
|
||||
session/checkpoint substitutions, missing sources, workspace/scope mismatches,
|
||||
unknown fields/schema, malformed/oversized envelopes and invalid host inputs.
|
||||
They start no server and require only Node built-ins. The conformance runner
|
||||
also saves two deliberately altered synthetic envelopes: core storage accepts
|
||||
unreviewed context, while this example's verifier rejects those recalled bodies.
|
||||
The verifier is not automatically enabled in core CLI/MCP save or recall paths.
|
||||
|
||||
The private-key rejection fixture is a deliberately incomplete marker containing
|
||||
no key material. It exercises the existing best-effort secret scanner, not a
|
||||
complete privacy classifier or permission system. Never substitute private
|
||||
transcripts, credentials or production records into the public example.
|
||||
|
||||
`targetHarnesses` constrains MCP routing, not same-user filesystem access. The
|
||||
CLI is an operator interface: direct CLI reads can access a targeted record
|
||||
without a harness target filter, and the CLI can choose source attribution.
|
||||
Separate OS accounts or equivalent filesystem isolation are necessary when
|
||||
local processes are mutually untrusted.
|
||||
|
||||
The example provides no unified OAuth, delegated credential lifecycle, plan
|
||||
token routing, cross-machine synchronization, Graphiti partition policy, or
|
||||
Hermes MemoryProvider integration. A future backend adapter must preserve the
|
||||
existing record contract and enforce its authenticated partition policy
|
||||
separately from routing metadata.
|
||||
|
||||
See [the memory vault design](../../docs/design/ecc-memory-vault.md) for the
|
||||
canonical storage and threat contract.
|
||||
@@ -0,0 +1,246 @@
|
||||
'use strict';
|
||||
|
||||
// Runs existing ECC code against disposable synthetic vaults. No service or SDK installs.
|
||||
const assert = require('node:assert/strict');
|
||||
const fs = require('node:fs');
|
||||
const os = require('node:os');
|
||||
const path = require('node:path');
|
||||
const crypto = require('node:crypto');
|
||||
const { spawnSync } = require('node:child_process');
|
||||
const { encodeEvidence, verifyEvidence } = require('./evidence.cjs');
|
||||
|
||||
const repo = path.resolve(__dirname, '../..');
|
||||
const sha256 = bytes => crypto.createHash('sha256').update(bytes).digest('hex');
|
||||
const cleanEnv = { PATH: process.env.PATH || '/usr/bin:/bin' };
|
||||
// Use the already installed Ajv; no package manager or network operation occurs.
|
||||
let dependencyRoot;
|
||||
try {
|
||||
dependencyRoot = path.dirname(path.dirname(require.resolve('ajv/package.json')));
|
||||
} catch {
|
||||
process.stderr.write('ECC memory example requires the existing Ajv runtime dependency.\n');
|
||||
process.exit(1);
|
||||
}
|
||||
const sourcePaths = [
|
||||
'scripts/memory.js', 'scripts/memory-mcp.mjs', 'scripts/lib/memory-vault.js',
|
||||
'scripts/lib/memory-vault-format.js', 'scripts/lib/path-safety.js',
|
||||
'scripts/lib/missing-dependency.js', 'schemas/memory.schema.json', 'package.json',
|
||||
'examples/unified-memory/evidence.cjs',
|
||||
];
|
||||
function snapshot() {
|
||||
return Object.fromEntries(sourcePaths.map(file => [file, sha256(fs.readFileSync(path.join(repo, file)))]));
|
||||
}
|
||||
function sourceHead() {
|
||||
const result = spawnSync('git', ['-C', repo, 'rev-parse', 'HEAD'], {
|
||||
encoding: 'utf8', env: cleanEnv, timeout: 5000, maxBuffer: 1024,
|
||||
});
|
||||
return result.status === 0 && /^[a-f0-9]{40}\s*$/.test(result.stdout) ? result.stdout.trim() : null;
|
||||
}
|
||||
const before = snapshot();
|
||||
const headBefore = sourceHead();
|
||||
const root = fs.mkdtempSync(path.join(os.tmpdir(), 'ecc-memory-conformance-'));
|
||||
const checks = [];
|
||||
const startedAt = new Date().toISOString();
|
||||
function envFor(partition = 'alpha', harness = 'codex', allowUser = false) {
|
||||
const cwd = path.join(root, partition);
|
||||
fs.mkdirSync(cwd, { recursive: true });
|
||||
return { cwd, env: { ...cleanEnv,
|
||||
NODE_PATH: dependencyRoot,
|
||||
ECC_MEMORY_PROJECT_ROOT: path.join(cwd, 'vault'),
|
||||
ECC_MEMORY_USER_ROOT: path.join(root, 'synthetic-user'),
|
||||
...(harness ? { ECC_MEMORY_HARNESS: harness } : {}),
|
||||
ECC_MEMORY_ALLOW_USER_SCOPE: allowUser ? '1' : '0',
|
||||
} };
|
||||
}
|
||||
function run(script, args, input, options) {
|
||||
return spawnSync(process.execPath, [path.join(repo, script), ...args], {
|
||||
...options, input, encoding: 'utf8', timeout: 10000, maxBuffer: 2 * 1024 * 1024,
|
||||
});
|
||||
}
|
||||
function cli(args, input = '', partition = 'alpha') {
|
||||
const result = run('scripts/memory.js', [...args, '--json'], input, envFor(partition));
|
||||
assert.equal(result.status, 0, 'Synthetic CLI operation failed; raw output withheld');
|
||||
return JSON.parse(result.stdout);
|
||||
}
|
||||
function mcp(harness, calls, partition = 'alpha', allowUser = false) {
|
||||
const frames = [
|
||||
{ jsonrpc: '2.0', id: 1, method: 'initialize', params: {
|
||||
protocolVersion: '2025-11-25', capabilities: {},
|
||||
clientInfo: { name: 'ecc-lane-conformance', version: '1.0.0' },
|
||||
} },
|
||||
{ jsonrpc: '2.0', method: 'notifications/initialized', params: {} },
|
||||
...calls.map(([name, args], index) => ({ jsonrpc: '2.0', id: index + 2,
|
||||
method: 'tools/call', params: { name, arguments: args } })),
|
||||
];
|
||||
const result = run('scripts/memory-mcp.mjs', [],
|
||||
frames.map(frame => JSON.stringify(frame)).join('\n') + '\n', envFor(partition, harness, allowUser));
|
||||
assert.equal(result.status, 0, 'Synthetic MCP process failed; raw output withheld');
|
||||
const responses = result.stdout.trim().split('\n').map(line => JSON.parse(line));
|
||||
assert.equal(responses.length, calls.length + 1, 'Missing or extra MCP response');
|
||||
assert.equal(responses[0].result.protocolVersion, '2025-11-25');
|
||||
return calls.map((_, index) => {
|
||||
const response = responses.find(item => item.id === index + 2);
|
||||
assert.ok(response, 'Missing correlated MCP response');
|
||||
return response;
|
||||
});
|
||||
}
|
||||
function payload(response) {
|
||||
assert.equal(response.error, undefined, 'Unexpected JSON-RPC error');
|
||||
assert.notEqual(response.result.isError, true, 'Unexpected tool rejection');
|
||||
return JSON.parse(response.result.content.find(item => item.type === 'text').text);
|
||||
}
|
||||
function check(name, fn) { fn(); checks.push({ name, passed: true }); }
|
||||
function save(title, scope = 'project', target = 'all', partition = 'alpha', body = 'Synthetic orbit evidence.') {
|
||||
return cli(['save', '--title', title, '--scope', scope, '--source-harness', 'codex',
|
||||
'--target', target, '--stdin'], body, partition).memory;
|
||||
}
|
||||
|
||||
try {
|
||||
const sourceText = 'Synthetic fixture only: orbit project uses scoped memory.';
|
||||
// Kept separately from recalled content; memory cannot supply its own source catalog.
|
||||
const sources = new Map([['fixture:orbit', Object.freeze({ workspace: 'alpha', scope: 'project', text: sourceText,
|
||||
observedAt: startedAt, sessionId: 'fixture-session', checkpointId: 'fixture-checkpoint' })]]);
|
||||
const evidenceContext = { workspace: 'alpha', scope: 'project' };
|
||||
const body = encodeEvidence('fixture:orbit', sources, evidenceContext);
|
||||
const shared = save('orbit shared evidence', 'project', 'all', 'alpha', body);
|
||||
const team = save('orbit team context', 'team');
|
||||
const targeted = save('orbit codex context', 'project', 'codex');
|
||||
const user = save('orbit user context', 'user');
|
||||
const other = save('orbit other project', 'project', 'all', 'beta');
|
||||
|
||||
for (const harness of ['codex', 'claude', 'hermes']) {
|
||||
const result = mcp(harness, [
|
||||
['memory_search', { query: 'orbit' }],
|
||||
['memory_read', { id: shared.id }],
|
||||
['memory_read', { id: targeted.id }],
|
||||
['memory_search', { query: 'orbit', scopes: ['user'] }],
|
||||
['memory_save', { title: 'spoof', body: 'Synthetic', sourceHarness: 'other' }],
|
||||
['memory_search', { query: 'orbit', targetHarness: 'codex' }],
|
||||
['memory_save', { title: 'trusted', body: 'Synthetic', trust: 'verified' }],
|
||||
['memory_read', { id: user.id, scope: 'user' }],
|
||||
['memory_save', { title: 'user write', body: 'Synthetic', scope: 'user' }],
|
||||
]);
|
||||
check(`${harness}: CLI/MCP ordered search parity`, () => {
|
||||
const expected = cli(['search', 'orbit', '--target-harness', harness]);
|
||||
assert.deepEqual(payload(result[0]).results, expected.results.map(({ memory, score, excerpt }) => ({ memory, score, excerpt })));
|
||||
const ids = payload(result[0]).results.map(item => item.memory.id);
|
||||
assert.ok(ids.includes(shared.id) && ids.includes(team.id));
|
||||
assert.equal(ids.includes(targeted.id), harness === 'codex');
|
||||
assert.ok(!ids.includes(user.id) && !ids.includes(other.id));
|
||||
});
|
||||
check(`${harness}: read preserves provenance and unreviewed trust`, () => {
|
||||
const read = payload(result[1]).memory;
|
||||
assert.equal(read.body, body);
|
||||
for (const field of ['id', 'scope', 'sourceHarness', 'targetHarnesses', 'createdAt', 'updatedAt', 'trust']) {
|
||||
assert.deepEqual(read[field], shared[field]);
|
||||
}
|
||||
assert.equal(read.trust, 'unreviewed');
|
||||
const cliRead = cli(['read', shared.id]).memory;
|
||||
assert.deepEqual(verifyEvidence(read.body, sources, { workspace: 'alpha', scope: read.scope }),
|
||||
verifyEvidence(cliRead.body, sources, { workspace: 'alpha', scope: cliRead.scope }));
|
||||
});
|
||||
check(`${harness}: direct target visibility enforced by MCP`, () => {
|
||||
if (harness === 'codex') assert.equal(payload(result[2]).memory.id, targeted.id);
|
||||
else assert.equal(result[2].result.isError, true);
|
||||
});
|
||||
check(`${harness}: scope elevation, identity spoofing and trust promotion rejected`, () => {
|
||||
for (const response of result.slice(3)) assert.equal(response.error?.code, -32602);
|
||||
});
|
||||
check(`${harness}: query reproducible across process restart`, () => {
|
||||
assert.deepEqual(payload(mcp(harness, [['memory_search', { query: 'orbit' }]])[0]), payload(result[0]));
|
||||
});
|
||||
}
|
||||
check('MCP write identity and evidence survive CLI handoff read', () => {
|
||||
sources.set('fixture:handoff', Object.freeze({ workspace: 'alpha', scope: 'project', text: 'Synthetic handoff.',
|
||||
observedAt: startedAt, sessionId: 'fixture-hermes-session', checkpointId: 'fixture-handoff' }));
|
||||
const handoffBody = encodeEvidence('fixture:handoff', sources, evidenceContext);
|
||||
const saved = payload(mcp('hermes', [['memory_save', { title: 'handoff fixture', body: handoffBody,
|
||||
kind: 'handoff', targetHarnesses: ['codex'], links: [shared.id] }]])[0]).memory;
|
||||
assert.equal(saved.sourceHarness, 'hermes');
|
||||
assert.equal(saved.trust, 'unreviewed');
|
||||
const read = payload(mcp('codex', [['memory_read', { id: saved.id }]])[0]).memory;
|
||||
assert.deepEqual(read.links, [shared.id]);
|
||||
const cliRead = cli(['read', saved.id]).memory;
|
||||
assert.equal(cliRead.body, handoffBody);
|
||||
assert.equal(cliRead.sourceHarness, 'hermes');
|
||||
assert.equal(cliRead.trust, 'unreviewed');
|
||||
assert.deepEqual(verifyEvidence(cliRead.body, sources, { workspace: 'alpha', scope: cliRead.scope }),
|
||||
verifyEvidence(read.body, sources, { workspace: 'alpha', scope: read.scope }));
|
||||
});
|
||||
check('operator opt-in enables only explicit user recall', () => {
|
||||
const result = mcp('hermes', [['memory_search', { query: 'orbit', scopes: ['user'] }],
|
||||
['memory_search', { query: 'orbit' }]], 'alpha', true);
|
||||
assert.deepEqual(payload(result[0]).results.map(item => item.memory.id), [user.id]);
|
||||
assert.ok(!payload(result[1]).results.some(item => item.memory.id === user.id));
|
||||
});
|
||||
check('separate project root excludes alpha records', () => {
|
||||
const read = mcp('hermes', [['memory_search', { query: 'orbit' }], ['memory_read', { id: shared.id }]], 'beta');
|
||||
assert.deepEqual(payload(read[0]).results.map(item => item.memory.id), [other.id]);
|
||||
assert.equal(read[1].result.isError, true);
|
||||
});
|
||||
check('CLI direct read is operator access, not target authorization', () => {
|
||||
assert.equal(cli(['read', targeted.id]).memory.id, targeted.id);
|
||||
});
|
||||
check('missing configured identity prevents MCP startup', () => {
|
||||
const result = run('scripts/memory-mcp.mjs', [], '', envFor('alpha', null));
|
||||
assert.equal(result.status, 1);
|
||||
assert.match(result.stderr, /ECC_MEMORY_HARNESS/);
|
||||
});
|
||||
check('recalled evidence rejects tamper, unavailable source and foreign context', () => {
|
||||
const read = payload(mcp('codex', [['memory_read', { id: shared.id }]])[0]).memory;
|
||||
const altered = JSON.stringify({ ...JSON.parse(read.body), text: 'Synthetic altered evidence.' });
|
||||
assert.throws(() => verifyEvidence(altered, sources, evidenceContext), { code: 'SOURCE_MISMATCH' });
|
||||
assert.throws(() => verifyEvidence(read.body, new Map(), evidenceContext), { code: 'SOURCE_UNAVAILABLE' });
|
||||
assert.throws(() => verifyEvidence(read.body, sources, { ...evidenceContext, workspace: 'beta' }),
|
||||
{ code: 'CONTEXT_MISMATCH' });
|
||||
assert.throws(() => verifyEvidence(read.body, sources, { ...evidenceContext, scope: 'user' }),
|
||||
{ code: 'CONTEXT_MISMATCH' });
|
||||
});
|
||||
check('stored altered content and digest fail evidence verification after MCP recall', () => {
|
||||
for (const change of [{ text: 'Synthetic altered content.' }, { sha256: '0'.repeat(64) }]) {
|
||||
const altered = JSON.stringify({ ...JSON.parse(body), ...change });
|
||||
const saved = save('evidence rejection fixture', 'project', 'all', 'alpha', altered);
|
||||
const read = payload(mcp('hermes', [['memory_read', { id: saved.id }]])[0]).memory;
|
||||
assert.equal(read.id, saved.id);
|
||||
assert.equal(read.body, altered);
|
||||
assert.equal(read.trust, 'unreviewed');
|
||||
assert.throws(() => verifyEvidence(read.body, sources, { workspace: 'alpha', scope: read.scope }),
|
||||
{ code: 'SOURCE_MISMATCH' });
|
||||
}
|
||||
});
|
||||
check('synthetic private-key marker rejected without changing recalled dataset', () => {
|
||||
// Deliberately incomplete synthetic marker; never a real key or private input.
|
||||
const marker = '-----BEGIN PRIVATE KEY-----\nSynthetic non-key fixture.';
|
||||
const beforePrivacy = cli(['search', 'orbit', '--target-harness', 'codex']).results;
|
||||
const cliDenied = run('scripts/memory.js', ['save', '--title', 'orbit rejected fixture', '--stdin', '--json'],
|
||||
marker, envFor());
|
||||
assert.equal(cliDenied.status, 1, 'Synthetic sensitive write must be rejected');
|
||||
assert.equal(cliDenied.error, undefined, 'CLI rejection must not be a subprocess failure');
|
||||
assert.match(cliDenied.stderr, /suspected secret/i);
|
||||
const mcpDenied = mcp('codex', [['memory_save', { title: 'orbit rejected fixture', body: marker }]])[0];
|
||||
assert.equal(mcpDenied.result.isError, true, 'Synthetic sensitive write must be a tool rejection');
|
||||
const rejection = JSON.parse(mcpDenied.result.content.find(item => item.type === 'text').text);
|
||||
assert.equal(rejection.error.code, 'MEMORY_WRITE_REJECTED');
|
||||
assert.equal(rejection.error.message, 'Memory operation rejected a suspected secret.');
|
||||
assert.deepEqual(cli(['search', 'orbit', '--target-harness', 'codex']).results, beforePrivacy);
|
||||
assert.deepEqual(payload(mcp('codex', [['memory_search', { query: 'orbit' }]])[0]).results, beforePrivacy);
|
||||
});
|
||||
check('source files and HEAD unchanged after execution', () => {
|
||||
assert.deepEqual(snapshot(), before);
|
||||
assert.equal(sourceHead(), headBefore);
|
||||
});
|
||||
process.stdout.write(JSON.stringify({ schemaVersion: 'ecc.memory.conformance.receipt.v1',
|
||||
status: 'passed', startedAt, completedAt: new Date().toISOString(), nodeVersion: process.version,
|
||||
source: { head: headBefore, files: before,
|
||||
executionMode: 'local source files with existing dependencies; no fetch performed',
|
||||
identityBoundary: 'File digests identify executed source; HEAD alone does not establish a clean tree.' },
|
||||
exampleSha256: sha256(fs.readFileSync(__filename)), checks,
|
||||
evidenceBoundary: 'Synthetic real CLI/stdio execution. No live harness, Graphiti, OAuth, replication or deployment verification.',
|
||||
}, null, 2) + '\n');
|
||||
} catch (error) {
|
||||
// Never print raw process output or assertion values into the receipt.
|
||||
process.stderr.write(JSON.stringify({ status: 'failed', passedChecks: checks.map(item => item.name),
|
||||
errorType: error.name, message: 'Conformance failed after the listed checks; inspect the next synthetic operation.' }) + '\n');
|
||||
process.exitCode = 1;
|
||||
} finally {
|
||||
fs.rmSync(root, { recursive: true, force: true });
|
||||
}
|
||||
@@ -0,0 +1,79 @@
|
||||
'use strict';
|
||||
|
||||
// Example-only integrity checks. A host-owned catalog is not an identity provider.
|
||||
const { createHash } = require('node:crypto');
|
||||
const SCHEMA = 'ecc.memory.example-evidence.v1';
|
||||
const MAX_BODY_BYTES = 16 * 1024;
|
||||
const MAX_TEXT_BYTES = 8 * 1024;
|
||||
const ENVELOPE_KEYS = ['schema', 'sourceRef', 'sha256', 'text', 'observedAt', 'sessionId', 'checkpointId'];
|
||||
const SOURCE_KEYS = ['workspace', 'scope', 'text', 'observedAt', 'sessionId', 'checkpointId'];
|
||||
const slug = value => typeof value === 'string' && /^[a-z][a-z0-9-]{0,63}$/.test(value);
|
||||
const sourceRefIsValid = value => typeof value === 'string' && /^fixture:[a-z][a-z0-9-]{0,63}$/.test(value);
|
||||
const digest = text => createHash('sha256').update(text, 'utf8').digest('hex');
|
||||
|
||||
function fail(code) {
|
||||
const error = new Error(`Memory example evidence: ${code}`);
|
||||
error.code = code;
|
||||
throw error;
|
||||
}
|
||||
function hasExactKeys(value, keys) {
|
||||
return value !== null && typeof value === 'object' && !Array.isArray(value)
|
||||
&& Object.keys(value).length === keys.length && keys.every(key => Object.hasOwn(value, key));
|
||||
}
|
||||
function validObservation(value) {
|
||||
if (typeof value !== 'string' || !/^\d{4}-\d{2}-\d{2}T\d{2}:\d{2}:\d{2}\.\d{3}Z$/.test(value)) return false;
|
||||
const date = new Date(value);
|
||||
return Number.isFinite(date.getTime()) && date.toISOString() === value;
|
||||
}
|
||||
function validSourceFields(value) {
|
||||
return typeof value.text === 'string' && value.text.length > 0 && value.text.length <= MAX_TEXT_BYTES
|
||||
&& Buffer.byteLength(value.text, 'utf8') <= MAX_TEXT_BYTES
|
||||
// eslint-disable-next-line no-control-regex -- Intentionally reject C0 except tab/LF/CR, and DEL.
|
||||
&& !/[\u0000-\u0008\u000b\u000c\u000e-\u001f\u007f]/.test(value.text)
|
||||
&& validObservation(value.observedAt) && slug(value.sessionId) && slug(value.checkpointId);
|
||||
}
|
||||
function validateEnvelope(value) {
|
||||
if (!hasExactKeys(value, ENVELOPE_KEYS) || value.schema !== SCHEMA || !sourceRefIsValid(value.sourceRef)
|
||||
|| typeof value.sha256 !== 'string' || !/^[a-f0-9]{64}$/.test(value.sha256) || !validSourceFields(value)) {
|
||||
fail('INVALID_ENVELOPE');
|
||||
}
|
||||
}
|
||||
function getSource(sourceRef, catalog, context) {
|
||||
if (!hasExactKeys(context, ['workspace', 'scope']) || !slug(context.workspace)
|
||||
|| !['project', 'team', 'user'].includes(context.scope)) fail('INVALID_CONTEXT');
|
||||
if (!(catalog instanceof Map) || !sourceRefIsValid(sourceRef)) fail('INVALID_SOURCE');
|
||||
const source = catalog.get(sourceRef);
|
||||
if (source === undefined) fail('SOURCE_UNAVAILABLE');
|
||||
if (!hasExactKeys(source, SOURCE_KEYS) || !validSourceFields(source) || !slug(source.workspace)
|
||||
|| !['project', 'team', 'user'].includes(source.scope)) fail('INVALID_SOURCE');
|
||||
if (source.workspace !== context.workspace || source.scope !== context.scope) fail('CONTEXT_MISMATCH');
|
||||
return source;
|
||||
}
|
||||
function decode(body) {
|
||||
if (typeof body !== 'string' || body.length > MAX_BODY_BYTES || Buffer.byteLength(body, 'utf8') > MAX_BODY_BYTES) {
|
||||
fail('INVALID_ENVELOPE');
|
||||
}
|
||||
let value;
|
||||
try { value = JSON.parse(body); } catch { fail('INVALID_ENVELOPE'); }
|
||||
validateEnvelope(value);
|
||||
return value;
|
||||
}
|
||||
|
||||
function encodeEvidence(sourceRef, catalog, context) {
|
||||
const source = getSource(sourceRef, catalog, context);
|
||||
const body = JSON.stringify({ schema: SCHEMA, sourceRef, sha256: digest(source.text), text: source.text,
|
||||
observedAt: source.observedAt, sessionId: source.sessionId, checkpointId: source.checkpointId });
|
||||
decode(body);
|
||||
return body;
|
||||
}
|
||||
|
||||
function verifyEvidence(body, catalog, context) {
|
||||
const value = decode(body);
|
||||
const source = getSource(value.sourceRef, catalog, context);
|
||||
if (value.sha256 !== digest(source.text) || value.text !== source.text
|
||||
|| value.observedAt !== source.observedAt || value.sessionId !== source.sessionId
|
||||
|| value.checkpointId !== source.checkpointId) fail('SOURCE_MISMATCH');
|
||||
return Object.freeze({ status: 'source-content-match', sourceRef: value.sourceRef, sha256: value.sha256 });
|
||||
}
|
||||
|
||||
module.exports = { encodeEvidence, verifyEvidence };
|
||||
@@ -0,0 +1,109 @@
|
||||
'use strict';
|
||||
|
||||
// Pure synthetic checks: no subprocess, filesystem fixture, provider or server.
|
||||
const assert = require('node:assert/strict');
|
||||
const { encodeEvidence, verifyEvidence } = require('./evidence.cjs');
|
||||
const sourceRef = 'fixture:orbit';
|
||||
const source = Object.freeze({ workspace: 'alpha', scope: 'project',
|
||||
text: 'Synthetic orbit evidence: calibration color is amber.',
|
||||
observedAt: '2026-01-01T00:00:00.000Z', sessionId: 'fixture-session', checkpointId: 'fixture-checkpoint' });
|
||||
const context = Object.freeze({ workspace: 'alpha', scope: 'project' });
|
||||
const catalog = new Map([[sourceRef, source]]);
|
||||
const body = () => encodeEvidence(sourceRef, catalog, context);
|
||||
const edit = change => JSON.stringify({ ...JSON.parse(body()), ...change });
|
||||
let passed = 0;
|
||||
function test(name, fn) {
|
||||
try { fn(); passed += 1; }
|
||||
catch { throw new Error(`Synthetic evidence check failed: ${name}`); }
|
||||
}
|
||||
function rejects(fn, code) {
|
||||
assert.throws(fn, error => error.code === code
|
||||
&& error.message === `Memory example evidence: ${code}`);
|
||||
}
|
||||
|
||||
test('valid source content and provenance match', () => {
|
||||
const result = verifyEvidence(body(), catalog, context);
|
||||
assert.equal(result.status, 'source-content-match');
|
||||
assert.equal(result.sourceRef, sourceRef);
|
||||
assert.equal(result.sha256, JSON.parse(body()).sha256);
|
||||
assert.ok(Object.isFrozen(result));
|
||||
});
|
||||
test('deterministic encoding preserves input catalog', () => {
|
||||
const before = JSON.stringify([...catalog]);
|
||||
assert.equal(body(), body());
|
||||
assert.equal(JSON.stringify([...catalog]), before);
|
||||
});
|
||||
for (const [name, change] of [
|
||||
['changed text', { text: 'Synthetic altered content.' }],
|
||||
['changed digest', { sha256: '0'.repeat(64) }],
|
||||
['changed observation', { observedAt: '2026-01-02T00:00:00.000Z' }],
|
||||
['changed session', { sessionId: 'other-session' }],
|
||||
['changed checkpoint', { checkpointId: 'other-checkpoint' }],
|
||||
]) {
|
||||
test(name, () => rejects(() => verifyEvidence(edit(change), catalog, context), 'SOURCE_MISMATCH'));
|
||||
}
|
||||
test('missing source never becomes successful empty evidence', () => {
|
||||
rejects(() => verifyEvidence(body(), new Map(), context), 'SOURCE_UNAVAILABLE');
|
||||
});
|
||||
test('same reference in another workspace is denied', () => {
|
||||
rejects(() => verifyEvidence(body(), catalog, { ...context, workspace: 'beta' }), 'CONTEXT_MISMATCH');
|
||||
});
|
||||
test('project evidence cannot be relabeled as user evidence', () => {
|
||||
rejects(() => verifyEvidence(body(), catalog, { ...context, scope: 'user' }), 'CONTEXT_MISMATCH');
|
||||
});
|
||||
test('creation enforces host context too', () => {
|
||||
rejects(() => encodeEvidence(sourceRef, catalog, { ...context, workspace: 'beta' }), 'CONTEXT_MISMATCH');
|
||||
});
|
||||
for (const [name, value] of [
|
||||
['unknown schema', () => edit({ schema: 'unrecognized' })],
|
||||
['unknown authority field', () => edit({ trust: 'verified' })],
|
||||
['external URL is not a source lookup', () => edit({ sourceRef: 'https://example.invalid/source' })],
|
||||
['path is not a source lookup', () => edit({ sourceRef: '../private-source' })],
|
||||
['invalid timestamp', () => edit({ observedAt: '2026-02-30T00:00:00.000Z' })],
|
||||
['missing checkpoint', () => { const value = JSON.parse(body()); delete value.checkpointId; return JSON.stringify(value); }],
|
||||
['malformed JSON', () => '{'],
|
||||
['non-object JSON', () => 'null'],
|
||||
['oversized body', () => 'x'.repeat(16385)],
|
||||
]) {
|
||||
test(name, () => rejects(() => verifyEvidence(value(), catalog, context), 'INVALID_ENVELOPE'));
|
||||
}
|
||||
test('unavailable source is also denied during creation', () => {
|
||||
rejects(() => encodeEvidence(sourceRef, new Map(), context), 'SOURCE_UNAVAILABLE');
|
||||
});
|
||||
test('changed catalog content invalidates a previously encoded body', () => {
|
||||
const changed = new Map([[sourceRef, { ...source, text: 'Synthetic revised evidence.' }]]);
|
||||
rejects(() => verifyEvidence(body(), changed, context), 'SOURCE_MISMATCH');
|
||||
});
|
||||
test('recomputed attacker digest does not replace host source binding', () => {
|
||||
const crypto = require('node:crypto');
|
||||
const text = 'Synthetic attacker replacement.';
|
||||
const sha256 = crypto.createHash('sha256').update(text).digest('hex');
|
||||
rejects(() => verifyEvidence(edit({ text, sha256 }), catalog, context), 'SOURCE_MISMATCH');
|
||||
});
|
||||
test('invalid host source is not a record success', () => {
|
||||
const invalid = new Map([[sourceRef, { ...source, text: '' }]]);
|
||||
rejects(() => encodeEvidence(sourceRef, invalid, context), 'INVALID_SOURCE');
|
||||
});
|
||||
test('invalid host context is denied before source lookup', () => {
|
||||
rejects(() => verifyEvidence(body(), catalog, { workspace: 'alpha', scope: 'all' }), 'INVALID_CONTEXT');
|
||||
});
|
||||
test('rejects forbidden C0 controls and DEL in source and recalled text', () => {
|
||||
const codes = [...Array.from({ length: 32 }, (_, code) => code), 127]
|
||||
.filter(code => ![9, 10, 13].includes(code));
|
||||
for (const code of codes) {
|
||||
const text = `Synthetic ${String.fromCodePoint(code)} content.`;
|
||||
const invalid = new Map([[sourceRef, { ...source, text }]]);
|
||||
rejects(() => encodeEvidence(sourceRef, invalid, context), 'INVALID_SOURCE');
|
||||
rejects(() => verifyEvidence(edit({ text }), catalog, context), 'INVALID_ENVELOPE');
|
||||
}
|
||||
});
|
||||
test('preserves allowed whitespace, printable boundaries and non-C0 Unicode', () => {
|
||||
for (const code of [9, 10, 13, 32, 126, 128, 0x2028, 0x1f642]) {
|
||||
const text = `Synthetic ${String.fromCodePoint(code)} content.`;
|
||||
const allowed = new Map([[sourceRef, { ...source, text }]]);
|
||||
const encoded = encodeEvidence(sourceRef, allowed, context);
|
||||
assert.equal(verifyEvidence(encoded, allowed, context).status, 'source-content-match');
|
||||
}
|
||||
});
|
||||
process.stdout.write(`${JSON.stringify({ status: 'passed', checks: passed,
|
||||
boundary: 'Synthetic in-memory evidence checks; no authentication or runtime-service verification.' })}\n`);
|
||||
Reference in New Issue
Block a user