# Research for judgment-based coding standards

Date: 2026-09-26. Scope: Python 3.14 and aiohttp 3.14.3, as pinned in [`pyproject.toml`](../pyproject.toml). This note informs `CODING_STANDARDS.md`; it does not replace the project contracts in [`docs/architecture.md`](architecture.md), [`CONTRIBUTING.md`](../CONTRIBUTING.md), and the [discovery module guide](wiki/How-to-add-a-new-module.md).

## Recommendations

| Judgment standard for review | Mechanical check to keep separate | Basis |
| --- | --- | --- |
| Trace the **whole provider conversation** before changing network, pagination, retry, proxy, cookie, or cancellation behavior. Name the owner of each session, response, connector, and spawned task; distinguish owned resources from borrowed ones. Confirm the state needed by a later request survives and no state crosses source executions or targets. | Ruff, typing, and a local lifecycle test can catch specific mistakes, but cannot infer the intended ownership boundary. | The [module guide](wiki/How-to-add-a-new-module.md#own-the-provider-conversation) defines this boundary. aiohttp says a session owns a connector by default, supports a private cookie jar, and closes via its context manager; `connector_owner=False` changes connector ownership ([client reference](https://docs.aiohttp.org/en/v3.14.3/client_reference.html#client-session)). |
| Preserve cancellation as cancellation after cleanup. Review every `await`, exception handler, and task it can interrupt. Verify cleanup on success, transport failure, and interruption, and preserve retained evidence according to the central runner's contract. | A focused offline cancellation test can assert resource closure and propagated `CancelledError`; passing it does not prove every related branch was audited. | Python recommends `try/finally` for cleanup and generally re-raising `CancelledError`; swallowing it can break `TaskGroup` and timeouts ([Python 3.14 asyncio tasks](https://docs.python.org/3.14/library/asyncio-task.html#task-cancellation)). The [architecture guide](architecture.md#execution-and-lifecycle) separates source and run outcomes and retained evidence. |
| For related pages, retries, or polling, decide whether provider state must persist. Reuse the borrowed session when cookies, headers, connection pool, or selected proxy are part of the conversation. Use a deliberately empty cookie jar for independent probes. Check proxy selection at both session creation and each request, including helpers called with a borrowed session; session reuse alone does not prove proxy identity stays fixed. | An offline second-page test can require a cookie or other first-page state, assert the selected proxy, and assert session closure. A linter cannot establish the provider's state contract. | aiohttp documents session cookie sharing and `DummyCookieJar` ([advanced usage](https://docs.aiohttp.org/en/v3.14.3/client_advanced.html#custom-cookies), [client reference](https://docs.aiohttp.org/en/v3.14.3/client_reference.html#client-session)); its proxy settings can be passed at session or request level ([proxy support](https://docs.aiohttp.org/en/v3.14.3/client_advanced.html#proxy-support)). Project-specific behavior is in [`AsyncFetcher.open_session()` and `fetch()`](../theHarvester/lib/core.py) and the [module guide](wiki/How-to-add-a-new-module.md#own-the-provider-conversation). |
| Make review claims proportional to evidence: identify the changed contract and concrete before/after behavior, name the exact test or check run, and distinguish static source inspection, offline tests, CI, publication, and live provider behavior. Treat a test as useful only if it would fail on the relevant regression. | CI can report command results, coverage gates, and diff metadata; a reviewer still decides whether assertions cover the behavior and whether operator documentation changed. | [`CONTRIBUTING.md`](../CONTRIBUTING.md#open-a-pull-request) asks for before/after behavior, exact checks, and risks. Google's first-party [review guidance](https://google.github.io/eng-practices/review/reviewer/looking-for.html) calls for checking edge cases, concurrency, test validity, and documentation. GitHub [separates diffs, discussion, and checks](https://docs.github.com/en/pull-requests/get-started/about-pull-requests). |

## Project fit and caveats

- The source catalog and runner own identifiers, selection, final status, and normalized evidence; adapter review should not invent a parallel outcome mechanism ([architecture](architecture.md#execution-and-lifecycle), [module guide](wiki/How-to-add-a-new-module.md#2-implement-the-adapter)).
- The project uses offline provider contract tests and blocks routine external socket traffic. Those tests establish local behavior; live provider drift remains a separate, authorized integration question ([CONTRIBUTING.md](../CONTRIBUTING.md#test-safely)).
- `AsyncFetcher.fetch()` currently resolves its `proxy` argument even when given a borrowed session. A standard should require caller-path inspection before asserting fixed proxy identity, rather than assume it from aiohttp session reuse alone ([core.py](../theHarvester/lib/core.py)).
- These are review judgments, not new automatic gates or claims that all existing adapters conform. The linked aiohttp pages are versioned 3.14.3 documentation; exact project behavior must still be checked in the pinned code path.

## Retrospective findings

1. Review guidance was embedded in the always-loaded agent instructions. Move it to `CODING_STANDARDS.md` and retain a review-triggered pointer. Preserve the four existing review subjects: compatibility, sensitive data, reconnaissance boundaries, and provider resource lifetime. The architecture and module guide remain the owners of detailed behavior contracts.
2. Mechanical checks already have a CI path: [Python CI](../.github/workflows/theHarvester.yml) runs lint, formatting, pytest, and typing. The [test harness](../tests/conftest.py) enforces offline Python networking, and [provider coverage](../tests/test_provider_contract_coverage.py) checks catalog registration. Add judgment about coverage quality without duplicating those checks in prose.
3. Test-code linting and typing remain a separate improvement candidate: [Ruff excludes tests and ty includes only the application package](../pyproject.toml). Browser verification has a [separate workflow](../.github/workflows/harvestview-e2e.yml). This documentation change does not expand those gates or claim whole-repository coverage.

## Agent-instruction research across tools

The initial OpenAI review was recorded on 2026-09-26. OpenAI and Anthropic documentation were compared on 2026-09-27. The project recommendations below apply to contributors using any coding assistant; instruction-file discovery remains client-specific.

| Source | Relevant guidance | Application here |
| --- | --- | --- |
| OpenAI: [Custom instructions with AGENTS.md](https://learn.chatgpt.com/docs/agent-configuration/agents-md#how-codex-discovers-guidance) | Codex layers personal and project instructions, with directory-specific guidance. | Keep repository policy in shared project documents and personal tool preferences in user configuration. |
| OpenAI: [Codex best practices](https://learn.chatgpt.com/guides/best-practices#make-guidance-reusable-with-agentsmd) | Keep guidance practical, concise, and current; link task-specific documents and state how to verify completion. | Retain the project introduction, `uv` commands, task-triggered reading pointers, and reporting of unrun checks. |
| Anthropic: [Write an effective CLAUDE.md](https://code.claude.com/docs/en/best-practices#write-an-effective-claudemd) | Keep broadly applicable instructions short, remove redundant information, and link detailed references. | Keep the common entry point concise and retain architecture, provider contracts, and review judgments in their existing documents. |
| Anthropic: [Give Claude a way to verify its work](https://code.claude.com/docs/en/best-practices#give-claude-a-way-to-verify-its-work) | Give the agent observable verification criteria and require evidence from the checks it runs. | Start with affected offline behavior, retain publication checks, and distinguish observed results from unverified claims. |
| Anthropic: [Automate actions with hooks](https://code.claude.com/docs/en/hooks-guide) | Command hooks can execute checks at defined lifecycle events; instruction text alone does not ensure an action runs. | Keep mechanical checks in the existing test harness and CI. This research does not introduce Claude-specific hooks or change permission settings. |

### Shared project conclusions

Keep [AGENTS.md](../AGENTS.md) as the shared repository entry point and [CODING_STANDARDS.md](../CODING_STANDARDS.md) as the review reference. The same confidentiality, reconnaissance, compatibility, and verification expectations apply regardless of the contributor's assistant. These are our project-specific conclusions from the sources, not a vendor-mandated template or evidence that every assistant follows instructions identically. Model choices, permission modes, and vendor workflow examples remain outside repository policy.

The initial review also used OpenAI's [Harness engineering](https://openai.com/index/harness-engineering/) and [GPT-6 Astra prompting article](https://developers.openai.com/blog/rethinking-skills-and-prompts-for-gpt-6-astra#up-to-date-agentsmd). They explain that review's background; model-specific recommendations are not requirements for Claude or other agents. The general guidance above supports the shared document structure independently of that model-specific advice.

### Instruction discovery and Claude compatibility

- Codex discovers `AGENTS.md` using its documented [directory and override rules](https://learn.chatgpt.com/docs/agent-configuration/agents-md#how-codex-discovers-guidance).
- Claude Code supports native `AGENTS.md` loading from v2.1.277, subject to settings and plugin availability. By default, a project or ancestor `CLAUDE.md`, `.claude/CLAUDE.md`, or `CLAUDE.local.md` takes precedence. See Anthropic's [discovery rules and availability caveats](https://code.claude.com/docs/en/memory#agentsmd).
- For sessions without native loading, Anthropic documents a neighboring `CLAUDE.md` containing `@AGENTS.md`. This optional import shares the existing instructions without copying them; it is not necessary for every current Claude Code setup. See [sharing instructions across tools](https://code.claude.com/docs/en/memory#share-one-file-with-other-coding-tools).

Other assistants should use their documented instruction-loading mechanism to reach the same project files. This review verified documentation, not instruction loading or compliance in installed clients. No global guidance, model configuration, publication authority, or test gate changed.