theHarvester/docs/coding-standards-research.md

Research for judgment-based coding standards

Date: 2026-09-26. Scope: Python 3.14 and aiohttp 3.14.3, as pinned in pyproject.toml. This note informs CODING_STANDARDS.md; it does not replace the project contracts in docs/architecture.md, CONTRIBUTING.md, and the discovery module guide.

Recommendations

Judgment standard for review	Mechanical check to keep separate	Basis
Trace the whole provider conversation before changing network, pagination, retry, proxy, cookie, or cancellation behavior. Name the owner of each session, response, connector, and spawned task; distinguish owned resources from borrowed ones. Confirm the state needed by a later request survives and no state crosses source executions or targets.	Ruff, typing, and a local lifecycle test can catch specific mistakes, but cannot infer the intended ownership boundary.	The module guide defines this boundary. aiohttp says a session owns a connector by default, supports a private cookie jar, and closes via its context manager; connector_owner=False changes connector ownership (client reference).
Preserve cancellation as cancellation after cleanup. Review every await, exception handler, and task it can interrupt. Verify cleanup on success, transport failure, and interruption, and preserve retained evidence according to the central runner's contract.	A focused offline cancellation test can assert resource closure and propagated CancelledError; passing it does not prove every related branch was audited.	Python recommends try/finally for cleanup and generally re-raising CancelledError; swallowing it can break TaskGroup and timeouts (Python 3.14 asyncio tasks). The architecture guide separates source and run outcomes and retained evidence.
For related pages, retries, or polling, decide whether provider state must persist. Reuse the borrowed session when cookies, headers, connection pool, or selected proxy are part of the conversation. Use a deliberately empty cookie jar for independent probes. Check proxy selection at both session creation and each request, including helpers called with a borrowed session; session reuse alone does not prove proxy identity stays fixed.	An offline second-page test can require a cookie or other first-page state, assert the selected proxy, and assert session closure. A linter cannot establish the provider's state contract.	aiohttp documents session cookie sharing and DummyCookieJar (advanced usage, client reference); its proxy settings can be passed at session or request level (proxy support). Project-specific behavior is in AsyncFetcher.open_session() and fetch() and the module guide.
Make review claims proportional to evidence: identify the changed contract and concrete before/after behavior, name the exact test or check run, and distinguish static source inspection, offline tests, CI, publication, and live provider behavior. Treat a test as useful only if it would fail on the relevant regression.	CI can report command results, coverage gates, and diff metadata; a reviewer still decides whether assertions cover the behavior and whether operator documentation changed.	CONTRIBUTING.md asks for before/after behavior, exact checks, and risks. Google's first-party review guidance calls for checking edge cases, concurrency, test validity, and documentation. GitHub separates diffs, discussion, and checks.

Project fit and caveats

• The source catalog and runner own identifiers, selection, final status, and normalized evidence; adapter review should not invent a parallel outcome mechanism (architecture, module guide).
• The project uses offline provider contract tests and blocks routine external socket traffic. Those tests establish local behavior; live provider drift remains a separate, authorized integration question (CONTRIBUTING.md).
• AsyncFetcher.fetch() currently resolves its proxy argument even when given a borrowed session. A standard should require caller-path inspection before asserting fixed proxy identity, rather than assume it from aiohttp session reuse alone (core.py).
• These are review judgments, not new automatic gates or claims that all existing adapters conform. The linked aiohttp pages are versioned 3.14.3 documentation; exact project behavior must still be checked in the pinned code path.

Retrospective findings

1. Review guidance was embedded in the always-loaded agent instructions. Move it to CODING_STANDARDS.md and retain a review-triggered pointer. Preserve the four existing review subjects: compatibility, sensitive data, reconnaissance boundaries, and provider resource lifetime. The architecture and module guide remain the owners of detailed behavior contracts.
2. Mechanical checks already have a CI path: Python CI runs lint, formatting, pytest, and typing. The test harness enforces offline Python networking, and provider coverage checks catalog registration. Add judgment about coverage quality without duplicating those checks in prose.
3. Test-code linting and typing remain a separate improvement candidate: Ruff excludes tests and ty includes only the application package. Browser verification has a separate workflow. This documentation change does not expand those gates or claim whole-repository coverage.

Agent-instruction research across tools

The initial OpenAI review was recorded on 2026-09-26. OpenAI and Anthropic documentation were compared on 2026-09-27. The project recommendations below apply to contributors using any coding assistant; instruction-file discovery remains client-specific.

Source	Relevant guidance	Application here
OpenAI: Custom instructions with AGENTS.md	Codex layers personal and project instructions, with directory-specific guidance.	Keep repository policy in shared project documents and personal tool preferences in user configuration.
OpenAI: Codex best practices	Keep guidance practical, concise, and current; link task-specific documents and state how to verify completion.	Retain the project introduction, uv commands, task-triggered reading pointers, and reporting of unrun checks.
Anthropic: Write an effective CLAUDE.md	Keep broadly applicable instructions short, remove redundant information, and link detailed references.	Keep the common entry point concise and retain architecture, provider contracts, and review judgments in their existing documents.
Anthropic: Give Claude a way to verify its work	Give the agent observable verification criteria and require evidence from the checks it runs.	Start with affected offline behavior, retain publication checks, and distinguish observed results from unverified claims.
Anthropic: Automate actions with hooks	Command hooks can execute checks at defined lifecycle events; instruction text alone does not ensure an action runs.	Keep mechanical checks in the existing test harness and CI. This research does not introduce Claude-specific hooks or change permission settings.

Shared project conclusions

Keep AGENTS.md as the shared repository entry point and CODING_STANDARDS.md as the review reference. The same confidentiality, reconnaissance, compatibility, and verification expectations apply regardless of the contributor's assistant. These are our project-specific conclusions from the sources, not a vendor-mandated template or evidence that every assistant follows instructions identically. Model choices, permission modes, and vendor workflow examples remain outside repository policy.

The initial review also used OpenAI's Harness engineering and GPT-6 Astra prompting article. They explain that review's background; model-specific recommendations are not requirements for Claude or other agents. The general guidance above supports the shared document structure independently of that model-specific advice.

Instruction discovery and Claude compatibility

• Codex discovers AGENTS.md using its documented directory and override rules.
• Claude Code supports native AGENTS.md loading from v2.1.277, subject to settings and plugin availability. By default, a project or ancestor CLAUDE.md, .claude/CLAUDE.md, or CLAUDE.local.md takes precedence. See Anthropic's discovery rules and availability caveats.
• For sessions without native loading, Anthropic documents a neighboring CLAUDE.md containing @AGENTS.md. This optional import shares the existing instructions without copying them; it is not necessary for every current Claude Code setup. See sharing instructions across tools.

Other assistants should use their documented instruction-loading mechanism to reach the same project files. This review verified documentation, not instruction loading or compliance in installed clients. No global guidance, model configuration, publication authority, or test gate changed.