`normalize_path` caps a path at MAX_PATH_LENGTH, but `split_path` takes the last
segment as the filename and that column is varchar(300) — a 1,400-character segment
with no slash in it failed the insert, poisoned the transaction and cost the whole
stage. gov.np hit it once the HTTP probe stopped truncating itself and handed URL
discovery 15,098 web assets instead of 2,687.
MAX_HOST_LENGTH and MAX_FILENAME_LENGTH join the caps already in
shared/definitions/endpoints.py, `split_path` applies the filename cap and
`parse_url` rejects an over-long host rather than storing a truncated one, and the
model's max_length values now read from those constants instead of repeating the
numbers. The batch insert runs in a savepoint and retries row by row, so a row
postgres refuses costs that row and lands in UpsertResult.rejected — the same rule
the HTTP probe now follows.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DPueZXQMbjLXMokXeA8LFP
- Updated highlight-text component to support multiple search terms.
- Modified results-pagination component to make onPageSize optional and conditionally render PageSizeSelector.
- Integrated HostStructure component into web-asset-detail-sheet for improved asset structure visualization.
- Added HostStructureDialog to manage host structure interactions.
- Enhanced web-assets-table to handle structure-related actions and dialogs.
- Introduced new folder glyphs and updated endpoint classification logic in endpoints configuration.
- Added endpoint_count to subdomain type for better tracking of discovered endpoints.
- Implemented new endpoint filtering capabilities in the backend to support static content visibility.
- Updated models and services to accommodate new endpoint structures and classifications.
