WEBDIENSTE/theHarvester
1
0
Fork 0
mirror of https://github.com/laramies/theHarvester.git synced 2026-08-17 19:35:40 +02:00
Code Issues Packages Projects Releases Wiki Activity

Fix passive provider failure attribution and parsing (#2518)

Browse Source 
* fix: harden passive provider reliability

* fix: restore HackerTarget reverse DNS coverage
This commit is contained in:
Matt
2026-08-11 17:04:11 -04:00
committed by GitHub
parent fa8afa5e46
commit 801460f835
29 changed files with 2791 additions and 588 deletions
Download Patch File Download Diff File Expand all files Collapse all files
CHANGELOG.md
+3 -2
View File
@@ -29,8 +29,9 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
- Added root contributor and security policies, structured issue forms, repository agent guidance, discovery terminology, and an operator-focused documentation wiki ([d090a29a](https://github.com/laramies/theHarvester/commit/d090a29a), [7c491ef5](https://github.com/laramies/theHarvester/commit/7c491ef5), [8b9d420b](https://github.com/laramies/theHarvester/commit/8b9d420b)).
### Changed
- Removed the transport-wide delay before reading ready HTTP responses, bounded Wayback Archive and Common Crawl collection to 30 seconds, kept Wayback within the requested result limit, and made long-source progress visible in verbose mode.
- Reported Baidu no-response and verification outcomes plus Common Crawl query failures as source evidence without aborting sibling sources.
- Removed the transport-wide delay before reading ready HTTP responses, bounded Wayback Archive to 30 seconds and Common Crawl to 120 seconds, kept both sources within the requested result limit, and made long-source progress visible in verbose mode. Common Crawl now requests one 50-record page at a time instead of bursting page batches.
- Made Baidu, crt.sh, HackerTarget, Have I Been Pwned, Mojeek, OTX, and Robtex report blocked, malformed, or transport failures truthfully. Also fixed HackerTarget CSV parsing and Robtex AAAA results.
- Hardened BufferOver, Chaos, DNSDumpster, ONYPHE, and URLScan parsing and result attribution, including scoped typed results and bounded URLScan pagination.
- Made `harvestview` the sole launcher for the local web application and REST API.
- Standardized SQLite, JSONL, API, and HarvestView result names on `hostname` and `ip` without a presentation alias.
- Standardized URL-producing adapters, JSON, JSONL, SQLite, and API evidence on one `url` result kind while preserving producer provenance.