Commit Graph
207 Commits
Author SHA1 Message Date
Alisson Moretto 2f5ba88b79 Sherlockeye Integration 2026-05-30 10:56:08 -03:00
L1ghtn1ng 98dbda9af7 Harden API auth and fix type lint issues 2026-05-10 05:04:06 +01:00
J.TownsendandGitHub e2e85902ff Merge pull request #2306 from ChrisJr404/feat/dymo-api-source-2300
feat(discovery): add Dymo API data verifier source (closes #2300)
2026-05-10 03:36:48 +01:00
J.TownsendandGitHub 255cd8b563 Fix package version layout in the README 2026-05-10 03:25:12 +01:00
Chris (ChrisJr404) 131e23813a feat(discovery): add Dymo API data verifier source (#2300)
Adds a new passive source that calls the Dymo data verifier endpoint
(https://api.tpeoficial.com/v1/private/secure/verify) with the target
domain. The response is stored on the source for downstream inspection
and any matching canonical domain or didYouMean suggestion is added to
the host set so the rest of the pipeline can resolve it.

The source follows the existing plugin shape: SearchDymo lives in
theHarvester/discovery/dymosearch.py, the API key is read from the
``dymo`` block in api-keys.yaml via Core.dymo_key(), and the engine is
registered in get_supportedengines() and the -b/--source dispatcher.

Tests cover the missing-key path, header/payload shape, the empty,
unrelated-suggestion and non-dict-response branches, and the
get_supportedengines() registration. Closes #2300.
2026-05-06 11:04:34 -04:00
cyberkimchi cd0f300b2e Add Mojeek search source with API and scraping support + README file update 2026-03-09 15:17:54 +01:00
AliceandGitHub eb9b6318fa Merge branch 'laramies:master' into feat-mojeek 2026-03-04 16:10:52 +01:00
L1ghtn1ng b7e3ca274d Add spacing between badges in README.md 2026-02-22 16:03:51 +00:00
L1ghtn1ng 912cfd5ac2 Add Repology packaging status badge to README.md 2026-02-22 16:03:04 +00:00
cyberkimchi ee4f870758 docs: add mojeek to passive modules list in README 2026-02-13 15:26:03 +01:00
RandGitHub 86ad5ad945 feat(discovery): add THC subdomain enumeration module (#2221)
* feat(discovery): add THC subdomain enumeration module

Add new THC module integrating ip.thc.org for subdomain discovery.

Features:
- Free service, no API key required
- No rate limits
- Fast plain-text response parsing

Includes 24 comprehensive tests covering:
- API endpoint validation
- Subdomain search functionality
- Edge cases (empty, unicode, special chars)
- Proxy support
- Response format validation
- Integration with theHarvester framework

Closes #2217

* fix: translate test comments to English

Accidentally pushed the Spanish version of the test file.
This commit updates all docstrings and comments to English.

* fix: correct rate limit documentation and add rate limit header test

- Remove 'no rate limits' claim from README.md
- Add test_api_returns_rate_limit_headers to verify x-ratelimit-* headers

* feat: add explicit rate limit handling

Based on bravesearch.py pattern:
- Detect HTTP 429 status code
- Read x-ratelimit-remaining header
- Implement retry logic with backoff (3 retries, 2s base delay)
- Handle rate limit exceptions

Added test for rate limit settings initialization.
2025-12-31 09:42:17 +00:00
L1ghtn1ng 873129d70c Update README: Replace uv sync --extra dev with uv sync --all-groups in installation instructions 2025-12-21 05:35:40 +00:00
Lee BairdandGitHub 372a5462a4 Added LeakIX info
Updated various service descriptions and pricing details in the README.
2025-10-08 08:41:38 -05:00
Lee BairdandGitHub 3007ad2ecd Add Windvane search engine and query details 2025-10-07 17:15:43 -05:00
Lee BairdandGitHub 27373e0ad3 Update pricing and account details for various services 2025-10-07 17:03:12 -05:00
Lee BairdandGitHub ef1f3cdff5 Add FOFA info. 2025-10-07 13:47:28 -05:00
L1ghtn1ng 914c710966 Add packaging status badge to README.md 2025-09-09 01:34:31 +01:00
ibrahmsql 843bb881a8 brave 2025-08-14 18:59:20 +03:00
Lee BairdandGitHub 34076a897c Removed stale modules 2025-08-09 17:15:35 -05:00
Lee BairdandGitHub ca3a90e50f Updated README to match modules. 2025-08-08 20:00:49 -05:00
Lee BairdandGitHub 872b40232a Updated README, removed bing (#2081)
* Removed bing.

* Added API info

* Update README.md

* Removed shodan from modules section.

* Fixed mistake

* Update README.md

* Removed comment.

* Resolved conflict.

* Added shodan
2025-08-04 22:45:19 +01:00
Lee BairdandGitHub a4dc7625b2 Removed bing. 2025-07-23 13:45:16 -05:00
Lee BairdandGitHub 8fb7c44d51 Updated README.md 2025-07-21 18:17:49 -05:00
Lee BairdandGitHub 7650b3995d Updated README.md with uv info 2025-07-21 13:39:50 -05:00
L1ghtn1ng b97661953e Fix #1999 2025-07-08 21:40:25 +01:00
Lee BairdandGitHub 5ffcca3b54 Noted modules that need API keys (#2038) 2025-06-17 00:56:58 +01:00
Lee BairdandGitHub 89bcdb74f6 Added 4 new modules to README.md 2025-06-13 18:49:19 -05:00
Lee BairdandGitHub 834fc666d6 Updated README with 2 new sources. 2025-06-13 17:27:34 -05:00
L1ghtn1ng e758a94f26 remove anubis module as the site has been down for a while now 2025-05-26 23:24:01 +01:00
Lee BairdandGitHub 75d9ec3ef4 Alphabetize various lists. (#2003) 2025-05-26 00:01:32 +01:00
L1ghtn1ng 6757c510cd Remove BinaryEdge integration from theHarvester.
This commit removes all code, references, and documentation related to the BinaryEdge module. The BinaryEdge API functionality is no longer supported, including its API key requirement and search capabilities. Updated dependencies, README, and source references to reflect this change.
2025-04-17 01:49:06 +01:00
Lee BairdandGitHub dfccf42527 Merge pull request #1972
* Alphabetized new module.

* Merge branch 'laramies:master' into master
2025-04-17 01:27:34 +01:00
4b82e2ee9a Added venacus data leak search engine plugin. (#1968)
* Added venacus data leak search engine plugin.

* Added venacus key to api-keys template.

* Run ruff check fix and format.

---------

Co-authored-by: Anis Mekacher <anis@venacus.com>
2025-04-13 19:06:56 +01:00
Lee BairdandGitHub 57c1ac1145 Added WhoisXML info. 2025-04-02 17:00:00 -05:00
L1ghtn1ng 773860e38d readme updates 2025-01-26 20:05:25 +00:00
J.TownsendandGitHub 4495dc8407 Update README.md 2024-10-26 17:56:31 +01:00
Elior Fureraj 811d1c3b6e * bump python version to 3.10 2024-06-16 20:38:47 +02:00
Bilel MEDIMEGHandJ.Townsend 7717f421ce Fix typo in README 2023-09-25 11:18:22 +01:00
Lee BairdandGitHub efa8dade85 Corrected module capitalization and added fuel URLs 2023-07-27 18:49:34 -05:00
Mohamed Ben 853e645882 Add tomba module 2023-07-20 20:19:45 +01:00
NotoriousRebel 99df21f1e7 Updated readme, removed junk file, and removed debug statement from takeover.py 2023-07-04 21:44:42 -04:00
NotoriousRebel 520b5e60fa Merge branch 'master' of https://github.com/laramies/theHarvester into freshredux 2023-07-02 19:17:15 -04:00
L1ghtn1ng 8e19b68a64 Doc updates 2023-05-31 00:44:03 +01:00
NotoriousRebel dc48fce88b Syncing branch with upstream. 2023-05-21 20:11:22 -04:00
Lee BairdandGitHub b1efc6ab3b Add new module Sitedossier (#1423)
* Create sitedossier.py

* Added new module.

* Added new module.

* Thanks to Jay for the help.

* Added new module to README.
2023-05-13 00:05:33 +01:00
Stefan BratescuandJ.Townsend d09392b5e0 update pentest-tools.com readme entry 2023-05-03 11:13:13 +01:00
Lee BairdandJ.Townsend 817e4bd502 Removed Quant. 2023-04-17 15:05:57 +01:00
MattandGitHub 90082dbaa0 Added 4 new sources, Removed 2 Sources, Added custom DNS resolver, backported to python3.9, and other misc changes (#1381)
* Added call to api endpoint to explicitly gather subdomains from zoomeye, updated user agents, replaced orjson with ujson, and fixed substring not found error.

* Updated orjson to ujson.

* Fixed semantic error in html check in google workaround.

* Fixed flake8 errors.

* Fixed VT to use API.

* Fixed virustotal module.

* Fixed possible edge case that could possibly cause an infinite loop.

* Removed broken modules.

* Added 4 new sources: brave, criminalip, hunterhow, and subdomainfinderc99, added dnsresolve, and other misc changes.

* Added TODO comment.

* Fixed bin\theHarvester to allow python3.9

* Pep8 newline at end of file.

* Fixed error when passing in comma seperated resolvers and allow for user to pass in --dns-resolve flag with no arguments to use default resolvers that user has.
2023-04-10 18:51:04 +01:00
NotoriousRebel fa57681871 Fixed error when passing in comma seperated resolvers and allow for user to pass in --dns-resolve flag with no arguments to use default resolvers that user has. 2023-04-08 12:44:43 -04:00
J.TownsendandGitHub acf49c778a use webp image in the readme 2023-02-05 21:50:14 +00:00