Commit Graph
197 Commits
Author SHA1 Message Date
RandGitHub 86ad5ad945 feat(discovery): add THC subdomain enumeration module (#2221)
* feat(discovery): add THC subdomain enumeration module

Add new THC module integrating ip.thc.org for subdomain discovery.

Features:
- Free service, no API key required
- No rate limits
- Fast plain-text response parsing

Includes 24 comprehensive tests covering:
- API endpoint validation
- Subdomain search functionality
- Edge cases (empty, unicode, special chars)
- Proxy support
- Response format validation
- Integration with theHarvester framework

Closes #2217

* fix: translate test comments to English

Accidentally pushed the Spanish version of the test file.
This commit updates all docstrings and comments to English.

* fix: correct rate limit documentation and add rate limit header test

- Remove 'no rate limits' claim from README.md
- Add test_api_returns_rate_limit_headers to verify x-ratelimit-* headers

* feat: add explicit rate limit handling

Based on bravesearch.py pattern:
- Detect HTTP 429 status code
- Read x-ratelimit-remaining header
- Implement retry logic with backoff (3 retries, 2s base delay)
- Handle rate limit exceptions

Added test for rate limit settings initialization.
2025-12-31 09:42:17 +00:00
L1ghtn1ng 873129d70c Update README: Replace uv sync --extra dev with uv sync --all-groups in installation instructions 2025-12-21 05:35:40 +00:00
Lee BairdandGitHub 372a5462a4 Added LeakIX info
Updated various service descriptions and pricing details in the README.
2025-10-08 08:41:38 -05:00
Lee BairdandGitHub 3007ad2ecd Add Windvane search engine and query details 2025-10-07 17:15:43 -05:00
Lee BairdandGitHub 27373e0ad3 Update pricing and account details for various services 2025-10-07 17:03:12 -05:00
Lee BairdandGitHub ef1f3cdff5 Add FOFA info. 2025-10-07 13:47:28 -05:00
L1ghtn1ng 914c710966 Add packaging status badge to README.md 2025-09-09 01:34:31 +01:00
ibrahmsql 843bb881a8 brave 2025-08-14 18:59:20 +03:00
Lee BairdandGitHub 34076a897c Removed stale modules 2025-08-09 17:15:35 -05:00
Lee BairdandGitHub ca3a90e50f Updated README to match modules. 2025-08-08 20:00:49 -05:00
Lee BairdandGitHub 872b40232a Updated README, removed bing (#2081)
* Removed bing.

* Added API info

* Update README.md

* Removed shodan from modules section.

* Fixed mistake

* Update README.md

* Removed comment.

* Resolved conflict.

* Added shodan
2025-08-04 22:45:19 +01:00
Lee BairdandGitHub a4dc7625b2 Removed bing. 2025-07-23 13:45:16 -05:00
Lee BairdandGitHub 8fb7c44d51 Updated README.md 2025-07-21 18:17:49 -05:00
Lee BairdandGitHub 7650b3995d Updated README.md with uv info 2025-07-21 13:39:50 -05:00
L1ghtn1ng b97661953e Fix #1999 2025-07-08 21:40:25 +01:00
Lee BairdandGitHub 5ffcca3b54 Noted modules that need API keys (#2038) 2025-06-17 00:56:58 +01:00
Lee BairdandGitHub 89bcdb74f6 Added 4 new modules to README.md 2025-06-13 18:49:19 -05:00
Lee BairdandGitHub 834fc666d6 Updated README with 2 new sources. 2025-06-13 17:27:34 -05:00
L1ghtn1ng e758a94f26 remove anubis module as the site has been down for a while now 2025-05-26 23:24:01 +01:00
Lee BairdandGitHub 75d9ec3ef4 Alphabetize various lists. (#2003) 2025-05-26 00:01:32 +01:00
L1ghtn1ng 6757c510cd Remove BinaryEdge integration from theHarvester.
This commit removes all code, references, and documentation related to the BinaryEdge module. The BinaryEdge API functionality is no longer supported, including its API key requirement and search capabilities. Updated dependencies, README, and source references to reflect this change.
2025-04-17 01:49:06 +01:00
Lee BairdandGitHub dfccf42527 Merge pull request #1972
* Alphabetized new module.

* Merge branch 'laramies:master' into master
2025-04-17 01:27:34 +01:00
4b82e2ee9a Added venacus data leak search engine plugin. (#1968)
* Added venacus data leak search engine plugin.

* Added venacus key to api-keys template.

* Run ruff check fix and format.

---------

Co-authored-by: Anis Mekacher <anis@venacus.com>
2025-04-13 19:06:56 +01:00
Lee BairdandGitHub 57c1ac1145 Added WhoisXML info. 2025-04-02 17:00:00 -05:00
L1ghtn1ng 773860e38d readme updates 2025-01-26 20:05:25 +00:00
J.TownsendandGitHub 4495dc8407 Update README.md 2024-10-26 17:56:31 +01:00
Elior Fureraj 811d1c3b6e * bump python version to 3.10 2024-06-16 20:38:47 +02:00
Bilel MEDIMEGHandJ.Townsend 7717f421ce Fix typo in README 2023-09-25 11:18:22 +01:00
Lee BairdandGitHub efa8dade85 Corrected module capitalization and added fuel URLs 2023-07-27 18:49:34 -05:00
Mohamed Ben 853e645882 Add tomba module 2023-07-20 20:19:45 +01:00
NotoriousRebel 99df21f1e7 Updated readme, removed junk file, and removed debug statement from takeover.py 2023-07-04 21:44:42 -04:00
NotoriousRebel 520b5e60fa Merge branch 'master' of https://github.com/laramies/theHarvester into freshredux 2023-07-02 19:17:15 -04:00
L1ghtn1ng 8e19b68a64 Doc updates 2023-05-31 00:44:03 +01:00
NotoriousRebel dc48fce88b Syncing branch with upstream. 2023-05-21 20:11:22 -04:00
Lee BairdandGitHub b1efc6ab3b Add new module Sitedossier (#1423)
* Create sitedossier.py

* Added new module.

* Added new module.

* Thanks to Jay for the help.

* Added new module to README.
2023-05-13 00:05:33 +01:00
Stefan BratescuandJ.Townsend d09392b5e0 update pentest-tools.com readme entry 2023-05-03 11:13:13 +01:00
Lee BairdandJ.Townsend 817e4bd502 Removed Quant. 2023-04-17 15:05:57 +01:00
MattandGitHub 90082dbaa0 Added 4 new sources, Removed 2 Sources, Added custom DNS resolver, backported to python3.9, and other misc changes (#1381)
* Added call to api endpoint to explicitly gather subdomains from zoomeye, updated user agents, replaced orjson with ujson, and fixed substring not found error.

* Updated orjson to ujson.

* Fixed semantic error in html check in google workaround.

* Fixed flake8 errors.

* Fixed VT to use API.

* Fixed virustotal module.

* Fixed possible edge case that could possibly cause an infinite loop.

* Removed broken modules.

* Added 4 new sources: brave, criminalip, hunterhow, and subdomainfinderc99, added dnsresolve, and other misc changes.

* Added TODO comment.

* Fixed bin\theHarvester to allow python3.9

* Pep8 newline at end of file.

* Fixed error when passing in comma seperated resolvers and allow for user to pass in --dns-resolve flag with no arguments to use default resolvers that user has.
2023-04-10 18:51:04 +01:00
NotoriousRebel fa57681871 Fixed error when passing in comma seperated resolvers and allow for user to pass in --dns-resolve flag with no arguments to use default resolvers that user has. 2023-04-08 12:44:43 -04:00
J.TownsendandGitHub acf49c778a use webp image in the readme 2023-02-05 21:50:14 +00:00
Alexandre ZANNIandGitHub fb40e0838c fix rawsec badge 2023-01-11 22:52:46 +01:00
L1ghtn1ng 81f982d278 Change bufferoverun to use api 2022-11-24 22:30:51 +00:00
J.TownsendandGitHub 81af1cb8d0 remove lgtm badge due to it shutting down 2022-08-21 02:51:02 +01:00
0b65a6380a BeVigil Source Added (#1180)
* BeVigil Source Added

* remove/revoke api key

* Remove unnecessary spaces

Co-authored-by: J.Townsend <L1ghtn1ng@users.noreply.github.com>

* Refactor for flake8

Co-authored-by: J.Townsend <L1ghtn1ng@users.noreply.github.com>

Co-authored-by: J.Townsend <L1ghtn1ng@users.noreply.github.com>
2022-08-13 23:45:47 +01:00
Lee BairdandGitHub 121e23b3f9 Updated README and misc. (#1171) 2022-08-06 21:12:08 +01:00
J.TownsendandGitHub 22e1eb9bdb A lot of changes and fixes (#1168)
* add pytest.ini file and fix test warning in qwantsearch

* ignore unsed imports

* Remove n45ht as the api is down

* Remove modules that use google as google blocks you after one go

* Remove missed code from when removing modules that use google

* Remove missed code from when removing modules that use google

* Add new vt-py dep and update pytest

* WIP virustotal migration to api

* Remove test that fails due to how the api returns not found entries

* Remove entries from myparser re google

* update fastapi and starlette

* update version

* Update dockerfile to use dev ubuntu version

* remove spyse module

* remove spyse dep

* fix tests by removing un-needed tests and indentation fix

* fix some mypy errors

* remove spyse and remove color output and fix mypy error

* flake8 fixes

* bump version and set it a dev version
2022-08-03 23:12:59 +01:00
Vipin Mahawar 99e1d01920 Binaryedge Placeholder Value Fix in README.md 2022-06-09 12:40:40 +05:30
MattandGitHub 2ccaf692dc Update README.md 2021-10-30 13:42:12 -04:00
J.TownsendandGitHub 15fe11130c Add new n45ht module, lots of results returned (#899)
* Add new fullhunt module

* Remove python checks in the run script as they are not needed anymore

* Remove netcraft from ci as it is not needed

* Update Dockerfile to fix security issues

* Add new n45ht module, lots of results returned
2021-10-24 04:10:37 +01:00
J.TownsendandGitHub 929a642bab Add new fullhunt module (#894) 2021-10-18 00:29:18 +01:00