WEBDIENSTE/theHarvester
1
0
Fork 0
mirror of https://github.com/laramies/theHarvester.git synced 2026-09-07 18:27:42 +02:00
Code Issues Packages Projects Releases Wiki Activity
4,198 Commits 2 Branches 38 Tags
fa2dedd38becaa68a3fe1daa68e7ba4e47e262f2
Commit Graph
4198 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
L1ghtn1ng fa2dedd38b Refactor and standardize output handling with new utilities in theHarvester.lib.output, add tests for print_linkedin_sections and sorted_unique. Fix regex inaccuracies and enhance CORS validation. 2026-01-24 11:28:20 +00:00
L1ghtn1ng 46865337aa update dev deps 2026-01-24 10:58:22 +00:00
L1ghtn1ng 15872350fd Remove unnecessary comments and improve BuiltWith URL handling logic in __main__.py 2026-01-24 10:37:19 +00:00
dependabot[bot]andJ.Townsend 56afc55bee Bump wheel from 0.45.1 to 0.46.2 
Bumps [wheel](https://github.com/pypa/wheel) from 0.45.1 to 0.46.2.
- [Release notes](https://github.com/pypa/wheel/releases)
- [Changelog](https://github.com/pypa/wheel/blob/main/docs/news.rst)
- [Commits](https://github.com/pypa/wheel/compare/0.45.1...0.46.2)

---
updated-dependencies:
- dependency-name: wheel
  dependency-version: 0.46.2
  dependency-type: direct:development
...

Signed-off-by: dependabot[bot] <support@github.com>
 2026-01-22 19:24:38 +00:00
L1ghtn1ng ffe1f3a832 Synchronize api-keys.yaml with Core class API key references and add a test to ensure consistency 4.10.0 2026-01-18 01:15:26 +00:00
Lee BairdandGitHub 6be673fad4 Add Bitbucket API key entry in api-keys.yaml 2026-01-17 18:30:34 -06:00
L1ghtn1ng 3a5d504b66 Dockerfile: Add apt-get update/upgrade steps and clean up cache 2026-01-17 03:32:31 +00:00
L1ghtn1ng 8ddcd1a8b2 Switch to ruff-action for linting and formatting in theHarvester GitHub workflow 2026-01-17 02:31:44 +00:00
L1ghtn1ng 31861c191b Add support for LeakIX API key and improve header configuration 2026-01-17 01:37:10 +00:00
L1ghtn1ng 40759146f1 Update dependencies: Bump aiodns to 4.0.0, ruff to 0.14.13, ty to 0.0.12, filelock to 3.20.3, and librt to 0.7.8 2026-01-17 01:36:47 +00:00
github-actions 7c6dec53ca Apply ruff fixes and formatting 2026-01-17 01:05:54 +00:00
L1ghtn1ng 3d7489c962 Add security tests and enhance input sanitization mechanisms 
- Introduced comprehensive security tests, including CORS configuration, XML injection prevention, path traversal prevention, and information disclosure protection.
- Improved input sanitization for filenames and XML content to mitigate injection attacks and ensure safe handling.
- Updated CORS middleware to restrict HTTP methods and block credentials with wildcard origins.
- Removed traceback exposure in API error responses, even in debug mode.
- Verified rate limiting and input validation for sensitive API endpoints.
- Ensured no hardcoded secrets are present in the main codebase.
 2026-01-17 01:05:40 +00:00
Lee BairdandGitHub e72876f427 Merge pull request #2225 from leebaird/master 
Remove bing_key method from Core class
 2026-01-16 07:48:36 -06:00
Lee BairdandGitHub 814c7811d8 Remove bing_key method from Core class 
Removed the bing_key method from Core class.
 2026-01-16 07:44:32 -06:00
L1ghtn1ng bc2fce07cc Update proxy parameter handling in tests: expand default structure to include both "http" and "socks5" fields 2026-01-06 22:14:02 +00:00
L1ghtn1ng 0991356bbd Fix mypy errors 2026-01-06 21:46:52 +00:00
L1ghtn1ng 422d62c2e9 Bump version to 4.10.0 2026-01-06 21:40:25 +00:00
L1ghtn1ng e38bb8fbb4 Fix issue #469 2026-01-06 21:37:49 +00:00
L1ghtn1ng da8a2c8561 Update dependencies: Bump aiohttp to 3.13.3, certifi to 2026.1.4, and ty to 0.0.9 2026-01-06 18:36:32 +00:00
L1ghtn1ng e853b81aa2 update dev deps 2025-12-31 10:52:55 +00:00
J.TownsendandGitHub ba85366644 Bump version from 4.9.1 to 4.9.2 for release 4.9.2 2025-12-31 09:43:47 +00:00
RandGitHub 86ad5ad945 feat(discovery): add THC subdomain enumeration module (#2221) 
* feat(discovery): add THC subdomain enumeration module

Add new THC module integrating ip.thc.org for subdomain discovery.

Features:
- Free service, no API key required
- No rate limits
- Fast plain-text response parsing

Includes 24 comprehensive tests covering:
- API endpoint validation
- Subdomain search functionality
- Edge cases (empty, unicode, special chars)
- Proxy support
- Response format validation
- Integration with theHarvester framework

Closes #2217

* fix: translate test comments to English

Accidentally pushed the Spanish version of the test file.
This commit updates all docstrings and comments to English.

* fix: correct rate limit documentation and add rate limit header test

- Remove 'no rate limits' claim from README.md
- Add test_api_returns_rate_limit_headers to verify x-ratelimit-* headers

* feat: add explicit rate limit handling

Based on bravesearch.py pattern:
- Detect HTTP 429 status code
- Read x-ratelimit-remaining header
- Implement retry logic with backoff (3 retries, 2s base delay)
- Handle rate limit exceptions

Added test for rate limit settings initialization.
 2025-12-31 09:42:17 +00:00
Branch VincentandGitHub 26c8d639be Merge pull request #2220 
* support python 3.14

* fix syntax warnings

* bump dockerfile to 3.14, optimize build time + size
4.9.1 		2025-12-30 09:36:19 +00:00
J.TownsendandGitHub d03790e6da Remove threatminer from ci 2025-12-23 00:12:58 +00:00
dependabot[bot]andJ.Townsend 2857c35141 Bump astral-sh/setup-uv from 5 to 7 
Bumps [astral-sh/setup-uv](https://github.com/astral-sh/setup-uv) from 5 to 7.
- [Release notes](https://github.com/astral-sh/setup-uv/releases)
- [Commits](https://github.com/astral-sh/setup-uv/compare/v5...v7)

---
updated-dependencies:
- dependency-name: astral-sh/setup-uv
  dependency-version: '7'
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
 2025-12-23 00:11:00 +00:00
github-actions 0e62293367 Apply ruff fixes and formatting 2025-12-21 07:00:42 +00:00
L1ghtn1ng 7f11a5c1b7 Remove ThreatMiner module and fix mypy errors, also update useragent list 2025-12-21 07:00:27 +00:00
L1ghtn1ng 54e9178851 Add uv.lock to manage dependencies and ensure consistent builds 4.9.0 2025-12-21 05:53:24 +00:00
L1ghtn1ng 9dd34fbfc9 Update CI: Replace setup-python with setup-uv and streamline dependency installation 2025-12-21 05:48:09 +00:00
L1ghtn1ng 873129d70c Update README: Replace uv sync --extra dev with uv sync --all-groups in installation instructions 2025-12-21 05:35:40 +00:00
L1ghtn1ng 2bb20d29ad Refactor hackertarget module follow core logic to make it consistent with the rest of the code base since the api key support got added. 2025-12-21 05:31:38 +00:00
Raghav ChowdaryandGitHub 4c803566a0 Merge pull request #2142 
* feat: add Hackertarget API key support (fixes #2122)

* feat(hackertarget): add API key support + tests (fixes #2122)

* style: run ruff fixes
 2025-12-21 05:13:39 +00:00
L1ghtn1ng f7c5443ff6 Update screenshot module to fix deprecation warnings 2025-12-21 05:11:41 +00:00
dependabot[bot]andJ.Townsend bf767daae4 Bump mypy from 1.19.0 to 1.19.1 
Bumps [mypy](https://github.com/python/mypy) from 1.19.0 to 1.19.1.
- [Changelog](https://github.com/python/mypy/blob/master/CHANGELOG.md)
- [Commits](https://github.com/python/mypy/compare/v1.19.0...v1.19.1)

---
updated-dependencies:
- dependency-name: mypy
  dependency-version: 1.19.1
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
 2025-12-19 07:56:40 +00:00
dependabot[bot]andJ.Townsend e6f5544060 Bump aiosqlite from 0.21.0 to 0.22.0 
Bumps [aiosqlite](https://github.com/omnilib/aiosqlite) from 0.21.0 to 0.22.0.
- [Changelog](https://github.com/omnilib/aiosqlite/blob/main/CHANGELOG.md)
- [Commits](https://github.com/omnilib/aiosqlite/compare/v0.21.0...v0.22.0)

---
updated-dependencies:
- dependency-name: aiosqlite
  dependency-version: 0.22.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
 2025-12-19 07:55:56 +00:00
dependabot[bot]andJ.Townsend 525d25f8ee Bump ruff from 0.14.8 to 0.14.10 
Bumps [ruff](https://github.com/astral-sh/ruff) from 0.14.8 to 0.14.10.
- [Release notes](https://github.com/astral-sh/ruff/releases)
- [Changelog](https://github.com/astral-sh/ruff/blob/main/CHANGELOG.md)
- [Commits](https://github.com/astral-sh/ruff/compare/0.14.8...0.14.10)

---
updated-dependencies:
- dependency-name: ruff
  dependency-version: 0.14.10
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
 2025-12-19 07:55:03 +00:00
dependabot[bot]andJ.Townsend 8fe590984e Bump fastapi from 0.124.4 to 0.125.0 
Bumps [fastapi](https://github.com/fastapi/fastapi) from 0.124.4 to 0.125.0.
- [Release notes](https://github.com/fastapi/fastapi/releases)
- [Commits](https://github.com/fastapi/fastapi/compare/0.124.4...0.125.0)

---
updated-dependencies:
- dependency-name: fastapi
  dependency-version: 0.125.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
 2025-12-18 03:40:32 +00:00
dependabot[bot]andJ.Townsend d8abc194bf Bump censys from 2.2.18 to 2.2.19 
Bumps [censys](https://github.com/censys/censys-python) from 2.2.18 to 2.2.19.
- [Release notes](https://github.com/censys/censys-python/releases)
- [Commits](https://github.com/censys/censys-python/compare/v2.2.18...v2.2.19)

---
updated-dependencies:
- dependency-name: censys
  dependency-version: 2.2.19
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
 2025-12-15 18:58:39 +00:00
dependabot[bot]andJ.Townsend 720167222c Bump fastapi from 0.124.2 to 0.124.4 
Bumps [fastapi](https://github.com/fastapi/fastapi) from 0.124.2 to 0.124.4.
- [Release notes](https://github.com/fastapi/fastapi/releases)
- [Commits](https://github.com/fastapi/fastapi/compare/0.124.2...0.124.4)

---
updated-dependencies:
- dependency-name: fastapi
  dependency-version: 0.124.4
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
 2025-12-12 23:13:05 +00:00
dependabot[bot]andJ.Townsend f04adcb1da Bump aiodns from 3.6.0 to 3.6.1 
Bumps [aiodns](https://github.com/saghul/aiodns) from 3.6.0 to 3.6.1.
- [Release notes](https://github.com/saghul/aiodns/releases)
- [Changelog](https://github.com/aio-libs/aiodns/blob/master/ChangeLog)
- [Commits](https://github.com/saghul/aiodns/compare/v3.6.0...v3.6.1)

---
updated-dependencies:
- dependency-name: aiodns
  dependency-version: 3.6.1
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
 2025-12-11 23:37:04 +00:00
dependabot[bot]andJ.Townsend fe5b37ee77 Bump fastapi from 0.124.0 to 0.124.2 
Bumps [fastapi](https://github.com/fastapi/fastapi) from 0.124.0 to 0.124.2.
- [Release notes](https://github.com/fastapi/fastapi/releases)
- [Commits](https://github.com/fastapi/fastapi/compare/0.124.0...0.124.2)

---
updated-dependencies:
- dependency-name: fastapi
  dependency-version: 0.124.2
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
 2025-12-11 01:07:42 +00:00
dependabot[bot]andJ.Townsend 1319533c59 Bump playwright from 1.56.0 to 1.57.0 
Bumps [playwright](https://github.com/microsoft/playwright-python) from 1.56.0 to 1.57.0.
- [Release notes](https://github.com/microsoft/playwright-python/releases)
- [Commits](https://github.com/microsoft/playwright-python/compare/v1.56.0...v1.57.0)

---
updated-dependencies:
- dependency-name: playwright
  dependency-version: 1.57.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
 2025-12-10 02:01:11 +00:00
dependabot[bot]andJ.Townsend 311d26d489 Bump fastapi from 0.123.10 to 0.124.0 
Bumps [fastapi](https://github.com/fastapi/fastapi) from 0.123.10 to 0.124.0.
- [Release notes](https://github.com/fastapi/fastapi/releases)
- [Commits](https://github.com/fastapi/fastapi/compare/0.123.10...0.124.0)

---
updated-dependencies:
- dependency-name: fastapi
  dependency-version: 0.124.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
 2025-12-08 22:45:48 +00:00
dependabot[bot]andJ.Townsend 54be41a1e6 Bump pytest from 9.0.1 to 9.0.2 
Bumps [pytest](https://github.com/pytest-dev/pytest) from 9.0.1 to 9.0.2.
- [Release notes](https://github.com/pytest-dev/pytest/releases)
- [Changelog](https://github.com/pytest-dev/pytest/blob/main/CHANGELOG.rst)
- [Commits](https://github.com/pytest-dev/pytest/compare/9.0.1...9.0.2)

---
updated-dependencies:
- dependency-name: pytest
  dependency-version: 9.0.2
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
 2025-12-08 22:45:22 +00:00
J.TownsendandGitHub f9c061b0dc Merge pull request #2204 from x9xhack/master 2025-12-06 04:59:50 +00:00
x9xhackandGitHub 410c20adc5 Fix winloop dependency condition for Windows 2025-12-06 07:55:24 +03:00
dependabot[bot]andJ.Townsend cb67b4837a Bump winloop from 0.3.1 to 0.4.0 
Bumps [winloop](https://github.com/Vizonex/Winloop) from 0.3.1 to 0.4.0.
- [Release notes](https://github.com/Vizonex/Winloop/releases)
- [Commits](https://github.com/Vizonex/Winloop/compare/v0.3.1...v0.4.0)

---
updated-dependencies:
- dependency-name: winloop
  dependency-version: 0.4.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
 2025-12-05 22:56:15 +00:00
dependabot[bot]andJ.Townsend bc8394af6f Bump ruff from 0.14.7 to 0.14.8 
Bumps [ruff](https://github.com/astral-sh/ruff) from 0.14.7 to 0.14.8.
- [Release notes](https://github.com/astral-sh/ruff/releases)
- [Changelog](https://github.com/astral-sh/ruff/blob/main/CHANGELOG.md)
- [Commits](https://github.com/astral-sh/ruff/compare/0.14.7...0.14.8)

---
updated-dependencies:
- dependency-name: ruff
  dependency-version: 0.14.8
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
 2025-12-05 22:54:21 +00:00
dependabot[bot]andJ.Townsend 018c131ac2 Bump fastapi from 0.123.5 to 0.123.10 
Bumps [fastapi](https://github.com/fastapi/fastapi) from 0.123.5 to 0.123.10.
- [Release notes](https://github.com/fastapi/fastapi/releases)
- [Commits](https://github.com/fastapi/fastapi/compare/0.123.5...0.123.10)

---
updated-dependencies:
- dependency-name: fastapi
  dependency-version: 0.123.10
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
 2025-12-05 22:54:04 +00:00
dependabot[bot]andJ.Townsend 47d27a733a Bump aiodns from 3.5.0 to 3.6.0 
Bumps [aiodns](https://github.com/saghul/aiodns) from 3.5.0 to 3.6.0.
- [Release notes](https://github.com/saghul/aiodns/releases)
- [Changelog](https://github.com/aio-libs/aiodns/blob/master/ChangeLog)
- [Commits](https://github.com/saghul/aiodns/compare/v3.5.0...v3.6.0)

---
updated-dependencies:
- dependency-name: aiodns
  dependency-version: 3.6.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
 2025-12-05 22:53:34 +00:00