- Introduced comprehensive security tests, including CORS configuration, XML injection prevention, path traversal prevention, and information disclosure protection.
- Improved input sanitization for filenames and XML content to mitigate injection attacks and ensure safe handling.
- Updated CORS middleware to restrict HTTP methods and block credentials with wildcard origins.
- Removed traceback exposure in API error responses, even in debug mode.
- Verified rate limiting and input validation for sensitive API endpoints.
- Ensured no hardcoded secrets are present in the main codebase.
* feat(discovery): add THC subdomain enumeration module
Add new THC module integrating ip.thc.org for subdomain discovery.
Features:
- Free service, no API key required
- No rate limits
- Fast plain-text response parsing
Includes 24 comprehensive tests covering:
- API endpoint validation
- Subdomain search functionality
- Edge cases (empty, unicode, special chars)
- Proxy support
- Response format validation
- Integration with theHarvester framework
Closes#2217
* fix: translate test comments to English
Accidentally pushed the Spanish version of the test file.
This commit updates all docstrings and comments to English.
* fix: correct rate limit documentation and add rate limit header test
- Remove 'no rate limits' claim from README.md
- Add test_api_returns_rate_limit_headers to verify x-ratelimit-* headers
* feat: add explicit rate limit handling
Based on bravesearch.py pattern:
- Detect HTTP 429 status code
- Read x-ratelimit-remaining header
- Implement retry logic with backoff (3 retries, 2s base delay)
- Handle rate limit exceptions
Added test for rate limit settings initialization.
- Implemented unittests to validate `process` method behavior under error, retry, and pagination conditions.
- Introduced safeguards against infinite loops in pagination and retries within `SearchGithubCode`.
Introduce the `next_page_or_end` method in `githubcode.py` to handle pagination. Additionally, modify the test case and upgrade pytest settings in `pyproject.toml` to the latest versions, ensuring compatibility and updated configurations.
* Added call to api endpoint to explicitly gather subdomains from zoomeye, updated user agents, replaced orjson with ujson, and fixed substring not found error.
* Updated orjson to ujson.
* Fixed semantic error in html check in google workaround.
* Fixed flake8 errors.
* Fixed VT to use API.
* Fixed virustotal module.
* Fixed possible edge case that could possibly cause an infinite loop.
* Removed broken modules.
* Added 4 new sources: brave, criminalip, hunterhow, and subdomainfinderc99, added dnsresolve, and other misc changes.
* Added TODO comment.
* Fixed bin\theHarvester to allow python3.9
* Pep8 newline at end of file.
* Fixed error when passing in comma seperated resolvers and allow for user to pass in --dns-resolve flag with no arguments to use default resolvers that user has.
* add pytest.ini file and fix test warning in qwantsearch
* ignore unsed imports
* Remove n45ht as the api is down
* Remove modules that use google as google blocks you after one go
* Remove missed code from when removing modules that use google
* Remove missed code from when removing modules that use google
* Add new vt-py dep and update pytest
* WIP virustotal migration to api
* Remove test that fails due to how the api returns not found entries
* Remove entries from myparser re google
* update fastapi and starlette
* update version
* Update dockerfile to use dev ubuntu version
* remove spyse module
* remove spyse dep
* fix tests by removing un-needed tests and indentation fix
* fix some mypy errors
* remove spyse and remove color output and fix mypy error
* flake8 fixes
* bump version and set it a dev version
* Add new fullhunt module
* Remove python checks in the run script as they are not needed anymore
* Remove netcraft from ci as it is not needed
* Update Dockerfile to fix security issues
* Add new n45ht module, lots of results returned
L1ghtn1ng