Commit Graph
145 Commits
Author SHA1 Message Date
L1ghtn1ng cc1c653ad4 Move continue-on-error for Python 3.13 to individual steps
Shifted the `continue-on-error` condition for Python 3.13 from the job level to each individual step within the GitHub Actions workflow. This ensures more granular control over error handling, allowing other steps to succeed or fail independently.
2024-10-16 05:51:41 +01:00
L1ghtn1ng 40ecb55c41 Revise error handling in CI workflow
Consolidate error handling by setting `continue-on-error` at the job level with a condition for Python 3.13. Remove redundant `continue-on-error` directives from individual steps to streamline the workflow file.
2024-10-16 05:29:30 +01:00
L1ghtn1ng 014ed70e1e ci tweak 2024-10-16 05:08:37 +01:00
L1ghtn1ng d4162486a4 ci tweak 2024-10-16 05:03:56 +01:00
L1ghtn1ng dd3bef6c40 Increase parallelism and add continue-on-error to steps
Updated the max-parallel setting from 8 to 10 to increase workflow efficiency. Also added the continue-on-error parameter to multiple steps to ensure the workflow continues despite errors. This improves the robustness and speed of the CI process.
2024-10-16 04:59:00 +01:00
L1ghtn1ng 62277a37ba Add Python 3.13 to GitHub workflow
Extend the CI matrix to include Python 3.13 alongside existing versions. This ensures compatibility and automated testing for the latest Python release.
2024-10-16 04:31:57 +01:00
L1ghtn1ng 2c7b2bb249 Fix proxy handling and re-enable mypy type checking as tests fixed
Updated aiohttp calls to handle proxies correctly by converting them to strings when necessary. Removed an unnecessary blank line in core.py, and re-enabled static type checking with mypy in the GitHub workflow now that mypy tests are passing
2024-10-16 04:27:20 +01:00
L1ghtn1ng 7550063c17 Update Python version requirements and clean up code
Upgraded the minimum required Python version to 3.11 across various files. Removed `pyre-check` from dependencies and temporarily disabled mypy static type checker in the CI configuration. Additionally, simplified several condition checks and fixed formatting inconsistencies in theHarvester modules.
2024-10-16 03:27:09 +01:00
Branch VincentandGitHub 1373857d36 fix: create static directory when missing (#1721) 2024-04-27 17:46:47 +01:00
L1ghtn1ng fbf9244bd8 Replace various linters with Ruff
This commit removes individual linters and formatters such as black, flake8 and isort and replaces them with Ruff. The relevant configuration files have been deleted and the settings have been merged into pyproject.toml. Also, python-version in the GitHub workflow file has been updated to include 3.12. Ruff's linter and formatter is now used in the GitHub workflows as well.
2024-03-02 22:27:55 +00:00
dependabot[bot]andJ.Townsend 8ee60f7db9 chore(deps): bump github/codeql-action from 2 to 3
Bumps [github/codeql-action](https://github.com/github/codeql-action) from 2 to 3.
- [Release notes](https://github.com/github/codeql-action/releases)
- [Changelog](https://github.com/github/codeql-action/blob/main/CHANGELOG.md)
- [Commits](https://github.com/github/codeql-action/compare/v2...v3)

---
updated-dependencies:
- dependency-name: github/codeql-action
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-12-13 23:23:48 +00:00
dependabot[bot]andJ.Townsend 06e36c3266 chore(deps): bump actions/setup-python from 4 to 5
Bumps [actions/setup-python](https://github.com/actions/setup-python) from 4 to 5.
- [Release notes](https://github.com/actions/setup-python/releases)
- [Commits](https://github.com/actions/setup-python/compare/v4...v5)

---
updated-dependencies:
- dependency-name: actions/setup-python
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-12-11 23:20:30 +00:00
Branch VincentandGitHub 4d825aaf33 build: migrate to pep517 backend (#1505)
* build: migrate to pep517 backend

* test loading config files
2023-10-08 19:45:23 +01:00
dependabot[bot]andJ.Townsend feb687b40b chore(deps): bump actions/checkout from 3 to 4
Bumps [actions/checkout](https://github.com/actions/checkout) from 3 to 4.
- [Release notes](https://github.com/actions/checkout/releases)
- [Changelog](https://github.com/actions/checkout/blob/main/CHANGELOG.md)
- [Commits](https://github.com/actions/checkout/compare/v3...v4)

---
updated-dependencies:
- dependency-name: actions/checkout
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
2023-09-04 22:34:10 +01:00
L1ghtn1ng 5b9e6b25f3 Update Python version in Github Actions
The Python version used in the Github Actions workflow, specifically in the 'theHarvester.yml' file, has been updated from 3.10.9 to 3.10.12. The reason for this change is to ensure the project files and the testing environment are using the latest stable version of the Python interpreter
2023-08-13 03:02:56 +01:00
yoonthegoonandJ.Townsend 7f4c3f6ca2 #1492 add black and isort to workflow 2023-07-27 01:34:21 +01:00
yoonthegoon b7aba8ed7f add flake8 config flag 2023-07-23 22:56:13 -04:00
J.TownsendandGitHub 28ac939796 remove un-needed ci test 2023-05-12 23:59:58 +01:00
J.TownsendandGitHub cd462b37f9 Multiple fixes (#1382)
* Fix api keys and proxies file in docker file and compose file

* update alpine version to 3.17.3

* flake8 fixes

* some mypy fixes

* Remove uneeded sublister ci test

* Remove another uneeded CI test and remove source files for uneeded modules
2023-04-10 20:27:38 +01:00
L1ghtn1ng 70cf550573 run static type checking last 2022-12-10 00:00:15 +00:00
L1ghtn1ng 57d082b09b fix ci python 3.10 testing 2022-12-09 23:49:48 +00:00
L1ghtn1ng 7e6e7eaba4 Update ci version numbers 2022-12-09 23:47:33 +00:00
L1ghtn1ng 3a2c180295 Remove missed omnisint ci removal and version import fix 2022-11-13 01:33:01 +00:00
L1ghtn1ng 46760915c3 Bump theHavester required python version to be 3.10+ 2022-11-13 01:21:28 +00:00
J.TownsendandGitHub 72d8dcad21 Flake8 fixes (#1181)
* Flake8 fixes

* Remove test due to bufferover run moving to api only
2022-08-14 01:23:56 +01:00
J.TownsendandGitHub 22e1eb9bdb A lot of changes and fixes (#1168)
* add pytest.ini file and fix test warning in qwantsearch

* ignore unsed imports

* Remove n45ht as the api is down

* Remove modules that use google as google blocks you after one go

* Remove missed code from when removing modules that use google

* Remove missed code from when removing modules that use google

* Add new vt-py dep and update pytest

* WIP virustotal migration to api

* Remove test that fails due to how the api returns not found entries

* Remove entries from myparser re google

* update fastapi and starlette

* update version

* Update dockerfile to use dev ubuntu version

* remove spyse module

* remove spyse dep

* fix tests by removing un-needed tests and indentation fix

* fix some mypy errors

* remove spyse and remove color output and fix mypy error

* flake8 fixes

* bump version and set it a dev version
2022-08-03 23:12:59 +01:00
dependabot[bot]andGitHub 04875206ba chore(deps): bump actions/setup-python from 3 to 4
Bumps [actions/setup-python](https://github.com/actions/setup-python) from 3 to 4.
- [Release notes](https://github.com/actions/setup-python/releases)
- [Commits](https://github.com/actions/setup-python/compare/v3...v4)

---
updated-dependencies:
- dependency-name: actions/setup-python
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
2022-06-08 21:03:39 +00:00
dependabot[bot]andGitHub 985e42d223 chore(deps): bump github/codeql-action from 1 to 2
Bumps [github/codeql-action](https://github.com/github/codeql-action) from 1 to 2.
- [Release notes](https://github.com/github/codeql-action/releases)
- [Changelog](https://github.com/github/codeql-action/blob/main/CHANGELOG.md)
- [Commits](https://github.com/github/codeql-action/compare/v1...v2)

---
updated-dependencies:
- dependency-name: github/codeql-action
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
2022-04-25 21:03:31 +00:00
dependabot[bot]andGitHub faaaf03a7b chore(deps): bump actions/checkout from 2 to 3
Bumps [actions/checkout](https://github.com/actions/checkout) from 2 to 3.
- [Release notes](https://github.com/actions/checkout/releases)
- [Changelog](https://github.com/actions/checkout/blob/main/CHANGELOG.md)
- [Commits](https://github.com/actions/checkout/compare/v2...v3)

---
updated-dependencies:
- dependency-name: actions/checkout
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
2022-03-01 22:14:55 +00:00
dependabot[bot]andGitHub 0c3b9daece chore(deps): bump actions/setup-python from 2.3.2 to 3
Bumps [actions/setup-python](https://github.com/actions/setup-python) from 2.3.2 to 3.
- [Release notes](https://github.com/actions/setup-python/releases)
- [Commits](https://github.com/actions/setup-python/compare/v2.3.2...v3)

---
updated-dependencies:
- dependency-name: actions/setup-python
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
2022-02-28 22:07:59 +00:00
dependabot[bot]andGitHub 9e6481b2e4 chore(deps): bump actions/setup-python from 2.3.1 to 2.3.2
Bumps [actions/setup-python](https://github.com/actions/setup-python) from 2.3.1 to 2.3.2.
- [Release notes](https://github.com/actions/setup-python/releases)
- [Commits](https://github.com/actions/setup-python/compare/v2.3.1...v2.3.2)

---
updated-dependencies:
- dependency-name: actions/setup-python
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2022-02-04 22:03:11 +00:00
J.TownsendandGitHub 42709a8c38 increase ci parallel for faster ci jobs 2021-12-10 01:39:30 +00:00
dependabot[bot]andGitHub 6a50ff2a52 chore(deps): bump actions/setup-python from 2.3.0 to 2.3.1
Bumps [actions/setup-python](https://github.com/actions/setup-python) from 2.3.0 to 2.3.1.
- [Release notes](https://github.com/actions/setup-python/releases)
- [Commits](https://github.com/actions/setup-python/compare/v2.3.0...v2.3.1)

---
updated-dependencies:
- dependency-name: actions/setup-python
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
2021-11-29 22:02:02 +00:00
J.TownsendandGitHub 7013a9ae45 Change ci to use py3.10 final (#927)
* Add new fullhunt module

* Remove python checks in the run script as they are not needed anymore

* Remove netcraft from ci as it is not needed

* Update Dockerfile to fix security issues

* Add new n45ht module, lots of results returned

* Update ci to use py3.10 final

* Update ci to use py3.10 final
2021-11-25 16:09:29 +00:00
dependabot[bot]andGitHub 20062f94b7 chore(deps): bump actions/setup-python from 2.2.2 to 2.3.0
Bumps [actions/setup-python](https://github.com/actions/setup-python) from 2.2.2 to 2.3.0.
- [Release notes](https://github.com/actions/setup-python/releases)
- [Commits](https://github.com/actions/setup-python/compare/v2.2.2...v2.3.0)

---
updated-dependencies:
- dependency-name: actions/setup-python
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
2021-11-17 22:02:12 +00:00
J.TownsendandGitHub 15fe11130c Add new n45ht module, lots of results returned (#899)
* Add new fullhunt module

* Remove python checks in the run script as they are not needed anymore

* Remove netcraft from ci as it is not needed

* Update Dockerfile to fix security issues

* Add new n45ht module, lots of results returned
2021-10-24 04:10:37 +01:00
J.TownsendandGitHub 35a9fae73d Remove python checks in the run script as they are not needed anymore and add py3.10 support (#895)
* Add new fullhunt module

* Remove python checks in the run script as they are not needed anymore

* Remove netcraft from ci as it is not needed
2021-10-18 01:16:29 +01:00
L1ghtn1ng 234d613c6c add python 3.10 to ci testing 2021-10-05 01:37:23 +01:00
L1ghtn1ng efe1852ec0 add python 3.10 to ci testing 2021-10-05 01:33:55 +01:00
J.TownsendandGitHub a66d913cfa Update theHarvester.yml 2021-08-02 21:43:08 +01:00
J.TownsendandGitHub 210eacf90c Update theHarvester.yml 2021-08-02 17:15:37 +01:00
L1ghtn1ng a0b807a02d remove shiftleft from ci 2021-07-01 23:00:57 +01:00
L1ghtn1ng 435414ea6a update codeql to fix it 2021-07-01 19:21:23 +01:00
L1ghtn1ng af304227ef update codeql to run on any branch 2021-07-01 19:16:51 +01:00
L1ghtn1ng aee2548c2e remove exalead as does not return many results and due to captcha blocks you 2021-06-28 23:19:20 +01:00
L1ghtn1ng d28393b9a2 Add new module anubis and unit test for it. Update shodan related things 2021-06-21 00:19:27 +01:00
L1ghtn1ng 7388188626 Merge remote-tracking branch 'upstream/master' into dev 2021-05-24 22:11:37 +01:00
L1ghtn1ng 7b66863892 Change flow of how github actions CI is done 2021-05-24 22:11:25 +01:00
dependabot[bot]andGitHub 440c8bbd56 chore(deps): bump actions/setup-python from 2 to 2.2.2
Bumps [actions/setup-python](https://github.com/actions/setup-python) from 2 to 2.2.2.
- [Release notes](https://github.com/actions/setup-python/releases)
- [Commits](https://github.com/actions/setup-python/compare/v2...v2.2.2)

Signed-off-by: dependabot[bot] <support@github.com>
2021-05-12 04:01:49 +00:00
L1ghtn1ng 654569e3d4 Initial commit of restAPI 2021-04-06 01:42:10 +01:00