Commit Graph
87 Commits
Author SHA1 Message Date
SoxojandGitHub f1337eb30d Add a privacy page and a code signing note (#3081) 2026-09-04 22:10:24 +02:00
SoxojandGitHub 88cbe0d487 Change links to the online website (#3079) 2026-09-04 20:21:06 +02:00
SoxojandGitHub 5059b4be55 Document the release process, with a Chinese translation (#3049)
Nothing in the repository says how a release is made, so the details live only in the workflows and in the history.

- `docs/source/release-process.rst`: the two version files that have to move together, cutting a release on its own branch, what `release: published` triggers, why the tag is not an ancestor of `main`, what each tag in the repository means, and where the artifacts land.
- `docs/source/locale/zh_CN/LC_MESSAGES/release-process.po`: the translation, following the existing gettext setup. 39 message ids, checked against the catalogue Sphinx extracts from the page, and rendered with `-D language=zh_CN`.
- `docs/source/index.rst`: one line in the toctree.

The tag section is the part that has cost time: `nightly-*` move on every push, and the legacy `main` and `dev` tags shadow the branches of the same name, so `git show main:path` answers from the tag.
2026-08-30 23:22:52 +02:00
SoxojandGitHub 939368977f Document pipx as an installation method (#3040)
New "Isolated installation with pipx" section in `installation.rst`, between the PyPI and GitHub ones: `pipx install maigret`, the `pdf` extra, upgrade, uninstall, and a note for uv users.

Until now pipx appeared only in the FAQ, as a fix for `maigret: command not found`.
2026-08-30 21:54:58 +02:00
SoxojandGitHub 3822390359 Drop the update_sitesmd console script (#3041)
`utils/` is in neither the sdist nor the wheel, so the declared entry point ships a command that fails on every install:

    $ update_sitesmd
    ModuleNotFoundError: No module named 'utils'

It only works in a dev checkout, where maigret is installed editable and the repository root is on sys.path.

- `pyproject.toml`: drop the `update_sitesmd` script.
- `.githooks/pre-commit`: run `python -m utils.update_site_data` instead.
- `installation.rst`: the quoted pip warning named both scripts, now one.
2026-08-30 21:54:42 +02:00
SoxojandGitHub 8cb4b308db Document the snap as an installation method (#3042)
The snap has been on `latest/stable` since 26.08, for amd64 and arm64, and is mentioned nowhere a user would look. The only match in the repository is `settings.rst`, in passing, as an example of an install whose directory is not writable.

- `README.md`: a "Snap (Linux)" subsection before the pip one, with the official Snap Store badge at the same height as the Cloud Shell buttons above it.
- `docs/source/installation.rst`: the same, between Cloud Shells and PyPI, with the badge under `only:: html` like the other buttons on that page, plus what strict confinement means in practice, the `removable-media` interface, and automatic updates through snapd.
2026-08-30 21:54:31 +02:00
SoxojandGitHub 80d9f73512 Bump to 0.6.5 (#3014) 2026-08-25 14:26:03 +02:00
SoxojandGitHub dbbc8adf5a Add system settings path and document offline test runs (#2974)
* Add /etc/maigret/settings.json to the settings search path

* Register the slow marker and document offline test runs
2026-08-15 19:41:06 +02:00
SoxojandGitHub d7f759a239 Keep runtime activation tokens out of the sites database (#2973)
* Keep runtime activation tokens out of the sites database

* Document the activation token cache
2026-08-15 12:43:30 +02:00
felipeandGitHub 46c90bd2cc docs: link to socid-extractor readthedocs pages (#2927) (#2967) 2026-08-12 17:15:58 +02:00
felipeandGitHub 77b565cfd5 fix: normalize SOCKS5 proxy scheme per HTTP transport (#2955) (#2966) 2026-08-12 17:04:32 +02:00
Soxoj e0dbef865c Improved release documentation 2026-08-11 18:57:43 +02:00
SoxojandGitHub 089f5abe9a Bump to 0.6.4 (#2960) 2026-08-11 18:18:35 +02:00
SoxojandGitHub 5387790810 Improved documentation, added quick link to deploy Maigret web to Readme (#2952)
* Improved documentation, added quick link to deploy Maigret web to Readme

* Move Render config file to utils/
2026-08-11 10:56:16 +02:00
Julio César SuásteguiandGitHub 34a1669974 Fix XMind reports for modern readers (#2930)
* fix(report): add XMind manifest for modern readers

* docs: document modern XMind reader compatibility
2026-08-10 22:54:35 +02:00
SoxojandGitHub 961680a81b Add support of Bilibili (#2891) 2026-07-20 20:28:02 +02:00
SoxojandGitHub d5cf4b15ef Add QQ search via qq_id id type; per-site encoding config (#2890) 2026-07-20 19:47:33 +02:00
SoxojandGitHub 6ced77f894 Bump to 0.6.3 (#2863) 2026-07-13 18:06:51 +02:00
SoxojandGitHub b11a62abc0 Fix site checks: 6 fixed, 1 disabled (#2834) 2026-07-04 19:03:49 +02:00
SoxojandGitHub cfb28b492a Bump to 0.6.2 (#2827)
* Bump to 0.6.2

* Release process automation
2026-07-01 17:32:50 +02:00
SoxojandGitHub 7b08793e9d Added docs for Neo4j integration (#2818) 2026-06-29 22:04:25 +02:00
AliandGitHub e41c4d7ee3 Add Neo4j Cypher export (--neo4j) (#2774)
Serialize the maigret graph (the same one --graph builds) into an idempotent Cypher script importable into Neo4j. Reuses MaigretGraph via an extracted _build_maigret_graph() helper, leaving save_graph_report behavior unchanged; no new runtime dependency. Adds the --neo4j flag, the neo4j_report setting, docs, and a unit test.

Closes #2630
2026-06-29 15:41:57 +02:00
SoxojandGitHub 3eaec01523 Code cleanup and refactoring after ponytail audit (#2789) 2026-06-21 12:51:01 +02:00
SoxojandGitHub 682059cf21 ORCID support (#2786)
* Added support of ORCID identifier

* ORCID documentation
2026-06-20 22:29:19 +02:00
Julio César SuásteguiandGitHub 0372bb3efc docs: fix typos (#2779) 2026-06-18 09:45:53 +02:00
SoxojandGitHub 9a2cce4d85 Add Cloudflare bypass (FlareSolverr) support to web UI + docs (#2700) 2026-05-25 18:53:18 +02:00
SoxojandGitHub b3f667d908 Fix docs build: dedup link targets, gate SVG badges to HTML (#2695) 2026-05-23 10:15:53 +02:00
SoxojandGitHub 070c086f29 Add docs FAQ for top search queries; document translation workflow; Chinese update (#2694) 2026-05-23 09:46:35 +02:00
SoxojandGitHub ec7f57eab7 Windows version improvements and docs (#2690) 2026-05-21 12:12:17 +02:00
SoxojandGitHub 3e77c13743 Document Tor/proxy usage, add Advanced usage docs section (closes #544) (#2663) 2026-05-16 14:42:36 +02:00
SoxojandGitHub c5885331d6 Bump to 0.6.1 (#2661) 2026-05-16 12:05:55 +02:00
SoxojandGitHub a7338e97f3 Make xhtml2pdf optional, fix install on Linux without libcairo (#2659)
* Make xhtml2pdf optional, fix install on Linux without libcairo

Move xhtml2pdf to the new [pdf] extra so default `pip install maigret`
no longer pulls pycairo (which has no Linux/macOS wheels and breaks the
build without libcairo2-dev). save_pdf_report now raises a clear
RuntimeError pointing to `pip install 'maigret[pdf]'`, and the CLI
turns it into a friendly warning instead of a crash. Adds tests
covering the missing-extra path, plus per-OS install docs.

Fix for #2657, #2534

* Make arabic-reshaper and python-bidi optional; idempotent update of db_meta.json and sites.md

* Regenerated poerty.lock

* Update CI workflow to cover minimal installation without PDF deps
2026-05-15 14:33:55 +02:00
SoxojandGitHub 203d712fa5 Update links to the community Telegram bot (#2641) 2026-05-11 19:41:43 +02:00
SoxojandGitHub 5c93b206e7 Cloudflare bypass webgate (#2628) 2026-05-09 10:48:43 +03:00
SoxojandGitHub 79e93ab715 AI mode documentation (#2620) 2026-05-05 22:21:00 +02:00
Sayon DeyandGitHub f897598f98 Fix outdated Google Colab setup instructions (#2591) 2026-05-02 15:21:16 +02:00
SoxojandGitHub 25026e21ea Fix site checks: 4 → ip_reputation, 9 fixed, 16 disabled, 3 dead dele… (#2555)
* Fix site checks: 4 → ip_reputation, 9 fixed, 16 disabled, 3 dead deleted; clarify ip_reputation tag semantics

* Improved test coverage
2026-04-23 21:17:07 +02:00
SoxojandGitHub 5e1cc45c17 Fix site checks: 12 fixed, 19 disabled; add new protection tags (#2550) 2026-04-22 20:25:41 +02:00
SoxojandGitHub d9b361b626 Fix site checks: 3 → ip_reputation, 10 fixed, 6 disabled, 2 dead deleted (#2549) 2026-04-22 12:46:53 +02:00
SoxojandGitHub 98f03c153b Add 3 crypto sites (Polymarket, Zora, Revolut.me), added crypto inves… (#2538)
* Add 3 crypto sites (Polymarket, Zora, Revolut.me), added crypto investigation use case page in docs

* Added fintech tag

* Updated sites metadata
2026-04-21 11:08:48 +02:00
SoxojandGitHub fb71f26fd0 Fix site checks: recover 6 CF sites via tls_fingerprint, 500px GraphQL, delete 4 dead domains (#2535) 2026-04-20 22:41:51 +02:00
SoxojandGitHub 37ce4fe728 Update of Readme and documentation (#2514)
* Big readme update

* Readme and documentation update

* Readme structure update

* Small fixes

* Changelog update
2026-04-17 17:42:36 +02:00
SoxojGitHubcopilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
6db1df2ddb Fix failing test for custom DB path resolution (#2468)
* Fix `--db` bug

* Fix test_resolve_db_path_custom_file to create the file before testing

Agent-Logs-Url: https://github.com/soxoj/maigret/sessions/3ea7b2e8-0565-4fca-8ec2-eff8eb4ee617

Co-authored-by: soxoj <31013580+soxoj@users.noreply.github.com>

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
2026-04-08 00:53:57 +02:00
CopilotGitHubsoxojcopilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
23adc178ea Fix crash on -a --self-check by adding exception handling to site check coroutines (#2466)
* Initial plan

* Fix crash on -a --self-check by adding exception handling in site_self_check and self_check

Wrap the body of site_self_check in try/except to catch unexpected errors
and always return a valid changes dict. Also add a safety-net try/except
in self_check around awaiting individual site check futures so that a
single site failure doesn't crash the entire self-check process.

Agent-Logs-Url: https://github.com/soxoj/maigret/sessions/5e27d620-5cbb-43d2-a9f9-ecb53a29904d

Co-authored-by: soxoj <31013580+soxoj@users.noreply.github.com>

* Restore @pytest.mark.slow on test_maigret_results

Agent-Logs-Url: https://github.com/soxoj/maigret/sessions/5e27d620-5cbb-43d2-a9f9-ecb53a29904d

Co-authored-by: soxoj <31013580+soxoj@users.noreply.github.com>

* Document --self-check error resilience, --auto-disable, and --diagnose in docs/

Update command-line-options.rst with expanded --self-check description
and new --auto-disable and --diagnose entries. Add a "Database self-check"
section to features.rst explaining error-resilient behaviour and usage
examples. Update usage-examples.rst to reference --auto-disable.

Agent-Logs-Url: https://github.com/soxoj/maigret/sessions/af1f0f09-9112-4902-8475-e81d235ff3ed

Co-authored-by: soxoj <31013580+soxoj@users.noreply.github.com>

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: soxoj <31013580+soxoj@users.noreply.github.com>
2026-04-07 19:44:09 +02:00
CopilotGitHubsoxojcopilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
6ed8fdefcc Add installation troubleshooting for missing system dependencies (#2465)
* Initial plan

* Add installation troubleshooting section for system dependency errors

Agent-Logs-Url: https://github.com/soxoj/maigret/sessions/6c3a5612-bdd5-4611-ba77-aea7ab52e304

Co-authored-by: soxoj <31013580+soxoj@users.noreply.github.com>

* Simplify README troubleshooting to a link to the full docs

Agent-Logs-Url: https://github.com/soxoj/maigret/sessions/6c557093-0643-4980-93ad-973e2d3141ef

Co-authored-by: soxoj <31013580+soxoj@users.noreply.github.com>

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: soxoj <31013580+soxoj@users.noreply.github.com>
2026-04-07 17:54:02 +02:00
SoxojandGitHub ad95302745 Add Markdown reports for LLM analysis (#2463) 2026-04-06 18:26:43 +02:00
SoxojandGitHub 269d50eedc DB update mechanism (#2458)
* Database update mechanism
2026-04-04 18:00:50 +02:00
SoxojGitHubcopilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
5d502eaef6 Add site protection tracking system, fix broken site checks (Instagra… (#2452)
* Add site protection tracking system, fix broken site checks (Instagram, StackOverflow, LeetCode, Boosty, LiveLib), preserve unicode in data.json

* Update poetry.lock by running poetry lock

Agent-Logs-Url: https://github.com/soxoj/maigret/sessions/14333f41-67d5-4e28-a782-9730b31fc667

Co-authored-by: soxoj <31013580+soxoj@users.noreply.github.com>

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
2026-04-02 20:28:20 +02:00
SoxojandGitHub a5d337b765 Tags and site names improvements (#2427)
- Added social tag to social networks (33 sites)
- Fixed wrong tags (8 sites)
- Filled empty tags for 213 sites in top-1000
- Country tag cleanup (~374 sites)
- Site naming normalization (75 sites)
- New tests (3)
- Documentation updates
2026-03-28 15:42:12 +01:00
SoxojGitHubcopilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
b145e7b26f feat(core): add POST request support, new sites, migrate to Majestic Million ranking (#2317)
* feat(core): add POST request support, new sites, migrate to Majestic Million ranking
- Added native POST request support to the Maigret engine (requestMethod, requestPayload) to enable querying modern JSON registration endpoints.
- Replaced the discontinued Alexa rank API with the Majestic Million dataset for global popularity sorting and automated CI updates.
- Fixed multiple false positives among top 500 sites and bypassed standard anti-bot protections using custom User-Agents.
- Updated public documentation and internal playbooks to reflect the new features.

* feat(data): apply all data.json site check updates from main branch

- Added CTFtime and PentesterLab (new sites added in main)
- Removed forums.imore.com (deleted in main as dead site)
- Disabled 5 sites per main branch fixes: Librusec, MirTesen, amateurvoyeurforum.com, forums.stevehoffman.tv, vegalab
- Fixed 5 site checks per main branch: SoundCloud, Taplink, Setlist, RoyalCams, club.cnews.ru (switched from status_code to message checkType with proper markers)

Co-authored-by: soxoj <31013580+soxoj@users.noreply.github.com>
Agent-Logs-Url: https://github.com/soxoj/maigret/sessions/a1d194d9-c0ff-4e2b-974c-c5e4b59548bf

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
2026-03-24 22:08:42 +01:00