WEBDIENSTE/maigret
1
0
Fork 0
mirror of https://github.com/soxoj/maigret.git synced 2026-08-17 19:25:41 +02:00
Code Issues Packages Projects Releases Wiki Activity
1,511 Commits 18 Branches 39 Tags
dbbc8adf5a953deeeb0b46069cce3bf5947b3bb9
Commit Graph
81 Commits
Author SHA1 Message Date
SoxojandGitHub dbbc8adf5a Add system settings path and document offline test runs (#2974) 
* Add /etc/maigret/settings.json to the settings search path

* Register the slow marker and document offline test runs
 2026-08-15 19:41:06 +02:00
SoxojandGitHub d7f759a239 Keep runtime activation tokens out of the sites database (#2973) 
* Keep runtime activation tokens out of the sites database

* Document the activation token cache
 2026-08-15 12:43:30 +02:00
felipeandGitHub 46c90bd2cc docs: link to socid-extractor readthedocs pages (#2927) (#2967) 2026-08-12 17:15:58 +02:00
felipeandGitHub 77b565cfd5 fix: normalize SOCKS5 proxy scheme per HTTP transport (#2955) (#2966) 2026-08-12 17:04:32 +02:00
Soxoj e0dbef865c Improved release documentation 2026-08-11 18:57:43 +02:00
SoxojandGitHub 089f5abe9a Bump to 0.6.4 (#2960) 2026-08-11 18:18:35 +02:00
SoxojandGitHub 5387790810 Improved documentation, added quick link to deploy Maigret web to Readme (#2952) 
* Improved documentation, added quick link to deploy Maigret web to Readme

* Move Render config file to utils/
 2026-08-11 10:56:16 +02:00
Julio César SuásteguiandGitHub 34a1669974 Fix XMind reports for modern readers (#2930) 
* fix(report): add XMind manifest for modern readers

* docs: document modern XMind reader compatibility
 2026-08-10 22:54:35 +02:00
SoxojandGitHub 961680a81b Add support of Bilibili (#2891) 2026-07-20 20:28:02 +02:00
SoxojandGitHub d5cf4b15ef Add QQ search via qq_id id type; per-site encoding config (#2890) 2026-07-20 19:47:33 +02:00
SoxojandGitHub 6ced77f894 Bump to 0.6.3 (#2863) 2026-07-13 18:06:51 +02:00
SoxojandGitHub b11a62abc0 Fix site checks: 6 fixed, 1 disabled (#2834) 2026-07-04 19:03:49 +02:00
SoxojandGitHub cfb28b492a Bump to 0.6.2 (#2827) 
* Bump to 0.6.2

* Release process automation
 2026-07-01 17:32:50 +02:00
SoxojandGitHub 7b08793e9d Added docs for Neo4j integration (#2818) 2026-06-29 22:04:25 +02:00
AliandGitHub e41c4d7ee3 Add Neo4j Cypher export (--neo4j) (#2774) 
Serialize the maigret graph (the same one --graph builds) into an idempotent Cypher script importable into Neo4j. Reuses MaigretGraph via an extracted _build_maigret_graph() helper, leaving save_graph_report behavior unchanged; no new runtime dependency. Adds the --neo4j flag, the neo4j_report setting, docs, and a unit test.

Closes #2630
 2026-06-29 15:41:57 +02:00
SoxojandGitHub 3eaec01523 Code cleanup and refactoring after ponytail audit (#2789) 2026-06-21 12:51:01 +02:00
SoxojandGitHub 682059cf21 ORCID support (#2786) 
* Added support of ORCID identifier

* ORCID documentation
 2026-06-20 22:29:19 +02:00
Julio César SuásteguiandGitHub 0372bb3efc docs: fix typos (#2779) 2026-06-18 09:45:53 +02:00
SoxojandGitHub 9a2cce4d85 Add Cloudflare bypass (FlareSolverr) support to web UI + docs (#2700) 2026-05-25 18:53:18 +02:00
SoxojandGitHub b3f667d908 Fix docs build: dedup link targets, gate SVG badges to HTML (#2695) 2026-05-23 10:15:53 +02:00
SoxojandGitHub 070c086f29 Add docs FAQ for top search queries; document translation workflow; Chinese update (#2694) 2026-05-23 09:46:35 +02:00
SoxojandGitHub ec7f57eab7 Windows version improvements and docs (#2690) 2026-05-21 12:12:17 +02:00
SoxojandGitHub 3e77c13743 Document Tor/proxy usage, add Advanced usage docs section (closes #544) (#2663) 2026-05-16 14:42:36 +02:00
SoxojandGitHub c5885331d6 Bump to 0.6.1 (#2661) 2026-05-16 12:05:55 +02:00
SoxojandGitHub a7338e97f3 Make xhtml2pdf optional, fix install on Linux without libcairo (#2659) 
* Make xhtml2pdf optional, fix install on Linux without libcairo

Move xhtml2pdf to the new [pdf] extra so default `pip install maigret`
no longer pulls pycairo (which has no Linux/macOS wheels and breaks the
build without libcairo2-dev). save_pdf_report now raises a clear
RuntimeError pointing to `pip install 'maigret[pdf]'`, and the CLI
turns it into a friendly warning instead of a crash. Adds tests
covering the missing-extra path, plus per-OS install docs.

Fix for #2657, #2534

* Make arabic-reshaper and python-bidi optional; idempotent update of db_meta.json and sites.md

* Regenerated poerty.lock

* Update CI workflow to cover minimal installation without PDF deps
 2026-05-15 14:33:55 +02:00
SoxojandGitHub 203d712fa5 Update links to the community Telegram bot (#2641) 2026-05-11 19:41:43 +02:00
SoxojandGitHub 5c93b206e7 Cloudflare bypass webgate (#2628) 2026-05-09 10:48:43 +03:00
SoxojandGitHub 79e93ab715 AI mode documentation (#2620) 2026-05-05 22:21:00 +02:00
Sayon DeyandGitHub f897598f98 Fix outdated Google Colab setup instructions (#2591) 2026-05-02 15:21:16 +02:00
SoxojandGitHub 25026e21ea Fix site checks: 4 → ip_reputation, 9 fixed, 16 disabled, 3 dead dele… (#2555) 
* Fix site checks: 4 → ip_reputation, 9 fixed, 16 disabled, 3 dead deleted; clarify ip_reputation tag semantics

* Improved test coverage
 2026-04-23 21:17:07 +02:00
SoxojandGitHub 5e1cc45c17 Fix site checks: 12 fixed, 19 disabled; add new protection tags (#2550) 2026-04-22 20:25:41 +02:00
SoxojandGitHub d9b361b626 Fix site checks: 3 → ip_reputation, 10 fixed, 6 disabled, 2 dead deleted (#2549) 2026-04-22 12:46:53 +02:00
SoxojandGitHub 98f03c153b Add 3 crypto sites (Polymarket, Zora, Revolut.me), added crypto inves… (#2538) 
* Add 3 crypto sites (Polymarket, Zora, Revolut.me), added crypto investigation use case page in docs

* Added fintech tag

* Updated sites metadata
 2026-04-21 11:08:48 +02:00
SoxojandGitHub fb71f26fd0 Fix site checks: recover 6 CF sites via tls_fingerprint, 500px GraphQL, delete 4 dead domains (#2535) 2026-04-20 22:41:51 +02:00
SoxojandGitHub 37ce4fe728 Update of Readme and documentation (#2514) 
* Big readme update

* Readme and documentation update

* Readme structure update

* Small fixes

* Changelog update
 2026-04-17 17:42:36 +02:00
SoxojGitHubcopilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
 		6db1df2ddb Fix failing test for custom DB path resolution (#2468) 
* Fix `--db` bug

* Fix test_resolve_db_path_custom_file to create the file before testing

Agent-Logs-Url: https://github.com/soxoj/maigret/sessions/3ea7b2e8-0565-4fca-8ec2-eff8eb4ee617

Co-authored-by: soxoj <31013580+soxoj@users.noreply.github.com>

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
 2026-04-08 00:53:57 +02:00
CopilotGitHubsoxojcopilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
 		23adc178ea Fix crash on -a --self-check by adding exception handling to site check coroutines (#2466) 
* Initial plan

* Fix crash on -a --self-check by adding exception handling in site_self_check and self_check

Wrap the body of site_self_check in try/except to catch unexpected errors
and always return a valid changes dict. Also add a safety-net try/except
in self_check around awaiting individual site check futures so that a
single site failure doesn't crash the entire self-check process.

Agent-Logs-Url: https://github.com/soxoj/maigret/sessions/5e27d620-5cbb-43d2-a9f9-ecb53a29904d

Co-authored-by: soxoj <31013580+soxoj@users.noreply.github.com>

* Restore @pytest.mark.slow on test_maigret_results

Agent-Logs-Url: https://github.com/soxoj/maigret/sessions/5e27d620-5cbb-43d2-a9f9-ecb53a29904d

Co-authored-by: soxoj <31013580+soxoj@users.noreply.github.com>

* Document --self-check error resilience, --auto-disable, and --diagnose in docs/

Update command-line-options.rst with expanded --self-check description
and new --auto-disable and --diagnose entries. Add a "Database self-check"
section to features.rst explaining error-resilient behaviour and usage
examples. Update usage-examples.rst to reference --auto-disable.

Agent-Logs-Url: https://github.com/soxoj/maigret/sessions/af1f0f09-9112-4902-8475-e81d235ff3ed

Co-authored-by: soxoj <31013580+soxoj@users.noreply.github.com>

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: soxoj <31013580+soxoj@users.noreply.github.com>
 2026-04-07 19:44:09 +02:00
CopilotGitHubsoxojcopilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
 		6ed8fdefcc Add installation troubleshooting for missing system dependencies (#2465) 
* Initial plan

* Add installation troubleshooting section for system dependency errors

Agent-Logs-Url: https://github.com/soxoj/maigret/sessions/6c3a5612-bdd5-4611-ba77-aea7ab52e304

Co-authored-by: soxoj <31013580+soxoj@users.noreply.github.com>

* Simplify README troubleshooting to a link to the full docs

Agent-Logs-Url: https://github.com/soxoj/maigret/sessions/6c557093-0643-4980-93ad-973e2d3141ef

Co-authored-by: soxoj <31013580+soxoj@users.noreply.github.com>

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: soxoj <31013580+soxoj@users.noreply.github.com>
 2026-04-07 17:54:02 +02:00
SoxojandGitHub ad95302745 Add Markdown reports for LLM analysis (#2463) 2026-04-06 18:26:43 +02:00
SoxojandGitHub 269d50eedc DB update mechanism (#2458) 
* Database update mechanism
 2026-04-04 18:00:50 +02:00
SoxojGitHubcopilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
 		5d502eaef6 Add site protection tracking system, fix broken site checks (Instagra… (#2452) 
* Add site protection tracking system, fix broken site checks (Instagram, StackOverflow, LeetCode, Boosty, LiveLib), preserve unicode in data.json

* Update poetry.lock by running poetry lock

Agent-Logs-Url: https://github.com/soxoj/maigret/sessions/14333f41-67d5-4e28-a782-9730b31fc667

Co-authored-by: soxoj <31013580+soxoj@users.noreply.github.com>

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
 2026-04-02 20:28:20 +02:00
SoxojandGitHub a5d337b765 Tags and site names improvements (#2427) 
- Added social tag to social networks (33 sites)
- Fixed wrong tags (8 sites)
- Filled empty tags for 213 sites in top-1000
- Country tag cleanup (~374 sites)
- Site naming normalization (75 sites)
- New tests (3)
- Documentation updates
 2026-03-28 15:42:12 +01:00
SoxojGitHubcopilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
 		b145e7b26f feat(core): add POST request support, new sites, migrate to Majestic Million ranking (#2317) 
* feat(core): add POST request support, new sites, migrate to Majestic Million ranking
- Added native POST request support to the Maigret engine (requestMethod, requestPayload) to enable querying modern JSON registration endpoints.
- Replaced the discontinued Alexa rank API with the Majestic Million dataset for global popularity sorting and automated CI updates.
- Fixed multiple false positives among top 500 sites and bypassed standard anti-bot protections using custom User-Agents.
- Updated public documentation and internal playbooks to reflect the new features.

* feat(data): apply all data.json site check updates from main branch

- Added CTFtime and PentesterLab (new sites added in main)
- Removed forums.imore.com (deleted in main as dead site)
- Disabled 5 sites per main branch fixes: Librusec, MirTesen, amateurvoyeurforum.com, forums.stevehoffman.tv, vegalab
- Fixed 5 site checks per main branch: SoundCloud, Taplink, Setlist, RoyalCams, club.cnews.ru (switched from status_code to message checkType with proper markers)

Co-authored-by: soxoj <31013580+soxoj@users.noreply.github.com>
Agent-Logs-Url: https://github.com/soxoj/maigret/sessions/a1d194d9-c0ff-4e2b-974c-c5e4b59548bf

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
 2026-03-24 22:08:42 +01:00
CopilotGitHubsoxojcopilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
 		2e430e5039 feat: add tag blacklisting via --exclude-tags (#2352) 
* Initial plan

* feat: add tag blacklisting support (--exclude-tags CLI flag, web UI, docs, tests)

Co-authored-by: soxoj <31013580+soxoj@users.noreply.github.com>
Agent-Logs-Url: https://github.com/soxoj/maigret/sessions/1a656af2-36bf-494f-9f03-1b5340f0357c

* fix: correct tag cloud label to match click-cycle interaction

Co-authored-by: soxoj <31013580+soxoj@users.noreply.github.com>
Agent-Logs-Url: https://github.com/soxoj/maigret/sessions/1a656af2-36bf-494f-9f03-1b5340f0357c

* feat: add all country tags to web interface tag cloud

Co-authored-by: soxoj <31013580+soxoj@users.noreply.github.com>
Agent-Logs-Url: https://github.com/soxoj/maigret/sessions/7e184b24-ff26-48fd-8a93-aea12b0a8d7b

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: soxoj <31013580+soxoj@users.noreply.github.com>
 2026-03-24 22:00:59 +01:00
SoxojandGitHub 97cc4b46d9 Improve site-check quality: fix broken site configs, add diagnostic utilities, and make self-check report-only by default with opt-in auto-disable. (#2301) 
- Fix VK and TradingView checkType; add Reddit and Microsoft Learn API-style probes where appropriate; adjust or disable entries that are unreliable under anti-bot protection.
- Self-check: stop aggressive auto-disable; default to reporting issues only; add --auto-disable and --diagnose for optional fixes and deeper output.
- Tooling: add utils/site_check.py and utils/check_top_n.py (and related helpers) to inspect and rank site behavior against the top-N list
- Scope: aligns with fixing top-traffic / high-impact sites and making diagnostics repeatable without silently flipping disabled flags
 2026-03-22 16:48:35 +01:00
SoxojandGitHub 227a25bfa1 Twitter fixed, mirrors mechanism improvement (#2299) 2026-03-22 01:14:17 +01:00
SoxojandGitHub f99091f5f7 Fixed false positives in top-500 (#2292) 2026-03-21 23:35:59 +01:00
SoxojandGitHub bebadb0362 Bump to 0.5.0 (#2108) 2025-08-10 13:10:50 +02:00
Ikko Eltociear AshimineandGitHub 4195a3ca21 docs: update usage-examples.rst (#1996) 
inital -> initial
 2025-02-18 10:50:29 +01:00
imgbot[bot]GitHubImgBotApp
 		900ed840b3 [ImgBot] Optimize images (#1974) 
*Total -- 2,762.12kb -> 2,382.30kb (13.75%)

/static/web_interface_screenshot_start.png -- 106.23kb -> 59.10kb (44.37%)
/docs/source/maigret_screenshot.png -- 375.21kb -> 233.98kb (37.64%)
/static/web_interface_screenshot.png -- 615.65kb -> 424.23kb (31.09%)
/static/recursive_search.svg -- 1,665.02kb -> 1,664.99kb (0%)

Signed-off-by: ImgBotApp <ImgBotHelp@gmail.com>
Co-authored-by: ImgBotApp <ImgBotHelp@gmail.com>
 2024-12-16 19:24:08 +01:00