WEBDIENSTE/maigret
1
0
Fork 0
mirror of https://github.com/soxoj/maigret.git synced 2026-08-17 19:25:41 +02:00
Code Issues Packages Projects Releases Wiki Activity
1,515 Commits 18 Branches 39 Tags
main
Commit Graph
100 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
SoxojandGitHub c983544557 Declare python-dateutil as a direct dependency (#2977) 
maigret/report.py imports dateutil.tz and dateutil.parser directly, but python-dateutil is not listed in pyproject.toml. It resolves today only because socid-extractor requires it transitively, so a first-party import is riding on a second-party dependency staying put.

Constraint is >=2.8.1, the same floor socid-extractor already asks for, with no upper bound. poetry.lock changes only its content-hash: python-dateutil 2.9.0.post0 was already locked in the main group, so no installed version moves.
 2026-08-16 11:57:56 +02:00
SoxojandGitHub dbbc8adf5a Add system settings path and document offline test runs (#2974) 
* Add /etc/maigret/settings.json to the settings search path

* Register the slow marker and document offline test runs
 2026-08-15 19:41:06 +02:00
SoxojandGitHub e1c80a7a42 Move snapcraft.yaml to snap/, migrate to core24 (#2975) 2026-08-15 19:40:31 +02:00
SoxojandGitHub d7f759a239 Keep runtime activation tokens out of the sites database (#2973) 
* Keep runtime activation tokens out of the sites database

* Document the activation token cache
 2026-08-15 12:43:30 +02:00
SoxojandGitHub 23077d2025 Update of README (#2971) 2026-08-14 16:09:17 +02:00
SoxojandGitHub b69f70f942 Add "Fork me on GitHub" ribbon to web interface (#2969) 2026-08-12 23:10:12 +02:00
SoxojandGitHub 089f5abe9a Bump to 0.6.4 (#2960) 2026-08-11 18:18:35 +02:00
SoxojandGitHub 8518622933 Fix site checks: 9 fixed, 4 disabled; git hook update (#2956) 
Closes #2941, #2950, #2918, #2913, #2905, #2897
 2026-08-11 15:51:31 +02:00
SoxojandGitHub fbda9cdfcc fix: retry transient network/proxy errors in site checkers (#2954) 
Rotating/residential proxies often drop a connection mid-request
(truncated body, closed socket, failed handshake) instead of failing
outright. These were previously lumped into "Unexpected" and never
retried, silently discarding otherwise-successful checks.

- Retry once on ClientPayloadError, ServerDisconnectedError, and
  aiohttp_socks' ProxyConnectionError/ProxyTimeoutError (aiohttp
  checker), and on CurlError (curl_cffi checker) — covers truncated
  responses, dropped connections, CONNECT-tunnel 502s, and TLS
  handshake failures.
- Fix the proxy-error except clause: it caught python_socks'
  ProxyError, but aiohttp_socks (the connector actually in use)
  raises its own unrelated same-named exceptions, so it never fired.
- Generic ProxyError (e.g. bad credentials) is classified but NOT
  retried — it fails identically every attempt.
 2026-08-11 13:16:13 +02:00
SoxojandGitHub af13058cfe Update sponsorship information (#2953) 2026-08-11 11:14:46 +02:00
SoxojandGitHub 5387790810 Improved documentation, added quick link to deploy Maigret web to Readme (#2952) 
* Improved documentation, added quick link to deploy Maigret web to Readme

* Move Render config file to utils/
 2026-08-11 10:56:16 +02:00
SoxojandGitHub 1171dbdb84 Sponsorship details update in README (#2939) 2026-08-08 02:07:11 +07:00
SoxojandGitHub d1b8b9fa5c Web interface update: live scan, settings perseistence, history, nagivation (#2925) 2026-08-03 08:41:07 +09:00
SoxojandGitHub c2e24f3218 Fix site checks: 9 fixed, 1 disabled (#2914) 2026-07-28 01:31:57 +05:00
SoxojandGitHub 8b263dc960 Fix site checks: 3 fixed, 2 dead deleted (#2909) 2026-07-27 12:02:24 +05:00
SoxojandGitHub a1d21dfffe Add 11 sites (#2893) 2026-07-21 00:15:38 +02:00
SoxojandGitHub aa6c16f1dc Tag 2210 sites by country and topic (#2892) 
- Countries: +783 sites (ccTLD-deterministic + LLM content classification),
cleaned 9 anomalous tags (uk->gb, eu region, domain hacks)
- Topics: +2165 sites; vocab 69->75 (+parenting, psychology, cooking,
esoteric, religion, firearms)
- Add tests/test_country_tags.py (ISO validity, no uk/eu, hack regression)
 2026-07-20 23:45:04 +02:00
SoxojandGitHub aa79917052 Updated AI report prompt for analysis and default model (#2888) 
* Updated AI report prompt for analysis and default model

* Fix test_cli.py DEFAULT_ARGS to match updated default AI model (gpt-4o → gpt-5.4)
 2026-07-20 20:38:09 +02:00
SoxojandGitHub 961680a81b Add support of Bilibili (#2891) 2026-07-20 20:28:02 +02:00
SoxojandGitHub d5cf4b15ef Add QQ search via qq_id id type; per-site encoding config (#2890) 2026-07-20 19:47:33 +02:00
SoxojandGitHub b745d33373 Fix site checks: 6 fixed, 4 disabled, 1 dead deleted (#2887) 2026-07-20 16:30:16 +02:00
SoxojandGitHub 3c7a0e0c43 Enrichment via socid_extractor url mutations (#2880) 2026-07-19 21:40:56 +02:00
SoxojandGitHub 2471538796 Add site checks: 21 new sites (#2878) 2026-07-18 01:04:58 +02:00
SoxojandGitHub c436174467 Improve CONTRIBUTING.md (#2871) 2026-07-15 17:23:53 +02:00
SoxojandGitHub 9d29061e2a Fix site checks: 2 fixed, 1 new added + proton activation method (#2869) 2026-07-15 12:30:00 +02:00
SoxojandGitHub 6ced77f894 Bump to 0.6.3 (#2863) 2026-07-13 18:06:51 +02:00
SoxojandGitHub 9842d8cf1f Fix site checks: 14 fixed, 5 disabled (#2860) 2026-07-11 16:02:16 +02:00
SoxojandGitHub 08d032a4a0 Update README with sponsor info (#2854) 2026-07-09 20:36:55 +02:00
SoxojandGitHub 3e23cae384 Update sponsor info (#2841) 2026-07-06 12:41:21 +02:00
SoxojandGitHub 6d57f58234 Add 5 new sites (#2840) 2026-07-05 20:49:14 +02:00
SoxojandGitHub e352e1dff7 Add 10 new sites (#2839) 2026-07-05 20:10:29 +02:00
SoxojandGitHub 897769dcc2 Fix site checks: 82 fixed, 11 disabled + DiscourseJson engine (#2836) 
* Fix site checks: 82 fixed, 11 disabled + DiscourseJson engine

* Fix test_hackernews_requires_profile_marker: update test body to JSON format matching Firebase API probe
 2026-07-05 13:28:46 +02:00
SoxojandGitHub c74bad1ebf Fix site checks: 13 fixed, 7 disabled + CN sites: Tuchong, GitCode, Yuque, LeetCode CN, Bangumi, NGA, Moegirl (#2835) 2026-07-04 22:51:50 +02:00
SoxojandGitHub b11a62abc0 Fix site checks: 6 fixed, 1 disabled (#2834) 2026-07-04 19:03:49 +02:00
SoxojandGitHub 28de12f361 Sponsorship update (#2833) 2026-07-04 15:41:47 +02:00
SoxojandGitHub cfb28b492a Bump to 0.6.2 (#2827) 
* Bump to 0.6.2

* Release process automation
 2026-07-01 17:32:50 +02:00
SoxojandGitHub e023dd441a Fix site checks: 6 fixed, 5 disabled (#2826) 2026-07-01 14:36:57 +02:00
SoxojandGitHub 7b08793e9d Added docs for Neo4j integration (#2818) 2026-06-29 22:04:25 +02:00
SoxojandGitHub 24845091c8 Refactor sponsor section in README.md (#2799) 
* Refactor sponsor section in README.md

* docs: update sponsor layout in Chinese README
 2026-06-24 09:12:01 +02:00
SoxojandGitHub 3eaec01523 Code cleanup and refactoring after ponytail audit (#2789) 2026-06-21 12:51:01 +02:00
SoxojandGitHub 682059cf21 ORCID support (#2786) 
* Added support of ORCID identifier

* ORCID documentation
 2026-06-20 22:29:19 +02:00
SoxojandGitHub 8f737d10c3 Update README with sponsorship details (#2785) 2026-06-19 13:12:20 +02:00
SoxojandGitHub 0b363a6ac9 Add AI Wiki links (#2778) 
* Add AI Wiki links

* Update Chinese README with AI Wiki links and badge reorganization
 2026-06-17 21:01:58 +02:00
SoxojandGitHub f2e4f8d3a2 Merge commit from fork 2026-06-17 15:15:32 +02:00
SoxojandGitHub 9fbe7ffd9b Fix site checks: 2 fixed, 2 disabled (#2745) 
Fixed: Weburg (#2740), Geeksfor Geeks (#2675)
Disabled: AppleDeveloper (#2738), Livios (#2724)
 2026-06-07 01:28:44 +02:00
SoxojandGitHub 79a612ba8d Sponsorship text update for README (#2730) 2026-06-03 00:28:58 +02:00
SoxojandGitHub 528b75336c Added sponsor logo and CLI proxy recommendation (#2729) 2026-06-02 18:28:15 +02:00
SoxojandGitHub 75e1015b05 Added sponsor logo and text to the README files (#2728) 2026-06-02 18:22:28 +02:00
SoxojandGitHub e8c7d5770f Fix site checks: 1 fixed, 8 disabled, 2 added (#2722) 2026-05-31 17:50:47 +02:00
SoxojandGitHub 4cc803bb39 Graceful Ctrl+C + error UX improvements (#2719) 2026-05-30 16:14:57 +02:00
SoxojandGitHub a89b7f09a7 Fix DNS resolver failures: classify aiodns errors, add --dns-resolver threaded fallback (#2688) (#2717) 
* Fix DNS resolver failures: classify aiodns errors, add --dns-resolver threaded fallback (#2688)

* test: include dns_resolver in CLI default args
 2026-05-30 16:09:54 +02:00
SoxojandGitHub 6babd43e8d Fix site checks: 4 fixed, 1 disabled (#2715) 2026-05-30 13:11:30 +02:00
SoxojandGitHub 0b49c8612a Dockerhub build update (#2713) 
* Dockerhub: update of label orders, fix for README update
 2026-05-29 18:41:11 +02:00
SoxojandGitHub bab78c96a6 Docker build workflow: update README in DockerHub, change order of labels (#2712) 2026-05-29 18:27:09 +02:00
SoxojandGitHub 9a2cce4d85 Add Cloudflare bypass (FlareSolverr) support to web UI + docs (#2700) 2026-05-25 18:53:18 +02:00
SoxojandGitHub fe93347da1 Fix Chinese name false positives: add CJK regexCheck to 27 sites (#2633) (#2699) 2026-05-25 15:32:43 +02:00
SoxojandGitHub b3f667d908 Fix docs build: dedup link targets, gate SVG badges to HTML (#2695) 2026-05-23 10:15:53 +02:00
SoxojandGitHub 070c086f29 Add docs FAQ for top search queries; document translation workflow; Chinese update (#2694) 2026-05-23 09:46:35 +02:00
SoxojandGitHub 1d67cbe2c7 Enhance README with AI profiling and analysis details (#2691) 2026-05-21 19:27:45 +02:00
SoxojandGitHub ec7f57eab7 Windows version improvements and docs (#2690) 2026-05-21 12:12:17 +02:00
SoxojandGitHub 2eef1d3d9d Added 3 sites (#2687) 2026-05-20 23:56:40 +02:00
SoxojandGitHub 992da86284 Added donation links (#2686) 2026-05-20 23:53:29 +02:00
SoxojandGitHub ceed9aa9cc fix(Instagram): refresh rate-limit marker for stale Login title (#2674) 2026-05-16 18:23:13 +02:00
SoxojandGitHub 51a5169987 fix(checking): reject URLs and emails extracted as usernames (#2673) 
Closes #1403
 2026-05-16 17:52:00 +02:00
SoxojandGitHub 3e77c13743 Document Tor/proxy usage, add Advanced usage docs section (closes #544) (#2663) 2026-05-16 14:42:36 +02:00
SoxojandGitHub c5885331d6 Bump to 0.6.1 (#2661) 2026-05-16 12:05:55 +02:00
SoxojandGitHub c1674529ad Fix site checks: 2 fixed, 3 disabled; add Faceit; fix utils import (#2660) 2026-05-15 18:15:38 +02:00
SoxojandGitHub a7338e97f3 Make xhtml2pdf optional, fix install on Linux without libcairo (#2659) 
* Make xhtml2pdf optional, fix install on Linux without libcairo

Move xhtml2pdf to the new [pdf] extra so default `pip install maigret`
no longer pulls pycairo (which has no Linux/macOS wheels and breaks the
build without libcairo2-dev). save_pdf_report now raises a clear
RuntimeError pointing to `pip install 'maigret[pdf]'`, and the CLI
turns it into a friendly warning instead of a crash. Adds tests
covering the missing-extra path, plus per-OS install docs.

Fix for #2657, #2534

* Make arabic-reshaper and python-bidi optional; idempotent update of db_meta.json and sites.md

* Regenerated poerty.lock

* Update CI workflow to cover minimal installation without PDF deps
 2026-05-15 14:33:55 +02:00
SoxojandGitHub 203d712fa5 Update links to the community Telegram bot (#2641) 2026-05-11 19:41:43 +02:00
SoxojandGitHub adbeb53618 Update download badge links in README.md (#2636) 2026-05-10 10:43:05 +03:00
SoxojandGitHub 264bae34d6 Add test for CheckError bug (#2631) 2026-05-09 16:11:07 +03:00
SoxojandGitHub 5c93b206e7 Cloudflare bypass webgate (#2628) 2026-05-09 10:48:43 +03:00
SoxojandGitHub 79e93ab715 AI mode documentation (#2620) 2026-05-05 22:21:00 +02:00
SoxojandGitHub 846feb6e7e Add web interface tests (#2619) 2026-05-05 19:32:01 +02:00
SoxojandGitHub 03b62027f6 Fixed duplicates of YouTube and Periscope (#2618) 2026-05-05 14:02:37 +02:00
SoxojandGitHub f293bff417 Fix site checks: 7 fixed, 1 disabled, 1 dead deleted (#2616) 2026-05-04 23:40:58 +02:00
SoxojandGitHub a77a8b3e84 Reddit fix (#2614) 2026-05-04 14:12:22 +02:00
SoxojandGitHub 3ff05b240a Fix site checks: 8 → ip_reputation, 6 fixed, 9 disabled, 1 dead deleted (#2611) 2026-05-03 20:02:45 +02:00
SoxojandGitHub 606fba01b4 Update CONTRIBUTING.md with instructions for developers (#2589) 2026-05-02 10:39:56 +02:00
SoxojandGitHub b79f8aca28 Add site checks: 18 new sites (#2575) 2026-04-29 16:55:47 +02:00
SoxojandGitHub 1352bd35c6 Fix site checks: 5 fixed, 4 disabled; fix UA leak bug (#2569) 2026-04-26 14:51:44 +02:00
SoxojandGitHub 3960510b63 Fix site checks: 7 fixed, 1 disabled (#2565) 
False-positive site probe issues #2531, #2542, #2556, #2559, #2560, #2561, #2563, #2496.
 2026-04-26 12:34:52 +02:00
SoxojandGitHub a7bda700b4 Add Docker web image with multi-stage building (#2564) 2026-04-26 11:45:08 +02:00
SoxojandGitHub e962b8c693 Fix site checks: 5 fixed; readme fix (#2562) 
* Fix site checks: 5 fixed; readme fix

* Logging improvements

* Improve YouTube data extraction
 2026-04-25 18:15:38 +02:00
SoxojandGitHub 77c11df119 Fix Google Cloud Shell launch (#2557) 2026-04-23 21:45:27 +02:00
SoxojandGitHub 25026e21ea Fix site checks: 4 → ip_reputation, 9 fixed, 16 disabled, 3 dead dele… (#2555) 
* Fix site checks: 4 → ip_reputation, 9 fixed, 16 disabled, 3 dead deleted; clarify ip_reputation tag semantics

* Improved test coverage
 2026-04-23 21:17:07 +02:00
SoxojandGitHub b1004588af AI mode (#2529) 
* Add AI mode
 2026-04-23 12:12:54 +02:00
SoxojandGitHub 5e1cc45c17 Fix site checks: 12 fixed, 19 disabled; add new protection tags (#2550) 2026-04-22 20:25:41 +02:00
SoxojandGitHub d9b361b626 Fix site checks: 3 → ip_reputation, 10 fixed, 6 disabled, 2 dead deleted (#2549) 2026-04-22 12:46:53 +02:00
SoxojandGitHub 0131f0b64c Add OnlyFans with activation mechanism; updated site ranks (#2546) 2026-04-21 19:03:45 +02:00
SoxojandGitHub e8393bfce3 Fix site checks: 3 fixed, 2 → ip_reputation, 7 disabled, 1 dead deleted (#2543) 2026-04-21 16:02:36 +02:00
SoxojandGitHub 98f03c153b Add 3 crypto sites (Polymarket, Zora, Revolut.me), added crypto inves… (#2538) 
* Add 3 crypto sites (Polymarket, Zora, Revolut.me), added crypto investigation use case page in docs

* Added fintech tag

* Updated sites metadata
 2026-04-21 11:08:48 +02:00
SoxojandGitHub 1f823e8322 Fix site checks: 3 fixed, 2 → ip_reputation, 7 disabled, 1 dead deleted (#2539) 2026-04-21 10:58:45 +02:00
SoxojandGitHub d6905a8fd8 Fix site checks: 4 fixed, 14 → ip_reputation, 8 disabled, 5 dead deleted (#2537) 2026-04-21 00:40:24 +02:00
SoxojandGitHub 7d216638fa fix site checks: 14 sites → ip_reputation, 7 disabled, 5 dead deleted (#2536) 2026-04-20 23:51:18 +02:00
SoxojandGitHub fb71f26fd0 Fix site checks: recover 6 CF sites via tls_fingerprint, 500px GraphQL, delete 4 dead domains (#2535) 2026-04-20 22:41:51 +02:00
SoxojandGitHub 37ce4fe728 Update of Readme and documentation (#2514) 
* Big readme update

* Readme and documentation update

* Readme structure update

* Small fixes

* Changelog update
 2026-04-17 17:42:36 +02:00
SoxojandGitHub f74f82ee13 Fixed: Hack MD, DailyKos, Mywed, WikimapiaSearch, TikTok Online Viewer (#2526, #2522, #2523, #2500, #2496); Disabled: Radiokot, Lurkmore, Mylespaul, AppleDiscussions, Loveplanet (#2524, #2511, #2498) (#2528) 2026-04-17 17:04:50 +02:00
SoxojandGitHub 8267367bed Support Python 3.14 in tests (#2515) 2026-04-11 15:11:09 +02:00
SoxojandGitHub 7fd9bb3692 Update workflow to trigger on published releases (#2508) 2026-04-10 12:34:00 +02:00