Files
OSINT-Framework/tools
s0lrayandPaperclip 74f037bd30 Remove 113 confirmed dead links from arf.json
Ran comprehensive link audit across all 1,113 URLs in arf.json.
Removed nodes where the URL has definitively failed (DNS resolution
failure, connection refused, or HTTP 404). Left in place: .onion
Tor hidden services, 403 bot-blocked sites (site still exists),
5xx server errors (transient), and timeouts.

Audit summary:
- Total URLs checked: 1,113
- OK: 829
- Dead (removed): 113  (DNS failure: 61, connection refused: 8, HTTP 404: 49, minus 5 already deduplicated)
- Redirected (kept, flagged in report): 374
- Slow >10s (kept): 1

Also adds tooling:
- tools/link_checker.py  -- async URL checker, writes JSON report
- tools/remove_dead_links.py  -- removes confirmed dead nodes from arf.json
- tools/link_audit_report.json  -- full audit results

Co-Authored-By: Paperclip <noreply@paperclip.ing>
2026-03-21 18:29:19 -04:00
..