s0lrayandPaperclip
74f037bd30
Ran comprehensive link audit across all 1,113 URLs in arf.json. Removed nodes where the URL has definitively failed (DNS resolution failure, connection refused, or HTTP 404). Left in place: .onion Tor hidden services, 403 bot-blocked sites (site still exists), 5xx server errors (transient), and timeouts. Audit summary: - Total URLs checked: 1,113 - OK: 829 - Dead (removed): 113 (DNS failure: 61, connection refused: 8, HTTP 404: 49, minus 5 already deduplicated) - Redirected (kept, flagged in report): 374 - Slow >10s (kept): 1 Also adds tooling: - tools/link_checker.py -- async URL checker, writes JSON report - tools/remove_dead_links.py -- removes confirmed dead nodes from arf.json - tools/link_audit_report.json -- full audit results Co-Authored-By: Paperclip <noreply@paperclip.ing>
OSINT Framework
Notes
OSINT framework focused on gathering information from free tools or resources. The intention is to help people find free OSINT resources. Some of the sites included might require registration or offer more data for $$$, but you should be able to get at least a portion of the available information for no cost.
I originally created this framework with an information security point of view. Since then, the response from other fields and disciplines has been incredible. I would love to be able to include any other OSINT resources, especially from fields outside of infosec. Please let me know about anything that might be missing!
Please visit the framework at the link below and good hunting!
Legend
(T) - Indicates a link to a tool that must be installed and run locally
(D) - Google Dork, for more information: Google Hacking
(R) - Requires registration
(M) - Indicates a URL that contains the search term and the URL itself must be edited manually
For Update Notifications
Follow me on Twitter: @jnordine - https://twitter.com/jnordine
Watch or star the project on Github: https://github.com/lockfale/osint-framework
Suggestions, Comments, Feedback
Feedback or new tool suggestions are extremely welcome! Please feel free to submit a pull request or open an issue on github or reach out on Twitter.
Contribute with a GitHub Pull Request
For new resources, please ensure that the site is available for public and free use.
- Update the arf.json file in the format shown below. If this isn't the first entry for a folder, add a comma to the last closing brace of the previous entry.
{
"name": "Example Name",
"type": "url",
"url": "http://example.com"
}
- Submit pull request!
Thank you!
OSINT Framework Website
Happy Hunting!