## Why
`docs/llms.txt` is written by hand and was last touched in February, so
the index served at <https://langchain-ai.github.io/langgraph/llms.txt>
had drifted badly:
- 12 pages listed, against the 43 the docs site publishes.
- One entry, `/oss/python/langgraph/why-langgraph`, now 404s.
The deploy workflow only fires on `docs/**` pushes, so nothing brought
the file back in line as the docs site changed. It is served verbatim:
`generate_redirects.py` copies it into `docs/_site` via a `static_files`
list, and `deploy-redirects.yml` uploads that directory to Pages.
The docs site already generates exactly this index, and regenerates it
on every deploy:
```
https://docs.langchain.com/oss/python/langgraph/llms.txt
```
## What changed
`generate_redirects.py` now fetches that index at build time and writes
it into `_site`, so the published file cannot drift from the docs site.
The committed `docs/llms.txt` stays as a fallback only, refreshed here
to current content.
`deploy-redirects.yml` gains a weekly `schedule:` trigger, so docs
changes that never touch this repo still reach the deployed file. The
`permissions:` block is unchanged.
## Fetching remote content in CI
The fetched body is published on a public Pages site, so it is validated
before it is written:
- The URL is a hardcoded module constant, never built from input.
- Both that URL and the post-redirect `response.url` are checked against
an HTTPS-plus-single-host allowlist. `urlopen` follows redirects, so
checking only the request URL would not be enough.
- 30 second timeout, response capped at 1MB, decoded as UTF-8.
- The body must open with a markdown heading and contain a
docs.langchain.com link, which rejects an error page or a truncated
response.
Any failure returns `None` and falls back to the committed copy, so a
docs.langchain.com outage degrades to a stale file rather than a broken
deploy or a published error page.
## Verification
Ran `python docs/generate_redirects.py`: 294 redirect files, `llms.txt`
fetched, 43 entries.
Each rejection path was exercised against a stubbed `urlopen` and all
fall back to the committed file:
| Case | Result |
|---|---|
| Network error | falls back |
| Timeout | falls back |
| Redirect to another host | falls back |
| Body over 1MB | falls back |
| Invalid UTF-8 | falls back |
| HTML error page | falls back |
| Empty body | falls back |
| Valid index | published |
Allowlist rejects `http://docs.langchain.com/...`,
`https://evil.com/...`, and `https://docs.langchain.com.evil.com/...`.
## Note on ordering
The fallback committed here is labelled "LangGraph (Python)", which is
what the docs site will serve once langchain-ai/docs#6032 deploys. Until
then the fetched file reads "Open source (Python)". No action needed; it
self-corrects.
---
Written with Claude Code; I reviewed the diff and ran the verification
above.
* docs: Add codespell for markdown files
* update
* remove path
* fix
* update linting guidelines
* chore[deps]: upgrade dependencies with `uv lock --upgrade` (#5471)
Co-authored-by: sydney-runkle <54324534+sydney-runkle@users.noreply.github.com>
* fix(checkpoint): correct logging call to use logger (#5458)
fix[checkpoint]: correct logging call to use logger
* release(langgraph): v0.5.3 (#5498)
bump
* extend to cover python files used for reference docs
* fix(docs): Update the graph image link (#5500)
Update the graph image link
Point to the correct image reference for Map-Reduce and the Send API example
* fix(docs): Update graph-api.md File to reflect correct image (#5499)
Update graph-api.md File to reflect correct image
Referencing to the correct image file
* docs(prebuilt): improve documentation in ToolNode module (#5497)
Update documentation in ToolNode module
* Update changelog via LangGraph Server Changelog Bot
* feat(sdk-py): Show is_studio_user (#5505)
* Update changelog via LangGraph Server Changelog Bot
* fix(docs): Node caching explanation code required a small fix,. (#5473)
fix(docs): Node caching explanation code required a small fix, to avoid confusion to readers. The code had `time.sleep(2)` but the note mentioned one second only.
Co-authored-by: ygicp <yagnesh@infocusp.com>
* fix(langgraph): add `stacklevel=2` to the warnings to point to the caller’s codes (#5457)
chore: add stacklevel=2 to the warnings to point to the caller’s codes
* chore(docs): Improve example in use mcp (#5480)
* Make example more explicit
* Update docs/docs/agents/mcp.md
* fix(docs): update examples link (#5515)
Co-authored-by: ahmed murtaza <ahmed.gmurtaza@gmail.com>
* docs: [LangGraph Server Changelog Bot] Changelog updates for new version(s) (#5514)
Update changelog via LangGraph Server Changelog Bot
* fix readmes
* fix
---------
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Co-authored-by: sydney-runkle <54324534+sydney-runkle@users.noreply.github.com>
Co-authored-by: Michael Li <michaelli65535@gmail.com>
Co-authored-by: Sakshi Gupta <64280320+sakshi1989@users.noreply.github.com>
Co-authored-by: Eugene Yurtsev <eyurtsev@gmail.com>
Co-authored-by: Sam Crowder <samecrowder@gmail.com>
Co-authored-by: William FH <13333726+hinthornw@users.noreply.github.com>
Co-authored-by: Yagnesh M. Bhadiyadra <35532869+yagneshmb@users.noreply.github.com>
Co-authored-by: ygicp <yagnesh@infocusp.com>
Co-authored-by: Ahmed Murtaza <ahmed.gmurtaza@hotmail.com>
Co-authored-by: ahmed murtaza <ahmed.gmurtaza@gmail.com>
* docs: static vs dynamic interrupts
* fixes based on feedback
* fix image
* Add section about debugging in Studio
* Reorg content based on feedback
* fix
* fix links
* fix wording
* fix wording
