mirror of
https://github.com/langchain-ai/langgraph.git
synced 2026-09-17 15:17:57 +02:00
Lauren Hirata SinghandGitHub
230927fb3a
## Why `docs/llms.txt` is written by hand and was last touched in February, so the index served at <https://langchain-ai.github.io/langgraph/llms.txt> had drifted badly: - 12 pages listed, against the 43 the docs site publishes. - One entry, `/oss/python/langgraph/why-langgraph`, now 404s. The deploy workflow only fires on `docs/**` pushes, so nothing brought the file back in line as the docs site changed. It is served verbatim: `generate_redirects.py` copies it into `docs/_site` via a `static_files` list, and `deploy-redirects.yml` uploads that directory to Pages. The docs site already generates exactly this index, and regenerates it on every deploy: ``` https://docs.langchain.com/oss/python/langgraph/llms.txt ``` ## What changed `generate_redirects.py` now fetches that index at build time and writes it into `_site`, so the published file cannot drift from the docs site. The committed `docs/llms.txt` stays as a fallback only, refreshed here to current content. `deploy-redirects.yml` gains a weekly `schedule:` trigger, so docs changes that never touch this repo still reach the deployed file. The `permissions:` block is unchanged. ## Fetching remote content in CI The fetched body is published on a public Pages site, so it is validated before it is written: - The URL is a hardcoded module constant, never built from input. - Both that URL and the post-redirect `response.url` are checked against an HTTPS-plus-single-host allowlist. `urlopen` follows redirects, so checking only the request URL would not be enough. - 30 second timeout, response capped at 1MB, decoded as UTF-8. - The body must open with a markdown heading and contain a docs.langchain.com link, which rejects an error page or a truncated response. Any failure returns `None` and falls back to the committed copy, so a docs.langchain.com outage degrades to a stale file rather than a broken deploy or a published error page. ## Verification Ran `python docs/generate_redirects.py`: 294 redirect files, `llms.txt` fetched, 43 entries. Each rejection path was exercised against a stubbed `urlopen` and all fall back to the committed file: | Case | Result | |---|---| | Network error | falls back | | Timeout | falls back | | Redirect to another host | falls back | | Body over 1MB | falls back | | Invalid UTF-8 | falls back | | HTML error page | falls back | | Empty body | falls back | | Valid index | published | Allowlist rejects `http://docs.langchain.com/...`, `https://evil.com/...`, and `https://docs.langchain.com.evil.com/...`. ## Note on ordering The fallback committed here is labelled "LangGraph (Python)", which is what the docs site will serve once langchain-ai/docs#6032 deploys. Until then the fetched file reads "Open source (Python)". No action needed; it self-corrects. --- Written with Claude Code; I reviewed the diff and ran the verification above.
54 lines
1.4 KiB
YAML
54 lines
1.4 KiB
YAML
name: Deploy Redirects to GitHub Pages
|
|
|
|
on:
|
|
push:
|
|
branches:
|
|
- main
|
|
paths:
|
|
- 'docs/**'
|
|
- '.github/workflows/deploy-redirects.yml'
|
|
# llms.txt is fetched from docs.langchain.com at build time, so redeploy on a
|
|
# schedule to pick up docs changes that never touch this repo.
|
|
schedule:
|
|
- cron: '17 6 * * 1'
|
|
workflow_dispatch:
|
|
|
|
permissions:
|
|
contents: read
|
|
pages: write
|
|
id-token: write
|
|
|
|
concurrency:
|
|
group: "pages"
|
|
cancel-in-progress: false
|
|
|
|
jobs:
|
|
deploy:
|
|
environment:
|
|
name: github-pages
|
|
url: ${{ steps.deployment.outputs.page_url }}
|
|
runs-on: ubuntu-latest
|
|
steps:
|
|
- name: Checkout
|
|
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
|
|
|
|
- name: Setup Python
|
|
uses: actions/setup-python@5fda3b95a4ea91299a34e894583c3862153e4b97 # v7.0.0
|
|
with:
|
|
python-version: '3.11'
|
|
|
|
- name: Generate redirect files
|
|
run: python docs/generate_redirects.py
|
|
|
|
- name: Setup Pages
|
|
uses: actions/configure-pages@45bfe0192ca1faeb007ade9deae92b16b8254a0d # v6.0.0
|
|
|
|
- name: Upload artifact
|
|
uses: actions/upload-pages-artifact@fc324d3547104276b827a68afc52ff2a11cc49c9 # v5.0.0
|
|
with:
|
|
path: 'docs/_site'
|
|
|
|
- name: Deploy to GitHub Pages
|
|
id: deployment
|
|
uses: actions/deploy-pages@cd2ce8fcbc39b97be8ca5fce6e763baed58fa128 # v5.0.0
|