docs: publish the generated llms.txt instead of a hand-written copy (#8922)

## Why

`docs/llms.txt` is written by hand and was last touched in February, so
the index served at <https://langchain-ai.github.io/langgraph/llms.txt>
had drifted badly:

- 12 pages listed, against the 43 the docs site publishes.
- One entry, `/oss/python/langgraph/why-langgraph`, now 404s.

The deploy workflow only fires on `docs/**` pushes, so nothing brought
the file back in line as the docs site changed. It is served verbatim:
`generate_redirects.py` copies it into `docs/_site` via a `static_files`
list, and `deploy-redirects.yml` uploads that directory to Pages.

The docs site already generates exactly this index, and regenerates it
on every deploy:

```
https://docs.langchain.com/oss/python/langgraph/llms.txt
```

## What changed

`generate_redirects.py` now fetches that index at build time and writes
it into `_site`, so the published file cannot drift from the docs site.
The committed `docs/llms.txt` stays as a fallback only, refreshed here
to current content.

`deploy-redirects.yml` gains a weekly `schedule:` trigger, so docs
changes that never touch this repo still reach the deployed file. The
`permissions:` block is unchanged.

## Fetching remote content in CI

The fetched body is published on a public Pages site, so it is validated
before it is written:

- The URL is a hardcoded module constant, never built from input.
- Both that URL and the post-redirect `response.url` are checked against
an HTTPS-plus-single-host allowlist. `urlopen` follows redirects, so
checking only the request URL would not be enough.
- 30 second timeout, response capped at 1MB, decoded as UTF-8.
- The body must open with a markdown heading and contain a
docs.langchain.com link, which rejects an error page or a truncated
response.

Any failure returns `None` and falls back to the committed copy, so a
docs.langchain.com outage degrades to a stale file rather than a broken
deploy or a published error page.

## Verification

Ran `python docs/generate_redirects.py`: 294 redirect files, `llms.txt`
fetched, 43 entries.

Each rejection path was exercised against a stubbed `urlopen` and all
fall back to the committed file:

| Case | Result |
|---|---|
| Network error | falls back |
| Timeout | falls back |
| Redirect to another host | falls back |
| Body over 1MB | falls back |
| Invalid UTF-8 | falls back |
| HTML error page | falls back |
| Empty body | falls back |
| Valid index | published |

Allowlist rejects `http://docs.langchain.com/...`,
`https://evil.com/...`, and `https://docs.langchain.com.evil.com/...`.

## Note on ordering

The fallback committed here is labelled "LangGraph (Python)", which is
what the docs site will serve once langchain-ai/docs#6032 deploys. Until
then the fetched file reads "Open source (Python)". No action needed; it
self-corrects.

---

Written with Claude Code; I reviewed the diff and ran the verification
above.

.github/workflows/deploy-redirects.yml

@@ -7,6 +7,10 @@ on:
     paths:
       - 'docs/**'
       - '.github/workflows/deploy-redirects.yml'
+  # llms.txt is fetched from docs.langchain.com at build time, so redeploy on a
+  # schedule to pick up docs changes that never touch this repo.
+  schedule:
+    - cron: '17 6 * * 1'
   workflow_dispatch:
 
 permissions:

docs/generate_redirects.py

@@ -12,13 +12,26 @@ which is SEO-friendly and treated similarly to 301 redirects by Google.
 To add new redirects, simply edit redirects.json and re-run this script.
 """
 
+import http.client
 import json
 import os
+import urllib.error
+import urllib.parse
+import urllib.request
 from pathlib import Path
 
 # Default fallback URL for any path not in the redirect map
 DEFAULT_REDIRECT = "https://docs.langchain.com/oss/python/langgraph/overview"
 
+# The docs site regenerates this index on every deploy, so fetching it here
+# keeps the published llms.txt from drifting. The URL is a hardcoded constant,
+# never built from input, and both it and the post-redirect URL are checked
+# against ALLOWED_LLMS_HOST before anything is read.
+CANONICAL_LLMS_URL = "https://docs.langchain.com/oss/python/langgraph/llms.txt"
+ALLOWED_LLMS_HOST = "docs.langchain.com"
+LLMS_FETCH_TIMEOUT = 30
+LLMS_MAX_BYTES = 1_000_000
+
 HTML_TEMPLATE = """<!doctype html>
 <html lang="en">
 <head>
@@ -75,6 +88,64 @@ CATCHALL_404_TEMPLATE = """<!doctype html>
 """
 
 
+def is_allowed_llms_url(url):
+    """Return True if url is HTTPS on the one host we accept content from."""
+    parsed = urllib.parse.urlsplit(url)
+    return parsed.scheme == "https" and parsed.hostname == ALLOWED_LLMS_HOST
+
+
+def fetch_canonical_llms_txt():
+    """Return the published LangGraph index, or None if it cannot be used.
+
+    Returning None leaves the caller on the committed docs/llms.txt, so a
+    docs.langchain.com outage degrades to a stale file rather than a broken
+    deploy or a published error page.
+    """
+    if not is_allowed_llms_url(CANONICAL_LLMS_URL):
+        print(f"Refusing to fetch {CANONICAL_LLMS_URL}: host not allowed")
+        return None
+
+    try:
+        with urllib.request.urlopen(  # noqa: S310 - constant, allowlisted URL
+            CANONICAL_LLMS_URL, timeout=LLMS_FETCH_TIMEOUT
+        ) as response:
+            # urlopen follows redirects, so re-check where it actually landed.
+            if not is_allowed_llms_url(response.url):
+                print(f"Refusing {CANONICAL_LLMS_URL}: redirected to {response.url}")
+                return None
+            body = response.read(LLMS_MAX_BYTES + 1)
+    # A connection dropped mid-body raises http.client.IncompleteRead, which
+    # descends from HTTPException rather than OSError, so catching only the
+    # urllib and OS errors would let it escape and fail the whole deploy.
+    except (
+        urllib.error.URLError,
+        http.client.HTTPException,
+        TimeoutError,
+        OSError,
+    ) as exc:
+        print(f"Could not fetch {CANONICAL_LLMS_URL}: {type(exc).__name__}: {exc}")
+        return None
+
+    if len(body) > LLMS_MAX_BYTES:
+        print(f"Refusing {CANONICAL_LLMS_URL}: larger than {LLMS_MAX_BYTES} bytes")
+        return None
+
+    try:
+        text = body.decode("utf-8")
+    except UnicodeDecodeError as exc:
+        print(f"Refusing {CANONICAL_LLMS_URL}: not valid UTF-8: {exc}")
+        return None
+
+    # An index opens with a markdown heading and links to the docs site. A
+    # body that does not is an error page or a truncated response, not content
+    # worth publishing.
+    if not text.startswith("# ") or f"https://{ALLOWED_LLMS_HOST}/" not in text:
+        print(f"Refusing {CANONICAL_LLMS_URL}: does not look like an llms.txt index")
+        return None
+
+    return text
+
+
 def generate_redirects():
     script_dir = Path(__file__).parent
     output_dir = script_dir / "_site"
@@ -126,14 +197,19 @@ def generate_redirects():
     catchall_404.write_text(CATCHALL_404_TEMPLATE.format(default_url=DEFAULT_REDIRECT))
     print(f"Created: {catchall_404}")
 
-    # Copy static files (like llms.txt) that can't be redirected via HTML
-    static_files = ["llms.txt"]
-    for static_file in static_files:
-        src = script_dir / static_file
+    # llms.txt can't be redirected via HTML, so publish the docs site's own
+    # generated index. The committed copy is only a fallback.
+    llms_txt = fetch_canonical_llms_txt()
+    if llms_txt is not None:
+        (output_dir / "llms.txt").write_text(llms_txt)
+        print(f"Fetched: {output_dir / 'llms.txt'} (from {CANONICAL_LLMS_URL})")
+    else:
+        src = script_dir / "llms.txt"
         if src.exists():
-            dst = output_dir / static_file
-            dst.write_text(src.read_text())
-            print(f"Copied: {dst}")
+            (output_dir / "llms.txt").write_text(src.read_text())
+            print(f"Copied: {output_dir / 'llms.txt'} (fallback, may be stale)")
+        else:
+            print("No llms.txt fetched and no committed fallback; skipping")
 
     print(f"\nGenerated {len(redirects)} redirect files in {output_dir}")
 

docs/llms.txt

@@ -1,35 +1,49 @@
-# LangGraph
+# Docs by LangChain: LangGraph (Python)
 
-LangGraph documentation has moved to docs.langchain.com.
+> Markdown index of the LangGraph (Python) documentation.
 
-## Overview
+## LangGraph (Python)
 
-- [LangGraph Overview](https://docs.langchain.com/oss/python/langgraph/overview): Introduction to LangGraph, a library for building stateful, multi-actor applications with LLMs.
-- [Why LangGraph?](https://docs.langchain.com/oss/python/langgraph/why-langgraph): Motivation for LangGraph and its key features.
-
-## Core Concepts
-
-- [Graph API](https://docs.langchain.com/oss/python/langgraph/graph-api): Learn how to define state, create nodes, and connect them with edges.
-- [Streaming](https://docs.langchain.com/oss/python/langgraph/streaming): Stream outputs from your graph for better UX.
-- [Persistence](https://docs.langchain.com/oss/python/langgraph/persistence): Add memory and checkpointing to your graphs.
-- [Add Memory](https://docs.langchain.com/oss/python/langgraph/add-memory): Implement short-term and long-term memory.
-- [Workflows & Agents](https://docs.langchain.com/oss/python/langgraph/workflows-agents): Build agents and workflows with LangGraph.
-
-## How-To Guides
-
-- [Use Subgraphs](https://docs.langchain.com/oss/python/langgraph/use-subgraphs): Compose graphs using subgraphs.
-- [Observability](https://docs.langchain.com/oss/python/langgraph/observability): Add tracing and debugging to your graphs.
-- [Common Errors](https://docs.langchain.com/oss/python/langgraph/common-errors): Troubleshoot common LangGraph errors.
-
-## Tutorials
-
-- [Agentic RAG](https://docs.langchain.com/oss/python/langgraph/agentic-rag): Build an agentic RAG system with LangGraph.
-- [SQL Agent](https://docs.langchain.com/oss/python/langgraph/sql-agent): Create a SQL agent with LangGraph.
-
-## Reference
-
-- [API Reference](https://reference.langchain.com/python/langgraph/): Complete API documentation for LangGraph.
-
-## LangGraph Platform
-
-For deploying LangGraph applications in production, see the [LangSmith documentation](https://docs.langchain.com/langsmith/agent-server).
+- [Memory](https://docs.langchain.com/oss/python/langgraph/add-memory.md)
+- [Build a custom RAG agent with LangGraph](https://docs.langchain.com/oss/python/langgraph/agentic-rag.md)
+- [Application structure](https://docs.langchain.com/oss/python/langgraph/application-structure.md)
+- [Backward compatibility](https://docs.langchain.com/oss/python/langgraph/backward-compatibility.md)
+- [Case studies](https://docs.langchain.com/oss/python/langgraph/case-studies.md)
+- [Changelog](https://docs.langchain.com/oss/python/langgraph/changelog-js.md)
+- [Changelog](https://docs.langchain.com/oss/python/langgraph/changelog-py.md)
+- [Checkpointers](https://docs.langchain.com/oss/python/langgraph/checkpointers.md)
+- [Choosing between Graph and Functional APIs](https://docs.langchain.com/oss/python/langgraph/choosing-apis.md)
+- [Deployment](https://docs.langchain.com/oss/python/langgraph/deploy.md)
+- [GRAPH_RECURSION_LIMIT](https://docs.langchain.com/oss/python/langgraph/errors/GRAPH_RECURSION_LIMIT.md)
+- [INVALID_CHAT_HISTORY](https://docs.langchain.com/oss/python/langgraph/errors/INVALID_CHAT_HISTORY.md)
+- [INVALID_CONCURRENT_GRAPH_UPDATE](https://docs.langchain.com/oss/python/langgraph/errors/INVALID_CONCURRENT_GRAPH_UPDATE.md)
+- [INVALID_GRAPH_NODE_RETURN_VALUE](https://docs.langchain.com/oss/python/langgraph/errors/INVALID_GRAPH_NODE_RETURN_VALUE.md)
+- [MISSING_CHECKPOINTER](https://docs.langchain.com/oss/python/langgraph/errors/MISSING_CHECKPOINTER.md)
+- [MULTIPLE_SUBGRAPHS](https://docs.langchain.com/oss/python/langgraph/errors/MULTIPLE_SUBGRAPHS.md)
+- [Event streaming](https://docs.langchain.com/oss/python/langgraph/event-streaming.md)
+- [Fault tolerance](https://docs.langchain.com/oss/python/langgraph/fault-tolerance.md)
+- [Custom stream channels](https://docs.langchain.com/oss/python/langgraph/frontend/custom-stream-channels.md)
+- [Graph execution](https://docs.langchain.com/oss/python/langgraph/frontend/graph-execution.md)
+- [Overview](https://docs.langchain.com/oss/python/langgraph/frontend/overview.md)
+- [Functional API overview](https://docs.langchain.com/oss/python/langgraph/functional-api.md)
+- [Graph API overview](https://docs.langchain.com/oss/python/langgraph/graph-api.md)
+- [Install LangGraph](https://docs.langchain.com/oss/python/langgraph/install.md)
+- [Interrupts](https://docs.langchain.com/oss/python/langgraph/interrupts.md)
+- [Run a local server](https://docs.langchain.com/oss/python/langgraph/local-server.md)
+- [LangSmith Observability](https://docs.langchain.com/oss/python/langgraph/observability.md)
+- [LangGraph overview](https://docs.langchain.com/oss/python/langgraph/overview.md)
+- [Persistence](https://docs.langchain.com/oss/python/langgraph/persistence.md)
+- [LangGraph runtime](https://docs.langchain.com/oss/python/langgraph/pregel.md)
+- [Quickstart](https://docs.langchain.com/oss/python/langgraph/quickstart.md)
+- [Build a custom SQL agent](https://docs.langchain.com/oss/python/langgraph/sql-agent.md)
+- [Stores](https://docs.langchain.com/oss/python/langgraph/stores.md)
+- [Streaming](https://docs.langchain.com/oss/python/langgraph/streaming.md)
+- [LangSmith Studio](https://docs.langchain.com/oss/python/langgraph/studio.md)
+- [Test](https://docs.langchain.com/oss/python/langgraph/test.md)
+- [Thinking in LangGraph](https://docs.langchain.com/oss/python/langgraph/thinking-in-langgraph.md)
+- [Agent Chat UI](https://docs.langchain.com/oss/python/langgraph/ui.md)
+- [Use the functional API](https://docs.langchain.com/oss/python/langgraph/use-functional-api.md)
+- [Use the graph API](https://docs.langchain.com/oss/python/langgraph/use-graph-api.md)
+- [Subgraphs](https://docs.langchain.com/oss/python/langgraph/use-subgraphs.md)
+- [Use time-travel](https://docs.langchain.com/oss/python/langgraph/use-time-travel.md)
+- [Workflows and agents](https://docs.langchain.com/oss/python/langgraph/workflows-agents.md)

libs/cli/langgraph_cli/deploy.py

@@ -51,7 +51,6 @@ RESERVED_ENV_VARS = frozenset(
         "LANGGRAPH_AUTH_TYPE",
         "LANGSMITH_AUTH_ENDPOINT",
         "LANGSMITH_TENANT_ID",
-        "LANGSMITH_WORKSPACE_ID",
         "LANGSMITH_AUTH_VERIFY_TENANT_ID",
         "LANGSMITH_HOST_PROJECT_ID",
         "LANGSMITH_HOST_PROJECT_NAME",
@@ -1230,24 +1229,6 @@ def _run_remote_build(
 # ---------------------------------------------------------------------------
 
 
-def _get_tenant_id(env_vars: dict[str, str]) -> str | None:
-    """Get the tenant ID from LANGSMITH_TENANT_ID or LANGSMITH_WORKSPACE_ID."""
-    tenant_id = env_vars.get("LANGSMITH_TENANT_ID") or os.environ.get(
-        "LANGSMITH_TENANT_ID"
-    )
-    fallback_tenant_id = env_vars.get("LANGSMITH_WORKSPACE_ID") or os.environ.get(
-        "LANGSMITH_WORKSPACE_ID"
-    )
-    if tenant_id and fallback_tenant_id:
-        raise click.UsageError(
-            "LANGSMITH_TENANT_ID and LANGSMITH_WORKSPACE_ID cannot both be set. "
-            "Set only one."
-        )
-    if tenant_id:
-        return tenant_id
-    return fallback_tenant_id or None
-
-
 def _create_host_backend_client(
     host_url: str | None,
     api_key: str | None,
@@ -1255,7 +1236,6 @@ def _create_host_backend_client(
 ) -> HostBackendClient:
     if env_vars is None:
         env_vars = _parse_env_from_config({}, pathlib.Path.cwd() / DEFAULT_CONFIG)
-    tenant_id = _get_tenant_id(env_vars)
     resolved_api_key = api_key
     if not resolved_api_key:
         for key_name in _API_KEY_ENV_NAMES:
@@ -1278,6 +1258,9 @@ def _create_host_backend_client(
             fg="yellow",
         )
         resolved_api_key = click.prompt("Enter LangSmith API key", hide_input=True)
+    tenant_id = env_vars.get("LANGSMITH_TENANT_ID") or os.environ.get(
+        "LANGSMITH_TENANT_ID"
+    )
     return HostBackendClient(host_url, resolved_api_key, tenant_id=tenant_id)
 
 

libs/cli/tests/unit_tests/test_deploy_helpers.py

@@ -21,7 +21,6 @@ from langgraph_cli.deploy import (
     _parse_env_from_config,
     _resolve_env_path,
     _resolve_pushed_image_digest,
-    _secrets_from_env,
     _smith_dashboard_base_url,
     _validate_prebuilt_image,
     normalize_image_tag,
@@ -541,63 +540,6 @@ class TestCreateHostBackendClientNoInput:
         assert client is not None
 
 
-@pytest.mark.parametrize("source", ["config", "shell"])
-@pytest.mark.parametrize("name", ["LANGSMITH_TENANT_ID", "LANGSMITH_WORKSPACE_ID"])
-def test_workspace_id_alias(monkeypatch, source, name):
-    monkeypatch.delenv("LANGSMITH_WORKSPACE_ID", raising=False)
-    monkeypatch.delenv("LANGSMITH_TENANT_ID", raising=False)
-    env_vars = {}
-    if source == "config":
-        env_vars[name] = "workspace"
-    else:
-        monkeypatch.setenv(name, "workspace")
-
-    def handler(request):
-        assert request.headers["X-Tenant-ID"] == "workspace"
-        return httpx.Response(200, json={"resources": []})
-
-    monkeypatch.setattr(
-        httpx, "HTTPTransport", lambda **kwargs: httpx.MockTransport(handler)
-    )
-    client = _create_host_backend_client(
-        "https://api.example.com", "test-key", env_vars
-    )
-    try:
-        client.list_deployments()
-    finally:
-        client._client.close()
-
-
-@pytest.mark.parametrize("tenant_source", ["config", "shell"])
-@pytest.mark.parametrize("workspace_source", ["config", "shell"])
-@pytest.mark.parametrize("workspace_id", ["tenant-id", "workspace-id"])
-def test_rejects_both_workspace_names(
-    monkeypatch, tenant_source, workspace_source, workspace_id
-):
-    env_vars = {}
-    for name, source, value in [
-        ("LANGSMITH_TENANT_ID", tenant_source, "tenant-id"),
-        ("LANGSMITH_WORKSPACE_ID", workspace_source, workspace_id),
-    ]:
-        monkeypatch.delenv(name, raising=False)
-        if source == "config":
-            env_vars[name] = value
-        else:
-            monkeypatch.setenv(name, value)
-
-    with pytest.raises(
-        click.UsageError,
-        match="LANGSMITH_TENANT_ID and LANGSMITH_WORKSPACE_ID cannot both be set",
-    ):
-        _create_host_backend_client("https://api.example.com", "test-key", env_vars)
-
-
-def test_workspace_id_is_not_uploaded_as_secret():
-    assert _secrets_from_env(
-        {"LANGSMITH_WORKSPACE_ID": "workspace", "APP_SETTING": "value"}
-    ) == [{"name": "APP_SETTING", "value": "value"}]
-
-
 class TestSmithDashboardBaseUrl:
     def test_none_returns_default(self):
         assert _smith_dashboard_base_url(None) == "https://smith.langchain.com"