[haik]: added in docs skeleton from other repo into this, still gotta map it onto the current struct

docs/.gitignore

@@ -0,0 +1,9 @@
+# Generated / local-only artifacts.
+.venv/
+site/
+
+# Doc pages generated from the codebase by gen_pages.py (real files on disk now,
+# since Zensical builds from disk rather than a plugin's virtual filesystem).
+content/reference/
+content/guides/
+content/frontend/

docs/README.md

@@ -0,0 +1,63 @@
+# docs
+
+A self-contained, local-only documentation generator built with
+[**Zensical**](https://zensical.org/) (the Material for MkDocs team's Rust-based
+successor to MkDocs). It builds a static site that **mirrors the codebase** —
+pages are generated from docstrings, READMEs, and the frontend source on every
+run, so the docs stay in sync as the code changes. Same "drop-in folder, re-run
+to refresh" spirit as `dependency-graph/`.
+
+## Use it
+
+```bash
+./docs/run.sh            # build the site and open it
+./docs/run.sh --serve    # live-reloading dev server (great while writing docstrings)
+```
+
+The first run creates an isolated `docs/.venv` and installs the doc tooling
+there. Output lands in `docs/site/` (git-ignored).
+
+## What it pulls in
+
+| Source | Becomes (`content/…`) |
+| --- | --- |
+| `backend/**/*.py` docstrings | `reference/` — one [mkdocstrings](https://mkdocstrings.github.io/) page per module. |
+| `frontend/src` (TSDoc) | `frontend/` — [TypeDoc](https://typedoc.org/) reference (best-effort; needs Node + one-time network). |
+| Every `README.md`, `implementation_plan.md`, `relevant_context.md`, `frontend/DESIGN.md` | `guides/` — copied verbatim. |
+
+Add a module or a README anywhere and it appears on the next run — Zensical
+**infers the navigation from the directory tree**, so there's no nav to maintain.
+
+## How it differs from a plain Zensical site
+
+Zensical doesn't run MkDocs plugins (no `mkdocs-gen-files` / `mkdocs-literate-nav`).
+So instead of generating pages inside the build, `gen_pages.py` runs **before** the
+build as a plain script and writes **real Markdown files** into `content/reference/`,
+`content/guides/`, and `content/frontend/`. Those three directories are wiped and rebuilt
+each run and are git-ignored; only `content/index.md` is hand-written.
+
+## Files
+
+| file | role |
+| --- | --- |
+| `run.sh` | bootstraps the venv, runs TypeDoc, generates pages, builds/serves, opens the site. |
+| `zensical.toml` | site config (theme, markdown extensions, mkdocstrings options). |
+| `gen_pages.py` | the engine: walks the repo and writes the `content/` page tree (pure stdlib). |
+| `content/index.md` | the one hand-written page (the landing page). |
+| `.venv/`, `site/`, `content/{reference,guides,frontend}/` | generated, git-ignored. |
+
+## Knobs
+
+- **Docstring style**: `zensical.toml` → `[project.plugins.mkdocstrings.handlers.python.options]` → `docstring_style` (currently `google`).
+- **Skip directories / featured loose docs**: `SKIP_DIRS` and the `curated` list in `gen_pages.py`.
+- **Theme & navigation features**: `[project.theme]` in `zensical.toml`.
+- **Markdown extensions**: the `[project.markdown_extensions.*]` tables.
+
+## Notes
+
+- The **HTTP API** reference is no longer baked into this site — Zensical can't run
+  the Swagger plugin yet. Use FastAPI's built-in `/docs` (Swagger) or `/redoc`.
+- mkdocstrings support in Zensical is **preliminary** (no cross-references/backlinks
+  yet); the rest renders normally. Track progress at
+  [zensical.org/docs/setup/extensions/mkdocstrings](https://zensical.org/docs/setup/extensions/mkdocstrings/).
+- The site is a static snapshot — re-run `run.sh` to refresh.

docs/content/index.md

@@ -0,0 +1,31 @@
+# Product Analytics — Documentation
+
+This site is **auto-generated from the codebase** and built with
+[Zensical](https://zensical.org/). Nothing here is written by hand except this
+landing page — every other page is generated from the sources below before each
+build, so the docs stay in sync as the code changes.
+
+Re-run `./docs/run.sh` (or `zensical serve -o` from `docs/` for a
+live preview) to refresh.
+
+## What's in here
+
+- **reference** — one page per Python module under `backend/`, rendered from the
+  module/class/function docstrings via mkdocstrings.
+- **frontend** — TypeDoc reference for the React/TypeScript app under `frontend/`
+  (present only when TypeDoc ran during generation).
+- **guides** — every Markdown doc in the repo: the root `README`, nested
+  `README.md` files, `implementation_plan.md`, `relevant_context.md`, and
+  `frontend/DESIGN.md`.
+
+## HTTP API
+
+The interactive HTTP API reference is served by the backend itself — run it and
+open **`/docs`** (FastAPI's built-in Swagger UI) or **`/redoc`**.
+
+## How it works
+
+`gen_pages.py` walks the repository on each build and writes real Markdown files
+into `content/reference/`, `content/guides/`, and `content/frontend/`. Zensical then infers
+the navigation from that directory tree. See `docs/README.md` for the full
+layout and knobs.

docs/content/javascripts/search-suggestions.js

@@ -0,0 +1,204 @@
+/*
+ * Default search suggestions.
+ *
+ * Zensical's search modal (a Preact component living in an open shadow root on a
+ * <div> appended to <body>) shows nothing until you type. This sprinkles a small
+ * curated list of "suggested pages" into the modal whenever the query is empty —
+ * the command-palette behaviour you get from most search modals.
+ *
+ * It only ever *adds* nodes (reusing the modal's own class names so they inherit
+ * its native styling) and re-attaches them after Preact re-renders. Everything is
+ * wrapped in try/catch so a future Zensical change can, at worst, fall back to the
+ * stock empty state — it can never break the real search.
+ */
+(function () {
+  "use strict";
+
+  // Curated entries shown on an empty query. `href` is resolved against the
+  // site's base, so these work from any (including deeply nested) page.
+  var SUGGESTIONS = [
+    { title: "Documentation home", path: ["Home"], href: "index.html" },
+    { title: "API Reference", path: ["reference"], href: "reference/index.html" },
+    { title: "Guides", path: ["guides"], href: "guides/index.html" },
+    { title: "Frontend reference", path: ["frontend"], href: "frontend/index.html" },
+    { title: "Implementation plan", path: ["guides"], href: "guides/implementation_plan.html" }
+  ];
+
+  var MARK = "data-os-suggest";
+
+  function siteBase() {
+    try {
+      var cfg = JSON.parse(document.getElementById("__config").textContent);
+      return String(cfg.base || ".").replace(/\/?$/, "/");
+    } catch (e) {
+      return "./";
+    }
+  }
+  var BASE = siteBase();
+
+  function resolve(p) {
+    try {
+      return new URL(BASE + p, location.href).href;
+    } catch (e) {
+      return p;
+    }
+  }
+
+  function el(doc, tag, cls) {
+    var n = doc.createElement(tag);
+    if (cls) n.className = cls;
+    n.setAttribute(MARK, "");
+    return n;
+  }
+
+  // Mirror the result-item markup the bundle emits: <ol class="b"> of
+  // <li><a class="a"><div class="B"><h2 class="x">title</h2>
+  // <menu class="t"><li>path…</li></menu></div></a></li>.
+  function buildNodes(doc) {
+    var frag = doc.createDocumentFragment();
+
+    var heading = el(doc, "h3", "A");
+    heading.style.opacity = "0.6";
+    heading.textContent = "Suggested pages";
+    frag.appendChild(heading);
+
+    var ol = el(doc, "ol", "b");
+    SUGGESTIONS.forEach(function (s) {
+      var li = el(doc, "li");
+      var a = el(doc, "a", "a");
+      a.href = resolve(s.href);
+
+      var wrap = el(doc, "div", "B");
+      var h2 = el(doc, "h2", "x");
+      h2.textContent = s.title;
+      var menu = el(doc, "menu", "t");
+      (s.path || []).forEach(function (seg) {
+        var pli = el(doc, "li");
+        pli.textContent = seg;
+        menu.appendChild(pli);
+      });
+
+      wrap.appendChild(h2);
+      wrap.appendChild(menu);
+      a.appendChild(wrap);
+      li.appendChild(a);
+      ol.appendChild(li);
+    });
+    frag.appendChild(ol);
+    return frag;
+  }
+
+  function wire(host) {
+    var root = host.shadowRoot;
+    if (!root || root.__osSuggestWired) return;
+    var input = root.querySelector("input[role=combobox]");
+    if (!input) return;
+    root.__osSuggestWired = true;
+
+    var doc = host.ownerDocument || document;
+    var observer;
+    var scheduled = false;
+
+    // `.e` is the modal's dialog container; its last child is the scrollable
+    // results body where real results render, so we slot suggestions in there.
+    function dialog() {
+      return root.querySelector(".e");
+    }
+    function body() {
+      var d = dialog();
+      return (d && d.lastElementChild) || d;
+    }
+    function isEmpty() {
+      return !input.value || !input.value.trim();
+    }
+
+    function apply() {
+      try {
+        var parent = body();
+        if (!parent) return;
+        var present = root.querySelector("ol[" + MARK + "]");
+        if (isEmpty()) {
+          if (!present) {
+            if (observer) observer.disconnect();
+            parent.appendChild(buildNodes(doc));
+            reobserve();
+          }
+        } else if (present) {
+          if (observer) observer.disconnect();
+          root.querySelectorAll("[" + MARK + "]").forEach(function (n) {
+            if (n.tagName !== "STYLE") n.remove();
+          });
+          reobserve();
+        }
+      } catch (e) {
+        /* never break native search */
+      }
+    }
+
+    function schedule() {
+      if (scheduled) return;
+      scheduled = true;
+      requestAnimationFrame(function () {
+        scheduled = false;
+        apply();
+      });
+    }
+
+    function reobserve() {
+      var d = dialog();
+      if (!observer || !d) return;
+      observer.observe(d, { childList: true, subtree: true });
+    }
+
+    // A little extra room so the suggestion-only state reads as intentional.
+    var style = doc.createElement("style");
+    style.setAttribute(MARK, "");
+    style.textContent =
+      "h3[" + MARK + "]{margin:0;padding:.5em .8em .25em;font-weight:600}" +
+      "[" + MARK + "] a{cursor:pointer}";
+    root.appendChild(style);
+
+    input.addEventListener("input", schedule, true);
+
+    observer = new MutationObserver(schedule);
+    reobserve();
+
+    schedule();
+    setTimeout(apply, 60);
+    setTimeout(apply, 300);
+  }
+
+  function scan() {
+    try {
+      var kids = document.body ? document.body.children : [];
+      for (var i = 0; i < kids.length; i++) {
+        var node = kids[i];
+        if (node.shadowRoot && node.shadowRoot.querySelector &&
+            node.shadowRoot.querySelector("input[role=combobox]")) {
+          wire(node);
+        }
+      }
+    } catch (e) {
+      /* ignore */
+    }
+  }
+
+  function start() {
+    scan();
+    // The overlay is created during bundle init; watch <body> for it (cheap:
+    // direct-children mutations only) and retry a few times as a safety net.
+    try {
+      new MutationObserver(scan).observe(document.body, { childList: true });
+    } catch (e) {
+      /* ignore */
+    }
+    setTimeout(scan, 300);
+    setTimeout(scan, 1200);
+  }
+
+  if (document.readyState === "loading") {
+    document.addEventListener("DOMContentLoaded", start);
+  } else {
+    start();
+  }
+})();

docs/content/stylesheets/openswarm.css

@@ -0,0 +1,135 @@
+/*
+ * Anthropic-inspired theme for the Zensical (Material) docs site.
+ *
+ * Brand palette: a warm clay/coral accent on a soft ivory canvas in light mode,
+ * and the same clay against a warm charcoal in dark mode. Editorial serif
+ * headings (Fraunces) echo Anthropic's display type; the body keeps a clean
+ * humanist sans. Both schemes are driven by `primary = "custom"` in
+ * zensical.toml, so the values below are what actually paint the UI.
+ */
+
+@import url("https://fonts.googleapis.com/css2?family=Fraunces:opsz,wght@9..144,400;9..144,500;9..144,600;9..144,700&family=Inter:wght@400;500;600&display=swap");
+
+:root {
+  /* Core brand tones (shared across schemes). */
+  --anthropic-clay: #d97757;
+  --anthropic-clay-dark: #bd5d3a;
+  --anthropic-clay-light: #e6a085;
+  --anthropic-ivory: #faf9f5;
+  --anthropic-cloud: #f0eee6;
+  --anthropic-ink: #1a1915;
+  --anthropic-slate-bg: #262624;
+  --anthropic-slate-surface: #30302d;
+  --anthropic-slate-border: #3d3d39;
+
+  --md-text-font: "Inter", -apple-system, BlinkMacSystemFont, "Segoe UI", Roboto, sans-serif;
+}
+
+/* ----------------------------------------------------------------------------
+ * Light scheme (default)
+ * ------------------------------------------------------------------------- */
+[data-md-color-scheme="default"] {
+  --md-primary-fg-color: var(--anthropic-clay);
+  --md-primary-fg-color--light: var(--anthropic-clay-light);
+  --md-primary-fg-color--dark: var(--anthropic-clay-dark);
+  --md-primary-bg-color: #ffffff;
+  --md-primary-bg-color--light: hsla(0, 0%, 100%, 0.72);
+
+  --md-accent-fg-color: var(--anthropic-clay-dark);
+  --md-accent-fg-color--transparent: hsla(15, 63%, 54%, 0.1);
+
+  --md-default-bg-color: var(--anthropic-ivory);
+  --md-default-fg-color: hsla(48, 12%, 9%, 0.92);
+  --md-default-fg-color--light: hsla(48, 12%, 9%, 0.62);
+  --md-default-fg-color--lighter: hsla(48, 12%, 9%, 0.34);
+  --md-default-fg-color--lightest: hsla(48, 12%, 9%, 0.1);
+
+  --md-typeset-a-color: var(--anthropic-clay-dark);
+  --md-code-bg-color: var(--anthropic-cloud);
+  --md-code-fg-color: #4a3a2e;
+  --md-footer-bg-color: #211f1a;
+  --md-footer-bg-color--dark: #1a1915;
+}
+
+/* ----------------------------------------------------------------------------
+ * Dark scheme (slate) — warmed toward Anthropic's charcoal
+ * ------------------------------------------------------------------------- */
+[data-md-color-scheme="slate"] {
+  /* Warm the derived greys away from Material's default blueish slate. */
+  --md-hue: 40;
+
+  --md-primary-fg-color: var(--anthropic-clay);
+  --md-primary-fg-color--light: var(--anthropic-clay-light);
+  --md-primary-fg-color--dark: var(--anthropic-clay-dark);
+  --md-primary-bg-color: #f7f4ee;
+  --md-primary-bg-color--light: hsla(40, 30%, 96%, 0.72);
+
+  --md-accent-fg-color: var(--anthropic-clay-light);
+
+  --md-default-bg-color: var(--anthropic-slate-bg);
+  --md-default-fg-color: hsla(44, 22%, 92%, 0.9);
+  --md-default-fg-color--light: hsla(44, 22%, 92%, 0.58);
+  --md-default-fg-color--lighter: hsla(44, 22%, 92%, 0.32);
+  --md-default-fg-color--lightest: hsla(44, 22%, 92%, 0.12);
+
+  --md-typeset-a-color: var(--anthropic-clay-light);
+  --md-code-bg-color: var(--anthropic-slate-surface);
+  --md-code-fg-color: #e8e2d6;
+  --md-footer-bg-color: #1c1b18;
+  --md-footer-bg-color--dark: #161512;
+}
+
+/* ----------------------------------------------------------------------------
+ * Typography — editorial serif headings, refined site title
+ * ------------------------------------------------------------------------- */
+.md-typeset h1,
+.md-typeset h2,
+.md-typeset h3,
+.md-header__topic > .md-ellipsis {
+  font-family: "Fraunces", Georgia, "Times New Roman", serif;
+  font-weight: 600;
+  letter-spacing: -0.015em;
+}
+
+.md-typeset h1 {
+  font-weight: 600;
+  color: var(--md-default-fg-color);
+}
+
+/* ----------------------------------------------------------------------------
+ * Header & tabs — subtle depth and a hairline divider
+ * ------------------------------------------------------------------------- */
+.md-header {
+  box-shadow: 0 1px 0 hsla(40, 12%, 50%, 0.16);
+}
+
+.md-header--shadow {
+  box-shadow: 0 2px 12px hsla(40, 20%, 10%, 0.16);
+}
+
+/* ----------------------------------------------------------------------------
+ * Links, code, admonitions — small clay accents
+ * ------------------------------------------------------------------------- */
+.md-typeset a {
+  text-underline-offset: 0.15em;
+}
+
+.md-typeset code {
+  border-radius: 0.3rem;
+}
+
+.md-typeset pre > code {
+  border-radius: 0.4rem;
+}
+
+/* Active nav item gets the clay accent bar. */
+.md-nav__link--active,
+.md-nav__item .md-nav__link--active {
+  color: var(--md-typeset-a-color);
+  font-weight: 600;
+}
+
+/* Search input rounded to match the soft brand feel. */
+.md-search__form {
+  border-radius: 0.4rem;
+}

docs/gen_pages.py

@@ -0,0 +1,144 @@
+"""Generate the doc-source tree from the repository (standalone pre-build step).
+
+Zensical doesn't run MkDocs plugins (no ``mkdocs-gen-files`` / ``literate-nav``),
+so instead of synthesizing virtual pages we write **real** Markdown files into
+``content/`` before ``zensical build`` runs. Zensical then infers the navigation
+from the directory structure, so the site still mirrors the codebase on every run.
+
+Pure standard library — run it with any Python ≥3.9:
+
+    python docs/gen_pages.py
+
+Sources:
+  1. ``backend/**/*.py``    -> ``content/reference/...``  (``::: module`` for mkdocstrings)
+  2. repo Markdown          -> ``content/guides/...``     (READMEs + loose docs, verbatim)
+  3. ``frontend/.typedoc``  -> ``content/frontend/...``   (TypeDoc output, if present)
+
+The three generated directories are wiped and rebuilt each run; ``content/index.md``
+(the hand-written landing page) is left untouched.
+"""
+
+from __future__ import annotations
+
+import shutil
+from pathlib import Path
+
+HERE = Path(__file__).resolve().parent
+REPO_ROOT = HERE.parent
+DOCS_DIR = HERE / "content"
+
+REFERENCE_DIR = DOCS_DIR / "reference"
+GUIDES_DIR = DOCS_DIR / "guides"
+FRONTEND_DIR = DOCS_DIR / "frontend"
+
+# Directory names we never walk into when collecting Markdown.
+SKIP_DIRS = {
+    ".git", ".venv", "venv", "node_modules", "__pycache__", "site",
+    ".pytest_cache", ".mypy_cache", "dist", "build", ".typedoc",
+}
+
+
+def _skipped(path: Path) -> bool:
+    return any(part in SKIP_DIRS for part in path.parts)
+
+
+def _write(dest: Path, text: str) -> None:
+    dest.parent.mkdir(parents=True, exist_ok=True)
+    dest.write_text(text, encoding="utf-8")
+
+
+def clean() -> None:
+    for d in (REFERENCE_DIR, GUIDES_DIR, FRONTEND_DIR):
+        if d.exists():
+            shutil.rmtree(d)
+
+
+# --- 1. Backend Python API -> mkdocstrings pages --------------------------
+
+def gen_backend_reference() -> int:
+    backend = REPO_ROOT / "backend"
+    if not backend.is_dir():
+        return 0
+    count = 0
+    for py in sorted(backend.rglob("*.py")):
+        if _skipped(py):
+            continue
+        parts = list(py.relative_to(REPO_ROOT).with_suffix("").parts)  # backend, apps, ...
+        is_package = parts[-1] == "__init__"
+        if is_package:
+            parts = parts[:-1]
+        module = ".".join(parts)                     # e.g. backend.apps.export.snapshot
+        rel = parts[1:]                              # drop the leading "backend" for a flatter nav
+
+        if is_package:
+            # Package → section landing page (works with navigation.indexes).
+            dest = REFERENCE_DIR.joinpath(*rel, "index.md") if rel else REFERENCE_DIR / "index.md"
+        else:
+            dest = REFERENCE_DIR.joinpath(*rel).with_suffix(".md")
+
+        # No hand-written H1: mkdocstrings renders the heading, and the nav label
+        # is derived from the file/dir name (clean, short labels).
+        _write(dest, f"::: {module}\n")
+        count += 1
+    return count
+
+
+# --- 2. Repo Markdown (READMEs + loose docs) ------------------------------
+
+def gen_guides() -> int:
+    curated = [
+        REPO_ROOT / "README.md",
+        REPO_ROOT / "implementation_plan.md",
+        REPO_ROOT / "relevant_context.md",
+        REPO_ROOT / "frontend" / "DESIGN.md",
+    ]
+    nested_readmes = sorted(REPO_ROOT.rglob("README.md"))
+
+    seen: set[Path] = set()
+    count = 0
+    for md in [*curated, *nested_readmes]:
+        if not md.is_file() or md in seen or _skipped(md):
+            continue
+        if DOCS_DIR in md.parents:       # never re-ingest our own generated tree
+            continue
+        seen.add(md)
+
+        rel = md.relative_to(REPO_ROOT)
+        # Map the project root README to the Guides landing page.
+        if rel == Path("README.md"):
+            dest = GUIDES_DIR / "index.md"
+        else:
+            dest = GUIDES_DIR / rel
+        _write(dest, md.read_text(encoding="utf-8"))
+        count += 1
+    return count
+
+
+# --- 3. Frontend TypeDoc output -------------------------------------------
+
+def gen_frontend() -> int:
+    typedoc = REPO_ROOT / "frontend" / ".typedoc"
+    if not typedoc.is_dir():
+        return 0
+    count = 0
+    for md in sorted(typedoc.rglob("*.md")):
+        rel = md.relative_to(typedoc)
+        if any(p in SKIP_DIRS for p in rel.parts):
+            continue
+        _write(FRONTEND_DIR / rel, md.read_text(encoding="utf-8"))
+        count += 1
+    return count
+
+
+def main() -> int:
+    clean()
+    n_api = gen_backend_reference()
+    n_guides = gen_guides()
+    n_front = gen_frontend()
+    print(f"gen_pages: {n_api} API pages, {n_guides} guides, {n_front} frontend pages "
+          f"written under {DOCS_DIR}")
+    return 0
+
+
+if __name__ == "__main__":
+    raise SystemExit(main())

docs/overrides/partials/header.html

@@ -0,0 +1,77 @@
+{#-
+  Local override of Zensical's partials/header.html.
+
+  The only change from the stock template is element order: the search block is
+  emitted *before* the palette (theme toggle) block, so the toggle renders to the
+  right of the search bar. Everything else mirrors the packaged partial — keep it
+  in sync if you upgrade Zensical (see
+  docs/.venv/.../zensical/templates/partials/header.html).
+-#}
+{% set class = "md-header" %}
+{% if "navigation.tabs.sticky" in features %}
+  {% set class = class ~ " md-header--shadow md-header--lifted" %}
+{% elif "navigation.tabs" not in features %}
+  {% set class = class ~ " md-header--shadow" %}
+{% endif %}
+<header class="{{ class }}" data-md-component="header">
+  <nav class="md-header__inner md-grid" aria-label="{{ lang.t('header') }}">
+    <a href="{{ config.extra.homepage | d(nav.homepage.url, true) | url }}" title="{{ config.site_name | e }}" class="md-header__button md-logo" aria-label="{{ config.site_name }}" data-md-component="logo">
+      {% include "partials/logo.html" %}
+    </a>
+    <label class="md-header__button md-icon" for="__drawer" aria-label="{{ lang.t('nav') }}">
+      {% set icon = config.theme.icon.menu or "material/menu" %}
+      {% include ".icons/" ~ icon ~ ".svg" %}
+    </label>
+    <div class="md-header__title" data-md-component="header-title">
+      <div class="md-header__ellipsis">
+        <div class="md-header__topic">
+          <span class="md-ellipsis">
+            {{ config.site_name }}
+          </span>
+        </div>
+        <div class="md-header__topic" data-md-component="header-topic">
+          <span class="md-ellipsis">
+            {% if page.meta and page.meta.title %}
+              {{ page.meta.title }}
+            {% else %}
+              {{ page.title }}
+            {% endif %}
+          </span>
+        </div>
+      </div>
+    </div>
+    {# Search first … #}
+    {% if "search" in config.plugins %}
+      {% set search = config.plugins["search"] | attr("config") %}
+      {% if search.enabled %}
+        <label class="md-header__button md-icon" for="__search" aria-label="{{ lang.t('search') }}">
+          {% set icon = config.theme.icon.search or "material/magnify" %}
+          {% include ".icons/" ~ icon ~ ".svg" %}
+        </label>
+        {% include "partials/search.html" %}
+      {% endif %}
+    {% endif %}
+    {# … then the theme toggle, so it lands to the right of the search bar. #}
+    {% if config.theme.palette %}
+      {% if not config.theme.palette is mapping %}
+        {% include "partials/palette.html" %}
+      {% endif %}
+    {% endif %}
+    {% if not config.theme.palette is mapping %}
+      {% include "partials/javascripts/palette.html" %}
+    {% endif %}
+    {% if config.extra.alternate %}
+      {% include "partials/alternate.html" %}
+    {% endif %}
+    <div class="md-header__source">
+      {% if config.repo_url %}
+        {% include "partials/source.html" %}
+      {% endif %}
+    </div>
+  </nav>
+  {% if "navigation.tabs.sticky" in features %}
+    {% if "navigation.tabs" in features %}
+      {% include "partials/tabs.html" %}
+    {% endif %}
+  {% endif %}
+</header>

docs/requirements.txt

@@ -0,0 +1,6 @@
+# Doc-site dependencies. Installed into docs/.venv by run.sh.
+# These never touch backend/.venv — docs tooling stays fully isolated.
+zensical                 # static site generator (Material for MkDocs team)
+mkdocstrings              # API-reference engine (preliminary Zensical support)
+mkdocstrings-python       # the Python handler (Griffe-based, ships separately)
+black                     # lets mkdocstrings pretty-format function signatures

docs/run.sh

@@ -0,0 +1,81 @@
+#!/usr/bin/env bash
+#
+# Build the documentation site and open it. Drop-in, local-only — same spirit as
+# dependency-graph/generate.sh. From the repo root (or anywhere), run:
+#
+#     ./docs/run.sh           # live-reloading dev server (default)
+#     ./docs/run.sh --build   # one-time build + open instead
+#
+# Built with Zensical (the Material for MkDocs team's successor to MkDocs).
+# Everything is isolated in docs/.venv.
+set -euo pipefail
+
+HERE="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
+REPO_ROOT="$(dirname "$HERE")"
+cd "$REPO_ROOT"
+
+VENV="$HERE/.venv"
+PY="$VENV/bin/python"
+ZENSICAL="$VENV/bin/zensical"
+
+# 1. Bootstrap an isolated venv for the doc tooling.
+# Zensical/mkdocstrings need Python >= 3.10; pick the newest available.
+pick_python() {
+  for c in python3.13 python3.12 python3.11 python3.10 python3; do
+    local bin
+    bin="$(command -v "$c" 2>/dev/null)" || continue
+    if "$bin" -c 'import sys; raise SystemExit(0 if sys.version_info[:2] >= (3, 10) else 1)' 2>/dev/null; then
+      echo "$bin"; return 0
+    fi
+  done
+  return 1
+}
+
+if [[ ! -x "$PY" ]]; then
+  BOOT_PY="$(pick_python)" || {
+    echo "docs: need Python >= 3.10 on PATH to build the docs" >&2
+    exit 1
+  }
+  echo "docs: creating venv at $VENV (using $BOOT_PY)"
+  "$BOOT_PY" -m venv "$VENV"
+fi
+echo "docs: installing/upgrading doc dependencies"
+"$PY" -m pip install -q --upgrade pip
+"$PY" -m pip install -q -r "$HERE/requirements.txt"
+
+# 2. Generate frontend TypeDoc markdown (best-effort; needs node + network once).
+if command -v npx >/dev/null 2>&1 && [[ -f "$REPO_ROOT/frontend/package.json" ]]; then
+  echo "docs: generating frontend TypeDoc reference"
+  ( cd "$REPO_ROOT/frontend" \
+      && npx -y -p typedoc -p typedoc-plugin-markdown typedoc \
+           --plugin typedoc-plugin-markdown \
+           --entryPointStrategy expand \
+           --readme none \
+           --skipErrorChecking \
+           --out .typedoc \
+           src \
+  ) >/dev/null 2>&1 || echo "docs: TypeDoc step failed/skipped; frontend section omitted" >&2
+else
+  echo "docs: npx/frontend not available; frontend section omitted" >&2
+fi
+
+# 3. Generate the doc-source tree from the codebase (pure stdlib; writes real files).
+echo "docs: generating pages from the codebase"
+"$PY" "$HERE/gen_pages.py"
+
+# 4. Build (or serve) the site. Run from the config dir so relative paths resolve.
+if [[ "${1:-}" == "--build" ]]; then
+  echo "docs: building site"
+  ( cd "$HERE" && "$ZENSICAL" build )
+
+  INDEX="$HERE/site/index.html"
+  echo "docs: built $INDEX"
+  if   command -v open     >/dev/null 2>&1; then open     "$INDEX" || true
+  elif command -v xdg-open >/dev/null 2>&1; then xdg-open "$INDEX" || true
+  else echo "docs: open $INDEX in a browser to view the site"
+  fi
+  exit 0
+fi
+
+# Default: live-reloading dev server.
+exec sh -c "cd '$HERE' && exec '$ZENSICAL' serve -o"

docs/zensical.toml

@@ -0,0 +1,111 @@
+# Zensical configuration. Build/preview via ./docs/run.sh, or directly:
+#   ( cd docs && .venv/bin/zensical build )
+#   ( cd docs && .venv/bin/zensical serve -o )
+#
+# Navigation is INFERRED from the directory tree under content/ (Zensical doesn't
+# run the mkdocs-literate-nav plugin). gen_pages.py writes that tree on each run,
+# so the nav mirrors the codebase automatically — top-level dirs become sections:
+#   content/reference/  -> "reference" (backend API, from docstrings via mkdocstrings)
+#   content/guides/     -> "guides"    (every README + loose Markdown doc)
+#   content/frontend/   -> "frontend"  (TypeDoc reference, when present)
+
+[project]
+site_name = "Open Swarm Analytics — Docs"
+site_description = "Auto-generated docs sourced from docstrings, READMEs, and the frontend."
+docs_dir = "content"
+site_dir = "site"
+use_directory_urls = false
+
+# Open Swarm - inspired theming (clay accent, ivory/charcoal canvases, serif
+# headings). The actual colors live in content/stylesheets/openswarm.css, keyed off
+# the "custom" palette entries below.
+extra_css = ["stylesheets/openswarm.css"]
+
+# Seed the search modal with a few "suggested pages" while the query is empty, so
+# it isn't blank on open (command-palette style). See the script header for how it
+# slots into the modal's shadow DOM.
+extra_javascript = ["javascripts/search-suggestions.js"]
+
+# Drop the "Made with Zensical" attribution from the footer (copyright.html
+# skips the generator block when this is false).
+[project.extra]
+generator = false
+
+[project.theme]
+# Template overrides (resolved relative to this file → docs/overrides/).
+# Currently just reorders the header so the theme toggle sits to the right of
+# the search bar; see overrides/partials/header.html.
+custom_dir = "overrides"
+# Brand logo (header, top-left) and browser-tab favicon. Both resolve relative
+# to docs_dir → content/assets/logo.png (copied in, not wiped by gen_pages.py).
+logo = "assets/logo.png"
+favicon = "assets/logo.png"
+features = [
+    "navigation.sections",
+    "navigation.indexes",
+    "navigation.top",
+    "navigation.tracking",
+    "navigation.footer",
+    "toc.follow",
+    "content.code.copy",
+    "search.suggest",
+    "search.highlight",
+]
+
+# Light/dark palettes with a toggle button in the header. Each entry follows the
+# system preference by default (`media`) and can be flipped manually via the
+# toggle, which persists the choice. `primary`/`accent` are "custom" so the clay
+# brand colors from openswarm.css take over.
+[[project.theme.palette]]
+media = "(prefers-color-scheme: light)"
+scheme = "default"
+primary = "custom"
+accent = "custom"
+[project.theme.palette.toggle]
+icon = "material/brightness-7"
+name = "Switch to dark mode"
+
+[[project.theme.palette]]
+media = "(prefers-color-scheme: dark)"
+scheme = "slate"
+primary = "custom"
+accent = "custom"
+[project.theme.palette.toggle]
+icon = "material/brightness-4"
+name = "Switch to light mode"
+
+# --- Markdown extensions (each table = one enabled extension) ----------------
+[project.markdown_extensions.admonition]
+[project.markdown_extensions.attr_list]
+[project.markdown_extensions.md_in_html]
+[project.markdown_extensions.tables]
+
+[project.markdown_extensions.toc]
+permalink = true
+
+[project.markdown_extensions.pymdownx.highlight]
+anchor_linenums = true
+
+[project.markdown_extensions.pymdownx.superfences]
+[project.markdown_extensions.pymdownx.inlinehilite]
+[project.markdown_extensions.pymdownx.details]
+
+# --- API reference from docstrings (mkdocstrings, preliminary in Zensical) ----
+# The Python handler ships separately (mkdocstrings-python, see requirements.txt).
+[project.plugins.mkdocstrings.handlers.python]
+# Relative to this config file → repo root, so `import backend` resolves under
+# Griffe's static analysis. (External paths aren't watched for live reload yet.)
+paths = [".."]
+inventories = ["https://docs.python.org/3/objects.inv"]
+
+[project.plugins.mkdocstrings.handlers.python.options]
+docstring_style = "google"
+show_source = true
+show_root_heading = true
+show_root_full_path = true
+show_if_no_docstring = true
+members_order = "source"
+separate_signature = true
+show_signature_annotations = true
+merge_init_into_class = true
+filters = ["!^_[^_]"]

linter/checks/ruff.py

@@ -56,7 +56,7 @@ def run_ruff(
         "--select", select,
         "--output-format", "concise",
         "--no-fix",
-        "--exclude", ".venv,__pycache__,data,uv-bin,webapp_template",
+        "--exclude", ".venv,__pycache__,data,uv-bin,webapp_template,.runner-venv",
     ]
 
     try:

linter/checks/vulture.py

@@ -130,7 +130,7 @@ def run_vulture(
         cmd.append(str(whitelist))
     cmd.extend([
         "--min-confidence", str(min_confidence),
-        "--exclude", ".venv,__pycache__,data,uv-bin",
+        "--exclude", ".venv,.runner-venv,__pycache__,data,uv-bin",
         "--ignore-decorators", "@*.router.*,@*.websocket,@app.*,@pytest.fixture,@pytest.fixture*",
         "--ignore-names", "cls",
     ])