openswarm/backend/apps/service/analytics/ANALYTICS_OVERVIEW.md

Analytics Overview (swarm-analytics SDK)

This document explains how the OpenSwarm product-analytics system works and how to use the swarm-analytics Python SDK to send analytics from the desktop app. It is written for an engineer/agent integrating the SDK into a separate codebase.

---

1. The big picture

• There is a standalone analytics ingest service (a FastAPI app, the product-analytics-v1 repo). It exposes a small set of typed POST endpoints under /public/* — one per event category.
• The desktop app's Python backend is the single network egress for analytics. The React frontend never talks to the analytics service directly; if the UI needs to record something it hands it to the local backend, which forwards it. (This doc only covers the backend SDK.)
• The backend talks to the service through the swarm-analytics pip package — a typed client that is auto-generated from the service's own pydantic models, so the client validates payloads against the exact schema the server enforces. If a call would be rejected by the server for being the wrong shape, it fails locally first, as a pydantic.ValidationError, before any network I/O.

Why it's "impossible to call wrong"

• Identity is never passed by the caller. No method takes install_id or user_id. The server resolves identity from the bearer token on every request. There is no way to spoof or forget it.
• Per-request metadata is auto-filled. ts (client timestamp) and submission_id (idempotency UUID) never appear in any method signature — the transport stamps them automatically.
• Enums are Literals. Fields like action and status only accept their allowed values; a typo raises immediately.
• Models are vendored verbatim from the service, so client and server can't drift (a generator + drift check guard this).

---

2. How a call flows (sync validate, async deliver)

The public API is fully synchronous and fire-and-forget:

1. You call e.g. client.logs.write(tag="app", subtag="started").
2. On the calling thread, the payload is validated against the pydantic model. Bad input raises pydantic.ValidationError here, in your stack.
3. A serialized record is handed to a background worker thread which does the actual HTTP POST, with retries and exponential backoff.
4. The call returns immediately. It never blocks on the network and (after validation) never raises for delivery problems.

Idempotency: submission_id is minted once at enqueue time and reused on every retry (including replays from a durable spool after a restart). The server dedups on (install_id, submission_id), so retries are no-ops, never double-writes.

Retry policy (handled for you):

• 2xx → success.
• 429 and 5xx → retried with backoff (up to max_attempts, default 8).
• other 4xx → permanent (bad data); dropped, not retried.
• network/timeout errors → retried.

---

3. Install

pip install swarm-analytics

(Or pip install ./sdk from the analytics repo root for a local build.)

---

4. Bootstrap: minting a token (register)

A fresh install has no token. register() is the one unauthenticated, blocking call — it mints an install token from an install_id you own.

from swarm_analytics import AnalyticsClient

token = AnalyticsClient.register(
    base_url="https://analytics.example.com",
    install_id=install_id,        # your app's stable per-install UUID
)
# Persist `token`. Reuse it on every subsequent run — never call register again
# once you have a token.

• It POSTs to /public/identify/create_install_token.
• Raises AuthError on 401, TransportError on other failures (and on network errors). Wrap it if you need to survive being offline on first launch.

---

5. Constructing the client

from swarm_analytics import AnalyticsClient

client = AnalyticsClient(
    base_url="https://analytics.example.com",
    token=token,                 # from register(), persisted
    mode="full",                 # or "minimal" (see opt-out below)
)

Constructor options:

Arg	Default	Meaning
base_url	(required)	Root URL of the analytics service.
token	(required)	Install token from register().
mode	"full"	"minimal" mutes product telemetry (see §7).
spool	None	Optional durable store for crash/offline survival (see §8).
max_attempts	8	Retry cap per record before it's dropped.
on_drop	None	Callback (record, status) when a record is permanently dropped.

The client starts a daemon worker thread on construction. Build one client per process and reuse it (a module-level singleton is ideal).

---

6. The full API surface

Every method is keyword-only and returns None. Identity, ts, and submission_id are intentionally absent — they're handled for you.

Logs (diagnostics)

client.logs.write(tag="agent", subtag="tool", data={"name": "shell"})
client.logs.write(tag="app", subtag="backend_started", data={"app_version": "1.2.0"})

• tag: str (required), subtag: str | None = None, data: Any = None (any JSON-serializable value — stored as opaque JSON server-side).

Product events

# App lifecycle
client.events.app_lifecycle.opened(os="darwin", os_version="25.3.0",
                                    app_version="1.2.0",
                                    timezone="America/Los_Angeles", locale="en-US")
client.events.app_lifecycle.closed()

# Agent sessions
client.events.agent.create(id="sess_123", name="Refactor auth", dashboard_id="dash_1")
client.events.agent.message(agent_id="sess_123", seq=0,
                            message=AgentMessage(id="m1", role="user", content="hello"))

# Dashboards
client.events.dashboard.event(dashboard_id="dash_1", action="create")  # open|close|create|delete

# Onboarding
client.events.onboarding.step(step_id="connect_provider", status="completed")  # started|completed|abandoned

AgentMessage is importable from the package:

from swarm_analytics import AgentMessage

Identity

client.identify.link_email(email="user@example.com")

Links an email to the current install (resolved from the token). Use it once the user provides an email; do not pass any id.

---

7. Categories and opt-out (mode)

Every endpoint has a category. mode="minimal" mutes only the product category; everything else still flows:

Category	Endpoints	Flows in minimal?
product	all client.events.*	No (muted)
diagnostic	client.logs.write	Yes
identity	client.identify.link_email	Yes
bootstrap	register()	Yes

So a log write always flows, even when the user opted out of product telemetry. Map your app's existing opt-out toggle onto mode: opted-out → "minimal", otherwise "full".

---

8. Durability (optional spool)

By default, in-flight records live in an in-memory queue and are lost if the process dies with deliveries pending. Pass a spool to persist them to disk and replay on next launch (with the same submission_id, so dedup still holds):

from swarm_analytics import SqliteSpool

client = AnalyticsClient(base_url=..., token=...,
                         spool=SqliteSpool("/path/to/service_spool.db"))

For the initial integration you can skip this; add it once the basic path works.

---

9. Shutdown

Flush pending records before the process exits so you don't lose the tail:

client.flush(timeout=2.0)   # block until drained, or give up after 2s
client.close()              # stop the worker thread

AnalyticsClient is also a context manager (__exit__ flushes + closes).

---

10. Error model

Where	What you get
Bad call arguments	pydantic.ValidationError, synchronously, on the calling thread.
register() rejected/fail	AuthError (401) or TransportError (other / network).
Delivery failures	Handled internally (retry/drop). Never raised to the caller.

Importable errors:

from swarm_analytics import AnalyticsError, AuthError, RateLimited, TransportError, ValidationRejected

---

11. Quick do / don't

Do

• Create exactly one AnalyticsClient per process and reuse it.
• Call register() once, persist the token, reuse it forever.
• Let the SDK fill ts/submission_id; let the server resolve identity.
• flush() + close() on shutdown.

Don't

• Don't pass install_id, user_id, ts, or submission_id — there's no parameter for them by design.
• Don't construct a new client per event.
• Don't call register() on every launch.
• Don't hand-edit anything under _generated/ (it's regenerated from the service).