Commit Graph
100 Commits
Author SHA1 Message Date
Elior Nataf LackritzandGitHub d569e18f4b fix(checkpoint-postgres): find plain-value seeds when walking delta history (#8535)
Fixes langchain-ai/langgraph#8534

`put` splits stored values in two: primitives stay inline in the
checkpoint's `channel_values`, everything else moves to
`checkpoint_blobs`, and only `_DeltaSnapshot` leaves an inline marker
behind when it moves. Stage-1 seed detection tested for that marker, so
a plain value — what a thread migrated from `BinaryOperatorAggregate`
leaves behind — was invisible to the walk.

### Effect

Migrated threads found no seed, walked to the root, and replayed every
write on every read. Values still came out correct, because replaying an
additive reducer from empty rebuilds the same list, which is why nothing
looked wrong. What was lost is early termination — the entire point of
`DeltaChannel`:

<!-- linear:table-colwidths:266,266,266 -->
| thread length | writes replayed, before | after |
| -- | -- | -- |
| 2 turns | 3 | 1 |
| 6 turns | 7 | 1 |
| 20 turns | 21 | 1 |

Read latency is flat at \~0.6ms across all three after the change.

### Approach

Stage 1 now checks both places a value can live rather than trusting the
marker. It probes `checkpoint_blobs`:

```sql
EXISTS (SELECT 1 FROM checkpoint_blobs b0
        WHERE b0.thread_id     = checkpoints.thread_id
          AND b0.checkpoint_ns = checkpoints.checkpoint_ns
          AND b0.channel       = %s
          AND b0.version       = checkpoint -> 'channel_versions' ->> %s
          AND b0.type         <> 'empty') AS hb_0
```

and selects the inline value alongside it, since `None`, `str`, `int`,
`float` and `bool` stay in `channel_values` with no blob row:

```sql
checkpoint -> 'channel_values' -> %s AS inline_0
```

The blob predicate matches `checkpoint_blobs`' primary key `(thread_id,
checkpoint_ns, channel, version)` exactly, so it is one index lookup per
row per channel, bounded by the 1024-row page.

I picked reading storage over the cheaper alternative — also writing the
marker for plain values — because **that would not fix any thread
already on disk.** Existing checkpoints have no marker and there is
nowhere to add one retroactively.

The seed resolves to the blob when one exists and the inline value
otherwise. That ordering is also what keeps a genuine inline `true` — a
`bool` channel holding `True` — distinguishable from the literal `true`
marker `put` inlines for a `_DeltaSnapshot`: only the snapshot has a
blob.

`None` is deliberately not treated as a seed; a JSON null is
indistinguishable from "nothing stored" at this layer, so the walk
continues and replay from empty is correct.

Params go from two to four per channel; both callers updated.

The inline half came out of review on this PR — a blob-only probe would
have left scalar-aggregate migrations (an integer sum, say) still
replaying their full history.

### On the `type <> 'empty'` predicate

Being upfront since it isn't demonstrable with a test: `put` does not
currently produce `empty` rows on this path — `blob_versions` is
filtered to keys present in `channel_values`, so `_dump_blobs`' empty
branch is unreachable from it. I confirmed there are no `empty` rows in
a populated test database.

I kept it because stage 2 already applies the same check when resolving
the seed blob. Without it the two stages could disagree: stage 1
terminates the walk on a row stage 2 then discards, producing no seed
*and* a truncated write chain — the same failure shape this function
exists to avoid. Rationale is in the docstring so the next reader
doesn't have to ask. Happy to drop it if you'd rather not carry an
unexercised predicate.

### Tests

`libs/checkpoint-postgres/tests/test_delta_plain_value_seed.py` —
blob-stored plain-value seed, `_DeltaSnapshot` seed, a version bump with
nothing stored (which must not stop the walk short of an older real
value), inline primitives (`int`, `str`, `float`, `None`), and inline
`True` versus the snapshot marker. Each fails against the behaviour it
fixes.

Verified: postgres suite 269 passed on PG 15 and 16; delta-channel
conformance against `AsyncPostgresSaver` went from 6 of 8 to 8 of 8,
including the pre-existing `test_history_migration_plain_value_as_seed`
failure this was causing; `make lint` clean.

### Not included

I wanted a Postgres conformance runner alongside `checkpoint-sqlite`'s,
but it needs `langgraph-checkpoint-conformance` as a dev dependency and
the contributing guide asks for maintainer sign-off before adding one.
The direct tests above cover the same ground without it.

Worth flagging separately: **conformance effectively runs against**
`InMemorySaver` **only today.** `libs/checkpoint-conformance/tests/`
contains just `test_validate_memory.py`, and `checkpoint-sqlite`'s
`test_conformance_delta.py` silently skips because the package isn't
installed in its test environment (`importorskip`). Wiring it up for
sqlite and postgres is what would have caught this bug, and
langchain-ai/langgraph#8534 notes it.

Sqlite is unaffected by the bug itself — it stores `channel_values`
inline and inspects them directly. `langgraph-api` already resolves
seeds by version rather than by marker.
2026-08-07 09:07:04 -04:00
Elior Nataf LackritzandGitHub f22af6248c chore: enable RUF100 and clear unused noqa directives (#8546)
Follow-up to review on #8540, where a stale `# noqa: E402` slipped past
me and Sydney spotted it by eye. This turns on the rule that catches
that automatically.

`RUF100` flags a `noqa` that suppresses nothing. `sdk-py` already had it
through its blanket `RUF` selection; this adds it to the other seven
packages and clears what it finds.

### The 33 it flags, all autofixed

**Blanket `# noqa` on docstring-closing lines** (4, in
`checkpoint-postgres` and `checkpoint-sqlite`). `E501` is in
`lint.ignore` for those packages, so nothing was being suppressed:

```diff
-        """  # noqa
+        """
```

**`# noqa: F821` on `anext(aiter_)`** (2). Left over from Python 3.9
support. `anext` became a builtin in 3.10, which is the floor now, so
`F821` no longer fires:

```diff
-                    anext(aiter_),  # type: ignore[arg-type]  # noqa: F821
+                    anext(aiter_),  # type: ignore[arg-type]
```

**Suppressions naming rules the package does not enable** (27), across
`langgraph`, `prebuilt` and `checkpoint-sqlite`: `FBT001`, `FBT002`,
`TC002`, `BLE001`, `ANN001`, `ANN002`, `ANN003`, `E501`, `F401`. Mostly
copied between packages whose rule sets differ.

### One measurement note

If you check these numbers yourself, use `--extend-select`:

```
ruff check --select RUF100 .          # 81, misleading
ruff check --extend-select RUF100 .   # 33, real
```

With a bare `--select`, ruff treats every other rule as disabled, so
every suppression for another rule looks unused. I quoted 81 before
catching that.

### Verified

`checkpoint-sqlite` 118 passed, `prebuilt` 284 passed, `langgraph` 1968
passed, `checkpoint-postgres` 264 passed on PG 15 and 16. `make lint`
clean in every package.

Independent of #8540 and #8537, so it can land in any order.
2026-08-06 17:38:31 -04:00
Sydney RunkleandGitHub 1a989f22bb fix(checkpoint-postgres): add column aliases to seed-blob branch of delta stage-2 UNION ALL (#7728)
## Summary

- Ports the langgraph API checkpointer fix to the OSS Postgres
checkpoint.
- The `'b'` (seed-blob) branch of `_build_delta_stage2_sql`'s `UNION
ALL` was missing column aliases (`AS _kind`, `AS checkpoint_id`, `AS
task_id`, `AS idx`).
- In PostgreSQL, `UNION ALL` column names are taken from the **first**
`SELECT`. When `channels_with_chain` is empty but `channels_with_seed`
is not (e.g. very first run of a DeltaChannel graph, or
`snapshot_frequency=1`), the `'b'` branch becomes the first `SELECT`, so
columns got default names (`text`, `null`) instead of the expected
aliases — causing `KeyError: '_kind'` in
`_build_delta_channels_writes_history`.

## Test plan

- [ ] Existing `checkpoint-postgres` delta channel tests pass (`make
test` in `libs/checkpoint-postgres`)
- [ ] Manually verified fix matches the correction suggested in the
upstream API PR review comment

🤖 Generated with [Claude Code](https://claude.com/claude-code)
2026-05-06 14:36:30 -04:00
0a53c385b2 feat: public get_writes_history saver API + delta cadence rework (#7699)
## Summary

- Promotes the private K-channel batched ancestor-walk to a stable
public `get_delta_channel_history` / `aget_delta_channel_history` API on
`BaseCheckpointSaver` (returns `Mapping[str, DeltaChannelHistory]`, a
TypedDict with `writes` always present and `seed` `NotRequired`)
- Removes `DELTA_SENTINEL` / `_DeltaSentinel` entirely — the saver layer
is now delta-agnostic on both write and read paths
- Reworks `DeltaChannel` snapshot cadence from "every Nth superstep" to
"every N updates to this channel," persisted in
`CheckpointMetadata.delta_updates_since_snapshot`
- Adds Postgres optimizations: paged stage-1 with cursor (1024-row
pages) and per-channel UNION ALL stage-2 (no over-fetch when channels
have different chain depths)
- Default `snapshot_frequency` becomes a positive int (default `1000`);
the previous `None` opt-out is removed

## Public API

```python
class DeltaChannelHistory(TypedDict):
    writes: list[PendingWrite]            # always present, possibly empty
    seed: NotRequired[Any]                # absent if walk reached root

def get_delta_channel_history(
    self, *, config: RunnableConfig, channels: Sequence[str]
) -> Mapping[str, DeltaChannelHistory]: ...

async def aget_delta_channel_history(
    self, *, config: RunnableConfig, channels: Sequence[str]
) -> Mapping[str, DeltaChannelHistory]: ...
```

`config` and `channels` are keyword-only so later additions (e.g.
`page_size`) don't shift the positional API.

The TypedDict-with-`NotRequired[seed]` shape matches the existing
checkpoint-package convention (`CheckpointMetadata` is
`TypedDict(total=False)`) — absence-via-key-omission rather than
introducing a new sentinel. Pregel translates `"seed" not in hist` to
`MISSING` on its side at consume time.

The default impl walks `get_tuple` + `parent_config` correctly but is
slow on long chains; savers that care override (`InMemorySaver`,
`PostgresSaver`).

## Sentinel removal

`DELTA_SENTINEL` and `_DeltaSentinel` are deleted entirely. The saver
layer becomes delta-agnostic:

- `DeltaChannel.checkpoint()` returns `MISSING` for non-snapshot steps;
pregel's `create_checkpoint` skips MISSING so delta channels without a
snapshot simply don't appear in `channel_values`
- `InMemorySaver.put` and Postgres `put` no longer filter sentinels
(they have nothing to filter)
- `_needs_replay` becomes `stored is MISSING`
- `DeltaChannel.from_checkpoint` accepts: `MISSING` → empty,
`_DeltaSnapshot(value)` → snapshot value, plain value → pre-migration
legacy

## Snapshot cadence

`DeltaChannel.snapshot_frequency: int` (default `1000`, positive). The
previous `None` opt-out is gone.

```python
def should_snapshot(ch_name, ch):
    if force_delta_snapshot:                                  # durability="exit"
        return True
    return updates_since_snapshot.get(ch_name, 0) >= ch.snapshot_frequency
```

Per-channel update counters are persisted in
`CheckpointMetadata.delta_updates_since_snapshot` (`NotRequired`,
`total=False`). The counter is incremented by `_put_checkpoint` for any
delta channel in `updated_channels` and reset to `0` by
`create_checkpoint` for channels that fire a snapshot this step.
Version-format-independent — works for `int`, `float`, and `str`
versioning schemes alike.

## Postgres optimization

Two improvements internal to the override:

**Stage-1 paged with cursor** (`LIMIT 1024` internal const, `AND
checkpoint_id < ?` for subsequent pages). The previous unpaged form
scanned every checkpoint in `(thread_id, ns)` and was pathological at
high thread depths.

**Stage-2 per-channel UNION ALL**: one `WHERE channel='X' AND
checkpoint_id = ANY(chain_X)` branch per channel plus one seed-blob
branch per channel with a seed. The previous form filtered by `channel =
ANY(channels) AND checkpoint_id = ANY(union_chain_cids)`, over-fetching
writes when channels had different chain depths (`K ×
max(chain_lengths)` vs the correct `sum(chain_lengths)`).

Both improvements stay internal to `PostgresSaver`/`AsyncPostgresSaver`;
the public contract returns a single `Mapping`.

## Benchmarks

`libs/langgraph/tests/test_delta_channel_benchmark.py`. Run via `python
libs/langgraph/tests/test_delta_channel_benchmark.py`. Postgres against
local pg:5441.

Results below trimmed to the high-signal cells. Sub-millisecond /
sub-100-turn rows omitted as warmup-bound; freq=1 omitted (chain depth =
1, nothing to optimize); peak read-time memory and Postgres storage are
flat between branches and omitted. Deep-thread reads and the
cadence-rework storage win are the load-bearing numbers.

### Postgres reads, 500 turns

| Scenario | main | branch | Δ |
|---|---:|---:|---:|
| Single-channel deep read | 17.7 ms | **6.1 ms** | **-66%** |
| Single-channel, 1000 turns | 35.0 ms | **14.3 ms** | **-59%** |
| K=3 channels, freq=50 uniform | 70.5 ms | **41.4 ms** | **-41%** |
| K=8 channels, freq=50 uniform | 214.2 ms | **139.4 ms** | **-35%** |
| K=8 channels, mixed freq (25/50/100/.../1000) | 295.6 ms | **214.4
ms** | **-27%** |

K-channel batching + paged stage-1 + per-channel UNION ALL stage-2 doing
exactly what they should at depth.

### InMemory reads, 500 turns

| Scenario | main | branch | Δ |
|---|---:|---:|---:|
| Single-channel deep read | 7.9 ms | **3.8 ms** | **-52%** |
| Single-channel, 1000 turns | 15.6 ms | **7.2 ms** | **-54%** |
| K=8 channels, freq=50 uniform | 112.3 ms | 94.6 ms | -16% |
| K=8 channels, mixed freq | 184.9 ms | **134.5 ms** | **-27%** |

### InMemory storage, 500 turns (cadence-rework win)

| Scenario | main | branch | Δ |
|---|---:|---:|---:|
| K=3, freq=50 uniform | 8.7 MB | **3.3 MB** | **-62%** |
| K=3 mixed freq | 3.8 MB | **1.3 MB** | **-66%** |
| K=8, freq=50 uniform | 23.1 MB | **8.7 MB** | **-62%** |
| K=8 mixed freq | 11.5 MB | **4.2 MB** | **-64%** |

Snapshot frequency now counts **channel updates** instead of
**supersteps**. On graphs where supersteps outpace per-channel updates
(e.g., input/end steps that don't write to channels), branch stores ~3×
fewer snapshot blobs.

### Tradeoff worth flagging

InMemory K=3 with mixed frequencies (50/200/1000) at 500 turns: **+64%
read latency** (46.6 → 76.5 ms). The mixed scenario has a channel with
`freq=1000` that goes the entire 500-turn run with no snapshot. On main,
the old superstep-counted cadence happened to fire at step=500 anyway.
New cadence gives users explicit control over walk depth via
`snapshot_frequency`. The K=8 mixed case still wins overall (-27%); this
regression is specific to the K=3 mixed shape.

Default `snapshot_frequency=1000` is the upper bound on walk depth —
it's a tunable knob.

## Tests

- New sqlite smoke test (`test_get_delta_channel_history.py`) exercises
the inherited default `BaseCheckpointSaver` impl via `SqliteSaver` /
`AsyncSqliteSaver` end-to-end with a real `DeltaChannel`-backed graph.
Sqlite uses the default unchanged — this validates the default path
actually works on a real second saver, not just on the optimized
override.
- Module-level `pytest.importorskip("langgraph.channels.delta")` guards
the test for sqlite's standalone CI environment (matches the postgres
pattern).

## Test plan

- [x] `libs/checkpoint`: 150 passed, 16 skipped
- [x] `libs/langgraph` (channels + delta migration): 41/41 (post-merge)
- [x] `libs/langgraph` (full pregel suite): 1784 passing — 6 "failures"
verified via `env -i` clean shell are local LangSmith env vars + `git
describe revision_id` polluting LangChain metadata fixtures; CI is
unaffected
- [x] `libs/checkpoint-postgres`: 40/40 saver tests + 3/3 delta channel
reconstruction tests against local Postgres
- [x] `libs/checkpoint-sqlite`: 105/105 (incl. retry-passed flake
`test_ttl_refresh`, unrelated to this PR)
- [x] Lint clean across all four libs (`ruff format`, `ruff check`,
`mypy`)
- [x] Branch-vs-main benchmarks — see results above

---------

Co-authored-by: Quanzheng Long <long@langchain.dev>
Co-authored-by: Cursor <cursoragent@cursor.com>
2026-05-04 15:18:43 -04:00
d8b7800183 chore(langgraph): use two phase read to avoid unnecessary data transport (#7660)
## Summary

Replaces the single-roundtrip `UNION ALL` DeltaChannel read with a
two-stage query that avoids fetching unused snapshot blobs, then removes
the old combined path entirely.

### Problem

`_get_channel_writes_history` used a single `UNION ALL` query that
fetched **all** checkpoint metadata, writes, and blobs for a
`(thread_id, channel)` in one shot. With `snapshot_frequency=N`, this
pulled back O(N/freq) full-size snapshot blobs even though only the
nearest one is needed to seed reconstruction. At 500 turns with
`snapshot_frequency=10`, this meant fetching ~100 complete
message-history snapshots per read.

### Solution

Two-stage read:
- **Stage 1** — lightweight scan of `checkpoints` only (no blob bytes):
walks the parent chain from the target checkpoint and stops at the first
ancestor with a snapshot, returning `chain_cids` and `seed_version`
- **Stage 2** — targeted fetch: only the writes for `chain_cids` and the
single seed blob at `seed_version`

The two-stage path is now unconditional — the old combined query and
`LG_DELTA_TWO_STAGE_QUERY` env-var gate have been removed.

### Sentinel cleanup

`DELTA_SENTINEL` is now a pure in-memory signal and is never written to
storage:
- Postgres `put()` already stripped it from `channel_values` before
writing blobs
- Memory saver `put()` now stores `"empty"` instead of serializing the
sentinel
- `EXT_DELTA_SENTINEL` (msgpack ext code 8) removed from
`JsonPlusSerializer`
- `DELTA_SENTINEL` is kept as an in-memory marker:
`DeltaChannel.checkpoint()` returns it so savers know to skip it, and
`_ChannelWritesHistory.seed` uses it to mean "no snapshot found, start
from empty"

## Performance

Benchmarked at `snapshot_frequency=10` on Postgres (`~100 tok/msg`):

| turns | old combined query | two-stage |
|------:|-------------------:|----------:|
| 50    | 6.0ms              | 2.8ms  (2.1x faster) |
| 100   | 10.1ms             | 5.6ms  (1.8x faster) |
| 500   | **216.1ms**        | 15.3ms (**14x faster**) |

The old query's read time grew super-linearly with turn count because
each read fetched O(N/freq) full snapshot blobs. Two-stage keeps read
depth bounded by `snapshot_frequency` regardless of thread length.

## Test plan

- `make test` in `libs/checkpoint`, `libs/checkpoint-postgres`,
`libs/langgraph`
- Removed `test_delta_sentinel_serde_round_trip` (sentinel no longer
serializable)
- Updated `test_memory.py` — delta channel blobs stored as `"empty"`,
not serialized sentinel
- Updated `test_channels.py` — `channel_values` no longer contains
sentinel key for DeltaChannels
- Deleted `test_delta_channel_two_stage_benchmark.py` (one-stage vs
two-stage comparison; path no longer exists)

---------

Co-authored-by: Sydney Runkle <54324534+sydney-runkle@users.noreply.github.com>
Co-authored-by: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
2026-05-01 11:06:54 -04:00
5c18bde0f8 feat(langgraph): DeltaChannel: store sentinel in blobs, reconstruct from checkpoint_writes (#7586)
# DeltaChannel: sentinel-based checkpoint blobs + write-replay
reconstruction

## Summary

`DeltaChannel` is a new fold-reducer channel that stores only a
zero-byte sentinel in checkpoint blobs instead of the full accumulated
value. On restore, the runtime replays ancestor writes through the
reducer to reconstruct state. For long-running threads with large
accumulating state (e.g. message histories), this delivers dramatically
smaller checkpoint blobs with configurable read-depth bounds.

```python
from typing import Annotated
from typing_extensions import TypedDict
from langgraph.channels.delta import DeltaChannel
from langgraph.graph.message import _messages_delta_reducer

class State(TypedDict):
    # blob per step: ~60 bytes (sentinel) instead of growing full list
    messages: Annotated[list, DeltaChannel(_messages_delta_reducer)]
    # bound read depth to 10 steps via periodic snapshots
    messages_bounded: Annotated[list, DeltaChannel(_messages_delta_reducer, snapshot_frequency=10)]
```

---

## Storage benchmarks (InMemory, ~400 char/msg)

**Messages blob storage** (`checkpoint_blobs` bytes for the messages
channel):

| turns | add\_messages | delta(inf) | delta(freq=50) | delta(freq=10) |
delta(freq=5) |

|------:|-------------:|-----------:|---------------:|---------------:|--------------:|
| 10 | 91.0 KB | 60 B (1517x) | 60 B (1517x) | 14.4 KB (6x) | 32.6 KB
(3x) |
| 50 | 2.20 MB | 300 B (7347x) | 67.1 KB (33x) | 423 KB (5x) | 864 KB
(3x) |
| 100 | 8.78 MB | 600 B (14636x) | 310 KB (28x) | 1.72 MB (5x) | 3.48 MB
(3x) |
| 250 | 54.80 MB | 1.5 KB (36536x) | 2.09 MB (26x) | 10.87 MB (5x) |
21.84 MB (3x) |
| 500 | 219.19 MB | 3.0 KB (73063x) | 8.56 MB (26x) | 43.67 MB (5x) |
87.50 MB (3x) |

**Total checkpoint storage** (blobs + writes + metadata):

| turns | add\_messages | delta(inf) | delta(freq=50) | delta(freq=10) |
delta(freq=5) |

|------:|-------------:|-----------:|---------------:|---------------:|--------------:|
| 10 | 129.7 KB | 38.7 KB (3.4x) | 38.7 KB (3.4x) | 53.1 KB (2.4x) |
71.2 KB (1.8x) |
| 50 | 2.40 MB | 196 KB (12x) | 263 KB (9x) | 620 KB (3.9x) | 1.06 MB
(2.3x) |
| 100 | 9.18 MB | 394 KB (23x) | 703 KB (13x) | 2.12 MB (4.3x) | 3.87 MB
(2.4x) |
| 250 | 55.79 MB | 987 KB (57x) | 3.07 MB (18x) | 11.86 MB (4.7x) |
22.82 MB (2.4x) |
| 500 | 221.16 MB | 1.98 MB (112x) | 10.53 MB (21x) | 45.64 MB (4.9x) |
89.48 MB (2.5x) |

**Write-phase peak heap**:

| turns | add\_messages | delta(inf) | delta(freq=50) | delta(freq=10) |
delta(freq=5) |

|------:|-------------:|-----------:|---------------:|---------------:|--------------:|
| 10 | 456 KB | 199 KB (2.3x) | 199 KB (2.3x) | 212 KB (2.2x) | 232 KB
(2.0x) |
| 50 | 3.04 MB | 742 KB (4.1x) | 805 KB (3.8x) | 1.21 MB (2.5x) | 1.67
MB (1.8x) |
| 100 | 10.70 MB | 1.41 MB (7.6x) | 1.82 MB (5.9x) | 3.42 MB (3.1x) |
5.25 MB (2.0x) |
| 250 | 60.44 MB | 3.36 MB (18x) | 5.67 MB (11x) | 14.87 MB (4.1x) |
26.31 MB (2.3x) |

**Read-phase avg `get_state` latency** (5 calls, InMemory):

| turns | add\_messages | delta(inf) | delta(freq=50) | delta(freq=10) |
delta(freq=5) |

|------:|-------------:|-----------:|---------------:|---------------:|--------------:|
| 10 | 0.7 ms | 1.1 ms (0.6x) | 1.1 ms (0.6x) | 0.8 ms (0.9x) | 0.6 ms
(1.1x) |
| 50 | 2.7 ms | 5.3 ms (0.5x) | 3.5 ms (0.8x) | 2.7 ms (1.0x) | 2.7 ms
(1.0x) |
| 100 | 5.5 ms | 11.1 ms (0.5x) | 6.0 ms (0.9x) | 5.2 ms (1.1x) | 5.4 ms
(1.0x) |
| 250 | 12.9 ms | 27.2 ms (0.5x) | 13.6 ms (0.9x) | 12.9 ms (1.0x) |
13.0 ms (1.0x) |

**Postgres `get_tuple` read latency** (~100 tok/msg per step):

| steps | full-list | delta(inf) | delta(freq=50) | delta(freq=10) |
delta(freq=5) |

|------:|----------:|-----------:|---------------:|---------------:|--------------:|
| 10 | 0.29 ms | 0.21 ms (1.4x) | 0.19 ms (1.6x) | 0.19 ms (1.5x) | 0.19
ms (1.6x) |
| 50 | 0.19 ms | 0.15 ms (1.3x) | 0.19 ms (1.0x) | 0.22 ms (0.8x) | 0.29
ms (0.7x) |
| 100 | 0.27 ms | 0.17 ms (1.6x) | 0.22 ms (1.2x) | 0.23 ms (1.2x) |
0.21 ms (1.3x) |
| 500 | 0.60 ms | 0.30 ms (2.0x) | 0.66 ms (0.9x) | 0.56 ms (1.1x) |
0.69 ms (0.9x) |

**Takeaway:** `snapshot_frequency=10` matches full-list read latency
while still saving 5x on blob storage and ~4x on total storage.

---

## How it works

### Checkpoint blobs

`checkpoint()` always returns `DELTA_SENTINEL` (a zero-byte msgpack ext
marker) instead of the accumulated value. On restore, the saver's
`_get_channel_writes_history` walks the ancestor chain collecting
`checkpoint_writes` entries and replays them through the reducer:

```python
# blob stored per step: ~1 byte (sentinel)
# vs. full list growing O(N) every step with BinaryOperatorAggregate
```

### Reducer interface

`DeltaChannel` takes a **batch reducer** `(state, list[writes]) ->
state` — all writes for a step arrive in one call, enabling single-pass
implementations:

```python
#  Don't use add_messages directly — it's a binary operator, not a batch reducer
messages: Annotated[list, DeltaChannel(add_messages)]  # wrong

#  Use _messages_delta_reducer — single pass, dedup by ID, RemoveMessage support
messages: Annotated[list, DeltaChannel(_messages_delta_reducer)]

#  Or write your own batch reducer for custom types
def my_dict_reducer(state: dict, writes: list[dict]) -> dict:
    result = dict(state)
    for w in writes:
        result.update(w)
    return result

files: Annotated[dict, DeltaChannel(my_dict_reducer)]
```

### Snapshot frequency

`snapshot_frequency=N` writes a full `_DeltaSnapshot` blob every N
pregel steps, bounding replay depth regardless of thread length.
Snapshots are eager — written even if the channel had no update that
step, so the depth bound always holds:

```python
# Replay walks at most 10 ancestors before hitting a snapshot
messages: Annotated[list, DeltaChannel(_messages_delta_reducer, snapshot_frequency=10)]
```

### Migration from `BinaryOperatorAggregate`

Pre-existing threads written under `BinaryOperatorAggregate` work
transparently after swapping the annotation — the saver detects a
plain-value ancestor blob and uses it as the reconstruction seed:

```python
# Before: BinaryOperatorAggregate stores full list every step
items: Annotated[list, add_messages]

# After: DeltaChannel — existing checkpoints still readable, new steps use sentinel
items: Annotated[list, DeltaChannel(_messages_delta_reducer)]
```

### Async write-ordering safety

In `durability="async"` mode (default), `put_writes` calls are
fire-and-forget. `AsyncPregelLoop` tracks in-flight `aput_writes`
futures for DeltaChannel channels in `_delta_write_futs` and drains them
via `await asyncio.gather()` in `_checkpointer_put_after_previous`
before `aput()` — ensuring `checkpoint_writes` are durable before the
sentinel blob is committed.

---

## What's in scope

- **`libs/langgraph/langgraph/channels/delta.py`** — `DeltaChannel`
implementation
- **`libs/langgraph/langgraph/graph/message.py`** —
`_messages_delta_reducer` (experimental)
- **`libs/checkpoint/`** — `_get_channel_writes_history` ancestor-walk
API on `BaseCheckpointSaver`, `InMemorySaver` optimized override
- **`libs/checkpoint-postgres/`** — `PostgresSaver` /
`AsyncPostgresSaver` single-roundtrip UNION ALL override
- **`libs/langgraph/langgraph/pregel/`** — `channels_from_checkpoint` /
`create_checkpoint` wiring, async write-ordering safety

---

## Follow-ups

- **Batch reconstruction**: each DeltaChannel field issues its own
`_get_channel_writes_history` call; a single walk collecting all
sentinel channels would reduce roundtrips proportionally to the number
of DeltaChannel fields.
- **Sync write ordering**: `BackgroundExecutor.__exit__` guarantees
completion before `invoke()` returns, but within a run there's no
explicit ordering between `put_writes` and `put`. Two-phase commit for
sync would close this gap.
- **`ShallowPostgresSaver` compatibility**: shallow savers keep only the
latest checkpoint and have no parent chain to walk; DeltaChannel is
currently incompatible and should raise or warn at compile time.
- Updating the writes table w/ delta epoch ids for more efficient reads
- follow up w/ LSD checkpointer implementations to support delta
channel! and update prune

---------

Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-authored-by: ccurme <chester.curme@gmail.com>
Co-authored-by: Will Fu-Hinthorn <will@langchain.dev>
2026-04-29 17:26:17 -04:00
William FHandGitHub e86b5f4da2 chore: pgqs (#6567)
Add more argument sanitization
2025-12-09 14:51:29 -08:00
Cole MurrayandGitHub b0a1029d55 fix(checkpoint-postgres): Replace f-string SQL formatting with parameterized queries in migration statements (#6328)
## Summary

Replace f-string SQL formatting with parameterized queries to prevent
potential SQL injection in checkpoint migration code.

## Changes

Updated the migration version tracking INSERT statements in all
checkpoint saver classes to use parameterized queries instead of
f-string formatting:

- `PostgresSaver`
(libs/checkpoint-postgres/langgraph/checkpoint/postgres/__init__.py:100)
- `AsyncPostgresSaver`
(libs/checkpoint-postgres/langgraph/checkpoint/postgres/aio.py:104-106)
- `ShallowPostgresSaver`
(libs/checkpoint-postgres/langgraph/checkpoint/postgres/shallow.py:255)
- `AsyncShallowPostgresSaver`
(libs/checkpoint-postgres/langgraph/checkpoint/postgres/shallow.py:617-619)

**Before (vulnerable to SQL injection):**
```python
cur.execute(f"INSERT INTO checkpoint_migrations (v) VALUES ({v})")
```

**After (using parameterized query):**
```python
cur.execute("INSERT INTO checkpoint_migrations (v) VALUES (%s)", (v,))
```

## Risk Assessment

The practical risk is low since `v` is an integer loop variable
controlled by the codebase. However, using string formatting in SQL
queries is a well-known anti-pattern that can lead to SQL injection
vulnerabilities, especially if the code is later refactored or copied to
other contexts.

## Testing

-  All 216 tests passing on PostgreSQL 15 and 16
-  Linting and type checking passing
-  No functional changes to behavior
2025-11-07 08:00:10 -05:00
Josh RogersandGitHub ff38f75594 fix(checkpoint-postgres): make async PG checkpoint migration idempotent (#6400)
- **Description:** The final migration for the postgres checkpointer is
not currently idempotent. That presents problems when migrating from one
checkpointer to another or if migrations otherwise get applied twice.
This makes the final migration idempotent to avoid this problem.
- **Issue:** N/A
- **Dependencies:** N/A
- **Twitter handle:** N/A
2025-11-06 09:49:51 -05:00
Sydney RunkleandGitHub 2d3121a17c chore: drop Python 3.9 (and syntax) (#6289)
* `strict=False` is the default, pyupgrade to min version 3.10 adds this
to be explicit w/ behavior
2025-10-16 20:17:46 -04:00
d9e3d83894 docs: style linting (#6260)
also fixes some links

---------

Co-authored-by: Sydney Runkle <54324534+sydney-runkle@users.noreply.github.com>
Co-authored-by: ccurme <chester.curme@gmail.com>
Co-authored-by: Sydney Runkle <sydneymarierunkle@gmail.com>
Co-authored-by: William FH <13333726+hinthornw@users.noreply.github.com>
2025-10-16 11:25:50 +00:00
1ba96f49bf fix(checkpoint): handle metadata.writes when serializing old checkpoints with Jsonb (#6236)
Issue

Support for `Checkpoint.metadata.writes` was dropped in `langgraph`
v0.5.x.

In `langgraph-checkpoint-postgres` v2.0.23, metadata was serialized with
`BasePostgresSaver._dump_metadata` -> `JsonPlusSerializer.dumps` which
handles `pydantic.BaseModel`.

In v2.0.23, metadata is serialized with `psycopg.types.json.Jsonb`,
which raises `TypeError: Object of type AIMessage is not JSON
serializable` when trying to serialize `writes`.

Solution

- Add `BaseCheckpointSaver.get_serializable_checkpoint_metadata` which
pops the `writes` key.
- Log deprecation warning when strange version combinations are used 

Solves https://github.com/langchain-ai/langgraph/issues/5769

---------

Co-authored-by: Alex Kondratev <56111142+soapun@users.noreply.github.com>
2025-10-06 11:27:34 -07:00
f0fced262a fix(langgraph): fix PostgresSaver crashing when loading older checkpoints (#6162)
### Description

https://github.com/langchain-ai/langgraph/issues/6137 and
https://github.com/langchain-ai/langgraph/issues/5677 reported issues
where older checkpoints read by AsyncPostgresSaver/PostgresSaver from
`langgraph-checkpoint-postgres==2.0.19` fail to read channel values,
throwing `NoneType object is not a mapping`. This was due to a bug in
how `channel_values` is assembled:
```python
"channel_values": {
    **value["checkpoint"].get("channel_values"),  # <--- if channel_values doesn't exist (old checkpoint), **None errors
    **self._load_blobs(value["channel_values"]),
},
```
This bug was observed for checkpoints generated by
`langgraph-checkpoint-postgres<=2.0.19`.

Fixed by providing a fallback to
`value["checkpoint"].get("channel_values")`:
```python
**value["checkpoint"],
"channel_values": {
    **(
        value["checkpoint"].get("channel_values") or {}
    ),  # 'or {}' needed for backwards compat with v3 checkpoints and below, as v4 introduced channel_values key
    **self._load_blobs(value["channel_values"]),
},
```

### Tests
Added test for AsyncPostgresSaver and test for PostgresSaver, using
monkeypatch to remove `channel_values` before CheckpointTuple is
assembled in `_load_checkpoint_tuple`.

### Solves
https://github.com/langchain-ai/langgraph/issues/6137 and
https://github.com/langchain-ai/langgraph/issues/5677

---------

Co-authored-by: Shahrukh Shaik <144558473+shahrukh-shaik@users.noreply.github.com>
2025-09-17 17:50:39 -07:00
8b55dff7a5 chore(deps): upgrade dependencies with uv lock --upgrade (#6146)
This PR updates the dependencies in all Python packages using `uv lock
--upgrade`.

This is an automated PR created by the UV Lock Upgrade workflow.

To make tests pass:
* linting fixes
* whitespace fixes in snapshots

---------

Co-authored-by: sydney-runkle <54324534+sydney-runkle@users.noreply.github.com>
Co-authored-by: Sydney Runkle <sydneymarierunkle@gmail.com>
2025-09-14 19:36:43 -04:00
cb7b924006 feat: Implement durability mode argument (#5432)
- Replaces checkpoint_during: bool
- checkpoint_during is deprecated but still respected
- We implement three durability modes (from least to most durable):
- "exit" - save checkpoint only when the graph exits (equivalent to
checkpoint_during=False)
- "async" - save checkpoint asynchronously while the next step executes
(the default, equivalent to old checkpoint_during=True)
- "sync" - save checkpoint synchronously before the next step starts
(new mode, slower but most durable)

Co-authored-by: Sydney Runkle <54324534+sydney-runkle@users.noreply.github.com>
2025-07-20 15:42:18 +01:00
Sydney Runkle c6d674cd3e Merge branch 'main' into v1 2025-07-16 18:32:07 -04:00
Nuno Campos e973e936c3 perf: checkpoint-postgres: Reduce writes to checkpoint_blobs table
- Channels containing primitive values don't need to be stored in separate rows in blobs table, as the overhead of a separate row will usually be higher than the size of the value
- This applies for instance to all internal channels used to manage edges, so it has a big impact just from that. It can also apply to user-managed channels depending on their values
- The same channel may switch storage between versions without any issue
2025-07-16 11:43:51 +02:00
Sydney Runkle a84b744eb6 Merge branch 'main' into v1 2025-07-08 09:55:13 -04:00
87f2e69395 chore[deps]: upgrade dependencies with uv lock --upgrade (#5358)
* chore: upgrade dependencies with `uv lock --upgrade`

* linting

* upgrade PR title

---------

Co-authored-by: sydney-runkle <54324534+sydney-runkle@users.noreply.github.com>
Co-authored-by: Sydney Runkle <sydneymarierunkle@gmail.com>
2025-07-07 23:24:32 +00:00
Sydney RunkleandGitHub c989f1c898 langgraph: remove support for thread_ts (old alias for checkpoint_id) (#5295)
* remove support for thread_ts

* docs and tests
2025-07-01 13:42:25 -04:00
Nuno CamposandGitHub 1134017d07 Preparation for 0.5 release: langgraph-checkpoint (#5124)
Prepare langgraph-checkpoint for 0.5

- Given we have no upper bound on langgraph-checkpoint dep need to undo all changes in langgraph-checkpoint that might break previous versions of langgraph
2025-06-16 21:57:11 +00:00
Nuno Campos 0cad7019cb Restore shallow checkpointer
- This should definitely be removed soon, but let's give people more time to update
2025-06-13 17:37:40 -07:00
Naohiro YoshidaandGitHub 1519b90414 Centralized CheckpointTuple creation into a shared function for checkpoint_postgres (#4970) 2025-06-09 18:40:17 +00:00
Sydney RunkleandGitHub 5e7566f4a3 lint: use pep 604 union syntax and pep 585 generic syntax (#4963)
* new union syntax

* fix test

* second round of conversions by injecting future annotations

* format + add top level makefile
2025-06-04 21:50:16 -04:00
Nuno Campos c31c940bbb checkpoint-postgres: Use lock also for pipeline mode 2025-06-03 14:59:56 -07:00
Nuno Campos 936176eb21 Remove Checkpoint.writes
- This has been superseded by saving the individual writes of each task through put_writes()
- Removing this speeds up checkpoint operations as it was duplicating data saved elsewhere already
2025-05-25 19:06:46 -07:00
Nuno Campos 4e8fbe4525 Remove Checkpoint.pending_sends
- Instead store sends in a Topic channel, removing the need to fetch sends as writes against the parent checkpoint
- Remove deprecated/unused functions in langgraph-checkpoint (will require bumping min range for langgraph-checkpoint in langgraph lib)
- Implement migration of old pending sends in langgraph-checkpoint-postgres
- Ensure parent config of `checkpoint_during=False` checkpoints always points to checkpoints that were also saved
2025-05-25 19:06:02 -07:00
Nuno Campos 8c11c1155a Remove postgres shallow checkpointer
- This was deprecated, and superseded by checkpoint_during=False, which is available for all checkpointers
2025-05-24 12:39:50 -07:00
Sydney Runkle 8ce33b948c docstrings for checkpoint-postgres 2025-05-01 13:40:05 -04:00
Vadym BardaandGitHub e25dde1df0 docs(reference): filter class methods and add missing docstrings (#4463) 2025-04-29 21:31:57 +00:00
18a9ae45f3 Add delete_thread method to Checkpointer class (#4328)
- Deletes all data associated with a thread_id
- Implemented in InMemory, Sqlite and Postgres checkpointers

Co-authored-by: Eugene Yurtsev <eyurtsev@gmail.com>
2025-04-17 16:38:58 +00:00
Vadym BardaandGitHub fdb9b9b8e0 checkpoint-postgres: add deprecation warning for ShallowPostgresSaver (#4219) 2025-04-09 11:31:33 -04:00
Nuno Campos 7f4822931e Update tests 2025-02-14 18:24:48 -08:00
Nuno Campos 9706211aca Exclude complex values from checkpoint metadata 2025-02-14 17:35:14 -08:00
Vadym BardaandGitHub 1377e3b6ba checkpoint: combine metadata when writing checkpoints (#3404) 2025-02-13 03:24:41 +00:00
Vadym BardaandGitHub 7ecda42b42 checkpoint-postgres: bring back missing migration (#3058) 2025-01-16 03:16:28 +00:00
Nuno Campos bba00506ea Lint 2025-01-15 10:57:26 -08:00
Nuno Campos fa12538a4e Lint 2025-01-15 10:50:11 -08:00
Nuno Campos c1a7bb8902 Lint 2025-01-15 10:49:47 -08:00
Nuno Campos 053a501db3 Add optional task_path arg for put_writes()
- Will be used for sorting pending_sends when available
2025-01-15 10:42:54 -08:00
Vadym BardaandGitHub 5db71a32bf checkpoint-duckdb: handle calling .list on async checkpointer (#3022) 2025-01-14 20:00:58 +00:00
Vadym BardaandGitHub b989502c24 checkpoint-sqlite/postgres: handle calling .list on async checkpointer (#3019) 2025-01-14 19:06:11 +00:00
William FHandGitHub e5b5f9510b Fix empty migration (#2978) 2025-01-09 23:14:02 +00:00
Vadym BardaandGitHub 44ee0199fd checkpoint postgres: add a shallow checkpointer (#2826)
This PR adds a "shallow" version of `PostgresSaver` checkpointer that
ONLY stores the most recent checkpoint and does NOT retain any history.
It is meant to be a light-weight drop-in replacement for the
PostgresSaver that supports most of the LangGraph persistence
functionality with the exception of time travel.
2024-12-20 17:51:20 +00:00
William FHandGitHub 93e4c8cc1f Create index concurrently (#2659) 2024-12-05 15:56:39 -08:00
Phoenix LoganandGitHub aca67107c1 fix: make database saver classes inheritance-friendly (#2615)
Replace hardcoded database saver class names with `cls` in
`from_conn_string` factory methods to improve subclassing support

## Changes
* Replaced direct class instantiations with `cls(conn)` in
`from_conn_string` classmethods across all database implementations
* Updated both synchronous and asynchronous variants for DuckDB,
PostgreSQL, and SQLite savers

## Why
This refactor makes the database saver classes more extensible by
following Python's convention of using `cls` in class methods. This
enables proper inheritance patterns where subclasses can reuse the
factory methods without needing to override them. Previously, the
hardcoded class names would always instantiate the parent class, even
when called from a subclass.

## Testing
The change is backward compatible and doesn't alter existing
functionality. All existing tests should continue to pass as this is
purely a structural refactoring that preserves the current behavior
while improving extensibility.

## Notes
This PR addresses follow up on comments from #2518 - AsyncPostgresSaver
didn't need to be fixed but many of the other DB saver classes did.
2024-12-03 20:26:06 -08:00
4332a9515d Fixup initial provisioning of aio postgres db (#2571) (#2600)
fixes #2570

---------

Co-authored-by: Tai Groot <tai@taigrr.com>
2024-12-03 01:55:26 +00:00
William FHandGitHub d767af421b feat: Add vector search (#2535)
- Initializing the store with an 'embedding config' -> this contains the
'dims' (used to create the table) and the encoder object (rn langchain
embeddings object, though that is ......)
- Call setup() -> creates the vector table.

Each document has 1 or more vectors associated with it for each json
path in the embedding config.

Would welcome critique and requests! 

Leaving the params as the defaults for pgvector but open to feedback if
you think it's important to be able to more transparently configure that
in setup()

```python
from typing import TypedDict, List, Dict, Any, Optional

from langchain_openai import OpenAIEmbeddings
from langgraph.graph import StateGraph
from langgraph.store.postgres import PostgresStore

emb_config = {
    "dims": 1536,  # OpenAI embedding dimensions
    "embed": OpenAIEmbeddings(model="text-embedding-3-small"),
    "distance_type": "cosine",
}
with PostgresStore.from_conn_string(
    "postgres://postgres:postgres@localhost:5441",
    embedding=emb_config,
) as store:
    store.setup()


# Define the state type for our graph
class State(TypedDict):
    query: str
    results: Optional[List[Dict[str, Any]]]


def put_stuff(state: State) -> State:
    docs = [
        ("doc1", {"text": "red apple in kitchen"}),
        ("doc2", {"text": "blue car in garage"}),
        ("doc3", {"text": "green apple on table"}),
    ]
    for key, value in docs:
        store.put(("docs",), key, value)


def search_stuff(state: State) -> State:
    """Search for documents using vector similarity."""
    results = store.search(("docs",), query=state["query"])

    return {"results": results}


builder = StateGraph(State)
builder.add_node(put_stuff)
builder.add_node(search_stuff)
builder.add_edge("__start__", "put_stuff")
builder.add_edge("put_stuff", "search_stuff")
# Compile
with PostgresStore.from_conn_string(
    "postgres://postgres:postgres@localhost:5441",
    embedding=emb_config,
) as store:
    chain = builder.compile(store=store)

    result = chain.invoke({"query": "sour apple"})

# Print results
for doc in result["results"]:
    print(doc.key)
    print(doc.value)
    print(doc.response_metadata)

```
2024-11-28 04:40:12 +00:00
William FHandGitHub 62a36befd5 Add in-mem vector search (#2547) 2024-11-27 14:53:24 -08:00
98935e1ffd fix: Fix race condition in PostgresSaver (#2494)
Signed-off-by: Tyler Ball <tyleraball@gmail.com>
Co-authored-by: Phoenix Logan <plogan@chanzuckerberg.com>
Co-authored-by: Tyler Ball <2481463+tyler-ball@users.noreply.github.com>
2024-11-25 20:19:52 +00:00