Commit Graph
35 Commits
Author SHA1 Message Date
d8b7800183 chore(langgraph): use two phase read to avoid unnecessary data transport (#7660)
## Summary

Replaces the single-roundtrip `UNION ALL` DeltaChannel read with a
two-stage query that avoids fetching unused snapshot blobs, then removes
the old combined path entirely.

### Problem

`_get_channel_writes_history` used a single `UNION ALL` query that
fetched **all** checkpoint metadata, writes, and blobs for a
`(thread_id, channel)` in one shot. With `snapshot_frequency=N`, this
pulled back O(N/freq) full-size snapshot blobs even though only the
nearest one is needed to seed reconstruction. At 500 turns with
`snapshot_frequency=10`, this meant fetching ~100 complete
message-history snapshots per read.

### Solution

Two-stage read:
- **Stage 1** — lightweight scan of `checkpoints` only (no blob bytes):
walks the parent chain from the target checkpoint and stops at the first
ancestor with a snapshot, returning `chain_cids` and `seed_version`
- **Stage 2** — targeted fetch: only the writes for `chain_cids` and the
single seed blob at `seed_version`

The two-stage path is now unconditional — the old combined query and
`LG_DELTA_TWO_STAGE_QUERY` env-var gate have been removed.

### Sentinel cleanup

`DELTA_SENTINEL` is now a pure in-memory signal and is never written to
storage:
- Postgres `put()` already stripped it from `channel_values` before
writing blobs
- Memory saver `put()` now stores `"empty"` instead of serializing the
sentinel
- `EXT_DELTA_SENTINEL` (msgpack ext code 8) removed from
`JsonPlusSerializer`
- `DELTA_SENTINEL` is kept as an in-memory marker:
`DeltaChannel.checkpoint()` returns it so savers know to skip it, and
`_ChannelWritesHistory.seed` uses it to mean "no snapshot found, start
from empty"

## Performance

Benchmarked at `snapshot_frequency=10` on Postgres (`~100 tok/msg`):

| turns | old combined query | two-stage |
|------:|-------------------:|----------:|
| 50    | 6.0ms              | 2.8ms  (2.1x faster) |
| 100   | 10.1ms             | 5.6ms  (1.8x faster) |
| 500   | **216.1ms**        | 15.3ms (**14x faster**) |

The old query's read time grew super-linearly with turn count because
each read fetched O(N/freq) full snapshot blobs. Two-stage keeps read
depth bounded by `snapshot_frequency` regardless of thread length.

## Test plan

- `make test` in `libs/checkpoint`, `libs/checkpoint-postgres`,
`libs/langgraph`
- Removed `test_delta_sentinel_serde_round_trip` (sentinel no longer
serializable)
- Updated `test_memory.py` — delta channel blobs stored as `"empty"`,
not serialized sentinel
- Updated `test_channels.py` — `channel_values` no longer contains
sentinel key for DeltaChannels
- Deleted `test_delta_channel_two_stage_benchmark.py` (one-stage vs
two-stage comparison; path no longer exists)

---------

Co-authored-by: Sydney Runkle <54324534+sydney-runkle@users.noreply.github.com>
Co-authored-by: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
2026-05-01 11:06:54 -04:00
5c18bde0f8 feat(langgraph): DeltaChannel: store sentinel in blobs, reconstruct from checkpoint_writes (#7586)
# DeltaChannel: sentinel-based checkpoint blobs + write-replay
reconstruction

## Summary

`DeltaChannel` is a new fold-reducer channel that stores only a
zero-byte sentinel in checkpoint blobs instead of the full accumulated
value. On restore, the runtime replays ancestor writes through the
reducer to reconstruct state. For long-running threads with large
accumulating state (e.g. message histories), this delivers dramatically
smaller checkpoint blobs with configurable read-depth bounds.

```python
from typing import Annotated
from typing_extensions import TypedDict
from langgraph.channels.delta import DeltaChannel
from langgraph.graph.message import _messages_delta_reducer

class State(TypedDict):
    # blob per step: ~60 bytes (sentinel) instead of growing full list
    messages: Annotated[list, DeltaChannel(_messages_delta_reducer)]
    # bound read depth to 10 steps via periodic snapshots
    messages_bounded: Annotated[list, DeltaChannel(_messages_delta_reducer, snapshot_frequency=10)]
```

---

## Storage benchmarks (InMemory, ~400 char/msg)

**Messages blob storage** (`checkpoint_blobs` bytes for the messages
channel):

| turns | add\_messages | delta(inf) | delta(freq=50) | delta(freq=10) |
delta(freq=5) |

|------:|-------------:|-----------:|---------------:|---------------:|--------------:|
| 10 | 91.0 KB | 60 B (1517x) | 60 B (1517x) | 14.4 KB (6x) | 32.6 KB
(3x) |
| 50 | 2.20 MB | 300 B (7347x) | 67.1 KB (33x) | 423 KB (5x) | 864 KB
(3x) |
| 100 | 8.78 MB | 600 B (14636x) | 310 KB (28x) | 1.72 MB (5x) | 3.48 MB
(3x) |
| 250 | 54.80 MB | 1.5 KB (36536x) | 2.09 MB (26x) | 10.87 MB (5x) |
21.84 MB (3x) |
| 500 | 219.19 MB | 3.0 KB (73063x) | 8.56 MB (26x) | 43.67 MB (5x) |
87.50 MB (3x) |

**Total checkpoint storage** (blobs + writes + metadata):

| turns | add\_messages | delta(inf) | delta(freq=50) | delta(freq=10) |
delta(freq=5) |

|------:|-------------:|-----------:|---------------:|---------------:|--------------:|
| 10 | 129.7 KB | 38.7 KB (3.4x) | 38.7 KB (3.4x) | 53.1 KB (2.4x) |
71.2 KB (1.8x) |
| 50 | 2.40 MB | 196 KB (12x) | 263 KB (9x) | 620 KB (3.9x) | 1.06 MB
(2.3x) |
| 100 | 9.18 MB | 394 KB (23x) | 703 KB (13x) | 2.12 MB (4.3x) | 3.87 MB
(2.4x) |
| 250 | 55.79 MB | 987 KB (57x) | 3.07 MB (18x) | 11.86 MB (4.7x) |
22.82 MB (2.4x) |
| 500 | 221.16 MB | 1.98 MB (112x) | 10.53 MB (21x) | 45.64 MB (4.9x) |
89.48 MB (2.5x) |

**Write-phase peak heap**:

| turns | add\_messages | delta(inf) | delta(freq=50) | delta(freq=10) |
delta(freq=5) |

|------:|-------------:|-----------:|---------------:|---------------:|--------------:|
| 10 | 456 KB | 199 KB (2.3x) | 199 KB (2.3x) | 212 KB (2.2x) | 232 KB
(2.0x) |
| 50 | 3.04 MB | 742 KB (4.1x) | 805 KB (3.8x) | 1.21 MB (2.5x) | 1.67
MB (1.8x) |
| 100 | 10.70 MB | 1.41 MB (7.6x) | 1.82 MB (5.9x) | 3.42 MB (3.1x) |
5.25 MB (2.0x) |
| 250 | 60.44 MB | 3.36 MB (18x) | 5.67 MB (11x) | 14.87 MB (4.1x) |
26.31 MB (2.3x) |

**Read-phase avg `get_state` latency** (5 calls, InMemory):

| turns | add\_messages | delta(inf) | delta(freq=50) | delta(freq=10) |
delta(freq=5) |

|------:|-------------:|-----------:|---------------:|---------------:|--------------:|
| 10 | 0.7 ms | 1.1 ms (0.6x) | 1.1 ms (0.6x) | 0.8 ms (0.9x) | 0.6 ms
(1.1x) |
| 50 | 2.7 ms | 5.3 ms (0.5x) | 3.5 ms (0.8x) | 2.7 ms (1.0x) | 2.7 ms
(1.0x) |
| 100 | 5.5 ms | 11.1 ms (0.5x) | 6.0 ms (0.9x) | 5.2 ms (1.1x) | 5.4 ms
(1.0x) |
| 250 | 12.9 ms | 27.2 ms (0.5x) | 13.6 ms (0.9x) | 12.9 ms (1.0x) |
13.0 ms (1.0x) |

**Postgres `get_tuple` read latency** (~100 tok/msg per step):

| steps | full-list | delta(inf) | delta(freq=50) | delta(freq=10) |
delta(freq=5) |

|------:|----------:|-----------:|---------------:|---------------:|--------------:|
| 10 | 0.29 ms | 0.21 ms (1.4x) | 0.19 ms (1.6x) | 0.19 ms (1.5x) | 0.19
ms (1.6x) |
| 50 | 0.19 ms | 0.15 ms (1.3x) | 0.19 ms (1.0x) | 0.22 ms (0.8x) | 0.29
ms (0.7x) |
| 100 | 0.27 ms | 0.17 ms (1.6x) | 0.22 ms (1.2x) | 0.23 ms (1.2x) |
0.21 ms (1.3x) |
| 500 | 0.60 ms | 0.30 ms (2.0x) | 0.66 ms (0.9x) | 0.56 ms (1.1x) |
0.69 ms (0.9x) |

**Takeaway:** `snapshot_frequency=10` matches full-list read latency
while still saving 5x on blob storage and ~4x on total storage.

---

## How it works

### Checkpoint blobs

`checkpoint()` always returns `DELTA_SENTINEL` (a zero-byte msgpack ext
marker) instead of the accumulated value. On restore, the saver's
`_get_channel_writes_history` walks the ancestor chain collecting
`checkpoint_writes` entries and replays them through the reducer:

```python
# blob stored per step: ~1 byte (sentinel)
# vs. full list growing O(N) every step with BinaryOperatorAggregate
```

### Reducer interface

`DeltaChannel` takes a **batch reducer** `(state, list[writes]) ->
state` — all writes for a step arrive in one call, enabling single-pass
implementations:

```python
#  Don't use add_messages directly — it's a binary operator, not a batch reducer
messages: Annotated[list, DeltaChannel(add_messages)]  # wrong

#  Use _messages_delta_reducer — single pass, dedup by ID, RemoveMessage support
messages: Annotated[list, DeltaChannel(_messages_delta_reducer)]

#  Or write your own batch reducer for custom types
def my_dict_reducer(state: dict, writes: list[dict]) -> dict:
    result = dict(state)
    for w in writes:
        result.update(w)
    return result

files: Annotated[dict, DeltaChannel(my_dict_reducer)]
```

### Snapshot frequency

`snapshot_frequency=N` writes a full `_DeltaSnapshot` blob every N
pregel steps, bounding replay depth regardless of thread length.
Snapshots are eager — written even if the channel had no update that
step, so the depth bound always holds:

```python
# Replay walks at most 10 ancestors before hitting a snapshot
messages: Annotated[list, DeltaChannel(_messages_delta_reducer, snapshot_frequency=10)]
```

### Migration from `BinaryOperatorAggregate`

Pre-existing threads written under `BinaryOperatorAggregate` work
transparently after swapping the annotation — the saver detects a
plain-value ancestor blob and uses it as the reconstruction seed:

```python
# Before: BinaryOperatorAggregate stores full list every step
items: Annotated[list, add_messages]

# After: DeltaChannel — existing checkpoints still readable, new steps use sentinel
items: Annotated[list, DeltaChannel(_messages_delta_reducer)]
```

### Async write-ordering safety

In `durability="async"` mode (default), `put_writes` calls are
fire-and-forget. `AsyncPregelLoop` tracks in-flight `aput_writes`
futures for DeltaChannel channels in `_delta_write_futs` and drains them
via `await asyncio.gather()` in `_checkpointer_put_after_previous`
before `aput()` — ensuring `checkpoint_writes` are durable before the
sentinel blob is committed.

---

## What's in scope

- **`libs/langgraph/langgraph/channels/delta.py`** — `DeltaChannel`
implementation
- **`libs/langgraph/langgraph/graph/message.py`** —
`_messages_delta_reducer` (experimental)
- **`libs/checkpoint/`** — `_get_channel_writes_history` ancestor-walk
API on `BaseCheckpointSaver`, `InMemorySaver` optimized override
- **`libs/checkpoint-postgres/`** — `PostgresSaver` /
`AsyncPostgresSaver` single-roundtrip UNION ALL override
- **`libs/langgraph/langgraph/pregel/`** — `channels_from_checkpoint` /
`create_checkpoint` wiring, async write-ordering safety

---

## Follow-ups

- **Batch reconstruction**: each DeltaChannel field issues its own
`_get_channel_writes_history` call; a single walk collecting all
sentinel channels would reduce roundtrips proportionally to the number
of DeltaChannel fields.
- **Sync write ordering**: `BackgroundExecutor.__exit__` guarantees
completion before `invoke()` returns, but within a run there's no
explicit ordering between `put_writes` and `put`. Two-phase commit for
sync would close this gap.
- **`ShallowPostgresSaver` compatibility**: shallow savers keep only the
latest checkpoint and have no parent chain to walk; DeltaChannel is
currently incompatible and should raise or warn at compile time.
- Updating the writes table w/ delta epoch ids for more efficient reads
- follow up w/ LSD checkpointer implementations to support delta
channel! and update prune

---------

Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-authored-by: ccurme <chester.curme@gmail.com>
Co-authored-by: Will Fu-Hinthorn <will@langchain.dev>
2026-04-29 17:26:17 -04:00
85cd64ed69 fix(checkpoint): revive lc=2 JSON blobs for safe types without allowlist (#7582)
## Summary

Fixes #7498 — `MESSAGE_COERCION_FAILURE` when resuming threads
checkpointed before v1.0.1.

**Root cause:** PR #6269 (v1.0.1) added an `_allowed_json_modules`
security gate to `JsonPlusSerializer._reviver`. The gate defaults to
`None`, so old `"json"`-format checkpoint blobs containing `lc=2`
constructor dicts (the pre-msgpack serialization format for pydantic
objects like `HumanMessage`) are now returned as raw dicts instead of
being reconstructed. Those raw dicts reach `add_messages →
convert_to_messages`, which sees `type="constructor"` and raises
`MESSAGE_COERCION_FAILURE`. Fresh first-turn messages are unaffected
because current `dumps_typed` only writes `"msgpack"` blobs.

**Fix:** `_reviver` now reconstructs `lc=2` blobs whose target class is
already in `SAFE_MSGPACK_TYPES` — the same curated allowlist already
used by the msgpack deserialization path (includes all standard
LangChain message types). Unknown classes are still blocked, preserving
the security intent of #6269.

## Changes

- `libs/checkpoint/langgraph/checkpoint/serde/jsonplus.py` — add
`_is_safe_json_type()` helper; update `_reviver` and
`_check_allowed_json_modules` to allow safe types without an explicit
allowlist
- `libs/checkpoint/tests/test_jsonplus.py` — two new regression tests:
safe-type `lc=2` blobs revive correctly; unknown-type `lc=2` blobs stay
blocked

## Test plan

- [ ] `test_lc2_json_safe_type_revives_without_allowlist` —
`HumanMessage`/`AIMessage` lc=2 JSON blobs round-trip to proper
`BaseMessage` objects with no allowlist configured
- [ ] `test_lc2_json_unknown_type_stays_blocked_without_allowlist` —
`pprint.pprint` lc=2 blob still returns raw dict (not reconstructed)
- [ ] `test_deserde_invalid_module` — existing behaviour unchanged
- [ ] Full `test_jsonplus.py` suite: 93/93 passing

Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-27 09:08:47 -04:00
f44b49b33d chore: dedup warnings (#7257)
Co-authored-by: Will Fu-Hinthorn <will@langchain.dev>
2026-04-17 10:13:03 -07:00
William FHandGitHub 901ab6b3f8 chore: add serde events (#6954) 2026-02-26 17:20:42 -08:00
William FHandGitHub 5ddfce1814 chore: support workflow dispatch on ci (#6952) 2026-02-26 14:45:02 -08:00
William FHandGitHub e2e90da5dc chore: improve subclass handling (#6948)
If subclass doesn't support the new parameter, we the current
implementation would create an error.
2026-02-26 13:08:00 -08:00
50df7d423a Merge commit from fork
* Patch

* Add more tests

* update idempotency tests

---------

Co-authored-by: William Fu-Hinthorn <13333726+hinthornw@users.noreply.github.com>
2026-02-26 12:01:58 -08:00
William FHandGitHub 092c9ecde6 chore: update ormsgpack minbound and add OPT_REPLACE_SURROGATES (#6395)
This lets the default `msgpack` serialization mode handle more cases
where user data contains invalid unicode.
2025-11-04 13:51:34 -08:00
Sydney RunkleandGitHub 41f8e61589 chore: bump core dep for prebuilt (#6323)
bumping core dependency for `langgraph-prebuilt` to `>1.0.0` so that we
can take advantage of internal utils that allow `ToolRuntime` injection.

We were previously bumping the version in lock step with prebuilt (prev
version was 0.3.67), so this pattern is in line with that.

Also updating snapshots accordingly:
* New mermaid syntax for a few graphs
* Removal of `examples` from `AIMessage`
2025-10-29 09:51:51 -07:00
c5744f583b chore: Restrict "json" type deserialization (#6269)
- Rm untyped loads/dumps
- Restrict to an allow list

---------

Co-authored-by: Sydney Runkle <sydneymarierunkle@gmail.com>
2025-10-20 10:18:36 -07:00
Sydney RunkleandGitHub 2c3e380a35 feat: adding cursory Python 3.14 support (#6298)
* catching error thrown by asyncio
* using 2nd check for annotations given Pydantic 2.12 changes
* skipping tests for remote graph bc langgraph-api is dependent on
`jsonschema-rs`
* skipping tests w/ pydantic v1 models

```bash
hint: This usually indicates a problem with the package or the build environment.
  help: `jsonschema-rs` (v0.29.1) was included because `langgraph:dev` (v1.0.0rc1) depends on `langgraph-cli[inmem]` which
        depends on `langgraph-api` (v0.4.29) which depends on `jsonschema-rs`
```

not yet testing for free threaded python, that'll be much more involved!

ended up separating lint / testing deps during this process bc I was
getting a ton of not required deps while testing that were complicating
things :/
2025-10-17 08:26:52 -04:00
Sydney RunkleandGitHub 2d3121a17c chore: drop Python 3.9 (and syntax) (#6289)
* `strict=False` is the default, pyupgrade to min version 3.10 adds this
to be explicit w/ behavior
2025-10-16 20:17:46 -04:00
Sydney RunkleandGitHub 3ad061f0d7 serialize/deserialize pandas with pickle fallback (#5057) 2025-06-12 15:14:00 -04:00
Nuno CamposandGitHub 2e33c520a5 Support numpy array serialization in JsonPlusSerializer (#5035)
* Handle numpy Fortran arrays

* Lint

* Lint

* Lint
2025-06-10 01:17:28 +00:00
Sydney RunkleandGitHub 228a08b966 ci: migrate to uv! (#4698)
* Migrate to `uv`
* Format `pyproject.toml` files properly
* Remove upper bounds on dependencies, and bounds on dev dependencies
(we should be using latest)
* Move to hatch for packaing

In the future we should:
* Set up dependabot / automate lockfile updates and tests
* Add tests for min compatible versions (I'll do this right after merge)
* Use dynamic versioning
* Bump `pydantic` to v2.11.4 in the lockfile, we have some tests failing
2025-05-15 17:39:14 -07:00
William Fu-Hinthorn d8cc9c9680 merge with ormsgpack 2025-03-24 16:21:59 -07:00
William Fu-Hinthorn 723d6f8a84 Add json mode msgpack unpacker 2025-03-24 16:15:48 -07:00
Nuno Campos 3d16acf3f5 Switch serialization lib to ormsgpack
- faster for large states
- supports tuples as dict keys
2025-03-24 09:01:55 -07:00
37344124e1 Fix updated_at timestamp loading (#3767)
Co-authored-by: Mohammad Mohtashim <45242107+keenborder786@users.noreply.github.com>
2025-03-10 22:20:12 +00:00
Nuno CamposandGitHub b1a4243055 Implement new Store interface (#1834) 2024-09-29 17:06:42 -07:00
vbarda 4271afeb42 checkpoint: support serde for SecretStr 2024-09-26 21:09:07 -04:00
Nuno CamposandGitHub 3b05279e32 Implement serialization with msgpack library (#1716)
* Implement serialization with msgpack library

- encode custom python objects with a msgpack extension type, with constructor path string, and args encoded as nested msgpack doc

* Smaller msgpack extension types

* Update lock files

* lock

* Don't delegate to pydantic json

* Fix kafka serde

- should use our serializer to load, as inputs to subgraphs are serialized using it
2024-09-16 14:06:42 -07:00
Nuno CamposandGitHub 66fc7c96aa More performance improvements in checkpointing and channels (#1685)
* Performance improvements in checkpointer libs

- Use sha1 instead of md5 for hashing (faster in python 3.x)
- Use orjson instead of json for json dumping (sadly can't use for json loading)

* Update tests

* Update

* Use random number instead of hash for get_version_number

* Avoid saving writes for the last task to complete in each step

- only when possible, exceptions for ERROR, INTERRUPT, SEND

* Make Channel.from_checkpoint a regular function

- context manager no longer needed since Context became a managed value

* Use __slots__ for Channels

* Fix for kafka
2024-09-12 21:08:30 -07:00
Nuno CamposandGitHub 95b97d69ec Nc/12sep/perf (#1702)
* Remove unused fil;e

* Add benchmark-fast command for running locally

* Small improvements to jsonplus serializer

* Don't use PregelNode.mapper when schema is a typed dict

- All it would do is create a new copy of same dict

* Avoid copying checkpoint when fetching at beginning of loop

* Fix needs array

* Update tests
2024-09-12 12:50:53 -07:00
Vadym BardaandGitHub 7bf99a5d2f langgraph: changes for compatibility pydantic v2 / langchain-core==0.3 (#1594) 2024-09-05 15:26:35 -04:00
Nuno Campos ca4ebe1031 Update tests 2024-08-30 15:51:57 -07:00
Nuno Campos ef790a57c6 Lint 2024-08-23 18:17:51 -07:00
Nuno Campos 728b283063 checkpoint: Ignore serialized values when constructor no longer available 2024-08-14 10:59:14 -07:00
Nuno Campos a64c0a2679 Add pathlib.Path and re.Pattern 2024-08-11 19:40:11 -07:00
Nuno Campos fdc5aece52 Add Decimal 2024-08-11 19:34:22 -07:00
Nuno Campos 685db62029 checkpoint: Add support for serializing more python types
- all ip interfaces
- deque,
- ZoneInfo
- date
- time
2024-08-11 19:27:29 -07:00
b37f78942d checkpoint-postgres: new library for postgres checkpointer implementation (#1236)
* checkpoint-postgres: new library for postgres checkpointer implementation

---------

Co-authored-by: Nuno Campos <nuno@langchain.dev>
2024-08-06 22:37:06 -04:00
4b2187c9a3 checkpoint: switch thread_ts -> checkpoint_id, add checkpoint_ns, change serializer protocol (#1185)
---------

Co-authored-by: Nuno Campos <nuno@langchain.dev>
2024-08-02 01:08:19 +00:00
ae74825ea7 langgraph checkpoint: new library for checkpoint interfaces (#1163)
---------

Co-authored-by: Nuno Campos <nuno@langchain.dev>
2024-07-31 16:45:52 -04:00