## Summary
Add a system-wide upper bound on supersteps-since-last-snapshot for
`DeltaChannel`, preventing unbounded ancestor walks on long-lived
threads where a delta channel stops receiving writes.
**Problem:** If a delta channel is written a few times (below
`snapshot_frequency`) and then never written again, it is never
snapshotted. Every subsequent run triggers an ancestor walk that grows
linearly with thread length — on long threads this becomes catastrophic.
**Solution:** Track a second counter (total supersteps) per delta
channel alongside the existing update count. Force a snapshot when
EITHER `updates >= snapshot_frequency` OR `supersteps >=
DELTA_MAX_SUPERSTEPS_SINCE_SNAPSHOT` (default 5000, overridable via env
`LANGGRAPH_DELTA_MAX_SUPERSTEPS_SINCE_SNAPSHOT`).
### Changes
- **`checkpoint` lib**: Rename metadata field
`delta_updates_since_snapshot: dict[str, int]` ->
`counters_since_last_snapshot: dict[str, tuple[int, int]]` where index 0
= updates, index 1 = supersteps.
- **`langgraph/_internal/_config.py`**: Add
`DELTA_MAX_SUPERSTEPS_SINCE_SNAPSHOT` constant with env override.
- **`langgraph/pregel/_checkpoint.py`**: Update
`delta_channels_to_snapshot()` predicate to fire on either threshold.
Rename reader helper to `read_counters_since_last_snapshot()`.
- **`langgraph/pregel/_loop.py`**: Iterate all delta channels each
superstep (not just updated ones) to bump the supersteps counter. Reset
both counters to `(0, 0)` on snapshot.
- **Tests**: Updated existing exit-mode tests for new field shape. Added
4 new tests covering forced snapshot (single run + multi-run
accumulation), predicate unit test, and counter reset.
## Test plan
- [x] `test_delta_channel_supersteps_bound.py` — 4 new tests all pass
- [x] `test_delta_channel_exit_mode.py` — 11 existing tests updated and
pass
- [x] `test_delta_channel_migration.py` — 11 tests pass
- [x] `test_channels.py` — 29 tests pass
- [x] `test_pregel.py` — 457 tests pass
- [x] `libs/checkpoint` test suite — 151 pass, 16 skipped
- [x] `make lint` clean (langgraph + checkpoint)
## Summary
Add a user-facing design doc and `get_delta_channel_keepset` helper for
third-party `BaseCheckpointSaver` authors who need to support graphs
using `DeltaChannel`.
**Deliverables:**
1. ~~**`docs/delta-channel-checkpointer-guide.md`** — comprehensive
guide covering~~:
moved to docs repo
2. **`BaseCheckpointSaver.get_delta_channel_keepset` /
`aget_delta_channel_keepset`** — returns the minimum set of ancestor
`checkpoint_id`s that must survive deletion for a given head's
`DeltaChannel` reconstruction to remain intact. Enables safe `prune`
implementations without silently corrupting delta history.
3. **Docstring warnings** on `prune`, `aprune`, `delete_for_runs`,
`adelete_for_runs`, `copy_thread`, `acopy_thread` explaining the
DeltaChannel pitfall (silent data loss if ancestor writes/snapshots are
deleted).
4. **Three new conformance capabilities** in
`libs/checkpoint-conformance`:
- `delta_channel_history` — validates the `aget_delta_channel_history`
walk contract
- `delta_channel_keepset` — validates the keep-set contract
- `delta_channel_reconstruction` — end-to-end round-trip (aput +
aput_writes + history + reconstruct)
## Test plan
- [x] `make format lint` passes in `libs/checkpoint`,
`libs/checkpoint-conformance`
- [x] All three new conformance capabilities pass against
`InMemorySaver`
- [x] Run conformance against SQLite saver
- [x] Run conformance against Postgres saver
---------
Co-authored-by: Cursor <cursoragent@cursor.com>
## Summary
Adds a Beta admonition to the `DeltaChannel` surface area so users can
distinguish stable from in-progress contracts. Single-sourced on the
base class; subclass overrides in `checkpoint-postgres` /
`checkpoint-sqlite` / memory inherit the marker via their existing
references back to the base.
Touched:
- `DeltaChannel` (`libs/langgraph/langgraph/channels/delta.py`)
- `BaseCheckpointSaver.get_delta_channel_history` and
`aget_delta_channel_history` (`libs/checkpoint/.../base/__init__.py`)
- `DeltaChannelHistory` TypedDict
- `CheckpointMetadata.delta_updates_since_snapshot`
## Why docstring admonitions, not `@beta`
The base `get_delta_channel_history` methods are designed to be
overridden by savers. Wrapping with `@beta` (from `langchain_core._api`)
would emit warnings whenever a subclass called
`super().get_delta_channel_history(...)` or whenever the default
ancestor walk fired. Docstring-only keeps the signal advisory and
noise-free. `_DeltaSnapshot` is already leading-underscore-private, so
it implicitly signals "internal."
## Versioning note
This intentionally stays a **minor** bump for the checkpoint releases
(4.0 → 4.1, 3.0 → 3.1):
- The new methods are strictly additive — defaults provided on the base,
no signatures changed, no removed APIs. Third-party savers keep working
without overriding anything.
- The beta marker and the version bump do orthogonal jobs: semver
answers "is this a breaking change?" (no), the marker answers "is this
contract stable?" (no).
- Bumping major now would consume the lever you want available for when
the delta contract actually changes in a breaking way.
## Test plan
- [ ] Docstring-only — no behavioral change
- [x] `make format && make lint` clean in `libs/checkpoint` and
`libs/langgraph`
## Summary
- Promotes the private K-channel batched ancestor-walk to a stable
public `get_delta_channel_history` / `aget_delta_channel_history` API on
`BaseCheckpointSaver` (returns `Mapping[str, DeltaChannelHistory]`, a
TypedDict with `writes` always present and `seed` `NotRequired`)
- Removes `DELTA_SENTINEL` / `_DeltaSentinel` entirely — the saver layer
is now delta-agnostic on both write and read paths
- Reworks `DeltaChannel` snapshot cadence from "every Nth superstep" to
"every N updates to this channel," persisted in
`CheckpointMetadata.delta_updates_since_snapshot`
- Adds Postgres optimizations: paged stage-1 with cursor (1024-row
pages) and per-channel UNION ALL stage-2 (no over-fetch when channels
have different chain depths)
- Default `snapshot_frequency` becomes a positive int (default `1000`);
the previous `None` opt-out is removed
## Public API
```python
class DeltaChannelHistory(TypedDict):
writes: list[PendingWrite] # always present, possibly empty
seed: NotRequired[Any] # absent if walk reached root
def get_delta_channel_history(
self, *, config: RunnableConfig, channels: Sequence[str]
) -> Mapping[str, DeltaChannelHistory]: ...
async def aget_delta_channel_history(
self, *, config: RunnableConfig, channels: Sequence[str]
) -> Mapping[str, DeltaChannelHistory]: ...
```
`config` and `channels` are keyword-only so later additions (e.g.
`page_size`) don't shift the positional API.
The TypedDict-with-`NotRequired[seed]` shape matches the existing
checkpoint-package convention (`CheckpointMetadata` is
`TypedDict(total=False)`) — absence-via-key-omission rather than
introducing a new sentinel. Pregel translates `"seed" not in hist` to
`MISSING` on its side at consume time.
The default impl walks `get_tuple` + `parent_config` correctly but is
slow on long chains; savers that care override (`InMemorySaver`,
`PostgresSaver`).
## Sentinel removal
`DELTA_SENTINEL` and `_DeltaSentinel` are deleted entirely. The saver
layer becomes delta-agnostic:
- `DeltaChannel.checkpoint()` returns `MISSING` for non-snapshot steps;
pregel's `create_checkpoint` skips MISSING so delta channels without a
snapshot simply don't appear in `channel_values`
- `InMemorySaver.put` and Postgres `put` no longer filter sentinels
(they have nothing to filter)
- `_needs_replay` becomes `stored is MISSING`
- `DeltaChannel.from_checkpoint` accepts: `MISSING` → empty,
`_DeltaSnapshot(value)` → snapshot value, plain value → pre-migration
legacy
## Snapshot cadence
`DeltaChannel.snapshot_frequency: int` (default `1000`, positive). The
previous `None` opt-out is gone.
```python
def should_snapshot(ch_name, ch):
if force_delta_snapshot: # durability="exit"
return True
return updates_since_snapshot.get(ch_name, 0) >= ch.snapshot_frequency
```
Per-channel update counters are persisted in
`CheckpointMetadata.delta_updates_since_snapshot` (`NotRequired`,
`total=False`). The counter is incremented by `_put_checkpoint` for any
delta channel in `updated_channels` and reset to `0` by
`create_checkpoint` for channels that fire a snapshot this step.
Version-format-independent — works for `int`, `float`, and `str`
versioning schemes alike.
## Postgres optimization
Two improvements internal to the override:
**Stage-1 paged with cursor** (`LIMIT 1024` internal const, `AND
checkpoint_id < ?` for subsequent pages). The previous unpaged form
scanned every checkpoint in `(thread_id, ns)` and was pathological at
high thread depths.
**Stage-2 per-channel UNION ALL**: one `WHERE channel='X' AND
checkpoint_id = ANY(chain_X)` branch per channel plus one seed-blob
branch per channel with a seed. The previous form filtered by `channel =
ANY(channels) AND checkpoint_id = ANY(union_chain_cids)`, over-fetching
writes when channels had different chain depths (`K ×
max(chain_lengths)` vs the correct `sum(chain_lengths)`).
Both improvements stay internal to `PostgresSaver`/`AsyncPostgresSaver`;
the public contract returns a single `Mapping`.
## Benchmarks
`libs/langgraph/tests/test_delta_channel_benchmark.py`. Run via `python
libs/langgraph/tests/test_delta_channel_benchmark.py`. Postgres against
local pg:5441.
Results below trimmed to the high-signal cells. Sub-millisecond /
sub-100-turn rows omitted as warmup-bound; freq=1 omitted (chain depth =
1, nothing to optimize); peak read-time memory and Postgres storage are
flat between branches and omitted. Deep-thread reads and the
cadence-rework storage win are the load-bearing numbers.
### Postgres reads, 500 turns
| Scenario | main | branch | Δ |
|---|---:|---:|---:|
| Single-channel deep read | 17.7 ms | **6.1 ms** | **-66%** |
| Single-channel, 1000 turns | 35.0 ms | **14.3 ms** | **-59%** |
| K=3 channels, freq=50 uniform | 70.5 ms | **41.4 ms** | **-41%** |
| K=8 channels, freq=50 uniform | 214.2 ms | **139.4 ms** | **-35%** |
| K=8 channels, mixed freq (25/50/100/.../1000) | 295.6 ms | **214.4
ms** | **-27%** |
K-channel batching + paged stage-1 + per-channel UNION ALL stage-2 doing
exactly what they should at depth.
### InMemory reads, 500 turns
| Scenario | main | branch | Δ |
|---|---:|---:|---:|
| Single-channel deep read | 7.9 ms | **3.8 ms** | **-52%** |
| Single-channel, 1000 turns | 15.6 ms | **7.2 ms** | **-54%** |
| K=8 channels, freq=50 uniform | 112.3 ms | 94.6 ms | -16% |
| K=8 channels, mixed freq | 184.9 ms | **134.5 ms** | **-27%** |
### InMemory storage, 500 turns (cadence-rework win)
| Scenario | main | branch | Δ |
|---|---:|---:|---:|
| K=3, freq=50 uniform | 8.7 MB | **3.3 MB** | **-62%** |
| K=3 mixed freq | 3.8 MB | **1.3 MB** | **-66%** |
| K=8, freq=50 uniform | 23.1 MB | **8.7 MB** | **-62%** |
| K=8 mixed freq | 11.5 MB | **4.2 MB** | **-64%** |
Snapshot frequency now counts **channel updates** instead of
**supersteps**. On graphs where supersteps outpace per-channel updates
(e.g., input/end steps that don't write to channels), branch stores ~3×
fewer snapshot blobs.
### Tradeoff worth flagging
InMemory K=3 with mixed frequencies (50/200/1000) at 500 turns: **+64%
read latency** (46.6 → 76.5 ms). The mixed scenario has a channel with
`freq=1000` that goes the entire 500-turn run with no snapshot. On main,
the old superstep-counted cadence happened to fire at step=500 anyway.
New cadence gives users explicit control over walk depth via
`snapshot_frequency`. The K=8 mixed case still wins overall (-27%); this
regression is specific to the K=3 mixed shape.
Default `snapshot_frequency=1000` is the upper bound on walk depth —
it's a tunable knob.
## Tests
- New sqlite smoke test (`test_get_delta_channel_history.py`) exercises
the inherited default `BaseCheckpointSaver` impl via `SqliteSaver` /
`AsyncSqliteSaver` end-to-end with a real `DeltaChannel`-backed graph.
Sqlite uses the default unchanged — this validates the default path
actually works on a real second saver, not just on the optimized
override.
- Module-level `pytest.importorskip("langgraph.channels.delta")` guards
the test for sqlite's standalone CI environment (matches the postgres
pattern).
## Test plan
- [x] `libs/checkpoint`: 150 passed, 16 skipped
- [x] `libs/langgraph` (channels + delta migration): 41/41 (post-merge)
- [x] `libs/langgraph` (full pregel suite): 1784 passing — 6 "failures"
verified via `env -i` clean shell are local LangSmith env vars + `git
describe revision_id` polluting LangChain metadata fixtures; CI is
unaffected
- [x] `libs/checkpoint-postgres`: 40/40 saver tests + 3/3 delta channel
reconstruction tests against local Postgres
- [x] `libs/checkpoint-sqlite`: 105/105 (incl. retry-passed flake
`test_ttl_refresh`, unrelated to this PR)
- [x] Lint clean across all four libs (`ruff format`, `ruff check`,
`mypy`)
- [x] Branch-vs-main benchmarks — see results above
---------
Co-authored-by: Quanzheng Long <long@langchain.dev>
Co-authored-by: Cursor <cursoragent@cursor.com>
## Summary
Replaces the single-roundtrip `UNION ALL` DeltaChannel read with a
two-stage query that avoids fetching unused snapshot blobs, then removes
the old combined path entirely.
### Problem
`_get_channel_writes_history` used a single `UNION ALL` query that
fetched **all** checkpoint metadata, writes, and blobs for a
`(thread_id, channel)` in one shot. With `snapshot_frequency=N`, this
pulled back O(N/freq) full-size snapshot blobs even though only the
nearest one is needed to seed reconstruction. At 500 turns with
`snapshot_frequency=10`, this meant fetching ~100 complete
message-history snapshots per read.
### Solution
Two-stage read:
- **Stage 1** — lightweight scan of `checkpoints` only (no blob bytes):
walks the parent chain from the target checkpoint and stops at the first
ancestor with a snapshot, returning `chain_cids` and `seed_version`
- **Stage 2** — targeted fetch: only the writes for `chain_cids` and the
single seed blob at `seed_version`
The two-stage path is now unconditional — the old combined query and
`LG_DELTA_TWO_STAGE_QUERY` env-var gate have been removed.
### Sentinel cleanup
`DELTA_SENTINEL` is now a pure in-memory signal and is never written to
storage:
- Postgres `put()` already stripped it from `channel_values` before
writing blobs
- Memory saver `put()` now stores `"empty"` instead of serializing the
sentinel
- `EXT_DELTA_SENTINEL` (msgpack ext code 8) removed from
`JsonPlusSerializer`
- `DELTA_SENTINEL` is kept as an in-memory marker:
`DeltaChannel.checkpoint()` returns it so savers know to skip it, and
`_ChannelWritesHistory.seed` uses it to mean "no snapshot found, start
from empty"
## Performance
Benchmarked at `snapshot_frequency=10` on Postgres (`~100 tok/msg`):
| turns | old combined query | two-stage |
|------:|-------------------:|----------:|
| 50 | 6.0ms | 2.8ms (2.1x faster) |
| 100 | 10.1ms | 5.6ms (1.8x faster) |
| 500 | **216.1ms** | 15.3ms (**14x faster**) |
The old query's read time grew super-linearly with turn count because
each read fetched O(N/freq) full snapshot blobs. Two-stage keeps read
depth bounded by `snapshot_frequency` regardless of thread length.
## Test plan
- `make test` in `libs/checkpoint`, `libs/checkpoint-postgres`,
`libs/langgraph`
- Removed `test_delta_sentinel_serde_round_trip` (sentinel no longer
serializable)
- Updated `test_memory.py` — delta channel blobs stored as `"empty"`,
not serialized sentinel
- Updated `test_channels.py` — `channel_values` no longer contains
sentinel key for DeltaChannels
- Deleted `test_delta_channel_two_stage_benchmark.py` (one-stage vs
two-stage comparison; path no longer exists)
---------
Co-authored-by: Sydney Runkle <54324534+sydney-runkle@users.noreply.github.com>
Co-authored-by: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
You can Send(..., timeout=...) now. Much fun.
This would allow us to do something like adding support for an
annotation to let the LLM to pick a timeout for a given tool call, etc.
---------
Co-authored-by: Will Fu-Hinthorn <will@langchain.dev>
## Summary
Fixes#7498 — `MESSAGE_COERCION_FAILURE` when resuming threads
checkpointed before v1.0.1.
**Root cause:** PR #6269 (v1.0.1) added an `_allowed_json_modules`
security gate to `JsonPlusSerializer._reviver`. The gate defaults to
`None`, so old `"json"`-format checkpoint blobs containing `lc=2`
constructor dicts (the pre-msgpack serialization format for pydantic
objects like `HumanMessage`) are now returned as raw dicts instead of
being reconstructed. Those raw dicts reach `add_messages →
convert_to_messages`, which sees `type="constructor"` and raises
`MESSAGE_COERCION_FAILURE`. Fresh first-turn messages are unaffected
because current `dumps_typed` only writes `"msgpack"` blobs.
**Fix:** `_reviver` now reconstructs `lc=2` blobs whose target class is
already in `SAFE_MSGPACK_TYPES` — the same curated allowlist already
used by the msgpack deserialization path (includes all standard
LangChain message types). Unknown classes are still blocked, preserving
the security intent of #6269.
## Changes
- `libs/checkpoint/langgraph/checkpoint/serde/jsonplus.py` — add
`_is_safe_json_type()` helper; update `_reviver` and
`_check_allowed_json_modules` to allow safe types without an explicit
allowlist
- `libs/checkpoint/tests/test_jsonplus.py` — two new regression tests:
safe-type `lc=2` blobs revive correctly; unknown-type `lc=2` blobs stay
blocked
## Test plan
- [ ] `test_lc2_json_safe_type_revives_without_allowlist` —
`HumanMessage`/`AIMessage` lc=2 JSON blobs round-trip to proper
`BaseMessage` objects with no allowlist configured
- [ ] `test_lc2_json_unknown_type_stays_blocked_without_allowlist` —
`pprint.pprint` lc=2 blob still returns raw dict (not reconstructed)
- [ ] `test_deserde_invalid_module` — existing behaviour unchanged
- [ ] Full `test_jsonplus.py` suite: 93/93 passing
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
## Summary
- Add `LANGGRAPH_STRICT_MSGPACK=true` guidance to `JsonPlusSerializer`
docstring and inline comments
- Update the warning message emitted for unregistered types to mention
the env var
- Add module docstring to `_msgpack.py` explaining the safety controls
- Add Security sections to checkpoint, checkpoint-postgres, and
checkpoint-sqlite READMEs
## Context
Multiple security advisories have reported the same msgpack
deserialization pattern (`ext_hook` → `importlib.import_module` →
`getattr` → call). The underlying behavior is documented in the repo's
threat model as T1, but the `LANGGRAPH_STRICT_MSGPACK` env var that
mitigates it is not surfaced in user-facing docs, docstrings, or warning
messages. This PR closes that gap.
## Test plan
- [x] Verify READMEs render correctly on GitHub (callout boxes use `>
[!IMPORTANT]` syntax)
- [x] Verify `JsonPlusSerializer` docstring renders in IDE tooltips
- [x] Confirm warning message format: `LANGGRAPH_STRICT_MSGPACK=true
PYTHON_CMD 2>&1 | grep -i strict`
---------
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Cleaning up CI for #6701
```md
I'm trying to help a customer with some issues related to their checkpointing in postgres. They have some timeouts and retries around the langgraph checkpoint queries and I suspect the queue may be filling up with cancelled tasks.
This PR adds some eager checks to not execute an operation if the future was already cancelled.
This is to prevent the queue executing tasks that might have already timed out, which would otherwise cause more tasks to timeout due to the longer execution delay.
Thank you for contributing to LangGraph! Follow these steps to mark your pull request as ready for review. **If any of these steps are not completed, your PR will not be considered for review.**
```
Co-authored-by: Conrad Ludgate <conradludgate@gmail.com>
Add some conformance tests for checkpointer implementations. Includes
additona methods that will be useful if you want to integarte in the
agent server.
some of these changes were obvious, and some were less obvious. In a few
spots, it felt like a judgement call if we should be saying LangSmith
Deployment of LangGraph Server. But hopefully either works.
Issue
Support for `Checkpoint.metadata.writes` was dropped in `langgraph`
v0.5.x.
In `langgraph-checkpoint-postgres` v2.0.23, metadata was serialized with
`BasePostgresSaver._dump_metadata` -> `JsonPlusSerializer.dumps` which
handles `pydantic.BaseModel`.
In v2.0.23, metadata is serialized with `psycopg.types.json.Jsonb`,
which raises `TypeError: Object of type AIMessage is not JSON
serializable` when trying to serialize `writes`.
Solution
- Add `BaseCheckpointSaver.get_serializable_checkpoint_metadata` which
pops the `writes` key.
- Log deprecation warning when strange version combinations are used
Solves https://github.com/langchain-ai/langgraph/issues/5769
---------
Co-authored-by: Alex Kondratev <56111142+soapun@users.noreply.github.com>
### Description
* Set `ensure_ascii=False` for all `json.dumps` calls in
`get_text_at_path`. Preserves non-ASCII text instead of embedding
`\uXXXX` escapes.
**Before**
```python
store.put(("user_123", "memories"), "1", {"text": "这是中文"})
# embeds {"text": "\\u8fd9\\u662f\\u4e2d\\u6587"}
```
**After**
```python
store.put(("user_123", "memories"), "1", {"text": "这是中文"})
# embeds {"text": "这是中文"}
```
### Tests & Docs
* Add unit test `test_non_ascii` that writes three records (Chinese,
Japanese, Korean) to an `InMemoryStore`, searches with the same strings,
and asserts the correct top hit with a score >= 0.15 for each.
### Issue
Fixes#5946
### Description
Adds Redis as a supported cache backend for LangGraph node-level
caching, enabling distributed caching across multiple processes/servers.
This implementation follows the same patterns as existing InMemoryCache
and SqliteCache.
### Key changes
- New RedisCache class implementing the BaseCache interface
- Support for TTL-based expiration and batch operations
- Worker-specific cache prefixes for parallel test isolation
### Dependencies
- redis package (already included in dev dependencies)
### Test Plan
- Unit tests: Added Redis cache tests covering basic operations, TTL,
batch operations, and error handling
- Integration tests: Redis cache integrated into existing LangGraph test
suite, tested with all checkpointer combinations
- This makes prepare_next_tasks constant on number of nodes in all
cases, whereas before we were falling back to node iteration when
resuming from an existing checkpoint
This commit fixes#5503
Gist of it is:
- `asyncio.exception.InvalidStateError` were being raised when the
future was cancelled
- this exception bubbled up and killed the background task
- `AsyncBatchedBaseStore` stopped doing queries because the background
task wasn't running anymore
This commit adds some "if future is not done" checks to guard against
this.
- Leave it up to each checkpointer implementation to decide whether to merge in configurable/metadata (previously PregelLoop would do some of this always)
- Never copy over internal langgraph keys into checkpoint.metadata (these are redundant/misleading to include)
Prepare langgraph-checkpoint for 0.5
- Given we have no upper bound on langgraph-checkpoint dep need to undo all changes in langgraph-checkpoint that might break previous versions of langgraph
- This has been superseded by saving the individual writes of each task through put_writes()
- Removing this speeds up checkpoint operations as it was duplicating data saved elsewhere already
- Instead store sends in a Topic channel, removing the need to fetch sends as writes against the parent checkpoint
- Remove deprecated/unused functions in langgraph-checkpoint (will require bumping min range for langgraph-checkpoint in langgraph lib)
- Implement migration of old pending sends in langgraph-checkpoint-postgres
- Ensure parent config of `checkpoint_during=False` checkpoints always points to checkpoints that were also saved
