Restores DeltaChannel as a standalone class in channels/delta.py and adds
a snapshot_frequency parameter that writes a full snapshot blob every N writes,
bounding the ancestor replay walk depth while preserving O(N) storage for
large N.
Key design decisions:
- Write-count based (not step-based): snapshot fires every N writes to the
channel, tracked via _write_count incremented in both update() and
replay_writes(). This ensures the snapshot always coincides with an actual
channel write (i.e., a new_versions entry in put()), so it is always stored.
- Snapshot blob format: {"__delta_v__": value, "__delta_wc__": n} embeds the
write count so from_checkpoint() can restore it across invocations, keeping
the cadence correct without any external state.
- _checkpoint.py simplified: DeltaChannel.checkpoint() now returns the right
thing (sentinel or snapshot dict) so create_checkpoint needs no special logic.
- _needs_replay updated: triggers on DELTA_SENTINEL / MISSING; snapshot dicts
and plain values (migration) resolve directly via from_checkpoint().
Benchmark shows correct tradeoffs across frequencies (500 turns):
freq=1 → 296 MB storage, ~7ms reads
freq=5 → 60 MB storage, ~4ms reads
freq=10 → 30 MB storage, ~4ms reads
freq=50 → 6.5 MB storage, ~3ms reads
freq=inf→ 290 KB storage, ~114ms reads
Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
Two fixes exposed by running the postgres test suite against a local
postgres instance:
1. `PostgresSaver._get_channel_writes_history` /
`AsyncPostgresSaver._aget_channel_writes_history` required
`checkpoint_id` in the passed config, raising `KeyError` when called
with just `thread_id` (e.g. `graph.aget_state({"thread_id": "..."})`).
Now resolves to the latest checkpoint via `get_tuple`/`aget_tuple`
when the id is missing.
2. `test_get_checkpoint_no_channel_values` (sync + async) monkeypatched
`_load_checkpoint_tuple` with the old `(value, cur)` signature. Method
now takes `(value)` only since delta reconstruction moved out of the
tuple-load path — updated both tests.
Local postgres (`brew install pgvector postgresql@16`, running on port
5441) now exercises all 40 non-vector postgres tests green.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Restructure DeltaChannel reconstruction so the hydration path matches
pregel's storage axes (blobs + writes) without leaking internal DTOs
into the public checkpoint contract.
Key changes:
* Deleted `DeltaChannelWrites` dataclass and `SEED_UNSET` sentinel.
Reconstruction data no longer flows through `Checkpoint.channel_values`
as a wrapped DTO — that field now carries a value or `DELTA_SENTINEL`,
never a reconstruction shape.
* Added private `_ChannelWritesHistory(seed: Any, writes: list[PendingWrite])`
NamedTuple as the return type for the new storage-level query.
* Added private, experimental `_get_channel_writes_history` /
`_aget_channel_writes_history` on `BaseCheckpointSaver` — reference
impl via `get_tuple` + `parent_config` walk, overridden on
`InMemorySaver` / `PostgresSaver` / `AsyncPostgresSaver` for perf.
Fixes a latent migration bug in the base fallback (now inspects
ancestor `channel_values` for pre-delta seed).
* `DeltaChannel.from_checkpoint(seed)` simplified to two cases
(sentinel/MISSING → empty, else → seed). New `replay_writes` method
folds `list[PendingWrite]` through the reducer.
* Delta hydration consolidated inside `channels_from_checkpoint` via
optional `saver` + `config` kwargs (+ async mirror
`achannels_from_checkpoint`). All six pregel call sites updated.
`get_tuple` no longer patches `channel_values` — removed
`_resolve_delta_channels` (memory) and per-tuple reconstruction from
`_load_checkpoint_tuple` (postgres sync + async).
* Hydration short-circuits on the target's own blob: if
`channel_values[k]` is a real value (pre-migration tip, `update_state`
result), use it directly. Only walks ancestors when the target holds
sentinel or is missing. Fixes a correctness bug where migration-tip
and `update_state` values would be lost.
* New test_delta_channel_migration.py: 10 scenarios covering
BinaryOperatorAggregate → DeltaChannel migration (basic + async,
time-travel, fork, `update_state`, tip-of-pre-migration, base-saver
fallback parity, cross-thread isolation).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
snapshot_every was a knob for bounding reconstruction cost on deep threads.
Benchmarks (notes/add_messages_replay_problem.md + scratch work on
sr/add-messages-replay-bench) showed the add_messages fast-path
(optimize/add-messages-fast-path) closes the quadratic replay cost for
threads under ~1000 turns, where the crossover to snapshots makes sense.
For deeper threads we'll ship a first-class compaction primitive instead.
Removals:
* DeltaChannel: snapshot_every ctor param, _writes_since_snapshot counter,
should_snapshot() / snapshot_write() methods, counter threading through
_apply_write / update / from_checkpoint / copy.
* Pregel loop: post-checkpoint snapshot-injection block and
SNAPSHOT_TASK_ID import + constant.
* Checkpoint base: _overwrite_types() helper and the ancestor-walk
short-circuit on user-emitted Overwrite in sync + async
get_channel_writes.
* InMemory + Postgres savers: same walk-terminator shortcut. The
pre-delta blob terminator (seed-from-ancestor-blob) stays — it's
required for migration correctness, not a snapshot optimization.
* Tests for all of the above.
Preserved:
* Channel-level Overwrite semantics in DeltaChannel / BinOpAggregate:
Overwrite still resets the value at reducer level; same-super-step
dedup and InvalidUpdateError on multiple Overwrites still enforced.
* Pre-delta migration seeding.
The recursive CTE was bottlenecked by a JSON-expression join
(`bl.version = checkpoint->'channel_versions'->>bl.channel`) that the
planner could not index, producing an O(ancestors x blobs) nested-loop.
At depth 1000 it ran ~275 ms and removed ~2M filter rows; the recursion
itself was 2.4 ms.
Switch to three plain indexed SELECTs per delta channel
(checkpoints, checkpoint_writes, checkpoint_blobs); a pure helper on
BasePostgresSaver walks the parent chain and assembles
DeltaChannelWrites. Sync (__init__.py) and async (aio.py) each own
their three-roundtrip I/O wrappers.
Bench numbers (notes/delta_channel_query_bench.md): 3x at depth 50,
15x at depth 200, ~100x at depth 1000. Plain over-fetches sibling rows
when the thread branches but still wins at every realistic depth on
both local and remote postgres.
Multi-channel coalescing dropped — reconstruction is per-channel now.
Same shape as InMemorySaver. Can come back as a SQL-level
optimization later if needed.
Four fixes from an independent review of the reconstruction pipeline, plus
a structural cleanup:
1. Ancestor walk excludes the target checkpoint itself (matches pregel:
writes stored under checkpoint_id=T are pending for the NEXT step and
applied separately via apply_writes). Memory saver previously included
them, diverging from Postgres and causing pending writes to be folded
into the reconstructed snapshot — visible via get_state during
interrupts and time-travel into a non-leaf checkpoint.
2. Pre-delta blob terminator. When the walk hits an ancestor whose blob
for the channel is a real value (not DELTA_SENTINEL), bind that blob
as DeltaChannelWrites.seed and stop. Without this, threads migrated
from pre-delta storage would replay ancestor writes to the root
forever AND lose any value that lived only in the old blob
(e.g. from update_state). Per-ancestor, the blob is checked BEFORE
its writes — a pre-delta blob subsumes writes at the same checkpoint,
so including them would double-count.
3. Base-fallback get_channel_writes follows parent_checkpoint_id instead
of list(before=...). The previous form returned every tuple with
id<target, including sibling branches on forked threads.
4. seed replaces the Overwrite-wrapping hack for pre-delta values.
DeltaChannelWrites(writes, seed=SEED_UNSET) makes the saver's
reconstruction terminator semantically explicit; drops the lazy
_make_overwrite import dance. User-emitted Overwrite still reset the
chain via _apply_write as before.
Postgres: recursive CTE enumerates on-path ancestors and joins once
against checkpoint_writes and once against checkpoint_blobs for every
delta channel in the get_tuple — one roundtrip instead of the previous
3 queries × N channels.
Tests added:
- Pre-delta blob seeding (seed binding, no double-counting of ancestor
writes at the terminator, pending-at-target excluded).
- Root checkpoint returns empty writes.
- Seed-based from_checkpoint replay (three scenarios: with writes,
seed-only, seed=None distinct from SEED_UNSET).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Annotated[NotRequired[dict[...]], DeltaChannel(reducer)] (the shape used
by deepagents' filesystem middleware) fell through type inference to
`list`, so the first operator call blew up with
"'list' object is not a mapping". `_is_field_channel` now unwraps a
parameterized Required[X]/NotRequired[X] before stripping extras, which
lets dict/set/mapping outer types reach the abc normalization block.
Also type-annotates the `new` locals in DeltaChannel.copy() and
from_checkpoint() so mypy can infer them through the abstract return
type.
Adds tests covering: dict Overwrite in update and in writes replay,
snapshot_write with a dict reducer, dict backwards-compat checkpoints,
NotRequired type inference, and a filesystem-shaped end-to-end graph.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Instead of a recursive SQL CTE, collect the ancestor checkpoint ID chain in
Python by fetching all (checkpoint_id, parent_checkpoint_id) for the thread
in one query, then fetch writes with a plain WHERE checkpoint_id = ANY(...).
Simpler, avoids recursive query planner overhead, and uses well-indexed lookups.
DeltaChannel.checkpoint() now returns a zero-byte DeltaChannelSentinel
instead of duplicating delta data in checkpoint_blobs. Reconstruction
walks the parent checkpoint chain via checkpoint_writes (which already
holds per-step writes) and replays them through the operator.
In-memory benchmark (100 turns, ~20K tokens):
storage: 10.2 MB → 40.5 KB (251x reduction)
read: 0.6ms → 7.9ms (reconstruction cost, amortized by storage savings)
InMemorySaver and PostgresSaver override get_channel_writes() with
efficient implementations (Python dict walk and recursive CTE respectively).
The base class fallback uses self.list() with a thread-local recursion guard.
Remove the positional `typ` parameter from `DeltaChannel.__init__`. The type is
now injected automatically from the `Annotated` outer type in `_is_field_channel`
(matching how `BinaryOperatorAggregate` receives its type). `copy()` and
`from_checkpoint()` propagate `self.typ` explicitly. Test helpers updated to
use `_get_channel` with the proper `Annotated` path.
Rely on the runtime raise in DeltaChannel.from_checkpoint() instead of
a compile-time boolean flag. Savers that assemble DeltaChainValue inside
_load_blobs work transparently; savers that don't will pass through a raw
DeltaValue and hit a clear ValueError on first reload.
Removes: BaseCheckpointSaver.supports_delta_channels, the attribute on
InMemorySaver / PostgresSaver / AsyncPostgresSaver, the compile-time
UserWarning in StateGraph.compile(), and the associated test.
Resolves https://github.com/langchain-ai/langchain/issues/35585
This would previously raise KeyError:
```python
from typing import Annotated
from langchain_core.tools import tool
from langchain.agents import create_agent
from typing_extensions import NotRequired
from langgraph.prebuilt import InjectedState
from langchain.agents import AgentState
class CustomAgentState(AgentState):
city: NotRequired[str]
@tool
def get_weather(city: Annotated[str | None, InjectedState("city")] = None) -> str:
"""Get weather for a given city."""
if city is None:
city = "Boston"
return f"It's always sunny in {city}!"
agent = create_agent(
model="claude-sonnet-4-6",
tools=[get_weather],
system_prompt="You are a helpful assistant",
state_schema=CustomAgentState,
)
input_message = {
"role": "user",
"content": "What's the weather?",
}
result = agent.invoke({"messages": [input_message]})
for m in result["messages"]:
m.pretty_print()
```
---------
Co-authored-by: Sydney Runkle <sydneymarierunkle@gmail.com>
Use typ() instead of [] throughout DeltaChannel so reducers over dict
(and other non-list types) work correctly. fromCheckpoint(MISSING) now
leaves value as typ() from __init__ instead of overwriting with MISSING.
copy() uses value.copy() to handle dicts. update() initialises base from
typ() when value is MISSING. Add four tests covering the deepagents-style
dict-merge / file-deletion reducer pattern.
- Rename `_steps_since_rehydrate` → `_steps_since_snapshot` in DeltaChannel
for clarity (counts steps since the last snapshot, not since rehydration)
- Pre-seed cycle-detection `visited` set with current checkpoint ID in both
sync and async `_assemble_delta_channels` to prevent self-referential chains
- Add 4 new unit tests:
- `test_delta_channel_snapshot_every_emits_plain_list`: verifies counter
semantics and snapshot/delta transitions
- `test_delta_channel_snapshot_every_end_to_end`: graph-level smoke test
- `test_delta_channel_assembly_fast_path_returns_delta_value`: exercises
chain traversal via get_channel_blob returning DeltaValue then plain list
- `test_delta_channel_assembly_broken_chain_logs_warning`: partial chain
when get_tuple returns None
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Add DeltaValue to SAFE_MSGPACK_TYPES so SQLite and other msgpack-based
savers don't emit "Deserializing unregistered type" warnings.
- Rename _is_diff_delta → _is_delta_value (leftover from DiffChannel rename).
- Parametrize benchmark by checkpointer: runs InMemory (fast-path) and
SQLite (get_tuple fallback) in the same table, sharing the _run_turns helper.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Fix InMemorySaver.get_channel_blob: use correct storage[thread_id][ns]
nesting and deserialize the checkpoint before extracting channel_versions.
- Pass checkpoint_id to after_checkpoint() in channels_from_checkpoint so
DeltaChannel seeds _last_checkpoint_id correctly on load; without this
every turn broke the chain at its boundary.
- Wire _assemble_delta_channels into _prepare_state_snapshot and
_aprepare_state_snapshot (get_state / get_state_history paths) and into
perform_superstep / aperform_superstep (update_state paths) — previously
only the loop __enter__ path did assembly.
- Fix test_get_channel_blob to use the correct storage structure.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Periodic full-snapshot checkpoints cap chain depth, trading a small
amount of extra storage for bounded reconstruction time.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Fixes a critical deadlock that occurs when _load_diff_chains calls self._cursor()
from within _load_blobs while the outer _load_checkpoint_tuple already holds
self._cursor(). On bare (non-pool) connections, the threading.Lock is not
reentrant, causing a deadlock.
Solution: Pass the cursor as a parameter to _load_diff_chains and _load_blobs
instead of acquiring a new cursor within those methods. Updated _load_checkpoint_tuple
to acquire a cursor once at the top level and pass it through the call chain.
Changes:
- Updated _load_blobs signature to accept optional cur parameter
- Updated _load_diff_chains signature (base and implementations) to accept optional cur parameter
- Modified _load_checkpoint_tuple in PostgresSaver to acquire cursor and pass it
- Modified _load_checkpoint_tuple_async to acquire cursor only when diff_payloads exist
- Removed nested self._cursor() calls in _load_diff_chains and _load_diff_chains_async
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
