mirror of
https://github.com/langchain-ai/langgraph.git
synced 2026-10-15 20:55:23 +02:00
Elior Nataf LackritzandGitHub
36bf3887f0
Closes #8531. `PostgresSaver` and `AsyncPostgresSaver` inherited `prune` / `aprune` from the base class, which raises `NotImplementedError`, so long-lived threads had no supported way to drop old checkpoints. This adds both strategies. Thanks to @wangjiawei-vegetable for proposing this scope in #8531 and #9247. - **`delete`** removes every checkpoint, write and blob of the threads, in one transaction. - **`keep_latest`** keeps the latest checkpoint of each namespace (the one `get_tuple` loads), with its pending writes and the blobs it references, and deletes the rest, including blobs only the deleted checkpoints used. A kept checkpoint whose parent is deleted is left with no parent. A subgraph call's namespace goes with the root checkpoint the call started from; a `checkpointer=True` subgraph keeps its latest checkpoint, since its state outlives each call. - **Each thread is pruned on one snapshot.** Its transaction runs at `REPEATABLE READ` and deletes everything that snapshot sees except the kept checkpoints, so a run appending to the thread at the same time is safe, whatever its checkpoint IDs. - **`keep_latest` refuses threads that use a `DeltaChannel`** (`NotImplementedError` naming them, nothing pruned). A `DeltaChannel` rebuilds its latest value from older checkpoints, so deleting them would empty it. The saver can't see channel types, so it detects one by what only a `DeltaChannel` leaves: `counters_since_delta_snapshot` in metadata, a snapshot (an inline `true` with a blob behind it), or `delta_writes_versioned`, which LangGraph.js sets on threads its `DeltaChannel` graphs start. The check runs for the whole batch first and again in each thread's transaction, so a thread that starts using a `DeltaChannel` mid-batch is refused at its turn. Unlike the `prune` in #6883, `keep_latest` here deletes the blobs older checkpoints no longer share with the kept ones (most of the storage on a long thread), and guards `DeltaChannel` threads. Measured locally, with or without planner statistics: a 6,000-checkpoint thread prunes in 100 to 200 ms, and one with 1,500 finished subgraph calls in about 110 ms. ## Known limitations - Two prunes of the same thread at once can fail one of them with a serialization failure or a deadlock; retrying is safe. - Forking or replaying from a checkpoint while it's being pruned is not safe. - Threads written by older releases can escape the `DeltaChannel` check: before #9142 (Python 1.2.13) and langchain-ai/langgraphjs#2973 (JS 1.5.0), some `update_state` paths saved checkpoints with neither counters nor a snapshot. - Postgres reuses the freed space but doesn't shrink the table files; that takes `VACUUM FULL` or `pg_repack`. - `DeltaChannel`-aware pruning (keeping ancestors back to the last snapshot) is a follow-up. ## Test plan - [x] `libs/checkpoint-postgres/tests/test_prune.py`: 71 tests over the sync and async savers (single connection, pool, pipeline), including the conformance `prune` capability. Passes on Postgres 15 and 16. - [x] `libs/langgraph/tests/test_prune_postgres.py`: 39 graph tests. A pruned thread loads the same state and keeps running like an unpruned twin (sync, async and exit durability, interrupts, a failed parallel step, interrupted and `checkpointer=True` subgraphs), plus deterministic races: a turn saved mid-prune, and threads switching to a `DeltaChannel` before and after their own check. - [x] Each guard (the snapshot, the writes and blob scopes, the subgraph call rules, the parent reset, the `DeltaChannel` checks, the single-string check, the pre-v4 pending sends, the pipeline sync) broken one at a time: every break fails at least one test. - [x] A thread pruned in a loop while it runs, in each durability mode, ends identical to its twin; threads written by LangGraph.js 1.5.0 and pruned from Python behave like their twins.