Commit Graph
65 Commits
Author SHA1 Message Date
Elior Nataf LackritzandGitHub 66ebe1a0da fix(checkpoint-postgres,checkpoint-sqlite): scope namespace matching to segment boundaries (#8478)
## Summary

Namespace scoping in the Postgres and SQLite stores matched the
dot-joined prefix with `LIKE '<path>%'`, which does not respect the `.`
separator — a search scoped to `("foo",)` also returned rows under
`("foobar",)`. Scoping now matches the namespace exactly or requires the
separator before any remainder, and pattern metacharacters in labels are
escaped.

`list_namespaces` moves to segment-aware matching for prefix and suffix
conditions, since neither `LIKE` nor `GLOB` can express "any character
except the separator".

Per-package reasoning is in the commit message.

## Compatibility

`*` in a `list_namespaces` match path now spans exactly one segment,
restoring the documented behavior (`NamespacePath` documents `("cache",
"*", "v1")` as "any cache category with v1 version") and matching
`InMemoryStore`. To match at any depth, combine both conditions, which
are ANDed: `list_namespaces(prefix=["uid"], suffix=["alice"])`.

## Test plan

- [x] `make format` / `make lint` / `make test` from
`libs/checkpoint-postgres` (224 passed) and `libs/checkpoint-sqlite`
(112 passed, 3 skipped)
2026-07-30 13:52:16 -04:00
Sydney RunkleandGitHub 86baa5d08e feat(checkpoint-sqlite): override get_delta_channel_history with streaming walk (#7702)
## Summary

Adds a sqlite-specific override of
`BaseCheckpointSaver.get_delta_channel_history` (and async). Before this
PR, `SqliteSaver` / `AsyncSqliteSaver` inherited the default impl, which
calls `get_tuple` once per ancestor — N round-trips, full pending-writes
fetch per step regardless of channel relevance.

The override mirrors the postgres two-stage shape (ancestor walk +
per-channel UNION ALL writes fetch) but adapted for sqlite:

- **No JSONB** → stage 1 streams the cursor row-by-row in
`checkpoint_id` DESC order. The merged walk advances one row at a time,
deserializing only on-path checkpoints and dropping each before
advancing — peak in-flight is one deserialized checkpoint, no
`fetchall()` materialization.
- **No separate blob table** → `channel_values` lives inline in the
checkpoint blob, so seeds come back from stage 1 with no second fetch.
- **Single merged walk (not K independent walks)**: each visited cid is
deserialized exactly once, regardless of how many channels are still
seeking their seed.
- **Stage 2** stays per-channel UNION ALL to avoid over-fetching writes
when channels have different chain depths — same rationale as postgres.

`AsyncSqliteSaver.get_delta_channel_history` bridges to its async form
via `run_coroutine_threadsafe`, matching the same cross-thread guard
used by `get_tuple` / `delete_thread`.

## Tests

- New `tests/test_delta_channel_migration.py`: covers the
`BinaryOperatorAggregate -> DeltaChannel` migration path on sqlite (sync
round-trip, sync continuation with post-migration delta folding, async
round-trip). Mirrors
`libs/langgraph/tests/test_delta_channel_migration.py` (which covered
`InMemorySaver`); without these, the override's behavior on
pre-migration threads was unverified — the override has to identify a
plain accumulated `channel_values[ch]` at a pre-migration ancestor as a
valid `seed`, not just `_DeltaSnapshot` sentinels.
- Existing `tests/test_get_delta_channel_history.py` (7 tests) continues
to pass and now exercises the optimized override end-to-end (previously
hit the inherited default impl).
- `make format`, `make lint`, `make test`: clean. 97/97 in the non-flaky
sqlite suite (the one ignored test, `test_async_asearch_refresh_ttl`, is
a known TTL-store timing flake on a separate module unrelated to this
PR).

## Benchmarks

### `get_delta_channel_history` micro-bench (override vs inherited
default impl)

1000-turn synthetic threads with sentinel snapshots + per-step writes;
`bench_sqlite_delta_history.py`. Per-call latency in microseconds.

| Scenario | min | median | mean |
|---|---:|---:|---:|
| S1 single channel, root-only snapshot | **4.60x** | **4.90x** |
**5.13x** |
| S2 mixed cadence (every-50 + root-only), 2 channels | **6.08x** |
**6.37x** | **6.84x** |
| S3 K=8 channels, root-only snapshot | 1.23x | 1.27x | 0.90x |

S2 wins biggest because per-channel UNION ALL avoids over-fetching
writes for the shallow channel. S3 is the worst case for sqlite (8
channels all walking to root, 1000 deserializations either way) — the
override still wins on min/median.

### Long-running thread mem/storage bench (delta vs no-delta)

`bench_sqlite_delta_memory.py`. `delta` mode uses `DeltaChannel` + the
override; `no_delta` uses `Annotated[list, _messages_delta_reducer]`
(full state in every blob). Same workload, file-backed sqlite. Latency
measured untraced (30 iterations); peak heap measured separately under
tracemalloc.

| Scenario | Turns | Storage Δ | Peak heap Δ | Read latency Δ |
|---|---:|---|---|---|
| K=1, freq=50 | 200 | **-96%** (942 KB vs 25.1 MB) | +21% (504 KB vs
418 KB) | **+13%** |
| K=1, freq=50 | 500 | **-98%** (2.9 MB vs 152.3 MB) | +20% (1.2 MB vs
1.0 MB) | **-6%** (delta wins) |
| K=3, freq=50 uniform | 200 | **-98%** (1.7 MB vs 73.5 MB) | +7% (1.3
MB vs 1.2 MB) | **+10%** |
| K=3, freq=50 uniform | 500 | **-99%** (6.0 MB vs 452.5 MB) | +7% (3.3
MB vs 3.0 MB) | **+6%** |
| K=3, freq=mixed | 200 | **-98%** (1.4 MB vs 73.5 MB) | +5% (1.3 MB vs
1.2 MB) | +190% (5.1 ms vs 1.7 ms abs) |
| K=3, freq=mixed | 500 | **-99%** (4.1 MB vs 452.5 MB) | +8% (3.3 MB vs
3.0 MB) | +377% (20.9 ms vs 4.4 ms abs) |

- **Storage**: -96 to -99% on long threads (a 500-turn K=3 thread
shrinks from 452 MB to 6 MB on disk). This is the headline win.
- **Peak heap**: within +5 to +21% of the no-delta path — the streaming
cursor + merged walk + drop-after-deserialize keep peak in-flight at one
checkpoint at a time.
- **Read latency**: equivalent-ish (within ~15%) on uniform-cadence
scenarios; at K=1/500 turns delta even wins by 6%. The mixed-cadence
rows have one channel with `snapshot_frequency=1000` walking to root on
a 500-turn thread — by configuration. Absolute mixed-delta latency is
still 5-21 ms per read.

Bench scripts (not committed; workspace-root convention matches other
`bench_*.py` files):
- `bench_sqlite_delta_history.py`
- `bench_sqlite_delta_memory.py`

## Test plan

- [x] `cd libs/checkpoint-sqlite && make format` clean
- [x] `cd libs/checkpoint-sqlite && make lint` clean
- [x] `cd libs/checkpoint-sqlite && make test` — 97 passed (1 known
flake unrelated)
- [x] `tests/test_get_delta_channel_history.py` — 7/7 (now exercises the
override)
- [x] `tests/test_delta_channel_migration.py` — 3/3 (new)
2026-05-05 15:30:57 -04:00
William FHandGitHub 30355a7a5d fix: aiosqlite's breaking change (#6699)
`aiosqlite` changed it's Connection type to no longer subclass
`threading.Thread`. This removed the is_alive method, which is called
proactively in setup().

This PR handles this in a backwards compat way.
2026-01-18 16:35:50 -08:00
Mason DaughertyandGitHub 196fbf2631 docs: storage nits (#6651) 2026-01-05 14:24:57 -05:00
Eugene YurtsevandGitHub 297242913f fix(checkpoint-sqlite): harden (#6565)
harden
2025-12-09 16:47:55 -05:00
c5744f583b chore: Restrict "json" type deserialization (#6269)
- Rm untyped loads/dumps
- Restrict to an allow list

---------

Co-authored-by: Sydney Runkle <sydneymarierunkle@gmail.com>
2025-10-20 10:18:36 -07:00
Sydney RunkleandGitHub 2d3121a17c chore: drop Python 3.9 (and syntax) (#6289)
* `strict=False` is the default, pyupgrade to min version 3.10 adds this
to be explicit w/ behavior
2025-10-16 20:17:46 -04:00
d9e3d83894 docs: style linting (#6260)
also fixes some links

---------

Co-authored-by: Sydney Runkle <54324534+sydney-runkle@users.noreply.github.com>
Co-authored-by: ccurme <chester.curme@gmail.com>
Co-authored-by: Sydney Runkle <sydneymarierunkle@gmail.com>
Co-authored-by: William FH <13333726+hinthornw@users.noreply.github.com>
2025-10-16 11:25:50 +00:00
shaktiman101GitHubgoogle-labs-jules[bot] <161369871+google-labs-jules[bot]@users.noreply.github.com>William FHCaspar Broekhuizen
9f969f5fe1 fix(checkpoint-sqlite): Handle TTL refresh correctly in AsyncSqliteStore.asearch (#5213)
The original implementation for `refresh_on_read=True` in `asearch` for
AsyncSqliteStore used a CTE with an UPDATE statement, which is not
well-supported by SQLite in that specific construction, leading to a
syntax error.

This commit changes the approach:
1. `_prepare_batch_search_queries` in `BaseSqliteStore` no longer
constructs a CTE-based UPDATE. Instead, it returns a flag indicating if
TTL refresh is needed for the searched items.
2. `_batch_search_ops` in both `AsyncSqliteStore` and `SqliteStore` now
check this flag. If true, they perform a separate UPDATE statement after
fetching the search results to refresh the TTL of those items.

Additionally, a new test case `test_async_asearch_refresh_ttl` was added
and existing test logic was refined to accurately verify this behavior.

---------

Co-authored-by: google-labs-jules[bot] <161369871+google-labs-jules[bot]@users.noreply.github.com>
Co-authored-by: William FH <13333726+hinthornw@users.noreply.github.com>
Co-authored-by: Caspar Broekhuizen <caspar@langchain.dev>
2025-09-23 10:25:06 -07:00
8b55dff7a5 chore(deps): upgrade dependencies with uv lock --upgrade (#6146)
This PR updates the dependencies in all Python packages using `uv lock
--upgrade`.

This is an automated PR created by the UV Lock Upgrade workflow.

To make tests pass:
* linting fixes
* whitespace fixes in snapshots

---------

Co-authored-by: sydney-runkle <54324534+sydney-runkle@users.noreply.github.com>
Co-authored-by: Sydney Runkle <sydneymarierunkle@gmail.com>
2025-09-14 19:36:43 -04:00
Eugene YurtsevandGitHub bc9d45b476 fix(checkpoint-sqlite): add validation to filter keys in sql store (#5666)
This PR adds validation to keys used in filtering logic in the SQLite store implementation.
2025-07-25 13:01:13 -04:00
79b4642e55 fix(docs): broken URL in _AIO_ERROR_MSG for AsyncSqliteSaver (#5483)
remove unreachable `yield` from unimplemented async methods

---------

Co-authored-by: Nuno Campos <nuno@langchain.dev>
2025-07-17 12:26:22 +00:00
87f2e69395 chore[deps]: upgrade dependencies with uv lock --upgrade (#5358)
* chore: upgrade dependencies with `uv lock --upgrade`

* linting

* upgrade PR title

---------

Co-authored-by: sydney-runkle <54324534+sydney-runkle@users.noreply.github.com>
Co-authored-by: Sydney Runkle <sydneymarierunkle@gmail.com>
2025-07-07 23:24:32 +00:00
Nuno Campos c406aede96 Fix deadlock in SqliteStore
- If setup wasnt called separately _cursor() and setup() would deadlock
- The call to setup() in _cursor() should be outside the lock block, as setup() also acquires the lock and re-checks the setup flag
2025-06-27 08:57:35 -07:00
Nuno CamposandGitHub 1134017d07 Preparation for 0.5 release: langgraph-checkpoint (#5124)
Prepare langgraph-checkpoint for 0.5

- Given we have no upper bound on langgraph-checkpoint dep need to undo all changes in langgraph-checkpoint that might break previous versions of langgraph
2025-06-16 21:57:11 +00:00
Sydney RunkleandGitHub 5e7566f4a3 lint: use pep 604 union syntax and pep 585 generic syntax (#4963)
* new union syntax

* fix test

* second round of conversions by injecting future annotations

* format + add top level makefile
2025-06-04 21:50:16 -04:00
Nuno Campos 4e8fbe4525 Remove Checkpoint.pending_sends
- Instead store sends in a Topic channel, removing the need to fetch sends as writes against the parent checkpoint
- Remove deprecated/unused functions in langgraph-checkpoint (will require bumping min range for langgraph-checkpoint in langgraph lib)
- Implement migration of old pending sends in langgraph-checkpoint-postgres
- Ensure parent config of `checkpoint_during=False` checkpoints always points to checkpoints that were also saved
2025-05-25 19:06:02 -07:00
William FHandGitHub 95f92069a7 sqlite: Add test for search with list filters (#4747) 2025-05-18 23:50:27 -07:00
William FHandGitHub 6b28319796 sqlite: update list_namespaces with max_depth (#4746)
sqlite: update on conflict
2025-05-18 23:27:06 -07:00
William FHandGitHub 025b634d98 SqliteStore (#3608) 2025-05-17 21:40:19 -07:00
Sydney RunkleandGitHub 228a08b966 ci: migrate to uv! (#4698)
* Migrate to `uv`
* Format `pyproject.toml` files properly
* Remove upper bounds on dependencies, and bounds on dev dependencies
(we should be using latest)
* Move to hatch for packaing

In the future we should:
* Set up dependabot / automate lockfile updates and tests
* Add tests for min compatible versions (I'll do this right after merge)
* Use dynamic versioning
* Bump `pydantic` to v2.11.4 in the lockfile, we have some tests failing
2025-05-15 17:39:14 -07:00
Nuno Campos 562d64bbb7 Lint 2025-05-09 12:07:59 -07:00
Nuno Campos 898f266f72 Overload clear method to delete all when called without args 2025-05-09 12:05:26 -07:00
Nuno Campos 83d2f93566 Lint 2025-05-08 16:57:21 -07:00
Nuno Campos 1a6395fd07 Move FileCache to sqlite package, add InMemoryCache 2025-05-08 16:50:36 -07:00
Sydney Runkle 6c6bfbc63a docstrings for checkpoint-sqlite 2025-05-01 13:40:21 -04:00
18a9ae45f3 Add delete_thread method to Checkpointer class (#4328)
- Deletes all data associated with a thread_id
- Implemented in InMemory, Sqlite and Postgres checkpointers

Co-authored-by: Eugene Yurtsev <eyurtsev@gmail.com>
2025-04-17 16:38:58 +00:00
11c71fef89 chore(docs): Improve documentation for AsyncSqliteSaver (#3858)
**Description:**
Make AsyncSqliteSaver examples workable.

**Issue:**
For "Usage within StateGraph" example,
SyntaxError: 'async with' outside async function

For "Raw usage" example
KeyError: 'checkpoint_ns' and KeyError: 'id'

**Dependencies:**
N/A

Co-authored-by: Eugene Yurtsev <eyurtsev@gmail.com>
2025-03-21 15:50:51 -04:00
8d33938173 Improve documentation string for SqliteSaver (#3857)
* Document that `check_same_thread` as an option when creating sqlite
connection.
* Document why it's OK to do that.

---------

Co-authored-by: Eugene Yurtsev <eyurtsev@gmail.com>
2025-03-18 21:29:05 +00:00
Vadym BardaandGitHub 9bf3fc2d0f checkpoint-sqlite: commit transactions in AsyncSqliteSaver.aput_writes (#3762) 2025-03-10 15:14:20 +00:00
Nuno Campos 7f4822931e Update tests 2025-02-14 18:24:48 -08:00
Nuno Campos 9706211aca Exclude complex values from checkpoint metadata 2025-02-14 17:35:14 -08:00
Vadym BardaandGitHub 1377e3b6ba checkpoint: combine metadata when writing checkpoints (#3404) 2025-02-13 03:24:41 +00:00
Nuno Campos 31734fb792 Lint 2025-01-15 11:02:10 -08:00
Nuno Campos eed577ee2a Update sqlite signature 2025-01-15 10:58:46 -08:00
Vadym BardaandGitHub b989502c24 checkpoint-sqlite/postgres: handle calling .list on async checkpointer (#3019) 2025-01-14 19:06:11 +00:00
Phoenix LoganandGitHub aca67107c1 fix: make database saver classes inheritance-friendly (#2615)
Replace hardcoded database saver class names with `cls` in
`from_conn_string` factory methods to improve subclassing support

## Changes
* Replaced direct class instantiations with `cls(conn)` in
`from_conn_string` classmethods across all database implementations
* Updated both synchronous and asynchronous variants for DuckDB,
PostgreSQL, and SQLite savers

## Why
This refactor makes the database saver classes more extensible by
following Python's convention of using `cls` in class methods. This
enables proper inheritance patterns where subclasses can reuse the
factory methods without needing to override them. Previously, the
hardcoded class names would always instantiate the parent class, even
when called from a subclass.

## Testing
The change is backward compatible and doesn't alter existing
functionality. All existing tests should continue to pass as this is
purely a structural refactoring that preserves the current behavior
while improving extensibility.

## Notes
This PR addresses follow up on comments from #2518 - AsyncPostgresSaver
didn't need to be fixed but many of the other DB saver classes did.
2024-12-03 20:26:06 -08:00
Vadym BardaandGitHub def3e06b4a move py.typed to submodules for namespace packages (#2177) 2024-10-24 13:43:43 -04:00
Vadym BardaandGitHub d32386f849 checkpoint: add DuckDB checkpointer (#2145) 2024-10-23 21:11:03 +00:00
Vadym BardaandGitHub 03be3b6567 docs: reorganize API refs (#1884) 2024-09-27 18:12:12 -04:00
Nuno Campos e71c24ea9b Raise exception if sync method on async checkpointer is called from main thread
- The thread running the event loop can block waiting for a coro to run, only background threads can
2024-09-23 08:41:57 -07:00
Nuno Campos 9a8cc75ea2 Fix some typing issues in langgraph lib 2024-09-19 11:38:41 -07:00
Nuno Campos b529365f5b Remove ignore for 3.9 2024-09-19 09:19:26 -07:00
Nuno Campos 95304d658a Fix 3.9 2024-09-19 08:51:15 -07:00
Nuno Campos 6c0339ca4b ci: Enable mypy checks for checkpoint-sqlite lib 2024-09-19 08:46:44 -07:00
Nuno Campos efb572f15b Add missing branch 2024-09-18 13:18:37 -07:00
Nuno CamposandGitHub 66fc7c96aa More performance improvements in checkpointing and channels (#1685)
* Performance improvements in checkpointer libs

- Use sha1 instead of md5 for hashing (faster in python 3.x)
- Use orjson instead of json for json dumping (sadly can't use for json loading)

* Update tests

* Update

* Use random number instead of hash for get_version_number

* Avoid saving writes for the last task to complete in each step

- only when possible, exceptions for ERROR, INTERRUPT, SEND

* Make Channel.from_checkpoint a regular function

- context manager no longer needed since Context became a managed value

* Use __slots__ for Channels

* Fix for kafka
2024-09-12 21:08:30 -07:00
Nuno Campos 270559b880 Implement changes in sqlite checkpointer 2024-09-10 16:17:22 -07:00
Nuno CamposandNuno Campos 4793b3f5e1 Use a lock for all operations on sqlite checkpointer
- Otherwise when used in multiple subgraphs in parallel separate queries can interfere w each other
2024-09-03 09:20:17 -07:00
Vadym BardaandGitHub d492475c61 checkpoint-sqlite: close connection in from_conn_string (#1589) 2024-09-03 11:00:37 -04:00