Commit Graph
66 Commits
Author SHA1 Message Date
Nuno Campos cb7b667e6f Update psycopg 2024-10-10 16:09:24 -07:00
William FHandGitHub 254b12a62d Use AsyncBatch for postgres store (#2020) 2024-10-08 06:58:26 +00:00
William FHandGitHub 6c0da426c6 [PostGres Checkpointer] Run CI on PG15 as well (#1953) 2024-10-02 19:16:58 +00:00
William FHandGitHub 5c3ac5d16d Update PG Implementation (#1948) 2024-10-01 13:11:19 -07:00
Vadym BardaandGitHub 4a45f6c99a langgraph, sqlite, postgres: update to use langgraph-checkpoint==2.0.0 (#1946) 2024-10-01 15:28:06 -04:00
William FHandGitHub dd88ac6224 Bump SDK Py (#1935) 2024-10-01 08:17:51 +00:00
William FHandGitHub 29f58fd9e5 Custom loads support in postgres checkpointer (#1930) 2024-09-30 16:42:23 -07:00
William FHandGitHub 97f79fc66b Add Postgres Store Implementation (#1906) 2024-09-30 21:18:58 +00:00
Vadym BardaandGitHub ea0418334b checkpoint-postgres: release 1.0.9 (#1890) 2024-09-27 16:12:06 -04:00
Vadym BardaandGitHub bf19dc7d08 checkpoint-postgres: handle null chars in metadata (#1885) 2024-09-27 16:02:03 +00:00
Vadym BardaandGitHub 659d127d2b checkpoint postgres,sqlite: update langgraph-checkpoint to 1.0.11 (#1815) 2024-09-23 22:09:28 +00:00
Vadym BardaandGitHub 7d513bcde2 checkpoint postgres,sqlite: update langgraph-checkpoint (#1813) 2024-09-23 21:47:26 +00:00
Vadym BardaandGitHub 515c619bc2 checkpoint postgres,sqlite: update version (#1812) 2024-09-23 17:33:19 -04:00
Nuno Campos e71c24ea9b Raise exception if sync method on async checkpointer is called from main thread
- The thread running the event loop can block waiting for a coro to run, only background threads can
2024-09-23 08:41:57 -07:00
Nuno CamposandGitHub 0bbe461785 Merge pull request #1771 from langchain-ai/nc/19sep/mypy-langgraph-partial
Enable mypy for langgraph lib
2024-09-19 16:53:42 -07:00
Nuno Campos cc318b1156 Fix 2024-09-19 11:43:30 -07:00
Nuno Campos fdf7bad988 Remove ignore for 3.9 2024-09-19 09:19:00 -07:00
Nuno Campos 495d258322 Fix 3.9 2024-09-19 08:50:17 -07:00
Nuno Campos b8a8651c23 ci: Enable mypy checks for checkpoint-postgres lib 2024-09-19 08:40:31 -07:00
Nuno Campos 4d0c12117b postgres 1.0.7 2024-09-16 14:10:19 -07:00
Nuno CamposandGitHub 3b05279e32 Implement serialization with msgpack library (#1716)
* Implement serialization with msgpack library

- encode custom python objects with a msgpack extension type, with constructor path string, and args encoded as nested msgpack doc

* Smaller msgpack extension types

* Update lock files

* lock

* Don't delegate to pydantic json

* Fix kafka serde

- should use our serializer to load, as inputs to subgraphs are serialized using it
2024-09-16 14:06:42 -07:00
c3b4acf3f1 checkpoint postgres: allow passing custom serde (#1699)
* Use correct serde in postgres checkpoint

* Revert metadata serde

* Revert

---------

Co-authored-by: Vadym Barda <vadym@langchain.dev>
2024-09-12 21:09:23 -07:00
Nuno CamposandGitHub 66fc7c96aa More performance improvements in checkpointing and channels (#1685)
* Performance improvements in checkpointer libs

- Use sha1 instead of md5 for hashing (faster in python 3.x)
- Use orjson instead of json for json dumping (sadly can't use for json loading)

* Update tests

* Update

* Use random number instead of hash for get_version_number

* Avoid saving writes for the last task to complete in each step

- only when possible, exceptions for ERROR, INTERRUPT, SEND

* Make Channel.from_checkpoint a regular function

- context manager no longer needed since Context became a managed value

* Use __slots__ for Channels

* Fix for kafka
2024-09-12 21:08:30 -07:00
Nuno Campos 270559b880 Implement changes in sqlite checkpointer 2024-09-10 16:17:22 -07:00
Nuno Campos 8f91402642 Use thread_id for partition key, ignore tasks for stale checkpoints 2024-09-10 16:17:22 -07:00
Nuno Campos b13db42568 WIP Subgraphs 2024-09-10 16:17:22 -07:00
Nuno Campos 6e9f54e5ae postgres 1.0.6 2024-09-02 17:57:55 -07:00
Nuno Campos dd4a15b569 Require latest version of langgraph-checkpoint 2024-09-02 17:57:33 -07:00
Nuno Campos 3b2a237deb Lint 2024-09-02 17:45:57 -07:00
Nuno Campos 0cc3799298 checkpoint: Implement sync methods in async checkpointers
- These are needed when subgraphs are invoked synchronously in nodes
2024-09-02 17:43:34 -07:00
Nuno Campos 3943637202 Bump them all 2024-08-30 14:18:35 -07:00
Nuno Campos e744db169a postgres/sqlite: Sort pending writes when fetching from db 2024-08-30 14:07:46 -07:00
Nuno Campos e338a4ec98 Fix filtering on ns 2024-08-27 16:09:11 -07:00
Nuno Campos 9b6100bc4f Avoid storing Sends twice in memory and postgres checkpointers
- Sends are stored through put_writes, so we don't need to also store them inside checkpoint object
- On reading checkpoint, reconstruct pending_sends from the stored writes
2024-08-27 11:10:18 -07:00
Nuno Campos 330c91f622 Update ruff 2024-08-27 10:50:39 -07:00
Vadym BardaandGitHub c88d289a73 checkpoint-postgres: release 1.0.4 (#1493) 2024-08-27 11:34:26 -04:00
Vadym BardaandGitHub d12f5c6d8b checkpoint-postgres: allow passing pool (#1452)
* checkpoint-postgres: allow passing pool

* make psycopg_pool a non-dev dependency

* code review

* lockfile

* move methods

* relax requirements, remove binary

* add binary to dev dependencies

* update readme
2024-08-27 15:30:13 +00:00
vbarda 85e698e20b filter on checkpoint NS 2024-08-26 19:26:35 -04:00
Vadym BardaandGitHub 4c4d7054ab Merge branch 'main' into vb/update-get-state 2024-08-23 14:27:40 -04:00
vbarda 4162be8119 optimize subgraph state lookups 2024-08-22 20:35:08 -04:00
Vadym BardaandGitHub 8090ca67c5 checkpoint-postgres: pass row_factory in cursor (#1433) 2024-08-22 17:39:21 -04:00
Nuno CamposandGitHub 7e32de9405 Remove current_tasks from checkpoint interface (#1440)
* Remove current_tasks from checkpoint interface

- Not used, now clear that it can be supported with put_writes(SCHEDULE)

* Add comment
2024-08-22 19:44:26 +00:00
vbarda 5654d8fc86 Merge branch 'main' into vb/update-get-state 2024-08-22 15:34:14 -04:00
Nuno CamposandGitHub a261e1a497 Fix semantics of put_writes/list (#1436)
* Fix semantics of put_writes/list

- put_writes(error) should not prevent saving future successful if task is retried successfully
- put_writes(writes) should be a no-op if non-error writes already exist for that task (this prevents tasks executed more than once from modifying writes previously saved / acted on)
- checkpoints should not include channel default values (ie those without a version)
- list() should fetch and return writes for each checkpoint

* Lint

* Rm print

* Fix import

* Lint
2024-08-22 19:04:42 +00:00
vbarda f51e7ea9a4 pass pending writes in checkpointers 2024-08-21 15:38:30 -04:00
vbarda a94168af7f Merge branch 'main' into vb/update-get-state 2024-08-21 15:07:52 -04:00
Nuno Campos 9cfb4b01c8 postgres: Fix args to put_writes 2024-08-19 15:54:49 -07:00
Nuno Campos c0c534cbe3 Add pipeline arg 2024-08-19 15:44:22 -07:00
Nuno Campos 7aaeedd7ba checkpoint-*: In put_writes clear any previously saved writes for this task_id
- Previously implementations were clearing only tasks if the index matched a previously saved one, which isn't enough to guarantee idempotency
2024-08-19 15:40:04 -07:00
vbarda 5033044587 update checkpointer tests 2024-08-12 16:22:27 -04:00