Compare commits

..
Author SHA1 Message Date
Nuno Campos 1b9b0a686e Remove more mentions of async 2025-03-01 17:25:58 -08:00
Nuno Campos 9e02a23682 Rm other mentions of stream_mode=messages 2025-03-01 14:20:15 -08:00
Nuno Campos 5e70e6f307 Rm docs 2025-03-01 14:10:02 -08:00
Nuno Campos eb57c06896 Remove features and dependencies
- rm langchain_core dependency
- replace callbacks w run tree
- rm Runnable dependency
- rm non-state Graph
- rm managed values
- rm entrypoint/task/call
- rm async methods
- rm shallow checkpointer
- rm messages stream mode
- rm debug flag
- rm remote graph
2025-03-01 13:53:27 -08:00
Nuno Campos 9284b57ba0 Remove prebuilt 2025-03-01 10:30:18 -08:00
Nuno Campos 25fea591b5 Remove sqlite 2025-03-01 10:14:10 -08:00
Nuno Campos b9fe53777f Remove cli 2025-03-01 10:13:58 -08:00
Nuno Campos 35c2e8a679 Remove kafka 2025-03-01 10:13:47 -08:00
Nuno Campos e0fb56c6a3 Remove sdks 2025-03-01 10:13:36 -08:00
Nuno Campos 3458a3cecb Remove examples 2025-03-01 10:13:24 -08:00
407 changed files with 22926 additions and 116070 deletions

No files matched your search

-6
View File
@@ -1,6 +0,0 @@
# Contributing to LangGraph
Hi there! Thank you for even being interested in contributing to LangGraph.
As an open-source project in a rapidly developing field, we are extremely open to contributions, whether they involve new features, improved infrastructure, better documentation, or bug fixes.
To learn how to contribute to LangGraph, please follow the [contribution guide here](https://docs.langchain.com/oss/python/contributing).
+15 -14
View File
@@ -1,28 +1,29 @@
name: "\U0001F41B Bug Report"
description: Report a bug in LangGraph. To report a security issue, please instead use the security option below. For questions, please use the LangChain Forum at forum.langchain.com.
labels: [pending, bug]
description: Report a bug in LangGraph. To report a security issue, please instead use the security option below. For questions, please use the GitHub Discussions.
labels: ["02 Bug Report"]
body:
- type: markdown
attributes:
value: |
value: >
Thank you for taking the time to file a bug report.
Use this to report BUGS in LangGraph. For usage questions, feature requests and general design questions, please use the [LangChain Forum](https://forum.langchain.com/).
Use this to report BUGS in LangGraph. For usage questions, feature requests and general design questions, please use [GitHub Discussions](https://github.com/langchain-ai/langgraph/discussions).
Relevant links to check before filing a bug report to see if your issue has already been reported, fixed or
if there's another way to solve your problem:
* [LangChain Forum](https://forum.langchain.com/),
* [LangGraph Github Issues](https://github.com/langchain-ai/langgraph/issues),
* [LangChain documentation with the integrated search](https://docs.langchain.com/),
* [GitHub search](https://github.com/langchain-ai/langgraph),
[LangGraph Github Discussions](https://github.com/langchain-ai/langgraph/discussions),
[LangGraph Github Issues](https://github.com/langchain-ai/langgraph/issues),
[LangGraph how-to guides](https://langchain-ai.github.io/langgraph/how-tos/).
[LangChain documentation with the integrated search](https://python.langchain.com/docs/get_started/introduction),
[GitHub search](https://github.com/langchain-ai/langgraph),
- type: checkboxes
id: checks
attributes:
label: Checked other resources
description: Before submitting this issue, please confirm that you have completed all the steps below by checking each option. These steps help ensure your issue is well-defined, relevant, and actionable.
options:
- label: This is a bug, not a usage question. For questions, please use the LangChain Forum (https://forum.langchain.com/).
- label: This is a bug, not a usage question. For questions, please use GitHub Discussions.
required: true
- label: I added a clear and detailed title that summarizes the issue.
required: true
@@ -37,7 +38,7 @@ body:
attributes:
label: Example Code
description: |
Please add a self-contained, [minimal, reproducible, example](https://stackoverflow.com/help/minimal-reproducible-example) with your use case. Replace this code with your own!
Please add a self-contained, [minimal, reproducible, example](https://stackoverflow.com/help/minimal-reproducible-example) with your use case.
placeholder: |
from langgraph.graph import StateGraph
@@ -77,7 +78,7 @@ body:
attributes:
label: System Info
description: |
Run on your machine: `python -m langchain_core.sys_info`
python -m langchain_core.sys_info
placeholder: |
python -m langchain_core.sys_info
validations:
+12 -6
View File
@@ -1,9 +1,15 @@
blank_issues_enabled: false
version: 2.1
contact_links:
- name: Documentation
url: https://github.com/langchain-ai/docs/issues/new?template=langgraph.yml
about: Report an issue related to the LangGraph documentation
- name: LangChain Forum
url: https://forum.langchain.com/
about: General community discussions and support
- name: 🤔 Question or Problem
about: Ask a question or ask about a problem in GitHub Discussions.
url: https://github.com/langchain-ai/langgraph/discussions/categories/q-a
- name: Feature Request
url: https://github.com/langchain-ai/langgraph/discussions/categories/ideas
about: Suggest a feature or an idea
- name: Show and tell
about: Show what you built with LangChain
url: https://github.com/langchain-ai/langgraph/discussions/categories/show-and-tell
- name: Slack
url: https://www.langchain.com/join-community
about: General community discussions
+19
View File
@@ -0,0 +1,19 @@
name: Documentation
description: Report an issue related to the LangGraph documentation.
title: "DOC: <Please write a comprehensive title after the 'DOC: ' prefix>"
labels: [03 - Documentation]
body:
- type: textarea
attributes:
label: "Issue with current documentation:"
description: >
Please make sure to leave a reference to the document/code you're
referring to.
- type: textarea
attributes:
label: "Idea or request for content:"
description: >
Please describe as clearly as possible what topics you think are missing
from the current documentation.
+8 -12
View File
@@ -1,29 +1,25 @@
name: 🔒 Privileged
description: You are a LangGraph maintainer, or was asked directly by a maintainer to create an issue here. If not, check the other options.
description: You are a LangChain maintainer, or was asked directly by a maintainer to create an issue here. If not, check the other options.
body:
- type: markdown
attributes:
value: |
Thanks for your interest in LangGraph! 🚀
If you are not a LangGraph maintainer or were not asked directly by a maintainer to create an issue, then please start the conversation on the [LangChain Forum](https://forum.langchain.com/) instead.
You are a LangGraph maintainer if you maintain any of the packages inside of the LangGraph repository
or are a regular contributor to LangGraph with previous merged merged pull requests.
Thanks for your interest in LangChain! 🚀
If you are not a LangChain maintainer or were not asked directly by a maintainer to create an issue, then please start the conversation in a [Question in GitHub Discussions](https://github.com/langchain-ai/langchain/discussions/categories/q-a) instead.
You are a LangChain maintainer if you maintain any of the packages inside of the LangChain repository
or are a regular contributor to LangChain with previous merged merged pull requests.
- type: checkboxes
id: privileged
attributes:
label: Privileged issue
description: Confirm that you are allowed to create an issue here.
options:
- label: I am a LangGraph maintainer, or was asked directly by a LangGraph maintainer to create an issue here.
- label: I am a LangChain maintainer, or was asked directly by a LangChain maintainer to create an issue here.
required: true
- type: textarea
id: content
attributes:
label: Issue Content
description: Add the content of the issue here.
- type: markdown
attributes:
value: |
Community members should **NOT** work on Privileged issues unless these issues have been explicitly marked with a "help-wanted" tag.
-31
View File
@@ -1,31 +0,0 @@
Thank you for contributing to LangGraph! Follow these steps to mark your pull request as ready for review. **If any of these steps are not completed, your PR will not be considered for review.**
- [ ] **PR title**: Follows the format: {TYPE}({SCOPE}): {DESCRIPTION}
- Examples:
- feat(core): add multi-tenant support
- fix(cli): resolve flag parsing error
- docs(openai): update API usage examples
- Allowed `{TYPE}` values:
- feat, fix, docs, style, refactor, perf, test, build, ci, chore, revert, release
- Allowed `{SCOPE}` values (optional):
- langgraph, docs, cli, checkpoint, checkpoint-postgres, checkpoint-sqlite, prebuilt, scheduler-kafka, sdk-py
- Once you've written the title, please delete this checklist item; do not include it in the PR.
- [ ] **PR message**: ***Delete this entire checklist*** and replace with
- **Description:** a description of the change. Include a [closing keyword](https://docs.github.com/en/issues/tracking-your-work-with-issues/using-issues/linking-a-pull-request-to-an-issue#linking-a-pull-request-to-an-issue-using-a-keyword) if applicable.
- **Issue:** the issue # it fixes, if applicable
- **Dependencies:** any dependencies required for this change
- **Twitter handle:** if your PR gets announced, and you'd like a mention, we'll gladly shout you out!
- [ ] **Add tests and docs**: If you're adding a new integration, you must include:
1. A test for the integration, preferably unit tests that do not rely on network access,
2. An example notebook showing its use. It lives in `docs/docs/integrations` directory.
- [ ] **Lint and test**: Run `make format`, `make lint` and `make test` from the root of the package(s) you've modified. We will not consider a PR unless these three are passing in CI. See [contribution guidelines](https://github.com/langchain-ai/langgraph/blob/main/CONTRIBUTING.md) for more.
Additional guidelines:
- Make sure optional dependencies are imported within a function.
- Please do not add dependencies to `pyproject.toml` files (even optional ones) unless they are **required** for unit tests.
- Most PRs should not touch more than one package.
- Changes should be backwards compatible.
+88
View File
@@ -0,0 +1,88 @@
# An action for setting up poetry install with caching.
# Using a custom action since the default action does not
# take poetry install groups into account.
# Action code from:
# https://github.com/actions/setup-python/issues/505#issuecomment-1273013236
name: poetry-install-with-caching
description: Poetry install with support for caching of dependency groups.
inputs:
python-version:
description: Python version, supporting MAJOR.MINOR only
required: true
poetry-version:
description: Poetry version
required: true
cache-key:
description: Cache key to use for manual handling of caching
required: true
runs:
using: composite
steps:
- uses: actions/setup-python@v5
name: Setup python ${{ inputs.python-version }}
id: setup-python
with:
python-version: ${{ inputs.python-version }}
- uses: actions/cache@v3
id: cache-bin-poetry
name: Cache Poetry binary - Python ${{ inputs.python-version }}
env:
SEGMENT_DOWNLOAD_TIMEOUT_MIN: "1"
with:
path: |
/opt/pipx/venvs/poetry
# This step caches the poetry installation, so make sure it's keyed on the poetry version as well.
key: bin-poetry-${{ runner.os }}-${{ runner.arch }}-py-${{ inputs.python-version }}-${{ inputs.poetry-version }}
- name: Refresh shell hashtable and fixup softlinks
if: steps.cache-bin-poetry.outputs.cache-hit == 'true'
shell: bash
env:
POETRY_VERSION: ${{ inputs.poetry-version }}
PYTHON_VERSION: ${{ inputs.python-version }}
run: |
set -eux
# Refresh the shell hashtable, to ensure correct `which` output.
hash -r
# `actions/cache@v3` doesn't always seem able to correctly unpack softlinks.
# Delete and recreate the softlinks pipx expects to have.
rm /opt/pipx/venvs/poetry/bin/python
cd /opt/pipx/venvs/poetry/bin
ln -s "$(which "python$PYTHON_VERSION")" python
chmod +x python
cd /opt/pipx_bin/
ln -s /opt/pipx/venvs/poetry/bin/poetry poetry
chmod +x poetry
# Ensure everything got set up correctly.
/opt/pipx/venvs/poetry/bin/python --version
/opt/pipx_bin/poetry --version
- name: Install poetry
if: steps.cache-bin-poetry.outputs.cache-hit != 'true'
shell: bash
env:
POETRY_VERSION: ${{ inputs.poetry-version }}
PYTHON_VERSION: ${{ inputs.python-version }}
# Install poetry using the python version installed by setup-python step.
run: pipx install "poetry==$POETRY_VERSION" --python '${{ steps.setup-python.outputs.python-path }}' --verbose
- name: Restore pip and poetry cached dependencies
uses: actions/cache@v3
env:
SEGMENT_DOWNLOAD_TIMEOUT_MIN: "4"
with:
path: |
~/.cache/pip
~/.cache/pypoetry/virtualenvs
~/.cache/pypoetry/cache
~/.cache/pypoetry/artifacts
./.venv
key: py-deps-${{ runner.os }}-${{ runner.arch }}-py-${{ inputs.python-version }}-poetry-${{ inputs.poetry-version }}-${{ inputs.cache-key }}-${{ hashFiles('./poetry.lock') }}
-18
View File
@@ -1,18 +0,0 @@
version: 2
updates:
- package-ecosystem: "github-actions"
directory: "/"
schedule:
interval: "weekly"
- package-ecosystem: "pip"
directories:
- "libs/checkpoint"
- "libs/checkpoint-postgres"
- "libs/checkpoint-sqlite"
- "libs/cli"
- "libs/langgraph"
- "libs/prebuilt"
- "libs/sdk-py"
schedule:
interval: "weekly"
+3 -8
View File
@@ -1,15 +1,10 @@
import ast
import os
from itertools import filterfalse
from typing import Dict, List, Tuple
from typing import List, Tuple
ROOT_PATH = os.path.abspath(os.path.join(__file__, "..", "..", ".."))
CLIENT_PATH = os.path.join(ROOT_PATH, "libs", "sdk-py", "langgraph_sdk", "client.py")
ASYNC_TO_SYNC_METHOD_MAP: Dict[str, str] = {
"aclose": "close",
"__aenter__": "__enter__",
"__aexit__": "__exit__",
}
def get_class_methods(node: ast.ClassDef) -> List[str]:
@@ -27,7 +22,7 @@ def find_classes(tree: ast.AST) -> List[Tuple[str, List[str]]]:
def compare_sync_async_methods(sync_methods: List[str], async_methods: List[str]) -> List[str]:
sync_set = set(sync_methods)
async_set = {ASYNC_TO_SYNC_METHOD_MAP.get(async_method, async_method) for async_method in async_methods}
async_set = set(async_methods)
missing_in_sync = list(async_set - sync_set)
missing_in_async = list(sync_set - async_set)
return missing_in_sync + missing_in_async
@@ -38,7 +33,7 @@ def main():
tree = ast.parse(file.read())
classes = find_classes(tree)
def is_sync(class_spec: Tuple[str, List[str]]) -> bool:
return class_spec[0].startswith("Sync")
+84 -146
View File
@@ -1,164 +1,108 @@
import logging
import asyncio
import json
import os
import pathlib
import sys
import time
from urllib import error, request
import langgraph_cli
import langgraph_cli.config
import langgraph_cli.docker
from langgraph_cli.cli import prepare_args_and_stdin
from langgraph_cli.constants import DEFAULT_PORT
import langgraph_cli.config
from langgraph_cli.exec import Runner, subp_exec
from langgraph_cli.progress import Progress
logger = logging.getLogger(__name__)
logging.basicConfig(level=logging.INFO)
from langgraph_cli.constants import DEFAULT_PORT
def test(config: pathlib.Path, port: int, tag: str, verbose: bool):
"""Spin up API with Postgres/Redis via docker compose and wait until ready."""
logger.info("Starting test...")
def test(
config: pathlib.Path,
port: int,
tag: str,
verbose: bool,
):
with Runner() as runner, Progress(message="Pulling...") as set:
# Detect docker/compose capabilities
# check docker available
capabilities = langgraph_cli.docker.check_capabilities(runner)
# Validate config and prepare compose stdin/args using built image
# open config
config_json = langgraph_cli.config.validate_config_file(config)
args, stdin = prepare_args_and_stdin(
capabilities=capabilities,
config_path=config,
config=config_json,
docker_compose=None,
port=port,
watch=False,
debugger_port=None,
debugger_base_url=f"http://127.0.0.1:{port}",
postgres_uri=None,
api_version=None,
image=tag,
base_image=None,
)
# Compose up with wait (implies detach), similar to `langgraph up --wait`
args_up = [*args, "up", "--remove-orphans", "--wait"]
compose_cmd = ["docker", "compose"]
if capabilities.compose_type == "standalone":
compose_cmd = ["docker-compose"]
set("Starting...")
try:
runner.run(
subp_exec(
*compose_cmd,
*args_up,
input=stdin,
verbose=verbose,
)
set("Running...")
args = [
"run",
"--rm",
"-p",
f"{port}:8000",
]
if isinstance(config_json["env"], str):
args.extend(
[
"--env-file",
str(config.parent / config_json["env"]),
]
)
except Exception as e: # noqa: BLE001
# On failure, show diagnostics then ensure clean teardown
sys.stderr.write(f"docker compose up failed: {e}\n")
try:
sys.stderr.write("\n== docker compose ps ==\n")
runner.run(
subp_exec(*compose_cmd, *args, "ps", input=stdin, verbose=False)
)
except Exception:
pass
try:
sys.stderr.write("\n== docker compose logs (api) ==\n")
runner.run(
subp_exec(
*compose_cmd,
*args,
"logs",
"langgraph-api",
input=stdin,
verbose=False,
)
)
except Exception:
pass
finally:
try:
runner.run(
subp_exec(
*compose_cmd,
*args,
"down",
"-v",
"--remove-orphans",
input=stdin,
verbose=False,
)
)
finally:
raise
set("")
base_url = f"http://localhost:{port}"
ok_url = f"{base_url}/ok"
logger.info(f"Waiting for {ok_url} to respond with 200...")
deadline = time.time() + 30
last_err: Exception | None = None
while time.time() < deadline:
try:
with request.urlopen(ok_url, timeout=2) as resp:
if resp.status == 200:
sys.stdout.write(
f"""Ready!\n- API: {base_url}\n- /ok: 200 OK\n"""
)
sys.stdout.flush()
break
else:
last_err = RuntimeError(f"Unexpected status: {resp.status}")
logger.error(f"Unexpected status: {resp.status}")
except error.URLError as e:
logger.error(f"URLError: {e}")
last_err = e
except Exception as e: # noqa: BLE001
logger.error(f"Exception: {e}")
last_err = e
time.sleep(0.5)
else:
logger.error("Timeout waiting for /ok to return 200")
# Bring stack down before raising
args_down = [*args, "down", "-v", "--remove-orphans"]
try:
runner.run(
subp_exec(
*compose_cmd,
*args_down,
input=stdin,
verbose=verbose,
)
)
finally:
raise SystemExit(
f"/ok did not return 202 within timeout. Last error: {last_err}"
for k, v in config_json["env"].items():
args.extend(
[
"-e",
f"{k}={v}",
]
)
if capabilities.healthcheck_start_interval:
args.extend(
[
"--health-interval",
"5s",
"--health-retries",
"1",
"--health-start-period",
"10s",
"--health-start-interval",
"1s",
]
)
else:
args.extend(
[
"--health-interval",
"5s",
"--health-retries",
"2",
]
)
_task = None
def on_stdout(line: str):
nonlocal _task
if "GET /ok" in line or "Uvicorn running on" in line:
set("")
sys.stdout.write(
f"""Ready!
- API: http://localhost:{port}
"""
)
sys.stdout.flush()
_task.cancel()
return True
return False
async def subp_exec_task(*args, **kwargs):
nonlocal _task
_task = asyncio.create_task(subp_exec(*args, **kwargs))
await _task
# Clean up: bring compose stack down to free ports for next test
logger.info("Test succeeded. Bringing down compose stack...")
try:
args_down = [*args, "down", "-v", "--remove-orphans"]
runner.run(
subp_exec(
*compose_cmd,
*args_down,
input=stdin,
subp_exec_task(
"docker",
*args,
tag,
verbose=verbose,
on_stdout=on_stdout,
)
)
logger.info("Compose stack down. Finishing...")
except Exception:
logger.exception("Failed to bring down compose stack")
except asyncio.CancelledError:
pass
logger.info("Test finished")
if __name__ == "__main__":
import argparse
@@ -166,12 +110,6 @@ if __name__ == "__main__":
parser = argparse.ArgumentParser()
parser.add_argument("-t", "--tag", type=str)
parser.add_argument("-c", "--config", type=str, default="./langgraph.json")
parser.add_argument("-p", "--port", type=int, default=DEFAULT_PORT)
parser.add_argument("-p", "--port", default=DEFAULT_PORT)
args = parser.parse_args()
try:
test(pathlib.Path(args.config), args.port, args.tag, verbose=True)
except BaseException:
logger.exception("Test failed")
raise
logger.info("Test execution finished")
test(pathlib.Path(args.config), args.port, args.tag, verbose=True)
+38 -82
View File
@@ -3,8 +3,8 @@ name: CLI integration test
on:
workflow_call:
permissions:
contents: read
env:
POETRY_VERSION: "1.7.1"
jobs:
build:
@@ -13,106 +13,62 @@ jobs:
matrix:
python-version:
- "3.10"
- "3.14"
example:
- name: A
workdir: libs/cli/examples
tag: langgraph-test-a
- name: B
workdir: libs/cli/examples/graphs
tag: langgraph-test-b
- name: C
workdir: libs/cli/examples/graphs_reqs_a
tag: langgraph-test-c
- name: D
workdir: libs/cli/examples/graphs_reqs_b
tag: langgraph-test-d
- "3.11"
name: "CLI integration test"
defaults:
run:
working-directory: libs/cli
steps:
- uses: actions/checkout@v6
- uses: actions/checkout@v4
- name: Get changed files
id: changed-files
uses: Ana06/get-changed-files@v2.3.0
with:
filter: "libs/cli/**"
- name: Set up Python ${{ matrix.python-version }}
- name: Set up Python ${{ matrix.python-version }} + Poetry ${{ env.POETRY_VERSION }}
if: steps.changed-files.outputs.all
uses: astral-sh/setup-uv@v7
uses: "./.github/actions/poetry_setup"
with:
python-version: ${{ matrix.python-version }}
enable-cache: true
cache-suffix: "cli-integration-test"
ignore-nothing-to-cache: true
poetry-version: ${{ env.POETRY_VERSION }}
cache-key: integration-test-cli
- name: Setup env
if: steps.changed-files.outputs.all
working-directory: libs/cli/examples
run: cat .env.example > .env
- name: Install cli globally
if: steps.changed-files.outputs.all
run: pip install -e .
- name: Build and test service ${{ matrix.example.name }}
- name: Build and test service A
if: steps.changed-files.outputs.all
working-directory: ${{ matrix.example.workdir }}
env:
LANGSMITH_API_KEY: ${{ secrets.LANGSMITH_API_KEY }}
working-directory: libs/cli/examples
run: |
# Build the image for this example
langgraph build -t ${{ matrix.example.tag }}
# Prepare environment file from local or parent example directory
if [ -f .env.example ]; then cp .env.example .env; elif [ -f ../.env.example ]; then cp ../.env.example .env && cp ../.env.example ../.env; fi
if [ -n "${{ secrets.LANGSMITH_API_KEY }}" ]; then echo "LANGSMITH_API_KEY=${{ secrets.LANGSMITH_API_KEY }}" >> .env; if [ -f ../.env ]; then echo "LANGSMITH_API_KEY=${{ secrets.LANGSMITH_API_KEY }}" >> ../.env; fi; fi
# Run the integration test using the built tag
# Compute repo root to reference the shared script robustly
REPO_ROOT=$(git rev-parse --show-toplevel)
timeout 60 python "$REPO_ROOT/.github/scripts/run_langgraph_cli_test.py" -t ${{ matrix.example.tag }}
# The build-arg isn't used; just testing that we accept other args
langgraph build -t langgraph-test-a --base-image "langchain/langgraph-trial"
cp .env.example .envg
timeout 60 python ../../../.github/scripts/run_langgraph_cli_test.py -c langgraph.json -t langgraph-test-a
- name: Build and test service B
if: steps.changed-files.outputs.all
working-directory: libs/cli/examples/graphs
run: |
langgraph build -t langgraph-test-b --base-image "langchain/langgraph-trial"
timeout 60 python ../../../../.github/scripts/run_langgraph_cli_test.py -t langgraph-test-b
- name: Build and test service C
if: steps.changed-files.outputs.all
working-directory: libs/cli/examples/graphs_reqs_a
run: |
langgraph build -t langgraph-test-c --base-image "langchain/langgraph-trial"
timeout 60 python ../../../../.github/scripts/run_langgraph_cli_test.py -t langgraph-test-c
- name: Build and test service D
if: steps.changed-files.outputs.all
working-directory: libs/cli/examples/graphs_reqs_b
run: |
langgraph build -t langgraph-test-d --base-image "langchain/langgraph-trial"
timeout 60 python ../../../../.github/scripts/run_langgraph_cli_test.py -t langgraph-test-d
- name: Build JS service
if: ${{ steps.changed-files.outputs.all && matrix.example.name == 'A' }}
if: steps.changed-files.outputs.all
working-directory: libs/cli/js-examples
run: |
langgraph build -t langgraph-test-e
- name: Build JS monorepo service
if: ${{ steps.changed-files.outputs.all && matrix.example.name == 'A' }}
working-directory: libs/cli/js-monorepo-example
run: |
langgraph build -t langgraph-test-f -c apps/agent/langgraph.json --build-command "yarn run turbo build" --install-command "yarn install"
- name: Build Python monorepo service
if: ${{ steps.changed-files.outputs.all && matrix.example.name == 'A' }}
working-directory: libs/cli/python-monorepo-example
run: |
langgraph build -t langgraph-test-g -c apps/agent/langgraph.json
cp apps/agent/.env.example apps/agent/.env
if [ -n "${{ secrets.LANGSMITH_API_KEY }}" ]; then echo "LANGSMITH_API_KEY=${{ secrets.LANGSMITH_API_KEY }}" >> apps/agent/.env; fi
timeout 60 python ../../../.github/scripts/run_langgraph_cli_test.py -t langgraph-test-g -c apps/agent/langgraph.json
- name: Build and test prerelease reqs service
if: ${{ steps.changed-files.outputs.all && matrix.example.name == 'A' }}
working-directory: libs/cli/examples/graph_prerelease_reqs
run: |
langgraph build -t langgraph-test-h
cp ../.env.example .env
if [ -n "${{ secrets.LANGSMITH_API_KEY }}" ]; then echo "LANGSMITH_API_KEY=${{ secrets.LANGSMITH_API_KEY }}" >> .env; fi
timeout 60 python ../../../../.github/scripts/run_langgraph_cli_test.py -t langgraph-test-h
echo "Finished starting up langgraph-test-h"
LANGGRAPH_VERSION=$(docker run --rm --entrypoint "" langgraph-test-h python -c "import sys; from importlib.metadata import version; v = version('langgraph'); print(v);")
if [ "$LANGGRAPH_VERSION" != "1.0.2" ]; then
echo "LANGGRAPH_VERSION != 1.0.2; $LANGGRAPH_VERSION"
exit 1
fi
LANGCHAIN_OPENAI_VERSION=$(docker run --rm --entrypoint "" langgraph-test-h python -c "import sys; from importlib.metadata import version; v = version('langchain-openai'); print(v);")
if [ "$LANGCHAIN_OPENAI_VERSION" != "1.0.1" ]; then
echo "LANGCHAIN_OPENAI_VERSION != 1.0.1; $LANGCHAIN_OPENAI_VERSION"
exit 1
fi
LANGCHAIN_ANTHROPIC_VERSION=$(docker run --rm --entrypoint "" langgraph-test-h python -c "import sys; from importlib.metadata import version; v = version('langchain-anthropic'); print(v);")
if [ "$LANGCHAIN_ANTHROPIC_VERSION" != "1.0.0a5" ]; then
echo "LANGCHAIN_ANTHROPIC_VERSION != 1.0.0a5; $LANGCHAIN_ANTHROPIC_VERSION"
exit 1
fi
- name: Build and test prerelease reqs fail service
if: ${{ steps.changed-files.outputs.all && matrix.example.name == 'A' }}
working-directory: libs/cli/examples/graph_prerelease_reqs_fail
run: |
langgraph build -t langgraph-test-i || [ $? -eq 1 ]
+42 -14
View File
@@ -8,10 +8,9 @@ on:
type: string
description: "From which folder this pipeline executes"
permissions:
contents: read
env:
POETRY_VERSION: "1.7.1"
# This env var allows us to get inline annotations when ruff has complaints.
RUFF_OUTPUT_FORMAT: github
@@ -31,34 +30,54 @@ jobs:
- "3.12"
name: "lint #${{ matrix.python-version }}"
steps:
- uses: actions/checkout@v6
- uses: actions/checkout@v4
- name: Get changed files
id: changed-files
uses: Ana06/get-changed-files@v2.3.0
with:
filter: "${{ inputs.working-directory }}/**"
- name: Set up Python ${{ matrix.python-version }}
- name: Set up Python ${{ matrix.python-version }} + Poetry ${{ env.POETRY_VERSION }}
if: steps.changed-files.outputs.all
uses: astral-sh/setup-uv@v7
uses: "./.github/actions/poetry_setup"
with:
python-version: ${{ matrix.python-version }}
enable-cache: true
cache-suffix: lint-${{ inputs.working-directory }}
poetry-version: ${{ env.POETRY_VERSION }}
cache-key: lint-${{ inputs.working-directory }}
- name: Check Poetry File
if: steps.changed-files.outputs.all
shell: bash
working-directory: ${{ inputs.working-directory }}
run: poetry check
- name: Check lock file
if: steps.changed-files.outputs.all
shell: bash
working-directory: ${{ inputs.working-directory }}
run: poetry lock --check
- name: Install dependencies
if: steps.changed-files.outputs.all
# Also installs dev/lint/test/typing dependencies, to ensure we have
# type hints for as many of our libraries as possible.
# This helps catch errors that require dependencies to be spotted, for example:
# https://github.com/langchain-ai/langchain/pull/10249/files#diff-935185cd488d015f026dcd9e19616ff62863e8cde8c0bee70318d3ccbca98341
#
# If you change this configuration, make sure to change the `cache-key`
# in the `poetry_setup` action above to stop using the old cache.
# It doesn't matter how you change it, any change will cause a cache-bust.
working-directory: ${{ inputs.working-directory }}
run: uv sync --frozen --group lint
run: poetry install --with dev
- name: Get .mypy_cache to speed up mypy
if: steps.changed-files.outputs.all
uses: actions/cache@v5
uses: actions/cache@v3
env:
SEGMENT_DOWNLOAD_TIMEOUT_MIN: "2"
with:
path: |
${{ inputs.working-directory }}/.mypy_cache
key: mypy-lint-${{ runner.os }}-${{ runner.arch }}-py${{ matrix.python-version }}-${{ inputs.working-directory }}-${{ hashFiles(format('{0}/uv.lock', inputs.working-directory)) }}
key: mypy-lint-${{ runner.os }}-${{ runner.arch }}-py${{ matrix.python-version }}-${{ inputs.working-directory }}-${{ hashFiles(format('{0}/poetry.lock', inputs.working-directory)) }}
- name: Analysing package code with our lint
if: steps.changed-files.outputs.all
@@ -73,18 +92,27 @@ jobs:
- name: Install test dependencies
if: steps.changed-files.outputs.all
# Also installs dev/lint/test/typing dependencies, to ensure we have
# type hints for as many of our libraries as possible.
# This helps catch errors that require dependencies to be spotted, for example:
# https://github.com/langchain-ai/langchain/pull/10249/files#diff-935185cd488d015f026dcd9e19616ff62863e8cde8c0bee70318d3ccbca98341
#
# If you change this configuration, make sure to change the `cache-key`
# in the `poetry_setup` action above to stop using the old cache.
# It doesn't matter how you change it, any change will cause a cache-bust.
working-directory: ${{ inputs.working-directory }}
run: uv sync --group lint
run: |
poetry install --with dev
- name: Get .mypy_cache_test to speed up mypy
if: steps.changed-files.outputs.all
uses: actions/cache@v5
uses: actions/cache@v3
env:
SEGMENT_DOWNLOAD_TIMEOUT_MIN: "2"
with:
path: |
${{ inputs.working-directory }}/.mypy_cache_test
key: mypy-test-${{ runner.os }}-${{ runner.arch }}-py${{ matrix.python-version }}-${{ inputs.working-directory }}-${{ hashFiles(format('{0}/uv.lock', inputs.working-directory)) }}
key: mypy-test-${{ runner.os }}-${{ runner.arch }}-py${{ matrix.python-version }}-${{ inputs.working-directory }}-${{ hashFiles(format('{0}/poetry.lock', inputs.working-directory)) }}
- name: Analysing tests with our lint
if: steps.changed-files.outputs.all
+12 -10
View File
@@ -8,8 +8,8 @@ on:
type: string
description: "From which folder this pipeline executes"
permissions:
contents: read
env:
POETRY_VERSION: "1.7.1"
jobs:
build:
@@ -17,21 +17,21 @@ jobs:
strategy:
matrix:
python-version:
- "3.9"
- "3.10"
- "3.11"
- "3.12"
- "3.13"
- "3.14"
name: "test #${{ matrix.python-version }}"
steps:
- uses: actions/checkout@v6
- name: Set up Python ${{ matrix.python-version }}
uses: astral-sh/setup-uv@v7
- uses: actions/checkout@v4
- name: Set up Python ${{ matrix.python-version }} + Poetry ${{ env.POETRY_VERSION }}
uses: "./.github/actions/poetry_setup"
with:
python-version: ${{ matrix.python-version }}
enable-cache: true
cache-suffix: test-${{ inputs.working-directory }}
poetry-version: ${{ env.POETRY_VERSION }}
cache-key: test-${{ inputs.working-directory }}
- name: Login to Docker Hub
uses: docker/login-action@v3
if: ${{ !github.event.pull_request.head.repo.fork }}
@@ -42,12 +42,14 @@ jobs:
- name: Install dependencies
shell: bash
working-directory: ${{ inputs.working-directory }}
run: uv sync --frozen --group test --no-dev
run: |
poetry install --with dev
- name: Run tests
shell: bash
working-directory: ${{ inputs.working-directory }}
run: make test
run: |
make test
- name: Ensure the tests did not create any additional files
shell: bash
+12 -10
View File
@@ -3,8 +3,8 @@ name: test
on:
workflow_call:
permissions:
contents: read
env:
POETRY_VERSION: "1.7.1"
jobs:
build:
@@ -12,24 +12,24 @@ jobs:
strategy:
matrix:
python-version:
- "3.9"
- "3.10"
- "3.11"
- "3.12"
- "3.13"
- "3.14"
defaults:
run:
working-directory: libs/langgraph
name: "test #${{ matrix.python-version }}"
steps:
- uses: actions/checkout@v6
- name: Set up Python ${{ matrix.python-version }}
uses: astral-sh/setup-uv@v7
- uses: actions/checkout@v4
- name: Set up Python ${{ matrix.python-version }} + Poetry ${{ env.POETRY_VERSION }}
uses: "./.github/actions/poetry_setup"
with:
python-version: ${{ matrix.python-version }}
enable-cache: true
cache-suffix: "test-langgraph"
poetry-version: ${{ env.POETRY_VERSION }}
cache-key: test-langgraph
- name: Login to Docker Hub
uses: docker/login-action@v3
if: ${{ !github.event.pull_request.head.repo.fork }}
@@ -39,11 +39,13 @@ jobs:
- name: Install dependencies
shell: bash
run: uv sync --frozen --group test --no-dev
run: |
poetry install --with dev
- name: Run tests
shell: bash
run: make test_parallel
run: |
make test_parallel
- name: Ensure the tests did not create any additional files
shell: bash
+13 -14
View File
@@ -9,13 +9,12 @@ on:
description: "From which folder this pipeline executes"
env:
POETRY_VERSION: "1.7.1"
PYTHON_VERSION: "3.10"
permissions:
contents: read
jobs:
build:
if: github.ref == 'refs/heads/main'
runs-on: ubuntu-latest
outputs:
@@ -23,14 +22,14 @@ jobs:
version: ${{ steps.check-version.outputs.version }}
steps:
- uses: actions/checkout@v6
- uses: actions/checkout@v4
- name: Set up Python $${ env.PYTHON_VERSION }}
uses: astral-sh/setup-uv@v7
- name: Set up Python + Poetry ${{ env.POETRY_VERSION }}
uses: "./.github/actions/poetry_setup"
with:
python-version: ${{ env.PYTHON_VERSION }}
enable-cache: true
cache-suffix: "release"
poetry-version: ${{ env.POETRY_VERSION }}
cache-key: release
# We want to keep this build stage *separate* from the release stage,
# so that there's no sharing of permissions between them.
@@ -44,11 +43,11 @@ jobs:
# > from the publish job.
# https://github.com/pypa/gh-action-pypi-publish#non-goals
- name: Build project for distribution
run: uv build
run: poetry build
working-directory: ${{ inputs.working-directory }}
- name: Upload build
uses: actions/upload-artifact@v6
uses: actions/upload-artifact@v4
with:
name: test-dist
path: ${{ inputs.working-directory }}/dist/
@@ -58,8 +57,8 @@ jobs:
shell: bash
working-directory: ${{ inputs.working-directory }}
run: |
echo pkg-name=$(grep -m 1 "^name = " pyproject.toml | cut -d '"' -f 2)
echo version=$(grep -m 1 "^version = " pyproject.toml | cut -d '"' -f 2)
echo pkg-name="$(poetry version | cut -d ' ' -f 1)" >> $GITHUB_OUTPUT
echo version="$(poetry version --short)" >> $GITHUB_OUTPUT
publish:
needs:
@@ -74,9 +73,9 @@ jobs:
id-token: write
steps:
- uses: actions/checkout@v6
- uses: actions/checkout@v4
- uses: actions/download-artifact@v7
- uses: actions/download-artifact@v4
with:
name: test-dist
path: ${{ inputs.working-directory }}/dist/
@@ -0,0 +1,57 @@
name: test
on:
workflow_call:
env:
POETRY_VERSION: "1.7.1"
jobs:
build:
runs-on: ubuntu-latest
strategy:
matrix:
python-version:
- "3.11"
- "3.12"
defaults:
run:
working-directory: libs/scheduler-kafka
name: "test #${{ matrix.python-version }}"
steps:
- uses: actions/checkout@v4
- name: Set up Python ${{ matrix.python-version }} + Poetry ${{ env.POETRY_VERSION }}
uses: "./.github/actions/poetry_setup"
with:
python-version: ${{ matrix.python-version }}
poetry-version: ${{ env.POETRY_VERSION }}
cache-key: test-scheduler-kafka
- name: Login to Docker Hub
uses: docker/login-action@v3
if: ${{ !github.event.pull_request.head.repo.fork }}
with:
username: ${{ secrets.DOCKERHUB_USERNAME }}
password: ${{ secrets.DOCKERHUB_RO_TOKEN }}
- name: Install dependencies
shell: bash
run: |
poetry install --with dev
- name: Run tests
shell: bash
run: |
make test
- name: Ensure the tests did not create any additional files
shell: bash
run: |
set -eu
STATUS="$(git status)"
echo "$STATUS"
# grep will exit non-zero if the target message isn't found,
# and `set -e` above will cause the step to fail.
echo "$STATUS" | grep 'nothing to commit, working tree clean'
+9 -9
View File
@@ -7,8 +7,8 @@ on:
paths:
- "libs/**"
permissions:
contents: read
env:
POETRY_VERSION: "1.7.1"
jobs:
benchmark:
@@ -17,20 +17,20 @@ jobs:
run:
working-directory: libs/langgraph
steps:
- uses: actions/checkout@v6
- uses: actions/checkout@v4
- run: SHA=$(git rev-parse HEAD) && echo "SHA=$SHA" >> $GITHUB_ENV
- name: Set up Python 3.11
uses: astral-sh/setup-uv@v7
- name: Set up Python 3.11 + Poetry ${{ env.POETRY_VERSION }}
uses: "./.github/actions/poetry_setup"
with:
python-version: "3.11"
enable-cache: true
cache-suffix: "bench"
poetry-version: ${{ env.POETRY_VERSION }}
cache-key: bench
- name: Install dependencies
run: uv sync --group test
run: poetry install --with dev
- name: Run benchmarks
run: OUTPUT=out/benchmark-baseline.json make -s benchmark
- name: Save outputs
uses: actions/cache/save@v5
uses: actions/cache/save@v4
with:
key: ${{ runner.os }}-benchmark-baseline-${{ env.SHA }}
path: |
+12 -12
View File
@@ -5,8 +5,8 @@ on:
paths:
- "libs/**"
permissions:
contents: read
env:
POETRY_VERSION: "1.7.1"
jobs:
benchmark:
@@ -15,22 +15,22 @@ jobs:
run:
working-directory: libs/langgraph
steps:
- uses: actions/checkout@v6
- uses: actions/checkout@v4
- id: files
name: Get changed files
uses: Ana06/get-changed-files@v2.3.0
with:
format: json
- name: Set up Python 3.11
uses: astral-sh/setup-uv@v7
- name: Set up Python 3.11 + Poetry ${{ env.POETRY_VERSION }}
uses: "./.github/actions/poetry_setup"
with:
python-version: "3.11"
enable-cache: true
cache-suffix: "bench"
poetry-version: ${{ env.POETRY_VERSION }}
cache-key: bench
- name: Install dependencies
run: uv sync --group test
run: poetry install --with dev
- name: Download baseline
uses: actions/cache/restore@v5
uses: actions/cache/restore@v4
with:
key: ${{ runner.os }}-benchmark-baseline
restore-keys: |
@@ -43,7 +43,7 @@ jobs:
run: |
{
echo 'OUTPUT<<EOF'
make -s benchmark-fast
make -s benchmark
echo EOF
} >> "$GITHUB_OUTPUT"
- name: Compare benchmarks
@@ -53,11 +53,11 @@ jobs:
echo 'OUTPUT<<EOF'
mv out/benchmark-baseline.json out/main.json
mv out/benchmark.json out/changes.json
uv run pyperf compare_to out/main.json out/changes.json --table --group-by-speed
poetry run pyperf compare_to out/main.json out/changes.json --table --group-by-speed
echo EOF
} >> "$GITHUB_OUTPUT"
- name: Annotation
uses: actions/github-script@v8
uses: actions/github-script@v7
with:
script: |
const file = JSON.parse(`${{ steps.files.outputs.added_modified_renamed }}`)[0]
+77 -56
View File
@@ -3,13 +3,9 @@ name: CI
on:
push:
branches:
- main
branches: [main]
pull_request:
permissions:
contents: read
# If another push to the same PR or branch happens while this workflow is still running,
# cancel the earlier run in favor of the next run.
#
@@ -20,14 +16,17 @@ concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: true
env:
POETRY_VERSION: "1.7.1"
jobs:
changes:
runs-on: ubuntu-latest
outputs:
python: ${{ steps.filter.outputs.python }}
deps: ${{ steps.filter.outputs.deps }}
sdk-js: ${{ steps.filter.outputs.sdk-js }}
steps:
- uses: actions/checkout@v6
- uses: actions/checkout@v4
- uses: dorny/paths-filter@v3
id: filter
with:
@@ -39,10 +38,10 @@ jobs:
- 'libs/checkpoint/**'
- 'libs/checkpoint-sqlite/**'
- 'libs/checkpoint-postgres/**'
- 'libs/scheduler-kafka/**'
- 'libs/prebuilt/**'
deps:
- '**/pyproject.toml'
- '**/uv.lock'
sdk-js:
- 'libs/sdk-js/**'
lint:
needs: changes
@@ -57,10 +56,10 @@ jobs:
"libs/checkpoint",
"libs/checkpoint-sqlite",
"libs/checkpoint-postgres",
"libs/scheduler-kafka",
"libs/prebuilt",
]
if: needs.changes.outputs.python == 'true' || needs.changes.outputs.deps == 'true'
if: needs.changes.outputs.python == 'true'
uses: ./.github/workflows/_lint.yml
with:
working-directory: ${{ matrix.working-directory }}
@@ -78,9 +77,8 @@ jobs:
"libs/checkpoint-sqlite",
"libs/checkpoint-postgres",
"libs/prebuilt",
"libs/sdk-py",
]
if: needs.changes.outputs.python == 'true' || needs.changes.outputs.deps == 'true'
if: needs.changes.outputs.python == 'true'
uses: ./.github/workflows/_test.yml
with:
working-directory: ${{ matrix.working-directory }}
@@ -89,78 +87,101 @@ jobs:
# NOTE: we're testing langgraph separately because it requires a different matrix
test-langgraph:
needs: changes
if: needs.changes.outputs.python == 'true' || needs.changes.outputs.deps == 'true'
if: needs.changes.outputs.python == 'true'
name: "cd libs/langgraph"
uses: ./.github/workflows/_test_langgraph.yml
secrets: inherit
# NOTE: we're testing scheduler-kafka separately because it requires a different matrix
test-scheduler-kafka:
needs: changes
if: needs.changes.outputs.python == 'true'
name: "cd libs/scheduler-kafka"
uses: ./.github/workflows/_test_scheduler_kafka.yml
secrets: inherit
check-sdk-methods:
needs: changes
if: needs.changes.outputs.python == 'true'
name: "Check SDK methods matching"
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v6
- uses: actions/checkout@v4
- name: Set up Python
uses: actions/setup-python@v6
uses: actions/setup-python@v5
with:
python-version: "3.11"
- name: Run check_sdk_methods script
run: python .github/scripts/check_sdk_methods.py
check-schema:
needs: changes
if: needs.changes.outputs.python == 'true'
name: "Check CLI schema hasn't changed #${{ matrix.python-version }}"
runs-on: ubuntu-latest
strategy:
matrix:
python-version:
- "3.13"
steps:
- uses: actions/checkout@v6
- name: Set up Python ${{ matrix.python-version }}
uses: astral-sh/setup-uv@v7
with:
python-version: "3.13"
enable-cache: true
cache-suffix: "schema-check-cli"
- name: Install CLI dependencies
run: |
cd libs/cli
uv sync
- name: Generate schema and check for changes
run: |
cd libs/cli
# Create a temporary copy of the current schema
cp schemas/schema.json schemas/schema.current.json
# Generate new schema
uv run python generate_schema.py
# Compare the new schema with the original
if ! diff -q schemas/schema.json schemas/schema.current.json > /dev/null; then
echo "Error: Langgraph.json configuration schema has changed. Please run 'uv run python generate_schema.py' in the libs/cli directory and commit the changes."
diff schemas/schema.json schemas/schema.current.json
exit 1
fi
echo "Schema check passed - no changes detected"
integration-test:
needs: changes
if: needs.changes.outputs.python == 'true' || needs.changes.outputs.deps == 'true'
if: needs.changes.outputs.python == 'true'
name: CLI integration test
uses: ./.github/workflows/_integration_test.yml
secrets: inherit
lint-js:
needs: changes
if: needs.changes.outputs.sdk-js == 'true'
runs-on: ubuntu-latest
strategy:
matrix:
working-directory:
- "libs/sdk-js"
defaults:
run:
working-directory: ${{ matrix.working-directory }}
steps:
- uses: actions/checkout@v3
- name: Setup Node.js (LTS)
uses: actions/setup-node@v3
with:
node-version: "20"
cache: "yarn"
cache-dependency-path: ${{ matrix.working-directory }}/yarn.lock
- name: Install dependencies
run: yarn install
- name: Run lint
run: yarn lint
- name: Build
run: yarn build
test-js:
needs: changes
if: needs.changes.outputs.sdk-js == 'true'
runs-on: ubuntu-latest
strategy:
matrix:
working-directory:
- "libs/sdk-js"
defaults:
run:
working-directory: ${{ matrix.working-directory }}
steps:
- uses: actions/checkout@v3
- name: Setup Node.js (LTS)
uses: actions/setup-node@v3
with:
node-version: "20"
cache: "yarn"
cache-dependency-path: ${{ matrix.working-directory }}/yarn.lock
- name: Install dependencies
run: yarn install
- name: Run tests
run: yarn test
ci_success:
name: "CI Success"
needs:
[
lint,
lint-js,
test,
test-langgraph,
check-sdk-methods,
check-schema,
test-scheduler-kafka,
integration-test,
test-js,
]
if: |
always()
+43
View File
@@ -0,0 +1,43 @@
---
name: CI / cd . / make spell_check
on:
push:
branches: [main]
pull_request:
branches: [main]
permissions:
contents: read
defaults:
run:
working-directory: docs
jobs:
codespell:
name: (Check for spelling errors)
runs-on: ubuntu-latest
steps:
- name: Checkout
uses: actions/checkout@v4
- name: Install Dependencies
run: |
pip install toml codespell==2.3.0 jupytext
- name: Extract Ignore Words List
run: |
# Use a Python script to extract the ignore words list from pyproject.toml
python ../.github/workflows/extract_ignored_words_list.py
id: extract_ignore_words
- name: Codespell
uses: codespell-project/actions-codespell@v2
with:
skip: '*.ambr,*.lock,*.ipynb,*.yaml,*.zlib,*.md'
ignore_words_list: ${{ steps.extract_ignore_words.outputs.ignore_words_list }}
# We do this to avoid spellchecking cell outputs
- name: Codespell Notebooks
run: make codespell
+170
View File
@@ -0,0 +1,170 @@
name: Deploy Docs
on:
push:
branches:
- main
pull_request:
branches:
- main
workflow_dispatch:
env:
POETRY_VERSION: "1.7.1"
permissions:
contents: read
pages: write
id-token: write
concurrency:
group: "pages"
cancel-in-progress: false
defaults:
run:
working-directory: docs
jobs:
get-changed-files:
runs-on: ubuntu-latest
outputs:
changed-files: ${{ steps.changed-files.outputs.added_modified }}
steps:
- uses: actions/checkout@v4
- name: Get changed files
id: changed-files
uses: Ana06/get-changed-files@v2.3.0
with:
filter: "docs/docs/**"
run-changed-notebooks:
needs: get-changed-files
uses: ./.github/workflows/run_notebooks.yml
secrets: inherit
with:
changed-files: ${{ needs.get-changed-files.outputs.changed-files }}
deploy:
# needs: run-changed-notebooks
runs-on: ubuntu-latest
timeout-minutes: 10 # Job will be cancelled if it runs for more than 10 minutes
env:
GITHUB_TOKEN: ${{ secrets.MKDOCS_GITHUB_TOKEN }}
steps:
- uses: actions/checkout@v4
with:
fetch-depth: 0
- name: Set up Python + Poetry ${{ env.POETRY_VERSION }}
uses: "./.github/actions/poetry_setup"
with:
python-version: "3.12"
poetry-version: ${{ env.POETRY_VERSION }}
cache-key: docs
- name: Use Node.js
uses: actions/setup-node@v3
with:
node-version: "22"
cache: "yarn"
cache-dependency-path: docs/yarn.lock
- name: Install dependencies
run: |
yarn
poetry install --with test --with docs --no-root
poetry run pip install -U \
pytest \
pytest-check-links \
GitPython \
"git+https://github.com/benjamincburns/markdown-exec.git@cc0d39d737e5ffd4b83d23cd8729d7ea16e363c8"
# we run this installation only for internal PRs
# as GITHUB_TOKEN is not available for PRs from outside contributors
if [ -n "${GITHUB_TOKEN}" ]; then
poetry run pip install "git+https://${GITHUB_TOKEN}@github.com/langchain-ai/mkdocs-material-insiders.git"
fi
poetry run jupyter kernelspec list
poetry run python3 -m ipykernel install --user --name=python3
npm install -g tslab
poetry run tslab install --python=python3
poetry run jupyter kernelspec list
- name: Run unit tests
# Run unit tests on the docs build pipeline
run: make tests
- name: Lint Docs
# This step lints the docs using the existing linting set up.
# It should be very fast and should not require any external services.
run: make lint-docs
- name: Build llms-text
run: make llms-text
- name: Build site
run: make build-docs
env:
MKDOCS_GIT_COMMITTERS_APIKEY: ${{ secrets.MKDOCS_GIT_COMMITTERS_APIKEY }}
OPENAI_API_KEY: sf-proj-1234567890 # fake placeholder, shouldn't actually be used
ANTHROPIC_API_KEY: sk-ant-api03-1234567890 # fake placeholder, shouldn't actually be used
- name: Check links in notebooks
env:
LANGCHAIN_API_KEY: test
run: |
if [ "${{ github.event_name }}" == "schedule" ] || [ "${{ github.event_name }}" == "workflow_dispatch" ] || ([ "${{ github.event_name }}" == "push" ] && [ "${{ github.ref }}" == "refs/heads/main" ]); then
echo "Running link check on all HTML files matching notebooks in docs directory..."
poetry run pytest -v \
--check-links-ignore "https://(api|web|docs)\.smith\.langchain\.com/.*" \
--check-links-ignore "https://academy\.langchain\.com/.*" \
--check-links-ignore "https://x.com/.*" \
--check-links-ignore "https://twitter.com/.*" \
--check-links-ignore "https://github\.com/.*" \
--check-links-ignore "http://localhost:8123/.*" \
--check-links-ignore "http://localhost:2024.*" \
--check-links-ignore "http://127.0.0.1:.*" \
--check-links-ignore "/.*\.(ipynb|html)$" \
--check-links-ignore "https://python\.langchain\.com/.*" \
--check-links-ignore "https://openai\.com/.*" \
--check-links-ignore "https://www\.uber\.com/.*" \
--check-links-ignore "https://pepy\.tech/.*" \
--check-links $(find site -name "index.html" | grep -v 'storm/index.html')
else
echo "Fetching changes from origin/main..."
git fetch origin main
echo "Checking for changed notebook files..."
CHANGED_FILES=$(git diff --name-only --diff-filter=d origin/main | grep 'docs/docs/.*\.ipynb$' | grep -v 'storm.ipynb' | sed -E 's|^docs/docs/|site/|; s/\.ipynb$/\/index.html/' || true)
echo "Changed files: ${CHANGED_FILES}"
if [ -n "${CHANGED_FILES}" ]; then
echo "Running link check on HTML files matching changed notebook files..."
poetry run pytest -v \
--check-links-ignore "https://(api|web|docs)\.smith\.langchain\.com/.*" \
--check-links-ignore "https://academy\.langchain\.com/.*" \
--check-links-ignore "http://localhost:8123/.*" \
--check-links-ignore "http://localhost:2024.*" \
--check-links-ignore "http://127.0.0.1:.*" \
--check-links-ignore "https://x.com/.*" \
--check-links-ignore "https://twitter.com/.*" \
--check-links-ignore "https://github\.com/.*" \
--check-links-ignore "/.*\.(ipynb|html)$" \
--check-links ${CHANGED_FILES} \
|| ([ $? = 5 ] && exit 0 || exit $?)
else
echo "No notebook files changed."
fi
fi
- name: Configure GitHub Pages
if: github.ref == 'refs/heads/main'
uses: actions/configure-pages@v4
- name: Upload Pages Artifact
# if: github.ref == 'refs/heads/main'
uses: actions/upload-pages-artifact@v3
with:
path: ./docs/site/
- name: Deploy to GitHub Pages
if: github.ref == 'refs/heads/main'
id: deployment
uses: actions/deploy-pages@v4
@@ -0,0 +1,10 @@
import toml
pyproject_toml = toml.load("pyproject.toml")
# Extract the ignore words list (adjust the key as per your TOML structure)
ignore_words_list = (
pyproject_toml.get("tool", {}).get("codespell", {}).get("ignore-words-list")
)
print(f"::set-output name=ignore_words_list::{ignore_words_list}") # noqa: T201
+49
View File
@@ -0,0 +1,49 @@
name: Check Docs & Links
on:
pull_request:
branches:
- main
push:
branches:
- main
schedule:
- cron: "0 5 * * *"
workflow_dispatch:
env:
POETRY_VERSION: "1.7.1"
jobs:
markdown-link-check:
runs-on: ubuntu-latest
steps:
- name: Checkout code
uses: actions/checkout@v4
with:
fetch-depth: 0
- name: Check links in Markdown files
uses: gaurav-nelson/github-action-markdown-link-check@v1
with:
folder-path: "docs/"
check-modified-files-only: ${{ github.event_name != 'schedule' }}
file-path: "./README.md"
config-file: "./.markdown-link-check.config.json"
check-readmes-synced:
# This checks that the repo README.md is identical to the libs/langgraph/README.md
runs-on: ubuntu-latest
steps:
- name: Checkout code
uses: actions/checkout@v4
with:
fetch-depth: 1
- name: Check README.md is in sync
run: |
if ! diff -q README.md libs/langgraph/README.md >/dev/null; then
echo "README.md is out of sync with libs/langgraph/README.md"
diff -C 3 README.md libs/langgraph/README.md
exit 1
fi
-46
View File
@@ -1,46 +0,0 @@
name: PR Title Lint
permissions:
pull-requests: read
on:
pull_request:
types: [opened, edited, synchronize]
jobs:
lint-pr-title:
runs-on: ubuntu-latest
steps:
- name: Validate PR Title
uses: amannn/action-semantic-pull-request@v6
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
with:
types: |
feat
fix
docs
style
refactor
perf
test
build
ci
chore
revert
release
scopes: |
checkpoint
checkpoint-postgres
checkpoint-sqlite
cli
langgraph
prebuilt
scheduler-kafka
sdk-py
docs
ci
deps
requireScope: false
ignoreLabels: |
ignore-lint-pr-title
+35 -45
View File
@@ -8,14 +8,13 @@ on:
type: string
default: "libs/langgraph"
permissions:
contents: read
env:
PYTHON_VERSION: "3.11"
POETRY_VERSION: "1.7.1"
jobs:
build:
if: github.ref == 'refs/heads/main'
runs-on: ubuntu-latest
outputs:
@@ -25,14 +24,14 @@ jobs:
tag: ${{ steps.check-version.outputs.tag }}
steps:
- uses: actions/checkout@v6
- uses: actions/checkout@v4
- name: Set up Python
uses: astral-sh/setup-uv@v7
- name: Set up Python + Poetry ${{ env.POETRY_VERSION }}
uses: "./.github/actions/poetry_setup"
with:
python-version: ${{ env.PYTHON_VERSION }}
enable-cache: true
cache-suffix: "release"
poetry-version: ${{ env.POETRY_VERSION }}
cache-key: release
# We want to keep this build stage *separate* from the release stage,
# so that there's no sharing of permissions between them.
@@ -46,11 +45,11 @@ jobs:
# > from the publish job.
# https://github.com/pypa/gh-action-pypi-publish#non-goals
- name: Build project for distribution
run: uv build
run: poetry build
working-directory: ${{ inputs.working-directory }}
- name: Upload build
uses: actions/upload-artifact@v6
uses: actions/upload-artifact@v4
with:
name: dist
path: ${{ inputs.working-directory }}/dist/
@@ -60,14 +59,8 @@ jobs:
shell: bash
working-directory: ${{ inputs.working-directory }}
run: |
PKG_NAME=$(grep -m 1 "^name = " pyproject.toml | cut -d '"' -f 2)
if grep -q 'dynamic.*=.*\[.*"version".*\]' pyproject.toml; then
# handle dynamic versioning
DIR_NAME=$(echo "$PKG_NAME" | tr '-' '_')
VERSION=$(grep -m 1 '^__version__' "${DIR_NAME}/__init__.py" | cut -d '"' -f 2)
else
VERSION=$(grep -m 1 "^version = " pyproject.toml | cut -d '"' -f 2)
fi
PKG_NAME="$(poetry version | cut -d ' ' -f 1)"
VERSION="$(poetry version --short)"
SHORT_PKG_NAME="$(echo "$PKG_NAME" | sed -e 's/langgraph//g' -e 's/-//g')"
if [ -z $SHORT_PKG_NAME ]; then
TAG="$VERSION"
@@ -86,7 +79,7 @@ jobs:
outputs:
release-body: ${{ steps.generate-release-body.outputs.release-body }}
steps:
- uses: actions/checkout@v6
- uses: actions/checkout@v4
with:
repository: langchain-ai/langgraph
path: langgraph
@@ -142,9 +135,7 @@ jobs:
needs:
- build
- release-notes
permissions:
contents: read
id-token: write
permissions: write-all
uses: ./.github/workflows/_test_release.yml
with:
working-directory: ${{ inputs.working-directory }}
@@ -157,7 +148,7 @@ jobs:
- test-pypi-publish
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v6
- uses: actions/checkout@v4
# We explicitly *don't* set up caching here. This ensures our tests are
# maximally sensitive to catching breakage.
@@ -172,11 +163,11 @@ jobs:
# - The package is published, and it breaks on the missing dependency when
# used in the real world.
- name: Set up Python
uses: astral-sh/setup-uv@v7
- name: Set up Python + Poetry ${{ env.POETRY_VERSION }}
uses: "./.github/actions/poetry_setup"
with:
python-version: ${{ env.PYTHON_VERSION }}
enable-cache: true
poetry-version: ${{ env.POETRY_VERSION }}
- name: Import published package
shell: bash
@@ -194,18 +185,18 @@ jobs:
# - attempt install again after 5 seconds if it fails because there is
# sometimes a delay in availability on test pypi
run: |
uv run pip install \
poetry run pip install \
--extra-index-url https://test.pypi.org/simple/ \
"$PKG_NAME==$VERSION" || \
( \
sleep 5 && \
uv run pip install \
poetry run pip install \
--extra-index-url https://test.pypi.org/simple/ \
"$PKG_NAME==$VERSION" \
)
if [[ "$PKG_NAME" == *prebuilt* ]]; then
uv run pip install langgraph
poetry run pip install langgraph
fi
if [[ "$PKG_NAME" == *checkpoint* || "$PKG_NAME" == *prebuilt* ]]; then
@@ -218,10 +209,10 @@ jobs:
IMPORT_NAME="$(echo "$PKG_NAME" | sed s/-/_/g)"
fi
uv run python -c "import $IMPORT_NAME; print(dir($IMPORT_NAME))"
poetry run python -c "import $IMPORT_NAME; print(dir($IMPORT_NAME))"
- name: Import test dependencies
run: uv sync --group test
run: poetry install --with dev
working-directory: ${{ inputs.working-directory }}
# Overwrite the local version of the package with the test PyPI version.
@@ -232,7 +223,7 @@ jobs:
PKG_NAME: ${{ needs.build.outputs.pkg-name }}
VERSION: ${{ needs.build.outputs.version }}
run: |
uv run pip install \
poetry run pip install \
--extra-index-url https://test.pypi.org/simple/ \
"$PKG_NAME==$VERSION"
@@ -260,16 +251,16 @@ jobs:
working-directory: ${{ inputs.working-directory }}
steps:
- uses: actions/checkout@v6
- uses: actions/checkout@v4
- name: Set up Python
uses: astral-sh/setup-uv@v7
- name: Set up Python + Poetry ${{ env.POETRY_VERSION }}
uses: "./.github/actions/poetry_setup"
with:
python-version: ${{ env.PYTHON_VERSION }}
enable-cache: true
cache-suffix: "release"
poetry-version: ${{ env.POETRY_VERSION }}
cache-key: release
- uses: actions/download-artifact@v7
- uses: actions/download-artifact@v4
with:
name: dist
path: ${{ inputs.working-directory }}/dist/
@@ -301,16 +292,16 @@ jobs:
working-directory: ${{ inputs.working-directory }}
steps:
- uses: actions/checkout@v6
- uses: actions/checkout@v4
- name: Set up Python
uses: astral-sh/setup-uv@v7
- name: Set up Python + Poetry ${{ env.POETRY_VERSION }}
uses: "./.github/actions/poetry_setup"
with:
python-version: ${{ env.PYTHON_VERSION }}
enable-cache: true
cache-suffix: "release"
poetry-version: ${{ env.POETRY_VERSION }}
cache-key: release
- uses: actions/download-artifact@v7
- uses: actions/download-artifact@v4
with:
name: dist
path: ${{ inputs.working-directory }}/dist/
@@ -322,6 +313,5 @@ jobs:
token: ${{ secrets.GITHUB_TOKEN }}
generateReleaseNotes: false
tag: ${{needs.build.outputs.tag}}
name: ${{ needs.build.outputs.pkg-name }}==${{ needs.build.outputs.version }}
body: ${{ needs.release-notes.outputs.release-body }}
commit: ${{ github.sha }}
+38
View File
@@ -0,0 +1,38 @@
name: JS Release
on:
workflow_dispatch:
jobs:
publish:
# Disallow publishing from branches that aren't `main`.
if: github.ref == 'refs/heads/main'
runs-on: ubuntu-latest
strategy:
matrix:
working-directory:
- "libs/sdk-js"
defaults:
run:
working-directory: ${{ matrix.working-directory }}
steps:
- uses: actions/checkout@v4
# JS Build
- name: Use Node.js
uses: actions/setup-node@v3
with:
node-version: "20"
cache: "yarn"
cache-dependency-path: ${{ matrix.working-directory }}/yarn.lock
- name: Install dependencies
run: yarn install
- name: Build
run: yarn build
- name: Publish package to NPM
run: |
echo "//registry.npmjs.org/:_authToken=${{ secrets.NPM_TOKEN }}" > .npmrc
npm publish
+82
View File
@@ -0,0 +1,82 @@
name: Run notebooks
on:
workflow_dispatch:
workflow_call:
inputs:
changed-files:
required: false
type: string
description: "JSON string of changed files"
schedule:
- cron: '0 13 * * *'
defaults:
run:
working-directory: docs
jobs:
build:
runs-on: ubuntu-latest
strategy:
matrix:
lib-version:
- "development"
- "latest"
steps:
- uses: actions/checkout@v4
- name: Set up Python + Poetry
uses: "./.github/actions/poetry_setup"
with:
python-version: 3.11
poetry-version: 1.7.1
cache-key: test-langgraph-notebooks
- name: Install dependencies
run: |
poetry install --with test
poetry run pip install jupyter
- name: Start services
run: make start-services
- name: Pre-download tiktoken files
run: |
poetry run python _scripts/download_tiktoken.py
- name: Prepare notebooks
run: |
if [ "${{ matrix.lib-version }}" = "development" ]; then
poetry run python _scripts/prepare_notebooks_for_ci.py --comment-install-cells
else
poetry run python _scripts/prepare_notebooks_for_ci.py
fi
- name: Run notebooks
env:
# these won't actually be used because of the VCR cassettes
# but need to set them to avoid triggering getpass()
OPENAI_API_KEY: ${{ secrets.OPENAI_API_KEY }}
ANTHROPIC_API_KEY: ${{ secrets.ANTHROPIC_API_KEY }}
TAVILY_API_KEY: ${{ secrets.TAVILY_API_KEY }}
LANGSMITH_API_KEY: ${{ secrets.LANGSMITH_API_KEY }}
NOMIC_API_KEY: ${{ secrets.NOMIC_API_KEY }}
COHERE_API_KEY: ${{ secrets.COHERE_API_KEY }}
FIREWORKS_API_KEY: ${{ secrets.FIREWORKS_API_KEY }}
run: |
if [ "${{ github.event_name }}" = "workflow_dispatch" ] || [ "${{ github.event_name }}" = "schedule" ]; then
echo "Running all notebooks"
./_scripts/execute_notebooks.sh
else
CHANGED_FILES=$(echo '${{ inputs.changed-files }}' | tr ' ' '\n' | sed 's|^docs/docs/|docs/|' | grep '\.ipynb$' || true)
if [ -n "$CHANGED_FILES" ]; then
echo "Running changed notebooks: $CHANGED_FILES"
./_scripts/execute_notebooks.sh $CHANGED_FILES
else
echo "No notebook files changed, skipping execution"
fi
fi
- name: Stop services
run: make stop-services
+29
View File
@@ -0,0 +1,29 @@
name: Check File Size
on:
push:
branches:
- main
pull_request:
branches:
- main
workflow_dispatch:
jobs:
file-size-check:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- name: Get changed files
id: changed-files
uses: tj-actions/changed-files@v44
- name: Filter by size
# TODO: roll back the web voyager hack
run: |
large_added_files=$(find ${{ steps.changed-files.outputs.added_files }} -maxdepth 0 -size +1M | grep -v "web_voyager" || true)
if [ -n "$large_added_files" ]; then
echo "Large files added: $large_added_files"
echo "# Large files added:" >> $GITHUB_STEP_SUMMARY
echo "$large_added_files" >> $GITHUB_STEP_SUMMARY
exit 1
fi
-45
View File
@@ -1,45 +0,0 @@
name: UV Lock Upgrade
on:
schedule:
# run at midnight every Sunday
- cron: '0 0 * * 0'
# allow manual triggering
workflow_dispatch:
permissions:
contents: write
pull-requests: write
jobs:
upgrade-dependencies:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v6
- name: Set up uv
uses: astral-sh/setup-uv@v7
with:
# use minimum supported Python version
python-version: "3.10"
enable-cache: true
cache-suffix: "uv-lock-upgrade"
- name: Run uv lock --upgrade in all Python packages
run: make lock-upgrade
- name: Create Pull Request
uses: peter-evans/create-pull-request@v8
with:
token: ${{ secrets.GITHUB_TOKEN }}
commit-message: "chore(deps): upgrade dependencies with `uv lock --upgrade`"
title: "chore(deps): upgrade dependencies with `uv lock --upgrade`"
body: |
This PR updates the dependencies in all Python packages using `uv lock --upgrade`.
This is an automated PR created by the UV Lock Upgrade workflow.
branch: deps/uv-lock-upgrade
delete-branch: true
labels: |
dependencies
+82 -2
View File
@@ -6,6 +6,9 @@ __pycache__/
*.py[cod]
*$py.class
# C extensions
*.so
# Distribution / packaging
.Python
build/
@@ -51,12 +54,27 @@ coverage.xml
.hypothesis/
.pytest_cache/
# Translations
*.mo
*.pot
# Django stuff:
*.log
local_settings.py
db.sqlite3
db.sqlite3-journal
# Flask stuff:
instance/
.webassets-cache
# Scrapy stuff:
.scrapy
# Sphinx documentation
docs/_build/
docs/docs/_build/
# PyBuilder
target/
@@ -71,9 +89,23 @@ ipython_config.py
# pyenv
.python-version
# pipenv
# According to pypa/pipenv#598, it is recommended to include Pipfile.lock in version control.
# However, in case of collaboration, if having platform-specific dependencies or dependencies
# having no cross-platform support, pipenv may install dependencies that don't work, or not
# install all needed dependencies.
#Pipfile.lock
# PEP 582; used by e.g. github.com/David-OConnor/pyflow
__pypackages__/
# Celery stuff
celerybeat-schedule
celerybeat.pid
# SageMath parsed files
*.sage.py
# Environments
.env
.envrc
@@ -85,6 +117,16 @@ ENV/
env.bak/
venv.bak/
# Spyder project settings
.spyderproject
.spyproject
# Rope project settings
.ropeproject
# mkdocs documentation
/site
# mypy
.mypy_cache/
.dmypy.json
@@ -96,7 +138,45 @@ dmypy.json
# macOS display setting files
.DS_Store
# Wandb directory
wandb/
# asdf tool versions
.tool-versions
/.ruff_cache/
*.pkl
*.bin
# integration test artifacts
data_map*
\[('_type', 'fake'), ('stop', None)]
# Replit files
*replit*
node_modules
docs/.yarn/
docs/node_modules/
docs/.docusaurus/
docs/.cache-loader/
docs/_dist
docs/api_reference/api_reference.rst
docs/api_reference/experimental_api_reference.rst
docs/api_reference/_build
docs/api_reference/*/
!docs/api_reference/_static/
!docs/api_reference/templates/
!docs/api_reference/themes/
docs/docs_skeleton/build
docs/docs_skeleton/node_modules
docs/docs_skeleton/yarn.lock
# Any new jupyter notebooks
# not intended for the repo
Untitled*.ipynb
Chinook.db
.vercel
.turbo
.editorconfig
.scratch
+4
View File
@@ -0,0 +1,4 @@
{
"aliveStatusCodes": [200, 206, 402],
"ignorePatterns": ["*dcbadge.vercel.app*"]
}
-55
View File
@@ -1,55 +0,0 @@
# AGENTS Instructions
This repository is a monorepo. Each library lives in a subdirectory under `libs/`.
When you modify code in any library, run the following commands in that library's directory before creating a pull request:
- `make format` – run code formatters
- `make lint` – run the linter
- `make test` – execute the test suite
To run a particular test file or to pass additional pytest options you can specify the `TEST` variable:
```txt
TEST=path/to/test.py make test
```
Other pytest arguments can also be supplied inside the `TEST` variable.
## Libraries
The repository contains several Python and JavaScript/TypeScript libraries.
Below is a high-level overview:
- **checkpoint** – base interfaces for LangGraph checkpointers.
- **checkpoint-postgres** – Postgres implementation of the checkpoint saver.
- **checkpoint-sqlite** – SQLite implementation of the checkpoint saver.
- **cli** – official command-line interface for LangGraph.
- **langgraph** – core framework for building stateful, multi-actor agents.
- **prebuilt** – high-level APIs for creating and running agents and tools.
- **sdk-js** – JS/TS SDK for interacting with the LangGraph REST API.
- **sdk-py** – Python SDK for the LangGraph Server API.
### Dependency map
The diagram below lists downstream libraries for each production dependency as
declared in that library's `pyproject.toml` (or `package.json`).
```text
checkpoint
├── checkpoint-postgres
├── checkpoint-sqlite
├── prebuilt
└── langgraph
prebuilt
└── langgraph
sdk-py
├── langgraph
└── cli
sdk-js (standalone)
```
Changes to a library may impact all of its dependents shown above.
-55
View File
@@ -1,55 +0,0 @@
# AGENTS Instructions
This repository is a monorepo. Each library lives in a subdirectory under `libs/`.
When you modify code in any library, run the following commands in that library's directory before creating a pull request:
- `make format` – run code formatters
- `make lint` – run the linter
- `make test` – execute the test suite
To run a particular test file or to pass additional pytest options you can specify the `TEST` variable:
```
TEST=path/to/test.py make test
```
Other pytest arguments can also be supplied inside the `TEST` variable.
## Libraries
The repository contains several Python and JavaScript/TypeScript libraries.
Below is a high-level overview:
- **checkpoint** – base interfaces for LangGraph checkpointers.
- **checkpoint-postgres** – Postgres implementation of the checkpoint saver.
- **checkpoint-sqlite** – SQLite implementation of the checkpoint saver.
- **cli** – official command-line interface for LangGraph.
- **langgraph** – core framework for building stateful, multi-actor agents.
- **prebuilt** – high-level APIs for creating and running agents and tools.
- **sdk-js** – JS/TS SDK for interacting with the LangGraph REST API.
- **sdk-py** – Python SDK for the LangGraph Server API.
### Dependency map
The diagram below lists downstream libraries for each production dependency as
declared in that library's `pyproject.toml` (or `package.json`).
```text
checkpoint
├── checkpoint-postgres
├── checkpoint-sqlite
├── prebuilt
└── langgraph
prebuilt
└── langgraph
sdk-py
├── langgraph
└── cli
sdk-js (standalone)
```
Changes to a library may impact all of its dependents shown above.
+293
View File
@@ -0,0 +1,293 @@
# Contributing to LangGraph
Thank you for being interested in contributing to LangGraph!
## General guidelines
Here are some things to keep in mind for all types of contributions:
- Follow the ["fork and pull request"](https://docs.github.com/en/get-started/exploring-projects-on-github/contributing-to-a-project) workflow.
- Fill out the checked-in pull request template when opening pull requests. Note related issues and tag relevant maintainers.
- Ensure your PR passes formatting, linting, and testing checks before requesting a review.
- If you would like comments or feedback, please open an issue or discussion and tag a maintainer.
- Backwards compatibility is key. Your changes must not be breaking, except in case of critical bug and security fixes.
- Look for duplicate PRs or issues that have already been opened before opening a new one.
- Keep scope as isolated as possible. As a general rule, your changes should not affect more than one package at a time.
### Bugfixes
For bug fixes, please open up an issue before proposing a fix to ensure the proposal properly addresses the underlying problem. In general, bug fixes should all have an accompanying unit test that fails before the fix.
### New features
For new features, please start a new [discussion](https://github.com/langchain-ai/langgraph/discussions), where the maintainers will help with scoping out the necessary changes.
## Contribute Documentation
Documentation is a vital part of LangGraph. We welcome both new documentation for new features and
community improvements to our current documentation. Please read the resources below before getting started:
- [Documentation style guide](#documentation-style-guide)
- [Documentation setup](#setup)
## Documentation Style Guide
As LangGraph continues to grow, the surface area of documentation required to cover it continues to grow too.
This page provides guidelines for anyone writing documentation for LangGraph, as well as some of our philosophies around organization and structure.
## Philosophy
LangGraph's documentation follows the [Diataxis framework](https://diataxis.fr).
Under this framework, all documentation falls under one of four categories: [Tutorials](#tutorials),
[How-to guides](#how-to-guides),
[References](#references), and [Explanations (aka conceptual guides)](#conceptual-guide).
### Tutorials
Tutorials are lessons that take the reader through a practical activity. Their purpose is to help the user
gain understanding of concepts and how they interact by showing one way to achieve some goal in a hands-on way.
They should **avoid** giving
multiple permutations of ways to achieve that goal in-depth. Choice is burdensome. Instead, they should guide a new user through a recommended path to accomplishing a concrete goal. While the end result of a tutorial does not necessarily need to
be completely production-ready, it should be useful and practically satisfy the goal that you clearly stated in the tutorial's introduction.
To quote the Diataxis website:
> A tutorial serves the user’s *acquisition* of skills and knowledge - their study. Its purpose is not to help the user get something done, but to help them learn.
In LangGraph, these are often higher level guides that show off end-to-end use cases.
Some examples include:
- [Build a Customer Support Bot](https://langchain-ai.github.io/langgraph/tutorials/customer-support/customer-support/)
- [Build a SQL Agent](https://langchain-ai.github.io/langgraph/tutorials/sql-agent/)
Here are some high-level tips on writing a good tutorial:
- Focus on guiding the user to get something done, but keep in mind the end-goal is more to impart principles than to create a perfect production system.
- Be specific, not abstract and follow one path.
- No need to go deeply into alternative approaches, but it’s ok to reference them, ideally with a link to an appropriate how-to guide.
- Get "a point on the board" as soon as possible - something the user can run that outputs something.
- You can iterate and expand afterwards.
- Try to frequently checkpoint at given steps where the user can run code and see progress.
- Focus on results, not technical explanation.
- Crosslink heavily to appropriate conceptual/reference pages
- The first time you mention a LangGraph concept, use its full name (e.g. "human-in-the-loop"), and link to its conceptual/other documentation page.
- It's also helpful to add a prerequisite callout that links to any pages with necessary background information.
- End with a recap/next steps section summarizing what the tutorial covered and future reading, such as related how-to guides.
- Use phrases like "Next we can run X & Y. We will expect Z.". Then afterwards, use language like "Notice Z" that recalls our expectations and directs the reader's attention to the topic we are trying to teach.
- Do not shy away from repetition.
### How-to guides
A how-to guide, as the name implies, demonstrates how to do something discrete and specific.
It should assume that the user is already familiar with underlying concepts, and is trying to solve an immediate problem, but
should still give some background or list the scenarios where the information contained within can be relevant.
They can and should discuss alternatives if one approach may be better than another in certain cases.
To quote the Diataxis website:
> A how-to guide serves the work of the already-competent user, whom you can assume to know what they want to do, and to be able to follow your instructions correctly.
Some examples include:
- [How to add persistence to your graph](https://langchain-ai.github.io/langgraph/how-tos/persistence/)
- [How to view and update past graph state](https://langchain-ai.github.io/langgraph/how-tos/human_in_the_loop/time-travel/)
Here are some high-level tips on writing a good how-to guide:
- Clearly explain what you are guiding the user through at the start
- Assume higher intent than a tutorial and show what the user needs to do to get that task done
- Assume familiarity of concepts, but explain why suggested actions are helpful
- Crosslink heavily to conceptual/reference pages
- Discuss alternatives and responses to real-world tradeoffs that may arise when solving a problem
- Use lots of example code, ideally within complete code blocks that the reader can copy and run.
- End with a recap/next steps section summarizing what the tutorial covered and future reading, such as other related how-to guides
### Conceptual guides
LangGraph's conceptual guides fall under the **Explanation** quadrant of Diataxis. They should cover LangChain terms and concepts
in a more abstract way than how-to guides or tutorials, and should be geared towards curious users interested in
gaining a deeper understanding of the framework. Try to avoid excessively large code examples. The goal here is to
impart perspective to the user rather than to finish a practical project. These guides should cover **why** things work they way they do.
To quote the Diataxis website:
> The perspective of explanation is higher and wider than that of the other types. It does not take the user’s eye-level view, as in a how-to guide, or a close-up view of the machinery, like reference material. Its scope in each case is a topic - “an area of knowledge”, that somehow has to be bounded in a reasonable, meaningful way.
Some examples include:
- [What does it mean to be agentic?](https://langchain-ai.github.io/langgraph/concepts/high_level/)
- [Tool calling](https://langchain-ai.github.io/langgraph/concepts/agentic_concepts/#tool-calling)
Here are some high-level tips on writing a good conceptual guide:
- Explain design decisions. Why does concept X exist and why was it designed this way?
- Use analogies and reference other concepts and alternatives
- Avoid blending in too much reference content
- You can and should reference content covered in other guides, but make sure to link to them
### References
References contain detailed, low-level information that describes exactly what functionality exists and how to use it.
In LangGraph, this is mainly our API reference pages, which are populated from docstrings within code.
References pages are generally not read end-to-end, but are consulted as necessary when a user needs to know
how to use something specific.
To quote the Diataxis website:
> The only purpose of a reference guide is to describe, as succinctly as possible, and in an orderly way. Whereas the content of tutorials and how-to guides are led by needs of the user, reference material is led by the product it describes.
Many of the reference pages in LangChain are automatically generated from code,
but here are some high-level tips on writing a good docstring:
- Be concise
- Discuss special cases and deviations from a user's expectations
- Go into detail on required inputs and outputs
- Light details on when one might use the feature are fine, but in-depth details belong in other sections.
Each category serves a distinct purpose and requires a specific approach to writing and structuring the content.
## General guidelines
Here are some other guidelines you should think about when writing and organizing documentation.
We generally do not merge new tutorials from outside contributors without an actue need.
We welcome updates as well as new integration docs, how-tos, and references.
### Avoid duplication
Multiple pages that cover the same material in depth are difficult to maintain and cause confusion. There should
be only one (very rarely two), canonical pages for a given concept or feature. Instead, you should link to other guides.
### Link to other sections
Because sections of the docs do not exist in a vacuum, it is important to link to other sections as often as possible
to allow a developer to learn more about an unfamiliar topic inline.
This includes linking to the API references as well as conceptual sections!
### Be concise
In general, take a less-is-more approach. If a section with a good explanation of a concept already exists, you should link to it rather than
re-explain it, unless the concept you are documenting presents some new wrinkle.
Be concise, including in code samples.
### General style
- Use active voice and present tense whenever possible
- Use examples and code snippets to illustrate concepts and usage
- Use appropriate header levels (`#`, `##`, `###`, etc.) to organize the content hierarchically
- Use fewer cells with more code to make copy/paste easier
- Use bullet points and numbered lists to break down information into easily digestible chunks
- Use tables (especially for **Reference** sections) and diagrams often to present information visually
- Include the table of contents for longer documentation pages to help readers navigate the content, but hide it for shorter pages
## Setup
LangChain documentation consists of two components:
1. Main Documentation: Hosted at [https://langchain-ai.github.io](https://langchain-ai.github.io/langgraph/),
this comprehensive resource serves as the primary user-facing documentation.
It covers a wide array of topics, including tutorials, use cases, integrations,
and more, offering extensive guidance on building with LangGraph.
The content for this documentation lives in the `/docs` directory of the monorepo.
2. In-code Documentation: This is documentation of the codebase itself, which is also
used to generate the externally facing [API Reference](https://langchain-ai.github.io/langgraph/reference/graphs/).
The content for the API reference is autogenerated by scanning the docstrings in the codebase. For this reason we ask that developers document their code well.
We appreciate all contributions to the documentation, whether it be fixing a typo,
adding a new tutorial or example and whether it be in the main documentation or the API Reference.
### 📜 Main Documentation
The content for the main documentation is located in the `/docs` directory of the monorepo.
The documentation is written using a combination of ipython notebooks (`.ipynb` files)
and markdown (`.md` files). The notebooks are converted to markdown
and then built using [MkDocs](https://www.mkdocs.org/).
Feel free to make contributions to the main documentation! 🥰
After modifying the documentation:
1. Run the linting and formatting commands (see below) to ensure that the documentation is well-formatted and free of errors.
2. Optionally build the documentation locally to verify that the changes look good.
3. Make a pull request with the changes.
### ⚒️ Linting and Building Documentation Locally
After writing up the documentation, you may want to lint and build the documentation
locally to ensure that it looks good and is free of errors.
If you're unable to build it locally that's okay as well, as you will be able to
see a preview of the documentation on the pull request page.
From the **monorepo root**, run the following command to install the dependencies:
```bash
poetry install --with docs --no-root
```
#### Building
The code that builds the documentation is located in the `/docs` directory of the monorepo.
Before building the documentation, it is always a good idea to clean the build directory:
```bash
make clean-docs
```
You can build and preview the documentation as outlined below:
```bash
make serve-docs
```
#### Linting
The documentation is linted from the **monorepo root**. To lint it, run the following from there:
```bash
make spellcheck
```
### ️In-code Documentation
The in-code documentation is autogenerated from docstrings.
For the API reference to be useful, the codebase must be well-documented. This means that all functions, classes, and methods should have a docstring that explains what they do, what the arguments are, and what the return value is. This is a good practice in general, but it is especially important for LangChain because the API reference is the primary resource for developers to understand how to use the codebase.
We generally follow the [Google Python Style Guide](https://google.github.io/styleguide/pyguide.html#38-comments-and-docstrings) for docstrings.
Here is an example of a well-documented function:
```python
def my_function(arg1: int, arg2: str) -> float:
"""This is a short description of the function. (It should be a single sentence.)
This is a longer description of the function. It should explain what
the function does, what the arguments are, and what the return value is.
It should wrap at 88 characters.
Examples:
This is a section for examples of how to use the function.
.. code-block:: python
my_function(1, "hello")
Args:
arg1: This is a description of arg1. We do not need to specify the type since
it is already specified in the function signature.
arg2: This is a description of arg2.
Returns:
This is a description of the return value.
"""
return 3.14
```
-68
View File
@@ -1,68 +0,0 @@
# Define the directories containing projects
LIBS_DIRS := $(wildcard libs/*)
# Default target
.PHONY: all
all: lint format lock test
# Install dependencies for all projects
.PHONY: install
install:
@echo "Creating virtual environment..."
@uv venv
@for dir in $(LIBS_DIRS); do \
if [ -f $$dir/pyproject.toml ]; then \
echo "Installing dependencies for $$dir"; \
uv pip install -e $$dir; \
fi; \
done
# Lint all projects
.PHONY: lint
lint:
@for dir in $(LIBS_DIRS); do \
if [ -f $$dir/Makefile ]; then \
echo "Running lint in $$dir"; \
$(MAKE) -C $$dir lint; \
fi; \
done
# Format all projects
.PHONY: format
format:
@for dir in $(LIBS_DIRS); do \
if [ -f $$dir/Makefile ]; then \
echo "Running format in $$dir"; \
$(MAKE) -C $$dir format; \
fi; \
done
# Lock all projects
.PHONY: lock
lock:
@for dir in $(LIBS_DIRS); do \
if [ -f $$dir/Makefile ]; then \
echo "Running lock in $$dir"; \
(cd $$dir && uv lock); \
fi; \
done
# Lock all projects and upgrade dependencies
.PHONY: lock-upgrade
lock-upgrade:
@for dir in $(LIBS_DIRS); do \
if [ -f $$dir/Makefile ]; then \
echo "Running lock-upgrade in $$dir"; \
(cd $$dir && uv lock --upgrade); \
fi; \
done
# Test all projects
.PHONY: test
test:
@for dir in $(LIBS_DIRS); do \
if [ -f $$dir/Makefile ]; then \
echo "Running test in $$dir"; \
$(MAKE) -C $$dir test; \
fi; \
done
+313 -65
View File
@@ -1,91 +1,339 @@
<picture class="github-only">
<source media="(prefers-color-scheme: light)" srcset="https://langchain-ai.github.io/langgraph/static/wordmark_dark.svg">
<source media="(prefers-color-scheme: dark)" srcset="https://langchain-ai.github.io/langgraph/static/wordmark_light.svg">
<img alt="LangGraph Logo" src="https://langchain-ai.github.io/langgraph/static/wordmark_dark.svg" width="80%">
</picture>
# 🦜🕸️LangGraph
<div>
<br>
</div>
[![Version](https://img.shields.io/pypi/v/langgraph.svg)](https://pypi.org/project/langgraph/)
![Version](https://img.shields.io/pypi/v/langgraph)
[![Downloads](https://static.pepy.tech/badge/langgraph/month)](https://pepy.tech/project/langgraph)
[![Open Issues](https://img.shields.io/github/issues-raw/langchain-ai/langgraph)](https://github.com/langchain-ai/langgraph/issues)
[![Docs](https://img.shields.io/badge/docs-latest-blue)](https://docs.langchain.com/oss/python/langgraph/overview)
[![Docs](https://img.shields.io/badge/docs-latest-blue)](https://langchain-ai.github.io/langgraph/)
Trusted by companies shaping the future of agents – including Klarna, Replit, Elastic, and more – LangGraph is a low-level orchestration framework for building, managing, and deploying long-running, stateful agents.
⚡ Building language agents as graphs ⚡
## Get started
> [!NOTE]
> Looking for the JS version? See the [JS repo](https://github.com/langchain-ai/langgraphjs) and the [JS docs](https://langchain-ai.github.io/langgraphjs/).
Install LangGraph:
## Overview
```
[LangGraph](https://langchain-ai.github.io/langgraph/) is a library for building
stateful, multi-actor applications with LLMs, used to create agent and multi-agent
workflows. Check out an introductory tutorial [here](https://langchain-ai.github.io/langgraph/tutorials/introduction/).
LangGraph is inspired by [Pregel](https://research.google/pubs/pub37252/) and [Apache Beam](https://beam.apache.org/). The public interface draws inspiration from [NetworkX](https://networkx.org/documentation/latest/). LangGraph is built by LangChain Inc, the creators of LangChain, but can be used without LangChain.
### Why use LangGraph?
LangGraph powers [production-grade agents](https://www.langchain.com/built-with-langgraph), trusted by Linkedin, Uber, Klarna, GitLab, and many more. LangGraph provides fine-grained control over both the flow and state of your agent applications. It implements a central [persistence layer](https://langchain-ai.github.io/langgraph/concepts/persistence/), enabling features that are common to most agent architectures:
- **Memory**: LangGraph persists arbitrary aspects of your application's state,
supporting memory of conversations and other updates within and across user
interactions;
- **Human-in-the-loop**: Because state is checkpointed, execution can be interrupted
and resumed, allowing for decisions, validation, and corrections at key stages via
human input.
Standardizing these components allows individuals and teams to focus on the behavior
of their agent, instead of its supporting infrastructure.
Through [LangGraph Platform](#langgraph-platform), LangGraph also provides tooling for
the development, deployment, debugging, and monitoring of your applications.
LangGraph integrates seamlessly with
[LangChain](https://python.langchain.com/docs/introduction/) and
[LangSmith](https://docs.smith.langchain.com/) (but does not require them).
To learn more about LangGraph, check out our first LangChain Academy
course, *Introduction to LangGraph*, available for free
[here](https://academy.langchain.com/courses/intro-to-langgraph).
### LangGraph Platform
[LangGraph Platform](https://langchain-ai.github.io/langgraph/concepts/langgraph_platform) is infrastructure for deploying LangGraph agents. It is a commercial solution for deploying agentic applications to production, built on the open-source LangGraph framework. The LangGraph Platform consists of several components that work together to support the development, deployment, debugging, and monitoring of LangGraph applications: [LangGraph Server](https://langchain-ai.github.io/langgraph/concepts/langgraph_server) (APIs), [LangGraph SDKs](https://langchain-ai.github.io/langgraph/concepts/sdk) (clients for the APIs), [LangGraph CLI](https://langchain-ai.github.io/langgraph/concepts/langgraph_cli) (command line tool for building the server), and [LangGraph Studio](https://langchain-ai.github.io/langgraph/concepts/langgraph_studio) (UI/debugger).
See deployment options [here](https://langchain-ai.github.io/langgraph/concepts/deployment_options/)
(includes a free tier).
Here are some common issues that arise in complex deployments, which LangGraph Platform addresses:
- **Streaming support**: LangGraph Server provides [multiple streaming modes](https://langchain-ai.github.io/langgraph/concepts/streaming) optimized for various application needs
- **Background runs**: Runs agents asynchronously in the background
- **Support for long running agents**: Infrastructure that can handle long running processes
- **[Double texting](https://langchain-ai.github.io/langgraph/concepts/double_texting)**: Handle the case where you get two messages from the user before the agent can respond
- **Handle burstiness**: Task queue for ensuring requests are handled consistently without loss, even under heavy loads
## Installation
```shell
pip install -U langgraph
```
Create a simple workflow:
## Example
```python
from langgraph.graph import START, StateGraph
from typing_extensions import TypedDict
Let's build a tool-calling [ReAct-style](https://langchain-ai.github.io/langgraph/concepts/agentic_concepts/#react-implementation) agent that uses a search tool!
class State(TypedDict):
text: str
def node_a(state: State) -> dict:
return {"text": state["text"] + "a"}
def node_b(state: State) -> dict:
return {"text": state["text"] + "b"}
graph = StateGraph(State)
graph.add_node("node_a", node_a)
graph.add_node("node_b", node_b)
graph.add_edge(START, "node_a")
graph.add_edge("node_a", "node_b")
print(graph.compile().invoke({"text": ""}))
# {'text': 'ab'}
```shell
pip install langchain-anthropic
```
Get started with the [LangGraph Quickstart](https://docs.langchain.com/oss/python/langgraph/quickstart).
```shell
export ANTHROPIC_API_KEY=sk-...
```
To quickly build agents with LangChain's `create_agent` (built on LangGraph), see the [LangChain Agents documentation](https://docs.langchain.com/oss/python/langchain/agents).
Optionally, we can set up [LangSmith](https://docs.smith.langchain.com/) for best-in-class observability.
## Core benefits
```shell
export LANGSMITH_TRACING=true
export LANGSMITH_API_KEY=lsv2_sk_...
```
LangGraph provides low-level supporting infrastructure for *any* long-running, stateful workflow or agent. LangGraph does not abstract prompts or architecture, and provides the following central benefits:
The simplest way to create a tool-calling agent in LangGraph is to use `create_react_agent`:
- [Durable execution](https://docs.langchain.com/oss/python/langgraph/durable-execution): Build agents that persist through failures and can run for extended periods, automatically resuming from exactly where they left off.
- [Human-in-the-loop](https://docs.langchain.com/oss/python/langgraph/interrupts): Seamlessly incorporate human oversight by inspecting and modifying agent state at any point during execution.
- [Comprehensive memory](https://docs.langchain.com/oss/python/langgraph/memory): Create truly stateful agents with both short-term working memory for ongoing reasoning and long-term persistent memory across sessions.
- [Debugging with LangSmith](http://www.langchain.com/langsmith): Gain deep visibility into complex agent behavior with visualization tools that trace execution paths, capture state transitions, and provide detailed runtime metrics.
- [Production-ready deployment](https://docs.langchain.com/langsmith/app-development): Deploy sophisticated agent systems confidently with scalable infrastructure designed to handle the unique challenges of stateful, long-running workflows.
<details open>
<summary>High-level implementation</summary>
## LangGraph’s ecosystem
```python
from langgraph.prebuilt import create_react_agent
from langgraph.checkpoint.memory import MemorySaver
from langchain_anthropic import ChatAnthropic
from langchain_core.tools import tool
While LangGraph can be used standalone, it also integrates seamlessly with any LangChain product, giving developers a full suite of tools for building agents. To improve your LLM application development, pair LangGraph with:
# Define the tools for the agent to use
@tool
def search(query: str):
"""Call to surf the web."""
# This is a placeholder, but don't tell the LLM that...
if "sf" in query.lower() or "san francisco" in query.lower():
return "It's 60 degrees and foggy."
return "It's 90 degrees and sunny."
- [LangSmith](http://www.langchain.com/langsmith) — Helpful for agent evals and observability. Debug poor-performing LLM app runs, evaluate agent trajectories, gain visibility in production, and improve performance over time.
- [LangSmith Deployment](https://docs.langchain.com/langsmith/deployments) — Deploy and scale agents effortlessly with a purpose-built deployment platform for long running, stateful workflows. Discover, reuse, configure, and share agents across teams — and iterate quickly with visual prototyping in [LangGraph Studio](https://docs.langchain.com/oss/python/langgraph/studio).
- [LangChain](https://docs.langchain.com/oss/python/langchain/overview) – Provides integrations and composable components to streamline LLM application development.
> [!NOTE]
> Looking for the JS version of LangGraph? See the [JS repo](https://github.com/langchain-ai/langgraphjs) and the [JS docs](https://docs.langchain.com/oss/javascript/langgraph/overview).
tools = [search]
model = ChatAnthropic(model="claude-3-5-sonnet-latest", temperature=0)
## Additional resources
# Initialize memory to persist state between graph runs
checkpointer = MemorySaver()
- [Guides](https://docs.langchain.com/oss/python/langgraph/guides): Quick, actionable code snippets for topics such as streaming, adding memory & persistence, and design patterns (e.g. branching, subgraphs, etc.).
- [Reference](https://reference.langchain.com/python/langgraph/): Detailed reference on core classes, methods, how to use the graph and checkpointing APIs, and higher-level prebuilt components.
- [Examples](https://docs.langchain.com/oss/python/langgraph/agentic-rag): Guided examples on getting started with LangGraph.
- [LangChain Forum](https://forum.langchain.com/): Connect with the community and share all of your technical questions, ideas, and feedback.
- [LangChain Academy](https://academy.langchain.com/courses/intro-to-langgraph): Learn the basics of LangGraph in our free, structured course.
- [Case studies](https://www.langchain.com/built-with-langgraph): Hear how industry leaders use LangGraph to ship AI applications at scale.
app = create_react_agent(model, tools, checkpointer=checkpointer)
## Acknowledgements
# Use the agent
final_state = app.invoke(
{"messages": [{"role": "user", "content": "what is the weather in sf"}]},
config={"configurable": {"thread_id": 42}}
)
final_state["messages"][-1].content
```
```
"Based on the search results, I can tell you that the current weather in San Francisco is:\n\nTemperature: 60 degrees Fahrenheit\nConditions: Foggy\n\nSan Francisco is known for its microclimates and frequent fog, especially during the summer months. The temperature of 60°F (about 15.5°C) is quite typical for the city, which tends to have mild temperatures year-round. The fog, often referred to as "Karl the Fog" by locals, is a characteristic feature of San Francisco\'s weather, particularly in the mornings and evenings.\n\nIs there anything else you\'d like to know about the weather in San Francisco or any other location?"
```
LangGraph is inspired by [Pregel](https://research.google/pubs/pub37252/) and [Apache Beam](https://beam.apache.org/). The public interface draws inspiration from [NetworkX](https://networkx.org/documentation/latest/). LangGraph is built by LangChain Inc, the creators of LangChain, but can be used without LangChain.
Now when we pass the same <code>"thread_id"</code>, the conversation context is retained via the saved state (i.e. stored list of messages)
```python
final_state = app.invoke(
{"messages": [{"role": "user", "content": "what about ny"}]},
config={"configurable": {"thread_id": 42}}
)
final_state["messages"][-1].content
```
```
"Based on the search results, I can tell you that the current weather in New York City is:\n\nTemperature: 90 degrees Fahrenheit (approximately 32.2 degrees Celsius)\nConditions: Sunny\n\nThis weather is quite different from what we just saw in San Francisco. New York is experiencing much warmer temperatures right now. Here are a few points to note:\n\n1. The temperature of 90°F is quite hot, typical of summer weather in New York City.\n2. The sunny conditions suggest clear skies, which is great for outdoor activities but also means it might feel even hotter due to direct sunlight.\n3. This kind of weather in New York often comes with high humidity, which can make it feel even warmer than the actual temperature suggests.\n\nIt's interesting to see the stark contrast between San Francisco's mild, foggy weather and New York's hot, sunny conditions. This difference illustrates how varied weather can be across different parts of the United States, even on the same day.\n\nIs there anything else you'd like to know about the weather in New York or any other location?"
```
</details>
> [!TIP]
> LangGraph is a **low-level** framework that allows you to implement any custom agent
architectures. Click on the low-level implementation below to see how to implement a
tool-calling agent from scratch.
<details>
<summary>Low-level implementation</summary>
```python
from typing import Literal
from langchain_anthropic import ChatAnthropic
from langchain_core.tools import tool
from langgraph.checkpoint.memory import MemorySaver
from langgraph.graph import END, START, StateGraph, MessagesState
from langgraph.prebuilt import ToolNode
# Define the tools for the agent to use
@tool
def search(query: str):
"""Call to surf the web."""
# This is a placeholder, but don't tell the LLM that...
if "sf" in query.lower() or "san francisco" in query.lower():
return "It's 60 degrees and foggy."
return "It's 90 degrees and sunny."
tools = [search]
tool_node = ToolNode(tools)
model = ChatAnthropic(model="claude-3-5-sonnet-latest", temperature=0).bind_tools(tools)
# Define the function that determines whether to continue or not
def should_continue(state: MessagesState) -> Literal["tools", END]:
messages = state['messages']
last_message = messages[-1]
# If the LLM makes a tool call, then we route to the "tools" node
if last_message.tool_calls:
return "tools"
# Otherwise, we stop (reply to the user)
return END
# Define the function that calls the model
def call_model(state: MessagesState):
messages = state['messages']
response = model.invoke(messages)
# We return a list, because this will get added to the existing list
return {"messages": [response]}
# Define a new graph
workflow = StateGraph(MessagesState)
# Define the two nodes we will cycle between
workflow.add_node("agent", call_model)
workflow.add_node("tools", tool_node)
# Set the entrypoint as `agent`
# This means that this node is the first one called
workflow.add_edge(START, "agent")
# We now add a conditional edge
workflow.add_conditional_edges(
# First, we define the start node. We use `agent`.
# This means these are the edges taken after the `agent` node is called.
"agent",
# Next, we pass in the function that will determine which node is called next.
should_continue,
)
# We now add a normal edge from `tools` to `agent`.
# This means that after `tools` is called, `agent` node is called next.
workflow.add_edge("tools", 'agent')
# Initialize memory to persist state between graph runs
checkpointer = MemorySaver()
# Finally, we compile it!
# This compiles it into a LangChain Runnable,
# meaning you can use it as you would any other runnable.
# Note that we're (optionally) passing the memory when compiling the graph
app = workflow.compile(checkpointer=checkpointer)
# Use the agent
final_state = app.invoke(
{"messages": [{"role": "user", "content": "what is the weather in sf"}]},
config={"configurable": {"thread_id": 42}}
)
final_state["messages"][-1].content
```
<b>Step-by-step Breakdown</b>:
<details>
<summary>Initialize the model and tools.</summary>
<ul>
<li>
We use <code>ChatAnthropic</code> as our LLM. <strong>NOTE:</strong> we need to make sure the model knows that it has these tools available to call. We can do this by converting the LangChain tools into the format for OpenAI tool calling using the <code>.bind_tools()</code> method.
</li>
<li>
We define the tools we want to use - a search tool in our case. It is really easy to create your own tools - see documentation here on how to do that <a href="https://python.langchain.com/docs/how_to/custom_tools/">here</a>.
</li>
</ul>
</details>
<details>
<summary>Initialize graph with state.</summary>
<ul>
<li>We initialize graph (<code>StateGraph</code>) by passing state schema (in our case <code>MessagesState</code>)</li>
<li><code>MessagesState</code> is a prebuilt state schema that has one attribute -- a list of LangChain <code>Message</code> objects, as well as logic for merging the updates from each node into the state.</li>
</ul>
</details>
<details>
<summary>Define graph nodes.</summary>
There are two main nodes we need:
<ul>
<li>The <code>agent</code> node: responsible for deciding what (if any) actions to take.</li>
<li>The <code>tools</code> node that invokes tools: if the agent decides to take an action, this node will then execute that action.</li>
</ul>
</details>
<details>
<summary>Define entry point and graph edges.</summary>
First, we need to set the entry point for graph execution - <code>agent</code> node.
Then we define one normal and one conditional edge. Conditional edge means that the destination depends on the contents of the graph's state (<code>MessagesState</code>). In our case, the destination is not known until the agent (LLM) decides.
<ul>
<li>Conditional edge: after the agent is called, we should either:
<ul>
<li>a. Run tools if the agent said to take an action, OR</li>
<li>b. Finish (respond to the user) if the agent did not ask to run tools</li>
</ul>
</li>
<li>Normal edge: after the tools are invoked, the graph should always return to the agent to decide what to do next</li>
</ul>
</details>
<details>
<summary>Compile the graph.</summary>
<ul>
<li>
When we compile the graph, we turn it into a LangChain
<a href="https://python.langchain.com/docs/concepts/runnables/">Runnable</a>,
which automatically enables calling <code>.invoke()</code>, <code>.stream()</code> and <code>.batch()</code>
with your inputs
</li>
<li>
We can also optionally pass checkpointer object for persisting state between graph runs, and enabling memory,
human-in-the-loop workflows, time travel and more. In our case we use <code>MemorySaver</code> -
a simple in-memory checkpointer
</li>
</ul>
</details>
<details>
<summary>Execute the graph.</summary>
<ol>
<li>LangGraph adds the input message to the internal state, then passes the state to the entrypoint node, <code>"agent"</code>.</li>
<li>The <code>"agent"</code> node executes, invoking the chat model.</li>
<li>The chat model returns an <code>AIMessage</code>. LangGraph adds this to the state.</li>
<li>Graph cycles the following steps until there are no more <code>tool_calls</code> on <code>AIMessage</code>:
<ul>
<li>If <code>AIMessage</code> has <code>tool_calls</code>, <code>"tools"</code> node executes</li>
<li>The <code>"agent"</code> node executes again and returns <code>AIMessage</code></li>
</ul>
</li>
<li>Execution progresses to the special <code>END</code> value and outputs the final state. And as a result, we get a list of all our chat messages as output.</li>
</ol>
</details>
</details>
## Documentation
* [Tutorials](https://langchain-ai.github.io/langgraph/tutorials/): Learn to build with LangGraph through guided examples.
* [How-to Guides](https://langchain-ai.github.io/langgraph/how-tos/): Accomplish specific things within LangGraph, from streaming, to adding memory & persistence, to common design patterns (branching, subgraphs, etc.), these are the place to go if you want to copy and run a specific code snippet.
* [Conceptual Guides](https://langchain-ai.github.io/langgraph/concepts/high_level/): In-depth explanations of the key concepts and principles behind LangGraph, such as nodes, edges, state and more.
* [API Reference](https://langchain-ai.github.io/langgraph/reference/graphs/): Review important classes and methods, simple examples of how to use the graph and checkpointing APIs, higher-level prebuilt components and more.
* [LangGraph Platform](https://langchain-ai.github.io/langgraph/concepts/#langgraph-platform): LangGraph Platform is a commercial solution for deploying agentic applications in production, built on the open-source LangGraph framework.
## Resources
* [Built with LangGraph](https://www.langchain.com/built-with-langgraph): Hear how industry leaders use LangGraph to ship powerful, production-ready AI applications.
## Contributing
For more information on how to contribute, see [here](https://github.com/langchain-ai/langgraph/blob/main/CONTRIBUTING.md).
-21
View File
@@ -1,21 +0,0 @@
MIT License
Copyright (c) 2024 LangChain, Inc.
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.
+8 -8
View File
@@ -18,7 +18,7 @@ POSTGRES_VERSIONS ?= 15 16
test_pg_version:
@echo "Testing PostgreSQL $(POSTGRES_VERSION)"
@POSTGRES_VERSION=$(POSTGRES_VERSION) make start-postgres
@uv run pytest $(TEST)
@poetry run pytest $(TEST)
@EXIT_CODE=$$?; \
make stop-postgres; \
echo "Finished testing PostgreSQL $(POSTGRES_VERSION); Exit code: $$EXIT_CODE"; \
@@ -36,7 +36,7 @@ test:
TEST ?= .
test_watch:
POSTGRES_VERSION=${POSTGRES_VERSION:-16} make start-postgres; \
uv run ptw $(TEST); \
poetry run ptw $(TEST); \
EXIT_CODE=$$?; \
make stop-postgres; \
exit $$EXIT_CODE
@@ -55,12 +55,12 @@ lint_tests: PYTHON_FILES=tests
lint_tests: MYPY_CACHE=.mypy_cache_test
lint lint_diff lint_package lint_tests:
uv run ruff check .
[ "$(PYTHON_FILES)" = "" ] || uv run ruff format $(PYTHON_FILES) --diff
[ "$(PYTHON_FILES)" = "" ] || uv run ruff check --select I $(PYTHON_FILES)
poetry run ruff check .
[ "$(PYTHON_FILES)" = "" ] || poetry run ruff format $(PYTHON_FILES) --diff
[ "$(PYTHON_FILES)" = "" ] || poetry run ruff check --select I $(PYTHON_FILES)
[ "$(PYTHON_FILES)" = "" ] || mkdir -p $(MYPY_CACHE)
[ "$(PYTHON_FILES)" = "" ] || uv run mypy $(PYTHON_FILES) --cache-dir $(MYPY_CACHE)
[ "$(PYTHON_FILES)" = "" ] || poetry run mypy $(PYTHON_FILES) --cache-dir $(MYPY_CACHE)
format format_diff:
uv run ruff format $(PYTHON_FILES)
uv run ruff check --select I --fix $(PYTHON_FILES)
poetry run ruff format $(PYTHON_FILES)
poetry run ruff check --select I --fix $(PYTHON_FILES)
+2 -56
View File
@@ -13,20 +13,6 @@ By default `langgraph-checkpoint-postgres` installs `psycopg` (Psycopg 3) withou
> [!IMPORTANT]
> When manually creating Postgres connections and passing them to `PostgresSaver` or `AsyncPostgresSaver`, make sure to include `autocommit=True` and `row_factory=dict_row` (`from psycopg.rows import dict_row`). See a full example in this [how-to guide](https://langchain-ai.github.io/langgraph/how-tos/persistence_postgres/).
>
> **Why these parameters are required:**
> - `autocommit=True`: Required for the `.setup()` method to properly commit the checkpoint tables to the database. Without this, table creation may not be persisted.
> - `row_factory=dict_row`: Required because the PostgresSaver implementation accesses database rows using dictionary-style syntax (e.g., `row["column_name"]`). The default `tuple_row` factory returns tuples that only support index-based access (e.g., `row[0]`), which will cause `TypeError` exceptions when the checkpointer tries to access columns by name.
>
> **Example of incorrect usage:**
> ```python
> # ❌ This will fail with TypeError during checkpointer operations
> with psycopg.connect(DB_URI) as conn: # Missing autocommit=True and row_factory=dict_row
> checkpointer = PostgresSaver(conn)
> checkpointer.setup() # May not persist tables properly
> # Any operation that reads from database will fail with:
> # TypeError: tuple indices must be integers or slices, not str
> ```
```python
from langgraph.checkpoint.postgres import PostgresSaver
@@ -39,7 +25,7 @@ with PostgresSaver.from_conn_string(DB_URI) as checkpointer:
# call .setup() the first time you're using the checkpointer
checkpointer.setup()
checkpoint = {
"v": 4,
"v": 1,
"ts": "2024-07-31T20:14:19.804150+00:00",
"id": "1ef4f797-8335-6428-8001-8a1503f9b875",
"channel_values": {
@@ -61,6 +47,7 @@ with PostgresSaver.from_conn_string(DB_URI) as checkpointer:
"start:node": 2
}
},
"pending_sends": [],
}
# store checkpoint
@@ -72,44 +59,3 @@ with PostgresSaver.from_conn_string(DB_URI) as checkpointer:
# list checkpoints
list(checkpointer.list(read_config))
```
### Async
```python
from langgraph.checkpoint.postgres.aio import AsyncPostgresSaver
async with AsyncPostgresSaver.from_conn_string(DB_URI) as checkpointer:
checkpoint = {
"v": 4,
"ts": "2024-07-31T20:14:19.804150+00:00",
"id": "1ef4f797-8335-6428-8001-8a1503f9b875",
"channel_values": {
"my_key": "meow",
"node": "node"
},
"channel_versions": {
"__start__": 2,
"my_key": 3,
"start:node": 3,
"node": 3
},
"versions_seen": {
"__input__": {},
"__start__": {
"__start__": 1
},
"node": {
"start:node": 2
}
},
}
# store checkpoint
await checkpointer.aput(write_config, checkpoint, {}, {})
# load checkpoint
await checkpointer.aget(read_config)
# list checkpoints
[c async for c in checkpointer.alist(read_config)]
```
@@ -1,44 +1,38 @@
from __future__ import annotations
import threading
from collections import defaultdict
from collections.abc import Iterator, Sequence
from contextlib import contextmanager
from typing import Any
from typing import Any, Optional
from langchain_core.runnables import RunnableConfig
from langgraph.checkpoint.base import (
WRITES_IDX_MAP,
ChannelVersions,
Checkpoint,
CheckpointMetadata,
CheckpointTuple,
get_checkpoint_id,
get_serializable_checkpoint_metadata,
)
from langgraph.checkpoint.serde.base import SerializerProtocol
from psycopg import Capabilities, Connection, Cursor, Pipeline
from psycopg.rows import DictRow, dict_row
from psycopg.types.json import Jsonb
from psycopg_pool import ConnectionPool
from langgraph.checkpoint.base import (
WRITES_IDX_MAP,
ChannelVersions,
Checkpoint,
CheckpointConfig,
CheckpointMetadata,
CheckpointTuple,
get_checkpoint_id,
get_checkpoint_metadata,
)
from langgraph.checkpoint.postgres import _internal
from langgraph.checkpoint.postgres.base import BasePostgresSaver
from langgraph.checkpoint.postgres.shallow import ShallowPostgresSaver
from langgraph.checkpoint.serde.base import SerializerProtocol
Conn = _internal.Conn # For backward compatibility
class PostgresSaver(BasePostgresSaver):
"""Checkpointer that stores checkpoints in a Postgres database."""
lock: threading.Lock
def __init__(
self,
conn: _internal.Conn,
pipe: Pipeline | None = None,
serde: SerializerProtocol | None = None,
pipe: Optional[Pipeline] = None,
serde: Optional[SerializerProtocol] = None,
) -> None:
super().__init__(serde=serde)
if isinstance(conn, ConnectionPool) and pipe is not None:
@@ -55,12 +49,12 @@ class PostgresSaver(BasePostgresSaver):
@contextmanager
def from_conn_string(
cls, conn_string: str, *, pipeline: bool = False
) -> Iterator[PostgresSaver]:
) -> Iterator["PostgresSaver"]:
"""Create a new PostgresSaver instance from a connection string.
Args:
conn_string: The Postgres connection info string.
pipeline: whether to use Pipeline
conn_string (str): The Postgres connection info string.
pipeline (bool): whether to use Pipeline
Returns:
PostgresSaver: A new PostgresSaver instance.
@@ -94,20 +88,19 @@ class PostgresSaver(BasePostgresSaver):
for v, migration in zip(
range(version + 1, len(self.MIGRATIONS)),
self.MIGRATIONS[version + 1 :],
strict=False,
):
cur.execute(migration)
cur.execute("INSERT INTO checkpoint_migrations (v) VALUES (%s)", (v,))
cur.execute(f"INSERT INTO checkpoint_migrations (v) VALUES ({v})")
if self.pipe:
self.pipe.sync()
def list(
self,
config: RunnableConfig | None,
config: Optional[CheckpointConfig],
*,
filter: dict[str, Any] | None = None,
before: RunnableConfig | None = None,
limit: int | None = None,
filter: Optional[dict[str, Any]] = None,
before: Optional[CheckpointConfig] = None,
limit: Optional[int] = None,
) -> Iterator[CheckpointTuple]:
"""List checkpoints from the database.
@@ -115,13 +108,13 @@ class PostgresSaver(BasePostgresSaver):
on the provided config. The checkpoints are ordered by checkpoint ID in descending order (newest first).
Args:
config: The config to use for listing the checkpoints.
filter: Additional filtering criteria for metadata.
before: If provided, only checkpoints before the specified checkpoint ID are returned.
limit: The maximum number of checkpoints to return.
config (CheckpointConfig): The config to use for listing the checkpoints.
filter (Optional[Dict[str, Any]]): Additional filtering criteria for metadata. Defaults to None.
before (Optional[CheckpointConfig]): If provided, only checkpoints before the specified checkpoint ID are returned. Defaults to None.
limit (Optional[int]): The maximum number of checkpoints to return. Defaults to None.
Yields:
An iterator of checkpoint tuples.
Iterator[CheckpointTuple]: An iterator of checkpoint tuples.
Examples:
>>> from langgraph.checkpoint.postgres import PostgresSaver
@@ -143,57 +136,53 @@ class PostgresSaver(BasePostgresSaver):
"""
where, args = self._search_where(config, filter, before)
query = self.SELECT_SQL + where + " ORDER BY checkpoint_id DESC"
params = list(args)
if limit is not None:
query += " LIMIT %s"
params.append(int(limit))
if limit:
query += f" LIMIT {limit}"
# if we change this to use .stream() we need to make sure to close the cursor
with self._cursor() as cur:
cur.execute(query, params)
values = cur.fetchall()
if not values:
return
# migrate pending sends if necessary
if to_migrate := [
v
for v in values
if v["checkpoint"]["v"] < 4 and v["parent_checkpoint_id"]
]:
cur.execute(
self.SELECT_PENDING_SENDS_SQL,
(
values[0]["thread_id"],
[v["parent_checkpoint_id"] for v in to_migrate],
cur.execute(query, args, binary=True)
for value in cur:
yield CheckpointTuple(
{
"configurable": {
"thread_id": value["thread_id"],
"checkpoint_ns": value["checkpoint_ns"],
"checkpoint_id": value["checkpoint_id"],
}
},
self._load_checkpoint(
value["checkpoint"],
value["channel_values"],
value["pending_sends"],
),
self._load_metadata(value["metadata"]),
(
{
"configurable": {
"thread_id": value["thread_id"],
"checkpoint_ns": value["checkpoint_ns"],
"checkpoint_id": value["parent_checkpoint_id"],
}
}
if value["parent_checkpoint_id"]
else None
),
self._load_writes(value["pending_writes"]),
)
grouped_by_parent = defaultdict(list)
for value in to_migrate:
grouped_by_parent[value["parent_checkpoint_id"]].append(value)
for sends in cur:
for value in grouped_by_parent[sends["checkpoint_id"]]:
if value["channel_values"] is None:
value["channel_values"] = []
self._migrate_pending_sends(
sends["sends"],
value["checkpoint"],
value["channel_values"],
)
for value in values:
yield self._load_checkpoint_tuple(value)
def get_tuple(self, config: RunnableConfig) -> CheckpointTuple | None:
def get_tuple(self, config: CheckpointConfig) -> Optional[CheckpointTuple]:
"""Get a checkpoint tuple from the database.
This method retrieves a checkpoint tuple from the Postgres database based on the
provided config. If the config contains a `checkpoint_id` key, the checkpoint with
provided config. If the config contains a "checkpoint_id" key, the checkpoint with
the matching thread ID and timestamp is retrieved. Otherwise, the latest checkpoint
for the given thread ID is retrieved.
Args:
config: The config to use for retrieving the checkpoint.
config (CheckpointConfig): The config to use for retrieving the checkpoint.
Returns:
The retrieved checkpoint tuple, or None if no matching checkpoint was found.
Optional[CheckpointTuple]: The retrieved checkpoint tuple, or None if no matching checkpoint was found.
Examples:
@@ -230,48 +219,58 @@ class PostgresSaver(BasePostgresSaver):
cur.execute(
self.SELECT_SQL + where,
args,
binary=True,
)
value = cur.fetchone()
if value is None:
return None
# migrate pending sends if necessary
if value["checkpoint"]["v"] < 4 and value["parent_checkpoint_id"]:
cur.execute(
self.SELECT_PENDING_SENDS_SQL,
(thread_id, [value["parent_checkpoint_id"]]),
)
if sends := cur.fetchone():
if value["channel_values"] is None:
value["channel_values"] = []
self._migrate_pending_sends(
sends["sends"],
for value in cur:
return CheckpointTuple(
{
"configurable": {
"thread_id": thread_id,
"checkpoint_ns": checkpoint_ns,
"checkpoint_id": value["checkpoint_id"],
}
},
self._load_checkpoint(
value["checkpoint"],
value["channel_values"],
)
return self._load_checkpoint_tuple(value)
value["pending_sends"],
),
self._load_metadata(value["metadata"]),
(
{
"configurable": {
"thread_id": thread_id,
"checkpoint_ns": checkpoint_ns,
"checkpoint_id": value["parent_checkpoint_id"],
}
}
if value["parent_checkpoint_id"]
else None
),
self._load_writes(value["pending_writes"]),
)
def put(
self,
config: RunnableConfig,
config: CheckpointConfig,
checkpoint: Checkpoint,
metadata: CheckpointMetadata,
new_versions: ChannelVersions,
) -> RunnableConfig:
) -> CheckpointConfig:
"""Save a checkpoint to the database.
This method saves a checkpoint to the Postgres database. The checkpoint is associated
with the provided config and its parent config (if any).
Args:
config: The config to associate with the checkpoint.
checkpoint: The checkpoint to save.
metadata: Additional metadata to save with the checkpoint.
new_versions: New channel versions as of this write.
config (CheckpointConfig): The config to associate with the checkpoint.
checkpoint (Checkpoint): The checkpoint to save.
metadata (CheckpointMetadata): Additional metadata to save with the checkpoint.
new_versions (ChannelVersions): New channel versions as of this write.
Returns:
RunnableConfig: Updated configuration after storing the checkpoint.
CheckpointConfig: Updated configuration after storing the checkpoint.
Examples:
@@ -287,9 +286,11 @@ class PostgresSaver(BasePostgresSaver):
configurable = config["configurable"].copy()
thread_id = configurable.pop("thread_id")
checkpoint_ns = configurable.pop("checkpoint_ns")
checkpoint_id = configurable.pop("checkpoint_id", None)
checkpoint_id = configurable.pop(
"checkpoint_id", configurable.pop("thread_ts", None)
)
copy = checkpoint.copy()
copy["channel_values"] = copy["channel_values"].copy()
next_config = {
"configurable": {
"thread_id": thread_id,
@@ -298,28 +299,16 @@ class PostgresSaver(BasePostgresSaver):
}
}
# inline primitive values in checkpoint table
# others are stored in blobs table
blob_values = {}
for k, v in checkpoint["channel_values"].items():
if v is None or isinstance(v, (str, int, float, bool)):
pass
else:
blob_values[k] = copy["channel_values"].pop(k)
with self._cursor(pipeline=True) as cur:
if blob_versions := {
k: v for k, v in new_versions.items() if k in blob_values
}:
cur.executemany(
self.UPSERT_CHECKPOINT_BLOBS_SQL,
self._dump_blobs(
thread_id,
checkpoint_ns,
blob_values,
blob_versions,
),
)
cur.executemany(
self.UPSERT_CHECKPOINT_BLOBS_SQL,
self._dump_blobs(
thread_id,
checkpoint_ns,
copy.pop("channel_values"), # type: ignore[misc]
new_versions,
),
)
cur.execute(
self.UPSERT_CHECKPOINTS_SQL,
(
@@ -327,15 +316,15 @@ class PostgresSaver(BasePostgresSaver):
checkpoint_ns,
checkpoint["id"],
checkpoint_id,
Jsonb(copy),
Jsonb(get_serializable_checkpoint_metadata(config, metadata)),
Jsonb(self._dump_checkpoint(copy)),
self._dump_metadata(get_checkpoint_metadata(config, metadata)),
),
)
return next_config
def put_writes(
self,
config: RunnableConfig,
config: CheckpointConfig,
writes: Sequence[tuple[str, Any]],
task_id: str,
task_path: str = "",
@@ -345,9 +334,9 @@ class PostgresSaver(BasePostgresSaver):
This method saves intermediate writes associated with a checkpoint to the Postgres database.
Args:
config: Configuration of the related checkpoint.
writes: List of writes to store.
task_id: Identifier for the task creating the writes.
config (CheckpointConfig): Configuration of the related checkpoint.
writes (List[Tuple[str, Any]]): List of writes to store.
task_id (str): Identifier for the task creating the writes.
"""
query = (
self.UPSERT_CHECKPOINT_WRITES_SQL
@@ -367,39 +356,16 @@ class PostgresSaver(BasePostgresSaver):
),
)
def delete_thread(self, thread_id: str) -> None:
"""Delete all checkpoints and writes associated with a thread ID.
Args:
thread_id: The thread ID to delete.
Returns:
None
"""
with self._cursor(pipeline=True) as cur:
cur.execute(
"DELETE FROM checkpoints WHERE thread_id = %s",
(str(thread_id),),
)
cur.execute(
"DELETE FROM checkpoint_blobs WHERE thread_id = %s",
(str(thread_id),),
)
cur.execute(
"DELETE FROM checkpoint_writes WHERE thread_id = %s",
(str(thread_id),),
)
@contextmanager
def _cursor(self, *, pipeline: bool = False) -> Iterator[Cursor[DictRow]]:
"""Create a database cursor as a context manager.
Args:
pipeline: whether to use pipeline for the DB operations inside the context manager.
pipeline (bool): whether to use pipeline for the DB operations inside the context manager.
Will be applied regardless of whether the PostgresSaver instance was initialized with a pipeline.
If pipeline mode is not supported, will fall back to using transaction context manager.
"""
with self.lock, _internal.get_connection(self.conn) as conn:
with _internal.get_connection(self.conn) as conn:
if self.pipe:
# a connection in pipeline mode can be used concurrently
# in multiple threads/coroutines, but only one cursor can be
@@ -415,6 +381,7 @@ class PostgresSaver(BasePostgresSaver):
# thread/coroutine at a time, so we acquire a lock
if self.supports_pipeline:
with (
self.lock,
conn.pipeline(),
conn.cursor(binary=True, row_factory=dict_row) as cur,
):
@@ -422,55 +389,14 @@ class PostgresSaver(BasePostgresSaver):
else:
# Use connection's transaction context manager when pipeline mode not supported
with (
self.lock,
conn.transaction(),
conn.cursor(binary=True, row_factory=dict_row) as cur,
):
yield cur
else:
with conn.cursor(binary=True, row_factory=dict_row) as cur:
with self.lock, conn.cursor(binary=True, row_factory=dict_row) as cur:
yield cur
def _load_checkpoint_tuple(self, value: DictRow) -> CheckpointTuple:
"""
Convert a database row into a CheckpointTuple object.
Args:
value: A row from the database containing checkpoint data.
Returns:
CheckpointTuple: A structured representation of the checkpoint,
including its configuration, metadata, parent checkpoint (if any),
and pending writes.
"""
return CheckpointTuple(
{
"configurable": {
"thread_id": value["thread_id"],
"checkpoint_ns": value["checkpoint_ns"],
"checkpoint_id": value["checkpoint_id"],
}
},
{
**value["checkpoint"],
"channel_values": {
**(value["checkpoint"].get("channel_values") or {}),
**self._load_blobs(value["channel_values"]),
},
},
value["metadata"],
(
{
"configurable": {
"thread_id": value["thread_id"],
"checkpoint_ns": value["checkpoint_ns"],
"checkpoint_id": value["parent_checkpoint_id"],
}
}
if value["parent_checkpoint_id"]
else None
),
self._load_writes(value["pending_writes"]),
)
__all__ = ["PostgresSaver", "BasePostgresSaver", "ShallowPostgresSaver", "Conn"]
__all__ = ["PostgresSaver", "BasePostgresSaver", "Conn"]
@@ -1,23 +0,0 @@
"""Shared async utility functions for the Postgres checkpoint & storage classes."""
from collections.abc import AsyncIterator
from contextlib import asynccontextmanager
from psycopg import AsyncConnection
from psycopg.rows import DictRow
from psycopg_pool import AsyncConnectionPool
Conn = AsyncConnection[DictRow] | AsyncConnectionPool[AsyncConnection[DictRow]]
@asynccontextmanager
async def get_connection(
conn: Conn,
) -> AsyncIterator[AsyncConnection[DictRow]]:
if isinstance(conn, AsyncConnection):
yield conn
elif isinstance(conn, AsyncConnectionPool):
async with conn.connection() as conn:
yield conn
else:
raise TypeError(f"Invalid connection type: {type(conn)}")
@@ -2,12 +2,13 @@
from collections.abc import Iterator
from contextlib import contextmanager
from typing import Union
from psycopg import Connection
from psycopg.rows import DictRow
from psycopg_pool import ConnectionPool
Conn = Connection[DictRow] | ConnectionPool[Connection[DictRow]]
Conn = Union[Connection[DictRow], ConnectionPool[Connection[DictRow]]]
@contextmanager
@@ -1,582 +0,0 @@
from __future__ import annotations
import asyncio
from collections import defaultdict
from collections.abc import AsyncIterator, Iterator, Sequence
from contextlib import asynccontextmanager
from typing import Any
from langchain_core.runnables import RunnableConfig
from langgraph.checkpoint.base import (
WRITES_IDX_MAP,
ChannelVersions,
Checkpoint,
CheckpointMetadata,
CheckpointTuple,
get_checkpoint_id,
get_serializable_checkpoint_metadata,
)
from langgraph.checkpoint.serde.base import SerializerProtocol
from psycopg import AsyncConnection, AsyncCursor, AsyncPipeline, Capabilities
from psycopg.rows import DictRow, dict_row
from psycopg.types.json import Jsonb
from psycopg_pool import AsyncConnectionPool
from langgraph.checkpoint.postgres import _ainternal
from langgraph.checkpoint.postgres.base import BasePostgresSaver
from langgraph.checkpoint.postgres.shallow import AsyncShallowPostgresSaver
Conn = _ainternal.Conn # For backward compatibility
class AsyncPostgresSaver(BasePostgresSaver):
"""Asynchronous checkpointer that stores checkpoints in a Postgres database."""
lock: asyncio.Lock
def __init__(
self,
conn: _ainternal.Conn,
pipe: AsyncPipeline | None = None,
serde: SerializerProtocol | None = None,
) -> None:
super().__init__(serde=serde)
if isinstance(conn, AsyncConnectionPool) and pipe is not None:
raise ValueError(
"Pipeline should be used only with a single AsyncConnection, not AsyncConnectionPool."
)
self.conn = conn
self.pipe = pipe
self.lock = asyncio.Lock()
self.loop = asyncio.get_running_loop()
self.supports_pipeline = Capabilities().has_pipeline()
@classmethod
@asynccontextmanager
async def from_conn_string(
cls,
conn_string: str,
*,
pipeline: bool = False,
serde: SerializerProtocol | None = None,
) -> AsyncIterator[AsyncPostgresSaver]:
"""Create a new AsyncPostgresSaver instance from a connection string.
Args:
conn_string: The Postgres connection info string.
pipeline: whether to use AsyncPipeline
Returns:
AsyncPostgresSaver: A new AsyncPostgresSaver instance.
"""
async with await AsyncConnection.connect(
conn_string, autocommit=True, prepare_threshold=0, row_factory=dict_row
) as conn:
if pipeline:
async with conn.pipeline() as pipe:
yield cls(conn=conn, pipe=pipe, serde=serde)
else:
yield cls(conn=conn, serde=serde)
async def setup(self) -> None:
"""Set up the checkpoint database asynchronously.
This method creates the necessary tables in the Postgres database if they don't
already exist and runs database migrations. It MUST be called directly by the user
the first time checkpointer is used.
"""
async with self._cursor() as cur:
await cur.execute(self.MIGRATIONS[0])
results = await cur.execute(
"SELECT v FROM checkpoint_migrations ORDER BY v DESC LIMIT 1"
)
row = await results.fetchone()
if row is None:
version = -1
else:
version = row["v"]
for v, migration in zip(
range(version + 1, len(self.MIGRATIONS)),
self.MIGRATIONS[version + 1 :],
strict=False,
):
await cur.execute(migration)
await cur.execute(
"INSERT INTO checkpoint_migrations (v) VALUES (%s)", (v,)
)
if self.pipe:
await self.pipe.sync()
async def alist(
self,
config: RunnableConfig | None,
*,
filter: dict[str, Any] | None = None,
before: RunnableConfig | None = None,
limit: int | None = None,
) -> AsyncIterator[CheckpointTuple]:
"""List checkpoints from the database asynchronously.
This method retrieves a list of checkpoint tuples from the Postgres database based
on the provided config. The checkpoints are ordered by checkpoint ID in descending order (newest first).
Args:
config: Base configuration for filtering checkpoints.
filter: Additional filtering criteria for metadata.
before: If provided, only checkpoints before the specified checkpoint ID are returned.
limit: Maximum number of checkpoints to return.
Yields:
An asynchronous iterator of matching checkpoint tuples.
"""
where, args = self._search_where(config, filter, before)
query = self.SELECT_SQL + where + " ORDER BY checkpoint_id DESC"
params = list(args)
if limit is not None:
query += " LIMIT %s"
params.append(int(limit))
# if we change this to use .stream() we need to make sure to close the cursor
async with self._cursor() as cur:
await cur.execute(query, params, binary=True)
values = await cur.fetchall()
if not values:
return
# migrate pending sends if necessary
if to_migrate := [
v
for v in values
if v["checkpoint"]["v"] < 4 and v["parent_checkpoint_id"]
]:
await cur.execute(
self.SELECT_PENDING_SENDS_SQL,
(
values[0]["thread_id"],
[v["parent_checkpoint_id"] for v in to_migrate],
),
)
grouped_by_parent = defaultdict(list)
for value in to_migrate:
grouped_by_parent[value["parent_checkpoint_id"]].append(value)
async for sends in cur:
for value in grouped_by_parent[sends["checkpoint_id"]]:
if value["channel_values"] is None:
value["channel_values"] = []
self._migrate_pending_sends(
sends["sends"],
value["checkpoint"],
value["channel_values"],
)
for value in values:
yield await self._load_checkpoint_tuple(value)
async def aget_tuple(self, config: RunnableConfig) -> CheckpointTuple | None:
"""Get a checkpoint tuple from the database asynchronously.
This method retrieves a checkpoint tuple from the Postgres database based on the
provided config. If the config contains a `checkpoint_id` key, the checkpoint with
the matching thread ID and "checkpoint_id" is retrieved. Otherwise, the latest checkpoint
for the given thread ID is retrieved.
Args:
config: The config to use for retrieving the checkpoint.
Returns:
The retrieved checkpoint tuple, or None if no matching checkpoint was found.
"""
thread_id = config["configurable"]["thread_id"]
checkpoint_id = get_checkpoint_id(config)
checkpoint_ns = config["configurable"].get("checkpoint_ns", "")
if checkpoint_id:
args: tuple[Any, ...] = (thread_id, checkpoint_ns, checkpoint_id)
where = "WHERE thread_id = %s AND checkpoint_ns = %s AND checkpoint_id = %s"
else:
args = (thread_id, checkpoint_ns)
where = "WHERE thread_id = %s AND checkpoint_ns = %s ORDER BY checkpoint_id DESC LIMIT 1"
async with self._cursor() as cur:
await cur.execute(
self.SELECT_SQL + where,
args,
binary=True,
)
value = await cur.fetchone()
if value is None:
return None
# migrate pending sends if necessary
if value["checkpoint"]["v"] < 4 and value["parent_checkpoint_id"]:
await cur.execute(
self.SELECT_PENDING_SENDS_SQL,
(thread_id, [value["parent_checkpoint_id"]]),
)
if sends := await cur.fetchone():
if value["channel_values"] is None:
value["channel_values"] = []
self._migrate_pending_sends(
sends["sends"],
value["checkpoint"],
value["channel_values"],
)
return await self._load_checkpoint_tuple(value)
async def aput(
self,
config: RunnableConfig,
checkpoint: Checkpoint,
metadata: CheckpointMetadata,
new_versions: ChannelVersions,
) -> RunnableConfig:
"""Save a checkpoint to the database asynchronously.
This method saves a checkpoint to the Postgres database. The checkpoint is associated
with the provided config and its parent config (if any).
Args:
config: The config to associate with the checkpoint.
checkpoint: The checkpoint to save.
metadata: Additional metadata to save with the checkpoint.
new_versions: New channel versions as of this write.
Returns:
RunnableConfig: Updated configuration after storing the checkpoint.
"""
configurable = config["configurable"].copy()
thread_id = configurable.pop("thread_id")
checkpoint_ns = configurable.pop("checkpoint_ns")
checkpoint_id = configurable.pop("checkpoint_id", None)
copy = checkpoint.copy()
copy["channel_values"] = copy["channel_values"].copy()
next_config = {
"configurable": {
"thread_id": thread_id,
"checkpoint_ns": checkpoint_ns,
"checkpoint_id": checkpoint["id"],
}
}
# inline primitive values in checkpoint table
# others are stored in blobs table
blob_values = {}
for k, v in checkpoint["channel_values"].items():
if v is None or isinstance(v, (str, int, float, bool)):
pass
else:
blob_values[k] = copy["channel_values"].pop(k)
async with self._cursor(pipeline=True) as cur:
if blob_versions := {
k: v for k, v in new_versions.items() if k in blob_values
}:
await cur.executemany(
self.UPSERT_CHECKPOINT_BLOBS_SQL,
await asyncio.to_thread(
self._dump_blobs,
thread_id,
checkpoint_ns,
blob_values,
blob_versions,
),
)
await cur.execute(
self.UPSERT_CHECKPOINTS_SQL,
(
thread_id,
checkpoint_ns,
checkpoint["id"],
checkpoint_id,
Jsonb(copy),
Jsonb(get_serializable_checkpoint_metadata(config, metadata)),
),
)
return next_config
async def aput_writes(
self,
config: RunnableConfig,
writes: Sequence[tuple[str, Any]],
task_id: str,
task_path: str = "",
) -> None:
"""Store intermediate writes linked to a checkpoint asynchronously.
This method saves intermediate writes associated with a checkpoint to the database.
Args:
config: Configuration of the related checkpoint.
writes: List of writes to store, each as (channel, value) pair.
task_id: Identifier for the task creating the writes.
"""
query = (
self.UPSERT_CHECKPOINT_WRITES_SQL
if all(w[0] in WRITES_IDX_MAP for w in writes)
else self.INSERT_CHECKPOINT_WRITES_SQL
)
params = await asyncio.to_thread(
self._dump_writes,
config["configurable"]["thread_id"],
config["configurable"]["checkpoint_ns"],
config["configurable"]["checkpoint_id"],
task_id,
task_path,
writes,
)
async with self._cursor(pipeline=True) as cur:
await cur.executemany(query, params)
async def adelete_thread(self, thread_id: str) -> None:
"""Delete all checkpoints and writes associated with a thread ID.
Args:
thread_id: The thread ID to delete.
Returns:
None
"""
async with self._cursor(pipeline=True) as cur:
await cur.execute(
"DELETE FROM checkpoints WHERE thread_id = %s",
(str(thread_id),),
)
await cur.execute(
"DELETE FROM checkpoint_blobs WHERE thread_id = %s",
(str(thread_id),),
)
await cur.execute(
"DELETE FROM checkpoint_writes WHERE thread_id = %s",
(str(thread_id),),
)
@asynccontextmanager
async def _cursor(
self, *, pipeline: bool = False
) -> AsyncIterator[AsyncCursor[DictRow]]:
"""Create a database cursor as a context manager.
Args:
pipeline: whether to use pipeline for the DB operations inside the context manager.
Will be applied regardless of whether the AsyncPostgresSaver instance was initialized with a pipeline.
If pipeline mode is not supported, will fall back to using transaction context manager.
"""
async with self.lock, _ainternal.get_connection(self.conn) as conn:
if self.pipe:
# a connection in pipeline mode can be used concurrently
# in multiple threads/coroutines, but only one cursor can be
# used at a time
try:
async with conn.cursor(binary=True, row_factory=dict_row) as cur:
yield cur
finally:
if pipeline:
await self.pipe.sync()
elif pipeline:
# a connection not in pipeline mode can only be used by one
# thread/coroutine at a time, so we acquire a lock
if self.supports_pipeline:
async with (
conn.pipeline(),
conn.cursor(binary=True, row_factory=dict_row) as cur,
):
yield cur
else:
# Use connection's transaction context manager when pipeline mode not supported
async with (
conn.transaction(),
conn.cursor(binary=True, row_factory=dict_row) as cur,
):
yield cur
else:
async with conn.cursor(binary=True, row_factory=dict_row) as cur:
yield cur
async def _load_checkpoint_tuple(self, value: DictRow) -> CheckpointTuple:
"""
Convert a database row into a CheckpointTuple object.
Args:
value: A row from the database containing checkpoint data.
Returns:
CheckpointTuple: A structured representation of the checkpoint,
including its configuration, metadata, parent checkpoint (if any),
and pending writes.
"""
return CheckpointTuple(
{
"configurable": {
"thread_id": value["thread_id"],
"checkpoint_ns": value["checkpoint_ns"],
"checkpoint_id": value["checkpoint_id"],
}
},
{
**value["checkpoint"],
"channel_values": {
**(value["checkpoint"].get("channel_values") or {}),
**self._load_blobs(value["channel_values"]),
},
},
value["metadata"],
(
{
"configurable": {
"thread_id": value["thread_id"],
"checkpoint_ns": value["checkpoint_ns"],
"checkpoint_id": value["parent_checkpoint_id"],
}
}
if value["parent_checkpoint_id"]
else None
),
await asyncio.to_thread(self._load_writes, value["pending_writes"]),
)
def list(
self,
config: RunnableConfig | None,
*,
filter: dict[str, Any] | None = None,
before: RunnableConfig | None = None,
limit: int | None = None,
) -> Iterator[CheckpointTuple]:
"""List checkpoints from the database.
This method retrieves a list of checkpoint tuples from the Postgres database based
on the provided config. The checkpoints are ordered by checkpoint ID in descending order (newest first).
Args:
config: Base configuration for filtering checkpoints.
filter: Additional filtering criteria for metadata.
before: If provided, only checkpoints before the specified checkpoint ID are returned.
limit: Maximum number of checkpoints to return.
Yields:
An iterator of matching checkpoint tuples.
"""
try:
# check if we are in the main thread, only bg threads can block
# we don't check in other methods to avoid the overhead
if asyncio.get_running_loop() is self.loop:
raise asyncio.InvalidStateError(
"Synchronous calls to AsyncPostgresSaver are only allowed from a "
"different thread. From the main thread, use the async interface. "
"For example, use `checkpointer.alist(...)` or `await "
"graph.ainvoke(...)`."
)
except RuntimeError:
pass
aiter_ = self.alist(config, filter=filter, before=before, limit=limit)
while True:
try:
yield asyncio.run_coroutine_threadsafe(
anext(aiter_), # type: ignore[arg-type] # noqa: F821
self.loop,
).result()
except StopAsyncIteration:
break
def get_tuple(self, config: RunnableConfig) -> CheckpointTuple | None:
"""Get a checkpoint tuple from the database.
This method retrieves a checkpoint tuple from the Postgres database based on the
provided config. If the config contains a `checkpoint_id` key, the checkpoint with
the matching thread ID and "checkpoint_id" is retrieved. Otherwise, the latest checkpoint
for the given thread ID is retrieved.
Args:
config: The config to use for retrieving the checkpoint.
Returns:
The retrieved checkpoint tuple, or None if no matching checkpoint was found.
"""
try:
# check if we are in the main thread, only bg threads can block
# we don't check in other methods to avoid the overhead
if asyncio.get_running_loop() is self.loop:
raise asyncio.InvalidStateError(
"Synchronous calls to AsyncPostgresSaver are only allowed from a "
"different thread. From the main thread, use the async interface. "
"For example, use `await checkpointer.aget_tuple(...)` or `await "
"graph.ainvoke(...)`."
)
except RuntimeError:
pass
return asyncio.run_coroutine_threadsafe(
self.aget_tuple(config), self.loop
).result()
def put(
self,
config: RunnableConfig,
checkpoint: Checkpoint,
metadata: CheckpointMetadata,
new_versions: ChannelVersions,
) -> RunnableConfig:
"""Save a checkpoint to the database.
This method saves a checkpoint to the Postgres database. The checkpoint is associated
with the provided config and its parent config (if any).
Args:
config: The config to associate with the checkpoint.
checkpoint: The checkpoint to save.
metadata: Additional metadata to save with the checkpoint.
new_versions: New channel versions as of this write.
Returns:
RunnableConfig: Updated configuration after storing the checkpoint.
"""
return asyncio.run_coroutine_threadsafe(
self.aput(config, checkpoint, metadata, new_versions), self.loop
).result()
def put_writes(
self,
config: RunnableConfig,
writes: Sequence[tuple[str, Any]],
task_id: str,
task_path: str = "",
) -> None:
"""Store intermediate writes linked to a checkpoint.
This method saves intermediate writes associated with a checkpoint to the database.
Args:
config: Configuration of the related checkpoint.
writes: List of writes to store, each as (channel, value) pair.
task_id: Identifier for the task creating the writes.
task_path: Path of the task creating the writes.
"""
return asyncio.run_coroutine_threadsafe(
self.aput_writes(config, writes, task_id, task_path), self.loop
).result()
def delete_thread(self, thread_id: str) -> None:
"""Delete all checkpoints and writes associated with a thread ID.
Args:
thread_id: The thread ID to delete.
Returns:
None
"""
try:
# check if we are in the main thread, only bg threads can block
# we don't check in other methods to avoid the overhead
if asyncio.get_running_loop() is self.loop:
raise asyncio.InvalidStateError(
"Synchronous calls to AsyncPostgresSaver are only allowed from a "
"different thread. From the main thread, use the async interface. "
"For example, use `await checkpointer.aget_tuple(...)` or `await "
"graph.ainvoke(...)`."
)
except RuntimeError:
pass
return asyncio.run_coroutine_threadsafe(
self.adelete_thread(thread_id), self.loop
).result()
__all__ = ["AsyncPostgresSaver", "AsyncShallowPostgresSaver", "Conn"]
@@ -1,34 +1,22 @@
from __future__ import annotations
import random
import warnings
from collections.abc import Sequence
from importlib.metadata import version as get_version
from typing import Any, cast
from typing import Any, Optional, cast
from psycopg.types.json import Jsonb
from langchain_core.runnables import RunnableConfig
from langgraph.checkpoint.base import (
WRITES_IDX_MAP,
BaseCheckpointSaver,
ChannelVersions,
Checkpoint,
CheckpointConfig,
CheckpointMetadata,
get_checkpoint_id,
)
from langgraph.checkpoint.serde.types import TASKS
from psycopg.types.json import Jsonb
from langgraph.checkpoint.serde.jsonplus import JsonPlusSerializer
from langgraph.checkpoint.serde.types import TASKS, ChannelProtocol
MetadataInput = dict[str, Any] | None
try:
major, minor = get_version("langgraph").split(".")[:2]
if int(major) == 0 and int(minor) < 5:
warnings.warn(
"You're using incompatible versions of langgraph and checkpoint-postgres. Please upgrade langgraph to avoid unexpected behavior.",
DeprecationWarning,
stacklevel=2,
)
except Exception:
# skip version check if running from source
pass
MetadataInput = Optional[dict[str, Any]]
"""
To add a new migration, add a new string to the MIGRATIONS list.
@@ -81,10 +69,10 @@ MIGRATIONS = [
"""
CREATE INDEX CONCURRENTLY IF NOT EXISTS checkpoint_writes_thread_id_idx ON checkpoint_writes(thread_id);
""",
"""ALTER TABLE checkpoint_writes ADD COLUMN IF NOT EXISTS task_path TEXT NOT NULL DEFAULT '';""",
"""ALTER TABLE checkpoint_writes ADD COLUMN task_path TEXT NOT NULL DEFAULT '';""",
]
SELECT_SQL = """
SELECT_SQL = f"""
select
thread_id,
checkpoint,
@@ -108,20 +96,17 @@ select
where cw.thread_id = checkpoints.thread_id
and cw.checkpoint_ns = checkpoints.checkpoint_ns
and cw.checkpoint_id = checkpoints.checkpoint_id
) as pending_writes
) as pending_writes,
(
select array_agg(array[cw.type::bytea, cw.blob] order by cw.task_path, cw.task_id, cw.idx)
from checkpoint_writes cw
where cw.thread_id = checkpoints.thread_id
and cw.checkpoint_ns = checkpoints.checkpoint_ns
and cw.checkpoint_id = checkpoints.parent_checkpoint_id
and cw.channel = '{TASKS}'
) as pending_sends
from checkpoints """
SELECT_PENDING_SENDS_SQL = f"""
select
checkpoint_id,
array_agg(array[type::bytea, blob] order by task_path, task_id, idx) as sends
from checkpoint_writes
where thread_id = %s
and checkpoint_id = any(%s)
and channel = '{TASKS}'
group by checkpoint_id
"""
UPSERT_CHECKPOINT_BLOBS_SQL = """
INSERT INTO checkpoint_blobs (thread_id, checkpoint_ns, channel, version, type, blob)
VALUES (%s, %s, %s, %s, %s, %s)
@@ -155,34 +140,31 @@ INSERT_CHECKPOINT_WRITES_SQL = """
class BasePostgresSaver(BaseCheckpointSaver[str]):
SELECT_SQL = SELECT_SQL
SELECT_PENDING_SENDS_SQL = SELECT_PENDING_SENDS_SQL
MIGRATIONS = MIGRATIONS
UPSERT_CHECKPOINT_BLOBS_SQL = UPSERT_CHECKPOINT_BLOBS_SQL
UPSERT_CHECKPOINTS_SQL = UPSERT_CHECKPOINTS_SQL
UPSERT_CHECKPOINT_WRITES_SQL = UPSERT_CHECKPOINT_WRITES_SQL
INSERT_CHECKPOINT_WRITES_SQL = INSERT_CHECKPOINT_WRITES_SQL
jsonplus_serde = JsonPlusSerializer()
supports_pipeline: bool
def _migrate_pending_sends(
def _load_checkpoint(
self,
pending_sends: list[tuple[bytes, bytes]],
checkpoint: dict[str, Any],
channel_values: list[tuple[bytes, bytes, bytes]],
) -> None:
if not pending_sends:
return
# add to values
enc, blob = self.serde.dumps_typed(
[self.serde.loads_typed((c.decode(), b)) for c, b in pending_sends],
)
channel_values.append((TASKS.encode(), enc.encode(), blob))
# add to versions
checkpoint["channel_versions"][TASKS] = (
max(checkpoint["channel_versions"].values())
if checkpoint["channel_versions"]
else self.get_next_version(None, None)
)
pending_sends: list[tuple[bytes, bytes]],
) -> Checkpoint:
return {
**checkpoint,
"pending_sends": [
self.serde.loads_typed((c.decode(), b)) for c, b in pending_sends or []
],
"channel_values": self._load_blobs(channel_values),
}
def _dump_checkpoint(self, checkpoint: Checkpoint) -> dict[str, Any]:
return {**checkpoint, "pending_sends": []}
def _load_blobs(
self, blob_values: list[tuple[bytes, bytes, bytes]]
@@ -201,7 +183,7 @@ class BasePostgresSaver(BaseCheckpointSaver[str]):
checkpoint_ns: str,
values: dict[str, Any],
versions: ChannelVersions,
) -> list[tuple[str, str, str, str, str, bytes | None]]:
) -> list[tuple[str, str, str, str, str, Optional[bytes]]]:
if not versions:
return []
@@ -259,7 +241,15 @@ class BasePostgresSaver(BaseCheckpointSaver[str]):
for idx, (channel, value) in enumerate(writes)
]
def get_next_version(self, current: str | None, channel: None) -> str:
def _load_metadata(self, metadata: dict[str, Any]) -> CheckpointMetadata:
return self.jsonplus_serde.loads(self.jsonplus_serde.dumps(metadata))
def _dump_metadata(self, metadata: CheckpointMetadata) -> str:
serialized_metadata = self.jsonplus_serde.dumps(metadata)
# NOTE: we're using JSON serializer (not msgpack), so we need to remove null characters before writing
return serialized_metadata.decode().replace("\\u0000", "")
def get_next_version(self, current: Optional[str], channel: ChannelProtocol) -> str:
if current is None:
current_v = 0
elif isinstance(current, int):
@@ -272,9 +262,9 @@ class BasePostgresSaver(BaseCheckpointSaver[str]):
def _search_where(
self,
config: RunnableConfig | None,
config: Optional[CheckpointConfig],
filter: MetadataInput,
before: RunnableConfig | None = None,
before: Optional[CheckpointConfig] = None,
) -> tuple[str, list[Any]]:
"""Return WHERE clause predicates for alist() given config, filter, before.
@@ -1,967 +0,0 @@
import asyncio
import threading
import warnings
from collections.abc import AsyncIterator, Iterator, Sequence
from contextlib import asynccontextmanager, contextmanager
from typing import Any
from langchain_core.runnables import RunnableConfig
from langgraph.checkpoint.base import (
WRITES_IDX_MAP,
ChannelVersions,
Checkpoint,
CheckpointMetadata,
CheckpointTuple,
get_serializable_checkpoint_metadata,
)
from langgraph.checkpoint.serde.base import SerializerProtocol
from langgraph.checkpoint.serde.types import TASKS
from psycopg import (
AsyncConnection,
AsyncCursor,
AsyncPipeline,
Capabilities,
Connection,
Cursor,
Pipeline,
)
from psycopg.rows import DictRow, dict_row
from psycopg.types.json import Jsonb
from psycopg_pool import AsyncConnectionPool, ConnectionPool
from langgraph.checkpoint.postgres import _ainternal, _internal
from langgraph.checkpoint.postgres.base import BasePostgresSaver
"""
To add a new migration, add a new string to the MIGRATIONS list.
The position of the migration in the list is the version number.
"""
MIGRATIONS = [
"""CREATE TABLE IF NOT EXISTS checkpoint_migrations (
v INTEGER PRIMARY KEY
);""",
"""CREATE TABLE IF NOT EXISTS checkpoints (
thread_id TEXT NOT NULL,
checkpoint_ns TEXT NOT NULL DEFAULT '',
type TEXT,
checkpoint JSONB NOT NULL,
metadata JSONB NOT NULL DEFAULT '{}',
PRIMARY KEY (thread_id, checkpoint_ns)
);""",
"""CREATE TABLE IF NOT EXISTS checkpoint_blobs (
thread_id TEXT NOT NULL,
checkpoint_ns TEXT NOT NULL DEFAULT '',
channel TEXT NOT NULL,
type TEXT NOT NULL,
blob BYTEA,
PRIMARY KEY (thread_id, checkpoint_ns, channel)
);""",
"""CREATE TABLE IF NOT EXISTS checkpoint_writes (
thread_id TEXT NOT NULL,
checkpoint_ns TEXT NOT NULL DEFAULT '',
checkpoint_id TEXT NOT NULL,
task_id TEXT NOT NULL,
idx INTEGER NOT NULL,
channel TEXT NOT NULL,
type TEXT,
blob BYTEA NOT NULL,
PRIMARY KEY (thread_id, checkpoint_ns, checkpoint_id, task_id, idx)
);""",
"""
CREATE INDEX CONCURRENTLY IF NOT EXISTS checkpoints_thread_id_idx ON checkpoints(thread_id);
""",
"""
CREATE INDEX CONCURRENTLY IF NOT EXISTS checkpoint_blobs_thread_id_idx ON checkpoint_blobs(thread_id);
""",
"""
CREATE INDEX CONCURRENTLY IF NOT EXISTS checkpoint_writes_thread_id_idx ON checkpoint_writes(thread_id);
""",
"""
ALTER TABLE checkpoint_writes ADD COLUMN IF NOT EXISTS task_path TEXT NOT NULL DEFAULT '';
""",
]
SELECT_SQL = f"""
select
thread_id,
checkpoint,
checkpoint_ns,
metadata,
(
select array_agg(array[bl.channel::bytea, bl.type::bytea, bl.blob])
from jsonb_each_text(checkpoint -> 'channel_versions')
inner join checkpoint_blobs bl
on bl.thread_id = checkpoints.thread_id
and bl.checkpoint_ns = checkpoints.checkpoint_ns
and bl.channel = jsonb_each_text.key
) as channel_values,
(
select
array_agg(array[cw.task_id::text::bytea, cw.channel::bytea, cw.type::bytea, cw.blob] order by cw.task_id, cw.idx)
from checkpoint_writes cw
where cw.thread_id = checkpoints.thread_id
and cw.checkpoint_ns = checkpoints.checkpoint_ns
and cw.checkpoint_id = (checkpoint->>'id')
) as pending_writes,
(
select array_agg(array[cw.type::bytea, cw.blob] order by cw.task_path, cw.task_id, cw.idx)
from checkpoint_writes cw
where cw.thread_id = checkpoints.thread_id
and cw.checkpoint_ns = checkpoints.checkpoint_ns
and cw.channel = '{TASKS}'
) as pending_sends
from checkpoints """
UPSERT_CHECKPOINT_BLOBS_SQL = """
INSERT INTO checkpoint_blobs (thread_id, checkpoint_ns, channel, type, blob)
VALUES (%s, %s, %s, %s, %s)
ON CONFLICT (thread_id, checkpoint_ns, channel) DO UPDATE SET
type = EXCLUDED.type,
blob = EXCLUDED.blob;
"""
UPSERT_CHECKPOINTS_SQL = """
INSERT INTO checkpoints (thread_id, checkpoint_ns, checkpoint, metadata)
VALUES (%s, %s, %s, %s)
ON CONFLICT (thread_id, checkpoint_ns)
DO UPDATE SET
checkpoint = EXCLUDED.checkpoint,
metadata = EXCLUDED.metadata;
"""
UPSERT_CHECKPOINT_WRITES_SQL = """
INSERT INTO checkpoint_writes (thread_id, checkpoint_ns, checkpoint_id, task_id, task_path, idx, channel, type, blob)
VALUES (%s, %s, %s, %s, %s, %s, %s, %s, %s)
ON CONFLICT (thread_id, checkpoint_ns, checkpoint_id, task_id, idx) DO UPDATE SET
channel = EXCLUDED.channel,
type = EXCLUDED.type,
blob = EXCLUDED.blob;
"""
INSERT_CHECKPOINT_WRITES_SQL = """
INSERT INTO checkpoint_writes (thread_id, checkpoint_ns, checkpoint_id, task_id, task_path, idx, channel, type, blob)
VALUES (%s, %s, %s, %s, %s, %s, %s, %s, %s)
ON CONFLICT (thread_id, checkpoint_ns, checkpoint_id, task_id, idx) DO NOTHING
"""
def _dump_blobs(
serde: SerializerProtocol,
thread_id: str,
checkpoint_ns: str,
values: dict[str, Any],
versions: ChannelVersions,
) -> list[tuple[str, str, str, str, bytes | None]]:
if not versions:
return []
return [
(
thread_id,
checkpoint_ns,
k,
*(serde.dumps_typed(values[k]) if k in values else ("empty", None)),
)
for k in versions
]
class ShallowPostgresSaver(BasePostgresSaver):
"""A checkpoint saver that uses Postgres to store checkpoints.
This checkpointer ONLY stores the most recent checkpoint and does NOT retain any history.
It is meant to be a light-weight drop-in replacement for the PostgresSaver that
supports most of the LangGraph persistence functionality with the exception of time travel.
"""
SELECT_SQL = SELECT_SQL
MIGRATIONS = MIGRATIONS
UPSERT_CHECKPOINT_BLOBS_SQL = UPSERT_CHECKPOINT_BLOBS_SQL
UPSERT_CHECKPOINTS_SQL = UPSERT_CHECKPOINTS_SQL
UPSERT_CHECKPOINT_WRITES_SQL = UPSERT_CHECKPOINT_WRITES_SQL
INSERT_CHECKPOINT_WRITES_SQL = INSERT_CHECKPOINT_WRITES_SQL
lock: threading.Lock
def __init__(
self,
conn: _internal.Conn,
pipe: Pipeline | None = None,
serde: SerializerProtocol | None = None,
) -> None:
warnings.warn(
"ShallowPostgresSaver is deprecated as of version 2.0.20 and will be removed in 3.0.0. "
"Use PostgresSaver instead, and invoke the graph with `graph.invoke(..., durability='exit')`.",
DeprecationWarning,
stacklevel=2,
)
super().__init__(serde=serde)
if isinstance(conn, ConnectionPool) and pipe is not None:
raise ValueError(
"Pipeline should be used only with a single Connection, not ConnectionPool."
)
self.conn = conn
self.pipe = pipe
self.lock = threading.Lock()
self.supports_pipeline = Capabilities().has_pipeline()
@classmethod
@contextmanager
def from_conn_string(
cls, conn_string: str, *, pipeline: bool = False
) -> Iterator["ShallowPostgresSaver"]:
"""Create a new ShallowPostgresSaver instance from a connection string.
Args:
conn_string: The Postgres connection info string.
pipeline: whether to use Pipeline
Returns:
ShallowPostgresSaver: A new ShallowPostgresSaver instance.
"""
with Connection.connect(
conn_string, autocommit=True, prepare_threshold=0, row_factory=dict_row
) as conn:
if pipeline:
with conn.pipeline() as pipe:
yield cls(conn, pipe)
else:
yield cls(conn)
def setup(self) -> None:
"""Set up the checkpoint database asynchronously.
This method creates the necessary tables in the Postgres database if they don't
already exist and runs database migrations. It MUST be called directly by the user
the first time checkpointer is used.
"""
with self._cursor() as cur:
cur.execute(self.MIGRATIONS[0])
results = cur.execute(
"SELECT v FROM checkpoint_migrations ORDER BY v DESC LIMIT 1"
)
row = results.fetchone()
if row is None:
version = -1
else:
version = row["v"]
for v, migration in zip(
range(version + 1, len(self.MIGRATIONS)),
self.MIGRATIONS[version + 1 :],
strict=False,
):
cur.execute(migration)
cur.execute("INSERT INTO checkpoint_migrations (v) VALUES (%s)", (v,))
if self.pipe:
self.pipe.sync()
def list(
self,
config: RunnableConfig | None,
*,
filter: dict[str, Any] | None = None,
before: RunnableConfig | None = None,
limit: int | None = None,
) -> Iterator[CheckpointTuple]:
"""List checkpoints from the database.
This method retrieves a list of checkpoint tuples from the Postgres database based
on the provided config. For ShallowPostgresSaver, this method returns a list with
ONLY the most recent checkpoint.
"""
where, args = self._search_where(config, filter, before)
query = self.SELECT_SQL + where
params = list(args)
if limit is not None:
query += " LIMIT %s"
params.append(int(limit))
with self._cursor() as cur:
cur.execute(query, params, binary=True)
for value in cur:
checkpoint: Checkpoint = {
**value["checkpoint"],
"channel_values": self._load_blobs(value["channel_values"]),
"pending_sends": [
self.serde.loads_typed((t.decode(), v))
for t, v in value["pending_sends"]
]
if value["pending_sends"]
else [],
}
yield CheckpointTuple(
config={
"configurable": {
"thread_id": value["thread_id"],
"checkpoint_ns": value["checkpoint_ns"],
"checkpoint_id": checkpoint["id"],
}
},
checkpoint=checkpoint,
metadata=value["metadata"],
pending_writes=self._load_writes(value["pending_writes"]),
)
def get_tuple(self, config: RunnableConfig) -> CheckpointTuple | None:
"""Get a checkpoint tuple from the database.
This method retrieves a checkpoint tuple from the Postgres database based on the
provided config (matching the thread ID in the config).
Args:
config: The config to use for retrieving the checkpoint.
Returns:
The retrieved checkpoint tuple, or None if no matching checkpoint was found.
Examples:
Basic:
>>> config = {"configurable": {"thread_id": "1"}}
>>> checkpoint_tuple = memory.get_tuple(config)
>>> print(checkpoint_tuple)
CheckpointTuple(...)
With timestamp:
>>> config = {
... "configurable": {
... "thread_id": "1",
... "checkpoint_ns": "",
... "checkpoint_id": "1ef4f797-8335-6428-8001-8a1503f9b875",
... }
... }
>>> checkpoint_tuple = memory.get_tuple(config)
>>> print(checkpoint_tuple)
CheckpointTuple(...)
""" # noqa
thread_id = config["configurable"]["thread_id"]
checkpoint_ns = config["configurable"].get("checkpoint_ns", "")
args = (thread_id, checkpoint_ns)
where = "WHERE thread_id = %s AND checkpoint_ns = %s"
with self._cursor() as cur:
cur.execute(
self.SELECT_SQL + where,
args,
binary=True,
)
for value in cur:
checkpoint: Checkpoint = {
**value["checkpoint"],
"channel_values": self._load_blobs(value["channel_values"]),
"pending_sends": [
self.serde.loads_typed((t.decode(), v))
for t, v in value["pending_sends"]
]
if value["pending_sends"]
else [],
}
return CheckpointTuple(
config={
"configurable": {
"thread_id": thread_id,
"checkpoint_ns": checkpoint_ns,
"checkpoint_id": checkpoint["id"],
}
},
checkpoint=checkpoint,
metadata=value["metadata"],
pending_writes=self._load_writes(value["pending_writes"]),
)
def put(
self,
config: RunnableConfig,
checkpoint: Checkpoint,
metadata: CheckpointMetadata,
new_versions: ChannelVersions,
) -> RunnableConfig:
"""Save a checkpoint to the database.
This method saves a checkpoint to the Postgres database. The checkpoint is associated
with the provided config. For ShallowPostgresSaver, this method saves ONLY the most recent
checkpoint and overwrites a previous checkpoint, if it exists.
Args:
config: The config to associate with the checkpoint.
checkpoint: The checkpoint to save.
metadata: Additional metadata to save with the checkpoint.
new_versions: New channel versions as of this write.
Returns:
RunnableConfig: Updated configuration after storing the checkpoint.
Examples:
>>> from langgraph.checkpoint.postgres import ShallowPostgresSaver
>>> DB_URI = "postgres://postgres:postgres@localhost:5432/postgres?sslmode=disable"
>>> with ShallowPostgresSaver.from_conn_string(DB_URI) as memory:
>>> config = {"configurable": {"thread_id": "1", "checkpoint_ns": ""}}
>>> checkpoint = {"ts": "2024-05-04T06:32:42.235444+00:00", "id": "1ef4f797-8335-6428-8001-8a1503f9b875", "channel_values": {"key": "value"}}
>>> saved_config = memory.put(config, checkpoint, {"source": "input", "step": 1, "writes": {"key": "value"}}, {})
>>> print(saved_config)
{'configurable': {'thread_id': '1', 'checkpoint_ns': '', 'checkpoint_id': '1ef4f797-8335-6428-8001-8a1503f9b875'}}
"""
configurable = config["configurable"].copy()
thread_id = configurable.pop("thread_id")
checkpoint_ns = configurable.pop("checkpoint_ns")
copy = checkpoint.copy()
next_config = {
"configurable": {
"thread_id": thread_id,
"checkpoint_ns": checkpoint_ns,
"checkpoint_id": checkpoint["id"],
}
}
with self._cursor(pipeline=True) as cur:
cur.execute(
"""DELETE FROM checkpoint_writes
WHERE thread_id = %s AND checkpoint_ns = %s AND checkpoint_id NOT IN (%s, %s)""",
(
thread_id,
checkpoint_ns,
checkpoint["id"],
configurable.get("checkpoint_id", ""),
),
)
cur.executemany(
self.UPSERT_CHECKPOINT_BLOBS_SQL,
_dump_blobs(
self.serde,
thread_id,
checkpoint_ns,
copy.pop("channel_values"), # type: ignore[misc]
new_versions,
),
)
cur.execute(
self.UPSERT_CHECKPOINTS_SQL,
(
thread_id,
checkpoint_ns,
Jsonb(copy),
Jsonb(get_serializable_checkpoint_metadata(config, metadata)),
),
)
return next_config
def put_writes(
self,
config: RunnableConfig,
writes: Sequence[tuple[str, Any]],
task_id: str,
task_path: str = "",
) -> None:
"""Store intermediate writes linked to a checkpoint.
This method saves intermediate writes associated with a checkpoint to the Postgres database.
Args:
config: Configuration of the related checkpoint.
writes: List of writes to store.
task_id: Identifier for the task creating the writes.
"""
query = (
self.UPSERT_CHECKPOINT_WRITES_SQL
if all(w[0] in WRITES_IDX_MAP for w in writes)
else self.INSERT_CHECKPOINT_WRITES_SQL
)
with self._cursor(pipeline=True) as cur:
cur.executemany(
query,
self._dump_writes(
config["configurable"]["thread_id"],
config["configurable"]["checkpoint_ns"],
config["configurable"]["checkpoint_id"],
task_id,
task_path,
writes,
),
)
@contextmanager
def _cursor(self, *, pipeline: bool = False) -> Iterator[Cursor[DictRow]]:
"""Create a database cursor as a context manager.
Args:
pipeline: whether to use pipeline for the DB operations inside the context manager.
Will be applied regardless of whether the ShallowPostgresSaver instance was initialized with a pipeline.
If pipeline mode is not supported, will fall back to using transaction context manager.
"""
with _internal.get_connection(self.conn) as conn:
if self.pipe:
# a connection in pipeline mode can be used concurrently
# in multiple threads/coroutines, but only one cursor can be
# used at a time
try:
with conn.cursor(binary=True, row_factory=dict_row) as cur:
yield cur
finally:
if pipeline:
self.pipe.sync()
elif pipeline:
# a connection not in pipeline mode can only be used by one
# thread/coroutine at a time, so we acquire a lock
if self.supports_pipeline:
with (
self.lock,
conn.pipeline(),
conn.cursor(binary=True, row_factory=dict_row) as cur,
):
yield cur
else:
# Use connection's transaction context manager when pipeline mode not supported
with (
self.lock,
conn.transaction(),
conn.cursor(binary=True, row_factory=dict_row) as cur,
):
yield cur
else:
with self.lock, conn.cursor(binary=True, row_factory=dict_row) as cur:
yield cur
class AsyncShallowPostgresSaver(BasePostgresSaver):
"""A checkpoint saver that uses Postgres to store checkpoints asynchronously.
This checkpointer ONLY stores the most recent checkpoint and does NOT retain any history.
It is meant to be a light-weight drop-in replacement for the AsyncPostgresSaver that
supports most of the LangGraph persistence functionality with the exception of time travel.
"""
SELECT_SQL = SELECT_SQL
MIGRATIONS = MIGRATIONS
UPSERT_CHECKPOINT_BLOBS_SQL = UPSERT_CHECKPOINT_BLOBS_SQL
UPSERT_CHECKPOINTS_SQL = UPSERT_CHECKPOINTS_SQL
UPSERT_CHECKPOINT_WRITES_SQL = UPSERT_CHECKPOINT_WRITES_SQL
INSERT_CHECKPOINT_WRITES_SQL = INSERT_CHECKPOINT_WRITES_SQL
lock: asyncio.Lock
def __init__(
self,
conn: _ainternal.Conn,
pipe: AsyncPipeline | None = None,
serde: SerializerProtocol | None = None,
) -> None:
warnings.warn(
"AsyncShallowPostgresSaver is deprecated as of version 2.0.20 and will be removed in 3.0.0. "
"Use AsyncPostgresSaver instead, and invoke the graph with `await graph.ainvoke(..., durability='exit')`.",
DeprecationWarning,
stacklevel=2,
)
super().__init__(serde=serde)
if isinstance(conn, AsyncConnectionPool) and pipe is not None:
raise ValueError(
"Pipeline should be used only with a single AsyncConnection, not AsyncConnectionPool."
)
self.conn = conn
self.pipe = pipe
self.lock = asyncio.Lock()
self.loop = asyncio.get_running_loop()
self.supports_pipeline = Capabilities().has_pipeline()
@classmethod
@asynccontextmanager
async def from_conn_string(
cls,
conn_string: str,
*,
pipeline: bool = False,
serde: SerializerProtocol | None = None,
) -> AsyncIterator["AsyncShallowPostgresSaver"]:
"""Create a new AsyncShallowPostgresSaver instance from a connection string.
Args:
conn_string: The Postgres connection info string.
pipeline: whether to use AsyncPipeline
Returns:
AsyncShallowPostgresSaver: A new AsyncShallowPostgresSaver instance.
"""
async with await AsyncConnection.connect(
conn_string, autocommit=True, prepare_threshold=0, row_factory=dict_row
) as conn:
if pipeline:
async with conn.pipeline() as pipe:
yield cls(conn=conn, pipe=pipe, serde=serde)
else:
yield cls(conn=conn, serde=serde)
async def setup(self) -> None:
"""Set up the checkpoint database asynchronously.
This method creates the necessary tables in the Postgres database if they don't
already exist and runs database migrations. It MUST be called directly by the user
the first time checkpointer is used.
"""
async with self._cursor() as cur:
await cur.execute(self.MIGRATIONS[0])
results = await cur.execute(
"SELECT v FROM checkpoint_migrations ORDER BY v DESC LIMIT 1"
)
row = await results.fetchone()
if row is None:
version = -1
else:
version = row["v"]
for v, migration in zip(
range(version + 1, len(self.MIGRATIONS)),
self.MIGRATIONS[version + 1 :],
strict=False,
):
await cur.execute(migration)
await cur.execute(
"INSERT INTO checkpoint_migrations (v) VALUES (%s)", (v,)
)
if self.pipe:
await self.pipe.sync()
async def alist(
self,
config: RunnableConfig | None,
*,
filter: dict[str, Any] | None = None,
before: RunnableConfig | None = None,
limit: int | None = None,
) -> AsyncIterator[CheckpointTuple]:
"""List checkpoints from the database asynchronously.
This method retrieves a list of checkpoint tuples from the Postgres database based
on the provided config. For ShallowPostgresSaver, this method returns a list with
ONLY the most recent checkpoint.
"""
where, args = self._search_where(config, filter, before)
query = self.SELECT_SQL + where
params = list(args)
if limit is not None:
query += " LIMIT %s"
params.append(int(limit))
async with self._cursor() as cur:
await cur.execute(query, params, binary=True)
async for value in cur:
checkpoint: Checkpoint = {
**value["checkpoint"],
"channel_values": self._load_blobs(value["channel_values"]),
"pending_sends": [
self.serde.loads_typed((t.decode(), v))
for t, v in value["pending_sends"]
]
if value["pending_sends"]
else [],
}
yield CheckpointTuple(
config={
"configurable": {
"thread_id": value["thread_id"],
"checkpoint_ns": value["checkpoint_ns"],
"checkpoint_id": checkpoint["id"],
}
},
checkpoint=checkpoint,
metadata=value["metadata"],
pending_writes=await asyncio.to_thread(
self._load_writes, value["pending_writes"]
),
)
async def aget_tuple(self, config: RunnableConfig) -> CheckpointTuple | None:
"""Get a checkpoint tuple from the database asynchronously.
This method retrieves a checkpoint tuple from the Postgres database based on the
provided config (matching the thread ID in the config).
Args:
config: The config to use for retrieving the checkpoint.
Returns:
The retrieved checkpoint tuple, or None if no matching checkpoint was found.
"""
thread_id = config["configurable"]["thread_id"]
checkpoint_ns = config["configurable"].get("checkpoint_ns", "")
args = (thread_id, checkpoint_ns)
where = "WHERE thread_id = %s AND checkpoint_ns = %s"
async with self._cursor() as cur:
await cur.execute(
self.SELECT_SQL + where,
args,
binary=True,
)
async for value in cur:
checkpoint: Checkpoint = {
**value["checkpoint"],
"channel_values": self._load_blobs(value["channel_values"]),
"pending_sends": [
self.serde.loads_typed((t.decode(), v))
for t, v in value["pending_sends"]
]
if value["pending_sends"]
else [],
}
return CheckpointTuple(
config={
"configurable": {
"thread_id": thread_id,
"checkpoint_ns": checkpoint_ns,
"checkpoint_id": checkpoint["id"],
}
},
checkpoint=checkpoint,
metadata=value["metadata"],
pending_writes=await asyncio.to_thread(
self._load_writes, value["pending_writes"]
),
)
async def aput(
self,
config: RunnableConfig,
checkpoint: Checkpoint,
metadata: CheckpointMetadata,
new_versions: ChannelVersions,
) -> RunnableConfig:
"""Save a checkpoint to the database asynchronously.
This method saves a checkpoint to the Postgres database. The checkpoint is associated
with the provided config. For AsyncShallowPostgresSaver, this method saves ONLY the most recent
checkpoint and overwrites a previous checkpoint, if it exists.
Args:
config: The config to associate with the checkpoint.
checkpoint: The checkpoint to save.
metadata: Additional metadata to save with the checkpoint.
new_versions: New channel versions as of this write.
Returns:
RunnableConfig: Updated configuration after storing the checkpoint.
"""
configurable = config["configurable"].copy()
thread_id = configurable.pop("thread_id")
checkpoint_ns = configurable.pop("checkpoint_ns")
copy = checkpoint.copy()
next_config = {
"configurable": {
"thread_id": thread_id,
"checkpoint_ns": checkpoint_ns,
"checkpoint_id": checkpoint["id"],
}
}
async with self._cursor(pipeline=True) as cur:
await cur.execute(
"""DELETE FROM checkpoint_writes
WHERE thread_id = %s AND checkpoint_ns = %s AND checkpoint_id NOT IN (%s, %s)""",
(
thread_id,
checkpoint_ns,
checkpoint["id"],
configurable.get("checkpoint_id", ""),
),
)
await cur.executemany(
self.UPSERT_CHECKPOINT_BLOBS_SQL,
_dump_blobs(
self.serde,
thread_id,
checkpoint_ns,
copy.pop("channel_values"), # type: ignore[misc]
new_versions,
),
)
await cur.execute(
self.UPSERT_CHECKPOINTS_SQL,
(
thread_id,
checkpoint_ns,
Jsonb(copy),
Jsonb(get_serializable_checkpoint_metadata(config, metadata)),
),
)
return next_config
async def aput_writes(
self,
config: RunnableConfig,
writes: Sequence[tuple[str, Any]],
task_id: str,
task_path: str = "",
) -> None:
"""Store intermediate writes linked to a checkpoint asynchronously.
This method saves intermediate writes associated with a checkpoint to the database.
Args:
config: Configuration of the related checkpoint.
writes: List of writes to store, each as (channel, value) pair.
task_id: Identifier for the task creating the writes.
"""
query = (
self.UPSERT_CHECKPOINT_WRITES_SQL
if all(w[0] in WRITES_IDX_MAP for w in writes)
else self.INSERT_CHECKPOINT_WRITES_SQL
)
params = await asyncio.to_thread(
self._dump_writes,
config["configurable"]["thread_id"],
config["configurable"]["checkpoint_ns"],
config["configurable"]["checkpoint_id"],
task_id,
task_path,
writes,
)
async with self._cursor(pipeline=True) as cur:
await cur.executemany(query, params)
@asynccontextmanager
async def _cursor(
self, *, pipeline: bool = False
) -> AsyncIterator[AsyncCursor[DictRow]]:
"""Create a database cursor as a context manager.
Args:
pipeline: whether to use pipeline for the DB operations inside the context manager.
Will be applied regardless of whether the AsyncShallowPostgresSaver instance was initialized with a pipeline.
If pipeline mode is not supported, will fall back to using transaction context manager.
"""
async with _ainternal.get_connection(self.conn) as conn:
if self.pipe:
# a connection in pipeline mode can be used concurrently
# in multiple threads/coroutines, but only one cursor can be
# used at a time
try:
async with conn.cursor(binary=True, row_factory=dict_row) as cur:
yield cur
finally:
if pipeline:
await self.pipe.sync()
elif pipeline:
# a connection not in pipeline mode can only be used by one
# thread/coroutine at a time, so we acquire a lock
if self.supports_pipeline:
async with (
self.lock,
conn.pipeline(),
conn.cursor(binary=True, row_factory=dict_row) as cur,
):
yield cur
else:
# Use connection's transaction context manager when pipeline mode not supported
async with (
self.lock,
conn.transaction(),
conn.cursor(binary=True, row_factory=dict_row) as cur,
):
yield cur
else:
async with (
self.lock,
conn.cursor(binary=True, row_factory=dict_row) as cur,
):
yield cur
def list(
self,
config: RunnableConfig | None,
*,
filter: dict[str, Any] | None = None,
before: RunnableConfig | None = None,
limit: int | None = None,
) -> Iterator[CheckpointTuple]:
"""List checkpoints from the database.
This method retrieves a list of checkpoint tuples from the Postgres database based
on the provided config. For ShallowPostgresSaver, this method returns a list with
ONLY the most recent checkpoint.
"""
aiter_ = self.alist(config, filter=filter, before=before, limit=limit)
while True:
try:
yield asyncio.run_coroutine_threadsafe(
anext(aiter_), # type: ignore[arg-type] # noqa: F821
self.loop,
).result()
except StopAsyncIteration:
break
def get_tuple(self, config: RunnableConfig) -> CheckpointTuple | None:
"""Get a checkpoint tuple from the database.
This method retrieves a checkpoint tuple from the Postgres database based on the
provided config (matching the thread ID in the config).
Args:
config: The config to use for retrieving the checkpoint.
Returns:
The retrieved checkpoint tuple, or None if no matching checkpoint was found.
"""
try:
# check if we are in the main thread, only bg threads can block
# we don't check in other methods to avoid the overhead
if asyncio.get_running_loop() is self.loop:
raise asyncio.InvalidStateError(
"Synchronous calls to AsyncShallowPostgresSaver are only allowed from a "
"different thread. From the main thread, use the async interface."
"For example, use `await checkpointer.aget_tuple(...)` or `await "
"graph.ainvoke(...)`."
)
except RuntimeError:
pass
return asyncio.run_coroutine_threadsafe(
self.aget_tuple(config), self.loop
).result()
def put(
self,
config: RunnableConfig,
checkpoint: Checkpoint,
metadata: CheckpointMetadata,
new_versions: ChannelVersions,
) -> RunnableConfig:
"""Save a checkpoint to the database.
This method saves a checkpoint to the Postgres database. The checkpoint is associated
with the provided config. For AsyncShallowPostgresSaver, this method saves ONLY the most recent
checkpoint and overwrites a previous checkpoint, if it exists.
Args:
config: The config to associate with the checkpoint.
checkpoint: The checkpoint to save.
metadata: Additional metadata to save with the checkpoint.
new_versions: New channel versions as of this write.
Returns:
RunnableConfig: Updated configuration after storing the checkpoint.
"""
return asyncio.run_coroutine_threadsafe(
self.aput(config, checkpoint, metadata, new_versions), self.loop
).result()
def put_writes(
self,
config: RunnableConfig,
writes: Sequence[tuple[str, Any]],
task_id: str,
task_path: str = "",
) -> None:
"""Store intermediate writes linked to a checkpoint.
This method saves intermediate writes associated with a checkpoint to the database.
Args:
config: Configuration of the related checkpoint.
writes: List of writes to store, each as (channel, value) pair.
task_id: Identifier for the task creating the writes.
task_path: Path of the task creating the writes.
"""
return asyncio.run_coroutine_threadsafe(
self.aput_writes(config, writes, task_id, task_path), self.loop
).result()
@@ -1,4 +1,3 @@
from langgraph.store.postgres.aio import AsyncPostgresStore
from langgraph.store.postgres.base import PoolConfig, PostgresStore
from langgraph.store.postgres.base import PostgresStore
__all__ = ["AsyncPostgresStore", "PoolConfig", "PostgresStore"]
__all__ = ["PostgresStore"]
@@ -1,586 +0,0 @@
from __future__ import annotations
import asyncio
import logging
from collections.abc import AsyncIterator, Callable, Iterable, Sequence
from contextlib import asynccontextmanager
from types import TracebackType
from typing import Any, cast
import orjson
from langgraph.store.base import (
GetOp,
ListNamespacesOp,
Op,
PutOp,
Result,
SearchOp,
)
from langgraph.store.base.batch import AsyncBatchedBaseStore
from psycopg import AsyncConnection, AsyncCursor, AsyncPipeline, Capabilities
from psycopg.rows import DictRow, dict_row
from psycopg_pool import AsyncConnectionPool
from langgraph.checkpoint.postgres import _ainternal
from langgraph.store.postgres.base import (
PLACEHOLDER,
BasePostgresStore,
PoolConfig,
PostgresIndexConfig,
Row,
TTLConfig,
_decode_ns_bytes,
_ensure_index_config,
_group_ops,
_row_to_item,
_row_to_search_item,
)
logger = logging.getLogger(__name__)
class AsyncPostgresStore(AsyncBatchedBaseStore, BasePostgresStore[_ainternal.Conn]):
"""Asynchronous Postgres-backed store with optional vector search using pgvector.
!!! example "Examples"
Basic setup and usage:
```python
from langgraph.store.postgres import AsyncPostgresStore
conn_string = "postgresql://user:pass@localhost:5432/dbname"
async with AsyncPostgresStore.from_conn_string(conn_string) as store:
await store.setup() # Run migrations. Done once
# Store and retrieve data
await store.aput(("users", "123"), "prefs", {"theme": "dark"})
item = await store.aget(("users", "123"), "prefs")
```
Vector search using LangChain embeddings:
```python
from langchain.embeddings import init_embeddings
from langgraph.store.postgres import AsyncPostgresStore
conn_string = "postgresql://user:pass@localhost:5432/dbname"
async with AsyncPostgresStore.from_conn_string(
conn_string,
index={
"dims": 1536,
"embed": init_embeddings("openai:text-embedding-3-small"),
"fields": ["text"] # specify which fields to embed. Default is the whole serialized value
}
) as store:
await store.setup() # Run migrations. Done once
# Store documents
await store.aput(("docs",), "doc1", {"text": "Python tutorial"})
await store.aput(("docs",), "doc2", {"text": "TypeScript guide"})
await store.aput(("docs",), "doc3", {"text": "Other guide"}, index=False) # don't index
# Search by similarity
results = await store.asearch(("docs",), query="programming guides", limit=2)
```
Using connection pooling for better performance:
```python
from langgraph.store.postgres import AsyncPostgresStore, PoolConfig
conn_string = "postgresql://user:pass@localhost:5432/dbname"
async with AsyncPostgresStore.from_conn_string(
conn_string,
pool_config=PoolConfig(
min_size=5,
max_size=20
)
) as store:
await store.setup() # Run migrations. Done once
# Use store with connection pooling...
```
Warning:
Make sure to:
1. Call `setup()` before first use to create necessary tables and indexes
2. Have the pgvector extension available to use vector search
3. Use Python 3.10+ for async functionality
Note:
Semantic search is disabled by default. You can enable it by providing an `index` configuration
when creating the store. Without this configuration, all `index` arguments passed to
`put` or `aput` will have no effect.
Note:
If you provide a TTL configuration, you must explicitly call `start_ttl_sweeper()` to begin
the background task that removes expired items. Call `stop_ttl_sweeper()` to properly
clean up resources when you're done with the store.
"""
__slots__ = (
"_deserializer",
"pipe",
"lock",
"supports_pipeline",
"index_config",
"embeddings",
"ttl_config",
"_ttl_sweeper_task",
"_ttl_stop_event",
)
supports_ttl: bool = True
def __init__(
self,
conn: _ainternal.Conn,
*,
pipe: AsyncPipeline | None = None,
deserializer: Callable[[bytes | orjson.Fragment], dict[str, Any]] | None = None,
index: PostgresIndexConfig | None = None,
ttl: TTLConfig | None = None,
) -> None:
if isinstance(conn, AsyncConnectionPool) and pipe is not None:
raise ValueError(
"Pipeline should be used only with a single AsyncConnection, not AsyncConnectionPool."
)
super().__init__()
self._deserializer = deserializer
self.conn = conn
self.pipe = pipe
self.lock = asyncio.Lock()
self.loop = asyncio.get_running_loop()
self.supports_pipeline = Capabilities().has_pipeline()
self.index_config = index
if self.index_config:
self.embeddings, self.index_config = _ensure_index_config(self.index_config)
else:
self.embeddings = None
self.ttl_config = ttl
self._ttl_sweeper_task: asyncio.Task[None] | None = None
self._ttl_stop_event = asyncio.Event()
async def abatch(self, ops: Iterable[Op]) -> list[Result]:
grouped_ops, num_ops = _group_ops(ops)
results: list[Result] = [None] * num_ops
async with _ainternal.get_connection(self.conn) as conn:
if self.pipe:
async with self.pipe:
await self._execute_batch(grouped_ops, results, conn)
else:
await self._execute_batch(grouped_ops, results, conn)
return results
@classmethod
@asynccontextmanager
async def from_conn_string(
cls,
conn_string: str,
*,
pipeline: bool = False,
pool_config: PoolConfig | None = None,
index: PostgresIndexConfig | None = None,
ttl: TTLConfig | None = None,
) -> AsyncIterator[AsyncPostgresStore]:
"""Create a new AsyncPostgresStore instance from a connection string.
Args:
conn_string: The Postgres connection info string.
pipeline: Whether to use AsyncPipeline (only for single connections)
pool_config: Configuration for the connection pool.
If provided, will create a connection pool and use it instead of a single connection.
This overrides the `pipeline` argument.
index: The embedding config.
Returns:
AsyncPostgresStore: A new AsyncPostgresStore instance.
"""
if pool_config is not None:
pc = pool_config.copy()
async with cast(
AsyncConnectionPool[AsyncConnection[DictRow]],
AsyncConnectionPool(
conn_string,
min_size=pc.pop("min_size", 1),
max_size=pc.pop("max_size", None),
kwargs={
"autocommit": True,
"prepare_threshold": 0,
"row_factory": dict_row,
**(pc.pop("kwargs", None) or {}),
},
**cast(dict, pc),
),
) as pool:
yield cls(conn=pool, index=index, ttl=ttl)
else:
async with await AsyncConnection.connect(
conn_string, autocommit=True, prepare_threshold=0, row_factory=dict_row
) as conn:
if pipeline:
async with conn.pipeline() as pipe:
yield cls(conn=conn, pipe=pipe, index=index, ttl=ttl)
else:
yield cls(conn=conn, index=index, ttl=ttl)
async def setup(self) -> None:
"""Set up the store database asynchronously.
This method creates the necessary tables in the Postgres database if they don't
already exist and runs database migrations. It MUST be called directly by the user
the first time the store is used.
"""
async def _get_version(cur: AsyncCursor[DictRow], table: str) -> int:
await cur.execute(
f"""
CREATE TABLE IF NOT EXISTS {table} (
v INTEGER PRIMARY KEY
)
"""
)
await cur.execute(f"SELECT v FROM {table} ORDER BY v DESC LIMIT 1")
row = cast(dict, await cur.fetchone())
if row is None:
version = -1
else:
version = row["v"]
return version
async with self._cursor() as cur:
version = await _get_version(cur, table="store_migrations")
for v, sql in enumerate(self.MIGRATIONS[version + 1 :], start=version + 1):
await cur.execute(sql)
await cur.execute("INSERT INTO store_migrations (v) VALUES (%s)", (v,))
if self.index_config:
version = await _get_version(cur, table="vector_migrations")
for v, migration in enumerate(
self.VECTOR_MIGRATIONS[version + 1 :], start=version + 1
):
sql = migration.sql
if migration.params:
params = {
k: v(self) if v is not None and callable(v) else v
for k, v in migration.params.items()
}
if "dims" in params:
try:
params["dims"] = int(params["dims"])
except Exception as e:
raise ValueError(
f"Invalid dims for vector index: {params['dims']}"
) from e
if "vector_type" in params:
vt = str(params["vector_type"])
if vt not in ("vector", "halfvec"):
raise ValueError(
f"Invalid vector_type for pgvector: {vt}"
)
params["vector_type"] = vt
if "index_type" in params:
it = str(params["index_type"])
if it not in ("hnsw", "ivfflat"):
raise ValueError(
f"Invalid index_type for pgvector: {it}"
)
params["index_type"] = it
sql = sql % params
await cur.execute(sql)
await cur.execute(
"INSERT INTO vector_migrations (v) VALUES (%s)", (v,)
)
async def sweep_ttl(self) -> int:
"""Delete expired store items based on TTL.
Returns:
int: The number of deleted items.
"""
async with self._cursor() as cur:
await cur.execute(
"""
DELETE FROM store
WHERE expires_at IS NOT NULL AND expires_at < NOW()
"""
)
deleted_count = cur.rowcount
return deleted_count
async def start_ttl_sweeper(
self, sweep_interval_minutes: int | None = None
) -> asyncio.Task[None]:
"""Periodically delete expired store items based on TTL.
Returns:
Task that can be awaited or cancelled.
"""
if not self.ttl_config:
return asyncio.create_task(asyncio.sleep(0))
if self._ttl_sweeper_task is not None and not self._ttl_sweeper_task.done():
return self._ttl_sweeper_task
self._ttl_stop_event.clear()
interval = float(
sweep_interval_minutes or self.ttl_config.get("sweep_interval_minutes") or 5
)
logger.info(f"Starting store TTL sweeper with interval {interval} minutes")
async def _sweep_loop() -> None:
while not self._ttl_stop_event.is_set():
try:
try:
await asyncio.wait_for(
self._ttl_stop_event.wait(),
timeout=interval * 60,
)
break
except asyncio.TimeoutError:
pass
expired_items = await self.sweep_ttl()
if expired_items > 0:
logger.info(f"Store swept {expired_items} expired items")
except asyncio.CancelledError:
break
except Exception as exc:
logger.exception("Store TTL sweep iteration failed", exc_info=exc)
task = asyncio.create_task(_sweep_loop())
task.set_name("ttl_sweeper")
self._ttl_sweeper_task = task
return task
async def stop_ttl_sweeper(self, timeout: float | None = None) -> bool:
"""Stop the TTL sweeper task if it's running.
Args:
timeout: Maximum time to wait for the task to stop, in seconds.
If `None`, wait indefinitely.
Returns:
bool: True if the task was successfully stopped or wasn't running,
False if the timeout was reached before the task stopped.
"""
if self._ttl_sweeper_task is None or self._ttl_sweeper_task.done():
return True
logger.info("Stopping TTL sweeper task")
self._ttl_stop_event.set()
if timeout is not None:
try:
await asyncio.wait_for(self._ttl_sweeper_task, timeout=timeout)
success = True
except asyncio.TimeoutError:
success = False
else:
await self._ttl_sweeper_task
success = True
if success:
self._ttl_sweeper_task = None
logger.info("TTL sweeper task stopped")
else:
logger.warning("Timed out waiting for TTL sweeper task to stop")
return success
async def __aenter__(self) -> AsyncPostgresStore:
return self
async def __aexit__(
self,
exc_type: type[BaseException] | None,
exc_val: BaseException | None,
exc_tb: TracebackType | None,
) -> None:
# Ensure the TTL sweeper task is stopped when exiting the context
if hasattr(self, "_ttl_sweeper_task") and self._ttl_sweeper_task is not None:
# Set the event to signal the task to stop
self._ttl_stop_event.set()
# We don't wait for the task to complete here to avoid blocking
# The task will clean up itself gracefully
async def _execute_batch(
self,
grouped_ops: dict,
results: list[Result],
conn: AsyncConnection[DictRow],
) -> None:
async with self._cursor(pipeline=True) as cur:
if GetOp in grouped_ops:
await self._batch_get_ops(
cast(Sequence[tuple[int, GetOp]], grouped_ops[GetOp]),
results,
cur,
)
if SearchOp in grouped_ops:
await self._batch_search_ops(
cast(Sequence[tuple[int, SearchOp]], grouped_ops[SearchOp]),
results,
cur,
)
if ListNamespacesOp in grouped_ops:
await self._batch_list_namespaces_ops(
cast(
Sequence[tuple[int, ListNamespacesOp]],
grouped_ops[ListNamespacesOp],
),
results,
cur,
)
if PutOp in grouped_ops:
await self._batch_put_ops(
cast(Sequence[tuple[int, PutOp]], grouped_ops[PutOp]),
cur,
)
async def _batch_get_ops(
self,
get_ops: Sequence[tuple[int, GetOp]],
results: list[Result],
cur: AsyncCursor[DictRow],
) -> None:
for query, params, namespace, items in self._get_batch_GET_ops_queries(get_ops):
await cur.execute(query, params)
rows = cast(list[Row], await cur.fetchall())
key_to_row = {row["key"]: row for row in rows}
for idx, key in items:
row = key_to_row.get(key)
if row:
results[idx] = _row_to_item(
namespace, row, loader=self._deserializer
)
else:
results[idx] = None
async def _batch_put_ops(
self,
put_ops: Sequence[tuple[int, PutOp]],
cur: AsyncCursor[DictRow],
) -> None:
queries, embedding_request = self._prepare_batch_PUT_queries(put_ops)
if embedding_request:
if self.embeddings is None:
# Should not get here since the embedding config is required
# to return an embedding_request above
raise ValueError(
"Embedding configuration is required for vector operations "
f"(for semantic search). "
f"Please provide an EmbeddingConfig when initializing the {self.__class__.__name__}."
)
query, txt_params = embedding_request
vectors = await self.embeddings.aembed_documents(
[param[-1] for param in txt_params]
)
queries.append(
(
query,
[
p
for (ns, k, pathname, _), vector in zip(
txt_params, vectors, strict=False
)
for p in (ns, k, pathname, vector)
],
)
)
for query, params in queries:
await cur.execute(query, params)
async def _batch_search_ops(
self,
search_ops: Sequence[tuple[int, SearchOp]],
results: list[Result],
cur: AsyncCursor[DictRow],
) -> None:
queries, embedding_requests = self._prepare_batch_search_queries(search_ops)
if embedding_requests and self.embeddings:
vectors = await self.embeddings.aembed_documents(
[query for _, query in embedding_requests]
)
for (idx, _), vector in zip(embedding_requests, vectors, strict=False):
_paramslist = queries[idx][1]
for i in range(len(_paramslist)):
if _paramslist[i] is PLACEHOLDER:
_paramslist[i] = vector
for (idx, _), (query, params) in zip(search_ops, queries, strict=False):
await cur.execute(query, params)
rows = cast(list[Row], await cur.fetchall())
items = [
_row_to_search_item(
_decode_ns_bytes(row["prefix"]), row, loader=self._deserializer
)
for row in rows
]
results[idx] = items
async def _batch_list_namespaces_ops(
self,
list_ops: Sequence[tuple[int, ListNamespacesOp]],
results: list[Result],
cur: AsyncCursor[DictRow],
) -> None:
queries = self._get_batch_list_namespaces_queries(list_ops)
for (query, params), (idx, _) in zip(queries, list_ops, strict=False):
await cur.execute(query, params)
rows = cast(list[dict], await cur.fetchall())
namespaces = [_decode_ns_bytes(row["truncated_prefix"]) for row in rows]
results[idx] = namespaces
@asynccontextmanager
async def _cursor(
self, *, pipeline: bool = False
) -> AsyncIterator[AsyncCursor[DictRow]]:
"""Create a database cursor as a context manager.
Args:
pipeline: whether to use pipeline for the DB operations inside the context manager.
Will be applied regardless of whether the PostgresStore instance was initialized with a pipeline.
If pipeline mode is not supported, will fall back to using transaction context manager.
"""
async with _ainternal.get_connection(self.conn) as conn:
if self.pipe:
# a connection in pipeline mode can be used concurrently
# in multiple threads/coroutines, but only one cursor can be
# used at a time
try:
async with conn.cursor(binary=True, row_factory=dict_row) as cur:
yield cur
finally:
if pipeline:
await self.pipe.sync()
elif pipeline:
# a connection not in pipeline mode can only be used by one
# thread/coroutine at a time, so we acquire a lock
if self.supports_pipeline:
async with (
self.lock,
conn.pipeline(),
conn.cursor(binary=True, row_factory=dict_row) as cur,
):
yield cur
else:
async with (
self.lock,
conn.transaction(),
conn.cursor(binary=True, row_factory=dict_row) as cur,
):
yield cur
else:
async with (
self.lock,
conn.cursor(binary=True, row_factory=dict_row) as cur,
):
yield cur
@@ -1,27 +1,34 @@
from __future__ import annotations
import asyncio
import concurrent.futures
import json
import logging
import threading
from collections import defaultdict
from collections.abc import Callable, Iterable, Iterator, Sequence
from collections.abc import Iterable, Iterator, Sequence
from contextlib import contextmanager
from datetime import datetime
from typing import (
TYPE_CHECKING,
Any,
Callable,
Generic,
Literal,
NamedTuple,
Optional,
TypeVar,
Union,
cast,
)
import orjson
from psycopg import Capabilities, Connection, Cursor, Pipeline
from psycopg.rows import DictRow, dict_row
from psycopg.types.json import Jsonb
from psycopg_pool import ConnectionPool
from typing_extensions import TypedDict
from langgraph.checkpoint.postgres import _internal as _pg_internal
from langgraph.store.base import (
BaseStore,
Embeddings,
GetOp,
IndexConfig,
Item,
@@ -31,22 +38,10 @@ from langgraph.store.base import (
Result,
SearchItem,
SearchOp,
TTLConfig,
ensure_embeddings,
get_text_at_path,
tokenize_path,
)
from psycopg import Capabilities, Connection, Cursor, Pipeline
from psycopg.rows import DictRow, dict_row
from psycopg.types.json import Jsonb
from psycopg_pool import ConnectionPool
from typing_extensions import TypedDict
from langgraph.checkpoint.postgres import _ainternal as _ainternal
from langgraph.checkpoint.postgres import _internal as _pg_internal
if TYPE_CHECKING:
from langchain_core.embeddings import Embeddings
logger = logging.getLogger(__name__)
@@ -55,8 +50,8 @@ class Migration(NamedTuple):
"""A database migration with optional conditions and parameters."""
sql: str
params: dict[str, Any] | None = None
condition: Callable[[BasePostgresStore], bool] | None = None
params: Optional[dict[str, Any]] = None
condition: Optional[Callable[["BasePostgresStore"], bool]] = None
MIGRATIONS: Sequence[str] = [
@@ -74,29 +69,13 @@ CREATE TABLE IF NOT EXISTS store (
"""
-- For faster lookups by prefix
CREATE INDEX CONCURRENTLY IF NOT EXISTS store_prefix_idx ON store USING btree (prefix text_pattern_ops);
""",
"""
-- Add expires_at column to store table
ALTER TABLE store
ADD COLUMN IF NOT EXISTS expires_at TIMESTAMP WITH TIME ZONE,
ADD COLUMN IF NOT EXISTS ttl_minutes INT;
""",
"""
-- Add indexes for efficient TTL sweeping
CREATE INDEX IF NOT EXISTS idx_store_expires_at ON store (expires_at)
WHERE expires_at IS NOT NULL;
""",
]
VECTOR_MIGRATIONS: Sequence[Migration] = [
Migration(
"""
DO $$
BEGIN
IF NOT EXISTS (SELECT 1 FROM pg_extension WHERE extname = 'vector') THEN
CREATE EXTENSION vector;
END IF;
END $$;
CREATE EXTENSION IF NOT EXISTS vector;
""",
),
Migration(
@@ -144,14 +123,13 @@ CREATE INDEX CONCURRENTLY IF NOT EXISTS store_vectors_embedding_idx ON store_vec
]
C = TypeVar("C", bound=_pg_internal.Conn | _ainternal.Conn)
C = TypeVar("C", bound=_pg_internal.Conn)
class PoolConfig(TypedDict, total=False):
"""Connection pool settings for PostgreSQL connections.
Controls connection lifecycle and resource utilization:
- Small pools (1-5) suit low-concurrency workloads
- Larger pools handle concurrent requests but consume more resources
- Setting max_size prevents resource exhaustion under load
@@ -160,14 +138,13 @@ class PoolConfig(TypedDict, total=False):
min_size: int
"""Minimum number of connections maintained in the pool. Defaults to 1."""
max_size: int | None
max_size: Optional[int]
"""Maximum number of connections allowed in the pool. None means unlimited."""
kwargs: dict
"""Additional connection arguments passed to each connection in the pool.
Default kwargs set automatically:
- autocommit: True
- prepare_threshold: 0
- row_factory: dict_row
@@ -236,62 +213,27 @@ class BasePostgresStore(Generic[C]):
MIGRATIONS = MIGRATIONS
VECTOR_MIGRATIONS = VECTOR_MIGRATIONS
conn: C
_deserializer: Callable[[bytes | orjson.Fragment], dict[str, Any]] | None
index_config: PostgresIndexConfig | None
_deserializer: Optional[Callable[[Union[bytes, orjson.Fragment]], dict[str, Any]]]
index_config: Optional[PostgresIndexConfig]
def _get_batch_GET_ops_queries(
self,
get_ops: Sequence[tuple[int, GetOp]],
) -> list[tuple[str, tuple, tuple[str, ...], list]]:
"""
Build queries to fetch (and optionally refresh the TTL of) multiple keys per namespace.
Each returned element is a tuple of:
(sql_query_string, sql_params, namespace, items_for_this_namespace)
where items_for_this_namespace is the original list of (idx, key, refresh_ttl).
"""
namespace_groups = defaultdict(list)
refresh_ttls = defaultdict(list)
for idx, op in get_ops:
namespace_groups[op.namespace].append((idx, op.key))
refresh_ttls[op.namespace].append(op.refresh_ttl)
results = []
for namespace, items in namespace_groups.items():
_, keys = zip(*items, strict=False)
this_refresh_ttls = refresh_ttls[namespace]
query = """
WITH passed_in AS (
SELECT unnest(%s::text[]) AS key,
unnest(%s::bool[]) AS do_refresh
),
updated AS (
UPDATE store s
SET expires_at = NOW() + (s.ttl_minutes || ' minutes')::interval
FROM passed_in p
WHERE s.prefix = %s
AND s.key = p.key
AND p.do_refresh = TRUE
AND s.ttl_minutes IS NOT NULL
RETURNING s.key
)
SELECT s.key, s.value, s.created_at, s.updated_at
FROM store s
JOIN passed_in p ON s.key = p.key
WHERE s.prefix = %s
_, keys = zip(*items)
keys_to_query = ",".join(["%s"] * len(keys))
query = f"""
SELECT key, value, created_at, updated_at
FROM store
WHERE prefix = %s AND key IN ({keys_to_query})
"""
ns_text = _namespace_to_text(namespace)
params = (
list(keys), # -> unnest(%s::text[])
list(this_refresh_ttls), # -> unnest(%s::bool[])
ns_text, # -> prefix = %s (for UPDATE)
ns_text, # -> prefix = %s (for final SELECT)
)
params = (_namespace_to_text(namespace), *keys)
results.append((query, params, namespace, items))
return results
def _prepare_batch_PUT_queries(
@@ -299,8 +241,9 @@ class BasePostgresStore(Generic[C]):
put_ops: Sequence[tuple[int, PutOp]],
) -> tuple[
list[tuple[str, Sequence]],
tuple[str, Sequence[tuple[str, str, str, str]]] | None,
Optional[tuple[str, Sequence[tuple[str, str, str, str]]]],
]:
# Last-write wins
dedupped_ops: dict[tuple[tuple[str, ...], str], PutOp] = {}
for _, op in put_ops:
dedupped_ops[(op.namespace, op.key)] = op
@@ -326,39 +269,25 @@ class BasePostgresStore(Generic[C]):
)
params = (_namespace_to_text(namespace), *keys)
queries.append((query, params))
embedding_request: tuple[str, Sequence[tuple[str, str, str, str]]] | None = None
embedding_request: Optional[tuple[str, Sequence[tuple[str, str, str, str]]]] = (
None
)
if inserts:
values = []
insertion_params: list[Any] = []
insertion_params = []
vector_values = []
embedding_request_params = []
# Handle TTL expiration
# First handle main store insertions
for op in inserts:
values.append("(%s, %s, %s, CURRENT_TIMESTAMP, CURRENT_TIMESTAMP)")
insertion_params.extend(
(
[
_namespace_to_text(op.namespace),
op.key,
Jsonb(cast(dict, op.value)),
)
]
)
if op.ttl is not None:
values.append(
"(%s, %s, %s, CURRENT_TIMESTAMP, CURRENT_TIMESTAMP, NOW() + %s::interval, %s)"
)
ttl_minutes = float(op.ttl)
insertion_params.extend(
(
f"{ttl_minutes * 60} seconds",
ttl_minutes,
)
)
else:
values.append(
"(%s, %s, %s, CURRENT_TIMESTAMP, CURRENT_TIMESTAMP, NULL, %s)"
)
insertion_params.append(None)
# Then handle embeddings if configured
if self.index_config:
@@ -370,7 +299,7 @@ class BasePostgresStore(Generic[C]):
k = op.key
if op.index is None:
paths = cast(dict, self.index_config)["__tokenized_fields"]
paths = self.index_config["__tokenized_fields"]
else:
paths = [(ix, tokenize_path(ix)) for ix in op.index]
@@ -385,13 +314,11 @@ class BasePostgresStore(Generic[C]):
values_str = ",".join(values)
query = f"""
INSERT INTO store (prefix, key, value, created_at, updated_at, expires_at, ttl_minutes)
INSERT INTO store (prefix, key, value, created_at, updated_at)
VALUES {values_str}
ON CONFLICT (prefix, key) DO UPDATE
SET value = EXCLUDED.value,
updated_at = CURRENT_TIMESTAMP,
expires_at = EXCLUDED.expires_at,
ttl_minutes = EXCLUDED.ttl_minutes
updated_at = CURRENT_TIMESTAMP
"""
queries.append((query, insertion_params))
@@ -412,156 +339,120 @@ class BasePostgresStore(Generic[C]):
self,
search_ops: Sequence[tuple[int, SearchOp]],
) -> tuple[
list[tuple[str, list[None | str | list[float]]]], # queries, params
list[tuple[str, list[Union[None, str, list[float]]]]], # queries, params
list[tuple[int, str]], # idx, query_text pairs to embed
]:
"""
Build per-SearchOp SQL queries (with optional TTL refresh) plus embedding requests.
Returns:
- queries: list of (SQL, param_list)
- embedding_requests: list of (original_index_in_search_ops, text_query)
"""
queries = []
embedding_requests = []
for idx, (_, op) in enumerate(search_ops):
# Build filter conditions first
filter_params = []
filter_clauses = []
filter_conditions = []
if op.filter:
for key, value in op.filter.items():
if isinstance(value, dict):
for op_name, val in value.items():
condition, params_ = self._get_filter_condition(
condition, filter_params_ = self._get_filter_condition(
key, op_name, val
)
filter_clauses.append(condition)
filter_params.extend(params_)
filter_conditions.append(condition)
filter_params.extend(filter_params_)
else:
filter_clauses.append("value->%s = %s::jsonb")
filter_params.extend([key, orjson.dumps(value).decode("utf-8")])
ns_condition = "TRUE"
ns_param: Sequence[str] | None = None
if op.namespace_prefix:
ns_condition = "store.prefix LIKE %s"
ns_param = (f"{_namespace_to_text(op.namespace_prefix)}%",)
else:
ns_param = ()
extra_filters = (
" AND " + " AND ".join(filter_clauses) if filter_clauses else ""
)
filter_conditions.append("value->%s = %s::jsonb")
filter_params.extend([key, json.dumps(value)])
# Vector search branch
if op.query and self.index_config:
# We'll embed the text later, so record the request.
embedding_requests.append((idx, op.query))
score_operator, post_operator = get_distance_operator(self)
post_operator = post_operator.replace("scored", "uniq")
vector_type = self.index_config.get("ann_index_config", {}).get(
"vector_type", "vector"
score_operator, post_operator = _get_distance_operator(self)
vector_type = (
cast(PostgresIndexConfig, self.index_config)
.get("ann_index_config", {})
.get("vector_type", "vector")
)
# For hamming bit vectors, or “regular” vectors
if (
vector_type == "bit"
and cast(dict, self.index_config).get("distance_type") == "hamming"
and self.index_config.get("distance_type") == "hamming"
):
score_operator = score_operator % (
"%s",
cast(dict, self.index_config)["dims"],
self.index_config["dims"],
)
else:
if vector_type not in ("vector", "halfvec"):
raise ValueError(
f"Invalid vector_type for pgvector: {vector_type}"
)
score_operator = score_operator % ("%s", vector_type)
score_operator = score_operator % (
"%s",
vector_type,
)
vectors_per_doc_estimate = cast(dict, self.index_config)[
"__estimated_num_vectors"
]
vectors_per_doc_estimate = self.index_config["__estimated_num_vectors"]
expanded_limit = (op.limit * vectors_per_doc_estimate * 2) + 1
# “sub_scored” does the main vector search
# Then we do DISTINCT ON to drop duplicates if your store can have them
# Finally we limit & offset
vector_search_cte = f"""
SELECT store.prefix, store.key, store.value, store.created_at, store.updated_at,
{score_operator} AS neg_score
FROM store
JOIN store_vectors sv ON store.prefix = sv.prefix AND store.key = sv.key
WHERE {ns_condition} {extra_filters}
ORDER BY {score_operator} ASC
LIMIT %s
"""
# Vector search with CTE for proper score handling
filter_str = (
""
if not filter_conditions
else " AND " + " AND ".join(filter_conditions)
)
if op.namespace_prefix:
prefix_filter_str = f"WHERE s.prefix LIKE %s {filter_str} "
ns_args: Sequence = (f"{_namespace_to_text(op.namespace_prefix)}%",)
else:
ns_args = ()
if filter_str:
prefix_filter_str = f"WHERE {filter_str} "
else:
prefix_filter_str = ""
search_results_sql = f"""
WITH scored AS (
{vector_search_cte}
)
SELECT uniq.prefix, uniq.key, uniq.value, uniq.created_at, uniq.updated_at,
{post_operator} AS score
FROM (
SELECT DISTINCT ON (scored.prefix, scored.key)
scored.prefix, scored.key, scored.value, scored.created_at, scored.updated_at, scored.neg_score
FROM scored
ORDER BY scored.prefix, scored.key, scored.neg_score ASC
) uniq
ORDER BY score DESC
base_query = f"""
WITH scored AS (
SELECT s.prefix, s.key, s.value, s.created_at, s.updated_at, {score_operator} AS neg_score
FROM store s
JOIN store_vectors sv ON s.prefix = sv.prefix AND s.key = sv.key
{prefix_filter_str}
ORDER BY {score_operator} ASC
LIMIT %s
OFFSET %s
"""
search_results_params = [
PLACEHOLDER,
*ns_param,
)
SELECT * FROM (
SELECT DISTINCT ON (prefix, key)
prefix, key, value, created_at, updated_at, {post_operator} as score
FROM scored
ORDER BY prefix, key, score DESC
) AS unique_docs
ORDER BY score DESC
LIMIT %s
OFFSET %s
"""
params = [
_PLACEHOLDER, # Vector placeholder
*ns_args,
*filter_params,
PLACEHOLDER,
_PLACEHOLDER,
expanded_limit,
op.limit,
op.offset,
]
# Regular search branch
else:
base_query = f"""
SELECT store.prefix, store.key, store.value, store.created_at, store.updated_at, NULL AS score
FROM store
WHERE {ns_condition} {extra_filters}
ORDER BY store.updated_at DESC
LIMIT %s
OFFSET %s
"""
search_results_sql = base_query
search_results_params = [
*ns_param,
*filter_params,
op.limit,
op.offset,
]
base_query = """
SELECT prefix, key, value, created_at, updated_at
FROM store
WHERE prefix LIKE %s
"""
params = [f"{_namespace_to_text(op.namespace_prefix)}%"]
if op.refresh_ttl:
# Wrap entire primary query in a CTE, then perform "update_at"
final_sql = f"""
WITH search_results AS (
{search_results_sql}
),
updated AS (
UPDATE store s
SET expires_at = NOW() + (s.ttl_minutes || ' minutes')::interval
FROM search_results sr
WHERE s.prefix = sr.prefix
AND s.key = sr.key
AND s.ttl_minutes IS NOT NULL
)
SELECT sr.prefix, sr.key, sr.value, sr.created_at, sr.updated_at, sr.score
FROM search_results sr
"""
final_params = search_results_params[:] # copy
else:
final_sql = search_results_sql
final_params = search_results_params
queries.append((final_sql, final_params))
if filter_conditions:
params.extend(filter_params)
base_query += " AND " + " AND ".join(filter_conditions)
base_query += " ORDER BY updated_at DESC"
base_query += " LIMIT %s OFFSET %s"
params.extend([op.limit, op.offset])
queries.append((base_query, params))
return queries, embedding_requests
@@ -571,7 +462,7 @@ class BasePostgresStore(Generic[C]):
) -> list[tuple[str, Sequence]]:
queries: list[tuple[str, Sequence]] = []
for _, op in list_ops:
query = r"""
query = """
SELECT DISTINCT ON (truncated_prefix) truncated_prefix, prefix
FROM (
SELECT
@@ -658,8 +549,7 @@ class PostgresStore(BaseStore, BasePostgresStore[_pg_internal.Conn]):
item = store.get(("users", "123"), "prefs")
```
Or using the convenient `from_conn_string` helper:
Or using the convenient from_conn_string helper:
```python
from langgraph.store.postgres import PostgresStore
@@ -696,7 +586,7 @@ class PostgresStore(BaseStore, BasePostgresStore[_pg_internal.Conn]):
store.put(("docs",), "doc2", {"text": "Other guide"}, index=False) # don't index
# Search by similarity
results = store.search(("docs",), query="programming guides", limit=2)
results = store.search(("docs",), "programming guides", limit=2)
```
Note:
@@ -708,11 +598,6 @@ class PostgresStore(BaseStore, BasePostgresStore[_pg_internal.Conn]):
Make sure to call `setup()` before first use to create necessary tables and indexes.
The pgvector extension must be available to use vector search.
Note:
If you provide a TTL configuration, you must explicitly call `start_ttl_sweeper()` to begin
the background thread that removes expired items. Call `stop_ttl_sweeper()` to properly
clean up resources when you're done with the store.
"""
__slots__ = (
@@ -722,19 +607,17 @@ class PostgresStore(BaseStore, BasePostgresStore[_pg_internal.Conn]):
"supports_pipeline",
"index_config",
"embeddings",
"_ttl_sweeper_thread",
"_ttl_stop_event",
)
supports_ttl: bool = True
def __init__(
self,
conn: _pg_internal.Conn,
*,
pipe: Pipeline | None = None,
deserializer: Callable[[bytes | orjson.Fragment], dict[str, Any]] | None = None,
index: PostgresIndexConfig | None = None,
ttl: TTLConfig | None = None,
pipe: Optional[Pipeline] = None,
deserializer: Optional[
Callable[[Union[bytes, orjson.Fragment]], dict[str, Any]]
] = None,
index: Optional[PostgresIndexConfig] = None,
) -> None:
super().__init__()
self._deserializer = deserializer
@@ -747,9 +630,6 @@ class PostgresStore(BaseStore, BasePostgresStore[_pg_internal.Conn]):
self.embeddings, self.index_config = _ensure_index_config(self.index_config)
else:
self.embeddings = None
self.ttl_config = ttl
self._ttl_sweeper_thread: threading.Thread | None = None
self._ttl_stop_event = threading.Event()
@classmethod
@contextmanager
@@ -758,20 +638,18 @@ class PostgresStore(BaseStore, BasePostgresStore[_pg_internal.Conn]):
conn_string: str,
*,
pipeline: bool = False,
pool_config: PoolConfig | None = None,
index: PostgresIndexConfig | None = None,
ttl: TTLConfig | None = None,
) -> Iterator[PostgresStore]:
pool_config: Optional[PoolConfig] = None,
index: Optional[PostgresIndexConfig] = None,
) -> Iterator["PostgresStore"]:
"""Create a new PostgresStore instance from a connection string.
Args:
conn_string: The Postgres connection info string.
pipeline: whether to use Pipeline
pool_config: Configuration for the connection pool.
conn_string (str): The Postgres connection info string.
pipeline (bool): whether to use Pipeline
pool_config (Optional[PoolArgs]): Configuration for the connection pool.
If provided, will create a connection pool and use it instead of a single connection.
This overrides the `pipeline` argument.
index: The index configuration for the store.
ttl: The TTL configuration for the store.
index (Optional[PostgresIndexConfig]): The index configuration for the store.
Returns:
PostgresStore: A new PostgresStore instance.
@@ -793,130 +671,23 @@ class PostgresStore(BaseStore, BasePostgresStore[_pg_internal.Conn]):
**cast(dict, pc),
),
) as pool:
yield cls(conn=pool, index=index, ttl=ttl)
yield cls(conn=pool, index=index)
else:
with Connection.connect(
conn_string, autocommit=True, prepare_threshold=0, row_factory=dict_row
) as conn:
if pipeline:
with conn.pipeline() as pipe:
yield cls(conn, pipe=pipe, index=index, ttl=ttl)
yield cls(conn, pipe=pipe, index=index)
else:
yield cls(conn, index=index, ttl=ttl)
def sweep_ttl(self) -> int:
"""Delete expired store items based on TTL.
Returns:
int: The number of deleted items.
"""
with self._cursor() as cur:
cur.execute(
"""
DELETE FROM store
WHERE expires_at IS NOT NULL AND expires_at < NOW()
"""
)
deleted_count = cur.rowcount
return deleted_count
def start_ttl_sweeper(
self, sweep_interval_minutes: int | None = None
) -> concurrent.futures.Future[None]:
"""Periodically delete expired store items based on TTL.
Returns:
Future that can be waited on or cancelled.
"""
if not self.ttl_config:
future: concurrent.futures.Future[None] = concurrent.futures.Future()
future.set_result(None)
return future
if self._ttl_sweeper_thread and self._ttl_sweeper_thread.is_alive():
logger.info("TTL sweeper thread is already running")
# Return a future that can be used to cancel the existing thread
future = concurrent.futures.Future()
future.add_done_callback(
lambda f: self._ttl_stop_event.set() if f.cancelled() else None
)
return future
self._ttl_stop_event.clear()
interval = float(
sweep_interval_minutes or self.ttl_config.get("sweep_interval_minutes") or 5
)
logger.info(f"Starting store TTL sweeper with interval {interval} minutes")
future = concurrent.futures.Future()
def _sweep_loop() -> None:
try:
while not self._ttl_stop_event.is_set():
if self._ttl_stop_event.wait(interval * 60):
break
try:
expired_items = self.sweep_ttl()
if expired_items > 0:
logger.info(f"Store swept {expired_items} expired items")
except Exception as exc:
logger.exception(
"Store TTL sweep iteration failed", exc_info=exc
)
future.set_result(None)
except Exception as exc:
future.set_exception(exc)
thread = threading.Thread(target=_sweep_loop, daemon=True, name="ttl-sweeper")
self._ttl_sweeper_thread = thread
thread.start()
future.add_done_callback(
lambda f: self._ttl_stop_event.set() if f.cancelled() else None
)
return future
def stop_ttl_sweeper(self, timeout: float | None = None) -> bool:
"""Stop the TTL sweeper thread if it's running.
Args:
timeout: Maximum time to wait for the thread to stop, in seconds.
If `None`, wait indefinitely.
Returns:
bool: True if the thread was successfully stopped or wasn't running,
False if the timeout was reached before the thread stopped.
"""
if not self._ttl_sweeper_thread or not self._ttl_sweeper_thread.is_alive():
return True
logger.info("Stopping TTL sweeper thread")
self._ttl_stop_event.set()
self._ttl_sweeper_thread.join(timeout)
success = not self._ttl_sweeper_thread.is_alive()
if success:
self._ttl_sweeper_thread = None
logger.info("TTL sweeper thread stopped")
else:
logger.warning("Timed out waiting for TTL sweeper thread to stop")
return success
def __del__(self) -> None:
"""Ensure the TTL sweeper thread is stopped when the object is garbage collected."""
if hasattr(self, "_ttl_stop_event") and hasattr(self, "_ttl_sweeper_thread"):
self.stop_ttl_sweeper(timeout=0.1)
yield cls(conn, index=index)
@contextmanager
def _cursor(self, *, pipeline: bool = False) -> Iterator[Cursor[DictRow]]:
"""Create a database cursor as a context manager.
Args:
pipeline: whether to use pipeline for the DB operations inside the context manager.
pipeline (bool): whether to use pipeline for the DB operations inside the context manager.
Will be applied regardless of whether the PostgresStore instance was initialized with a pipeline.
If pipeline mode is not supported, will fall back to using transaction context manager.
"""
@@ -1029,9 +800,7 @@ class PostgresStore(BaseStore, BasePostgresStore[_pg_internal.Conn]):
query,
[
p
for (ns, k, pathname, _), vector in zip(
txt_params, vectors, strict=False
)
for (ns, k, pathname, _), vector in zip(txt_params, vectors)
for p in (ns, k, pathname, vector)
],
)
@@ -1052,15 +821,13 @@ class PostgresStore(BaseStore, BasePostgresStore[_pg_internal.Conn]):
embeddings = self.embeddings.embed_documents(
[query for _, query in embedding_requests]
)
for (idx, _), embedding in zip(
embedding_requests, embeddings, strict=False
):
for (idx, _), embedding in zip(embedding_requests, embeddings):
_paramslist = queries[idx][1]
for i in range(len(_paramslist)):
if _paramslist[i] is PLACEHOLDER:
if _paramslist[i] is _PLACEHOLDER:
_paramslist[i] = embedding
for (idx, _), (query, params) in zip(search_ops, queries, strict=False):
for (idx, _), (query, params) in zip(search_ops, queries):
cur.execute(query, params)
rows = cast(list[Row], cur.fetchall())
results[idx] = [
@@ -1077,14 +844,11 @@ class PostgresStore(BaseStore, BasePostgresStore[_pg_internal.Conn]):
cur: Cursor[DictRow],
) -> None:
for (query, params), (idx, _) in zip(
self._get_batch_list_namespaces_queries(list_ops), list_ops, strict=False
self._get_batch_list_namespaces_queries(list_ops), list_ops
):
cur.execute(query, params)
results[idx] = [_decode_ns_bytes(row["truncated_prefix"]) for row in cur]
async def abatch(self, ops: Iterable[Op]) -> list[Result]:
return await asyncio.get_running_loop().run_in_executor(None, self.batch, ops)
def setup(self) -> None:
"""Set up the store database.
@@ -1112,14 +876,8 @@ class PostgresStore(BaseStore, BasePostgresStore[_pg_internal.Conn]):
with self._cursor() as cur:
version = _get_version(cur, table="store_migrations")
for v, sql in enumerate(self.MIGRATIONS[version + 1 :], start=version + 1):
try:
cur.execute(sql)
cur.execute("INSERT INTO store_migrations (v) VALUES (%s)", (v,))
except Exception as e:
logger.error(
f"Failed to apply migration {v}.\nSql={sql}\nError={e}"
)
raise
cur.execute(sql)
cur.execute("INSERT INTO store_migrations (v) VALUES (%s)", (v,))
if self.index_config:
version = _get_version(cur, table="vector_migrations")
@@ -1134,27 +892,6 @@ class PostgresStore(BaseStore, BasePostgresStore[_pg_internal.Conn]):
k: v(self) if v is not None and callable(v) else v
for k, v in migration.params.items()
}
if "dims" in params:
try:
params["dims"] = int(params["dims"])
except Exception as e:
raise ValueError(
f"Invalid dims for vector index: {params['dims']}"
) from e
if "vector_type" in params:
vt = str(params["vector_type"])
if vt not in ("vector", "halfvec"):
raise ValueError(
f"Invalid vector_type for pgvector: {vt}"
)
params["vector_type"] = vt
if "index_type" in params:
it = str(params["index_type"])
if it not in ("hnsw", "ivfflat"):
raise ValueError(
f"Invalid index_type for pgvector: {it}"
)
params["index_type"] = it
sql = sql % params
cur.execute(sql)
cur.execute("INSERT INTO vector_migrations (v) VALUES (%s)", (v,))
@@ -1180,7 +917,7 @@ def _get_vector_type_ops(store: BasePostgresStore) -> str:
if not store.index_config:
return "vector_cosine_ops"
config = store.index_config
config = cast(PostgresIndexConfig, store.index_config)
index_config = config.get("ann_index_config", _DEFAULT_ANN_CONFIG).copy()
vector_type = cast(str, index_config.get("vector_type", "vector"))
if vector_type not in ("vector", "halfvec"):
@@ -1208,44 +945,15 @@ def _get_vector_type_ops(store: BasePostgresStore) -> str:
def _get_index_params(store: Any) -> tuple[str, dict[str, Any]]:
"""Get a sanitized index type and configuration based on config.
Only allow known-safe kinds and integer parameters to avoid SQL injection
when constructing DDL strings for index creation.
"""
"""Get the index type and configuration based on config."""
if not store.index_config:
return "hnsw", {}
config = cast(PostgresIndexConfig, store.index_config)
raw = config.get("ann_index_config", _DEFAULT_ANN_CONFIG).copy()
kind = str(raw.pop("kind", "hnsw"))
if kind not in ("hnsw", "ivfflat", "flat"):
raise ValueError(
f"Invalid index kind for pgvector: {kind}. Expected 'hnsw', 'ivfflat', or 'flat'."
)
raw.pop("vector_type", None)
if kind == "hnsw":
allowed_keys = {"m", "ef_construction"}
else: # ivfflat/flat
allowed_keys = {"lists", "nlist"}
sanitized: dict[str, int] = {}
for k, v in list(raw.items()):
if k not in allowed_keys:
continue
key = "lists" if k == "nlist" else k
try:
ivalue = int(v) # type: ignore[call-overload]
except Exception as e:
raise ValueError(f"Invalid index parameter value for {k}: {v}") from e
if ivalue <= 0:
continue
sanitized[key] = ivalue
return kind, sanitized
index_config = config.get("ann_index_config", _DEFAULT_ANN_CONFIG).copy()
kind = index_config.pop("kind", "hnsw")
index_config.pop("vector_type", None)
return kind, index_config
def _namespace_to_text(
@@ -1261,7 +969,7 @@ def _row_to_item(
namespace: tuple[str, ...],
row: Row,
*,
loader: Callable[[bytes | orjson.Fragment], dict[str, Any]] | None = None,
loader: Optional[Callable[[Union[bytes, orjson.Fragment]], dict[str, Any]]] = None,
) -> Item:
"""Convert a row from the database into an Item.
@@ -1289,7 +997,7 @@ def _row_to_search_item(
namespace: tuple[str, ...],
row: Row,
*,
loader: Callable[[bytes | orjson.Fragment], dict[str, Any]] | None = None,
loader: Optional[Callable[[Union[bytes, orjson.Fragment]], dict[str, Any]]] = None,
) -> SearchItem:
"""Convert a row from the database into an Item."""
loader = loader or _json_loads
@@ -1320,7 +1028,7 @@ def _group_ops(ops: Iterable[Op]) -> tuple[dict[type, list[tuple[int, Op]]], int
return grouped_ops, tot
def _json_loads(content: bytes | orjson.Fragment) -> Any:
def _json_loads(content: Union[bytes, orjson.Fragment]) -> Any:
if isinstance(content, orjson.Fragment):
if hasattr(content, "buf"):
content = content.buf
@@ -1332,7 +1040,7 @@ def _json_loads(content: bytes | orjson.Fragment) -> Any:
return orjson.loads(cast(bytes, content))
def _decode_ns_bytes(namespace: str | bytes | list) -> tuple[str, ...]:
def _decode_ns_bytes(namespace: Union[str, bytes, list]) -> tuple[str, ...]:
if isinstance(namespace, list):
return tuple(namespace)
if isinstance(namespace, bytes):
@@ -1340,7 +1048,7 @@ def _decode_ns_bytes(namespace: str | bytes | list) -> tuple[str, ...]:
return tuple(namespace.split("."))
def get_distance_operator(store: Any) -> tuple[str, str]:
def _get_distance_operator(store: Any) -> tuple[str, str]:
"""Get the distance operator and score expression based on config."""
# Note: Today, we are not using ANN indices due to restrictions
# on PGVector's support for mixing vector and non-vector filters
@@ -1381,16 +1089,16 @@ def get_distance_operator(store: Any) -> tuple[str, str]:
def _ensure_index_config(
index_config: PostgresIndexConfig,
) -> tuple[Embeddings | None, PostgresIndexConfig]:
) -> tuple[Optional["Embeddings"], PostgresIndexConfig]:
index_config = index_config.copy()
tokenized: list[tuple[str, Literal["$"] | list[str]]] = []
tokenized: list[tuple[str, Union[Literal["$"], list[str]]]] = []
tot = 0
fields = index_config.get("fields") or ["$"]
if isinstance(fields, str):
fields = [fields]
if not isinstance(fields, list):
raise ValueError(f"Text fields must be a list or a string. Got {fields}")
for p in fields:
text_fields = index_config.get("text_fields") or ["$"]
if isinstance(text_fields, str):
text_fields = [text_fields]
if not isinstance(text_fields, list):
raise ValueError(f"Text fields must be a list or a string. Got {text_fields}")
for p in text_fields:
if p == "$":
tokenized.append((p, "$"))
tot += 1
@@ -1406,4 +1114,4 @@ def _ensure_index_config(
return embeddings, index_config
PLACEHOLDER = object()
_PLACEHOLDER = object()
File diff suppressed because it is too large. Load diff
+34 -51
View File
@@ -1,62 +1,47 @@
[build-system]
requires = ["hatchling"]
build-backend = "hatchling.build"
[project]
[tool.poetry]
name = "langgraph-checkpoint-postgres"
version = "3.0.3"
version = "2.0.15"
description = "Library with a Postgres implementation of LangGraph checkpoint saver."
authors = []
requires-python = ">=3.10"
readme = "README.md"
license = "MIT"
license-files = ['LICENSE']
dependencies = [
"langgraph-checkpoint>=2.1.2,<5.0.0",
"orjson>=3.10.1",
"psycopg>=3.2.0",
"psycopg-pool>=3.2.0",
]
readme = "README.md"
repository = "https://www.github.com/langchain-ai/langgraph"
packages = [{ include = "langgraph" }]
[project.urls]
Source = "https://github.com/langchain-ai/langgraph/tree/main/libs/checkpoint-postgres"
Twitter = "https://x.com/LangChainAI"
Slack = "https://www.langchain.com/join-community"
Reddit = "https://www.reddit.com/r/LangChain/"
[tool.poetry.dependencies]
python = "^3.9.0,<4.0"
langgraph-checkpoint = "^2.0.15"
orjson = ">=3.10.1"
psycopg = "^3.2.0"
psycopg-pool = "^3.2.0"
[dependency-groups]
test = [
"pytest",
"anyio",
"pytest-asyncio",
"pytest-mock",
"psycopg[binary]",
"langgraph-checkpoint",
"pytest-watcher",
]
lint = [
"ruff",
"codespell",
"mypy",
]
dev = [
{include-group = "test"},
{include-group = "lint"},
]
[tool.uv]
default-groups = ['dev']
[tool.uv.sources]
langgraph-checkpoint = { path = "../checkpoint", editable = true }
[tool.hatch.build.targets.wheel]
include = ["langgraph"]
[tool.poetry.group.dev.dependencies]
ruff = "^0.6.2"
codespell = "^2.2.0"
pytest = "^7.2.1"
anyio = "^4.4.0"
pytest-asyncio = "^0.21.1"
pytest-mock = "^3.11.1"
mypy = "^1.10.0"
psycopg = {extras = ["binary"], version = ">=3.0.0"}
langgraph-checkpoint = {path = "../checkpoint", develop = true}
pytest-watcher = "^0.4.3"
[tool.pytest.ini_options]
# --strict-markers will raise errors on unknown marks.
# https://docs.pytest.org/en/7.1.x/how-to/mark.html#raising-errors-on-unknown-marks
#
# https://docs.pytest.org/en/7.1.x/reference/reference.html
# --strict-config any warnings encountered while parsing the `pytest`
# section of the configuration file raise errors.
addopts = "--strict-markers --strict-config --durations=5 -vv"
asyncio_mode = "auto"
[build-system]
requires = ["poetry-core"]
build-backend = "poetry.core.masonry.api"
[tool.ruff]
lint.select = [
"E", # pycodestyle
@@ -64,10 +49,8 @@ lint.select = [
"UP", # pyupgrade
"B", # flake8-bugbear
"I", # isort
"UP", # pyupgrade
]
lint.ignore = ["E501", "B008"]
target-version = "py310"
lint.ignore = ["E501", "B008", "UP007", "UP006"]
[tool.mypy]
# https://mypy.readthedocs.io/en/stable/config_file.html
@@ -1,373 +0,0 @@
# type: ignore
from contextlib import asynccontextmanager
from typing import Any
from uuid import uuid4
import pytest
from langchain_core.runnables import RunnableConfig
from langgraph.checkpoint.base import (
EXCLUDED_METADATA_KEYS,
Checkpoint,
CheckpointMetadata,
create_checkpoint,
empty_checkpoint,
)
from langgraph.checkpoint.serde.types import TASKS
from psycopg import AsyncConnection
from psycopg.rows import dict_row
from psycopg_pool import AsyncConnectionPool
from langgraph.checkpoint.postgres.aio import (
AsyncPostgresSaver,
AsyncShallowPostgresSaver,
)
from tests.conftest import DEFAULT_POSTGRES_URI
def _exclude_keys(config: dict[str, Any]) -> dict[str, Any]:
return {k: v for k, v in config.items() if k not in EXCLUDED_METADATA_KEYS}
@asynccontextmanager
async def _pool_saver():
"""Fixture for pool mode testing."""
database = f"test_{uuid4().hex[:16]}"
# create unique db
async with await AsyncConnection.connect(
DEFAULT_POSTGRES_URI, autocommit=True
) as conn:
await conn.execute(f"CREATE DATABASE {database}")
try:
# yield checkpointer
async with AsyncConnectionPool(
DEFAULT_POSTGRES_URI + database,
max_size=10,
kwargs={"autocommit": True, "row_factory": dict_row},
) as pool:
checkpointer = AsyncPostgresSaver(pool)
await checkpointer.setup()
yield checkpointer
finally:
# drop unique db
async with await AsyncConnection.connect(
DEFAULT_POSTGRES_URI, autocommit=True
) as conn:
await conn.execute(f"DROP DATABASE {database}")
@asynccontextmanager
async def _pipe_saver():
"""Fixture for pipeline mode testing."""
database = f"test_{uuid4().hex[:16]}"
# create unique db
async with await AsyncConnection.connect(
DEFAULT_POSTGRES_URI, autocommit=True
) as conn:
await conn.execute(f"CREATE DATABASE {database}")
try:
async with await AsyncConnection.connect(
DEFAULT_POSTGRES_URI + database,
autocommit=True,
prepare_threshold=0,
row_factory=dict_row,
) as conn:
checkpointer = AsyncPostgresSaver(conn)
await checkpointer.setup()
async with conn.pipeline() as pipe:
checkpointer = AsyncPostgresSaver(conn, pipe=pipe)
yield checkpointer
finally:
# drop unique db
async with await AsyncConnection.connect(
DEFAULT_POSTGRES_URI, autocommit=True
) as conn:
await conn.execute(f"DROP DATABASE {database}")
@asynccontextmanager
async def _base_saver():
"""Fixture for regular connection mode testing."""
database = f"test_{uuid4().hex[:16]}"
# create unique db
async with await AsyncConnection.connect(
DEFAULT_POSTGRES_URI, autocommit=True
) as conn:
await conn.execute(f"CREATE DATABASE {database}")
try:
async with await AsyncConnection.connect(
DEFAULT_POSTGRES_URI + database,
autocommit=True,
prepare_threshold=0,
row_factory=dict_row,
) as conn:
checkpointer = AsyncPostgresSaver(conn)
await checkpointer.setup()
yield checkpointer
finally:
# drop unique db
async with await AsyncConnection.connect(
DEFAULT_POSTGRES_URI, autocommit=True
) as conn:
await conn.execute(f"DROP DATABASE {database}")
@asynccontextmanager
async def _shallow_saver():
"""Fixture for shallow connection mode testing."""
database = f"test_{uuid4().hex[:16]}"
# create unique db
async with await AsyncConnection.connect(
DEFAULT_POSTGRES_URI, autocommit=True
) as conn:
await conn.execute(f"CREATE DATABASE {database}")
try:
async with await AsyncConnection.connect(
DEFAULT_POSTGRES_URI + database,
autocommit=True,
prepare_threshold=0,
row_factory=dict_row,
) as conn:
checkpointer = AsyncShallowPostgresSaver(conn)
await checkpointer.setup()
yield checkpointer
finally:
# drop unique db
async with await AsyncConnection.connect(
DEFAULT_POSTGRES_URI, autocommit=True
) as conn:
await conn.execute(f"DROP DATABASE {database}")
@asynccontextmanager
async def _saver(name: str):
if name == "base":
async with _base_saver() as saver:
yield saver
elif name == "shallow":
async with _shallow_saver() as saver:
yield saver
elif name == "pool":
async with _pool_saver() as saver:
yield saver
elif name == "pipe":
async with _pipe_saver() as saver:
yield saver
@pytest.fixture
def test_data():
"""Fixture providing test data for checkpoint tests."""
config_1: RunnableConfig = {
"configurable": {
"thread_id": "thread-1",
"checkpoint_id": "1",
"checkpoint_ns": "",
}
}
config_2: RunnableConfig = {
"configurable": {
"thread_id": "thread-2",
"checkpoint_id": "2",
"checkpoint_ns": "",
}
}
config_3: RunnableConfig = {
"configurable": {
"thread_id": "thread-2",
"checkpoint_id": "2-inner",
"checkpoint_ns": "inner",
}
}
chkpnt_1: Checkpoint = empty_checkpoint()
chkpnt_2: Checkpoint = create_checkpoint(chkpnt_1, {}, 1)
chkpnt_3: Checkpoint = empty_checkpoint()
metadata_1: CheckpointMetadata = {
"source": "input",
"step": 2,
"score": 1,
}
metadata_2: CheckpointMetadata = {
"source": "loop",
"step": 1,
"score": None,
}
metadata_3: CheckpointMetadata = {}
return {
"configs": [config_1, config_2, config_3],
"checkpoints": [chkpnt_1, chkpnt_2, chkpnt_3],
"metadata": [metadata_1, metadata_2, metadata_3],
}
@pytest.mark.parametrize("saver_name", ["base", "pool", "pipe", "shallow"])
async def test_combined_metadata(saver_name: str, test_data) -> None:
async with _saver(saver_name) as saver:
config = {
"configurable": {
"thread_id": "thread-2",
"checkpoint_ns": "",
"__super_private_key": "super_private_value",
},
"metadata": {"run_id": "my_run_id"},
}
chkpnt: Checkpoint = create_checkpoint(empty_checkpoint(), {}, 1)
metadata: CheckpointMetadata = {
"source": "loop",
"step": 1,
"score": None,
}
await saver.aput(config, chkpnt, metadata, {})
checkpoint = await saver.aget_tuple(config)
assert checkpoint.metadata == {
**metadata,
"run_id": "my_run_id",
}
@pytest.mark.parametrize("saver_name", ["base", "pool", "pipe", "shallow"])
async def test_asearch(saver_name: str, test_data) -> None:
async with _saver(saver_name) as saver:
configs = test_data["configs"]
checkpoints = test_data["checkpoints"]
metadata = test_data["metadata"]
await saver.aput(configs[0], checkpoints[0], metadata[0], {})
await saver.aput(configs[1], checkpoints[1], metadata[1], {})
await saver.aput(configs[2], checkpoints[2], metadata[2], {})
# call method / assertions
query_1 = {"source": "input"} # search by 1 key
query_2 = {
"step": 1,
} # search by multiple keys
query_3: dict[str, Any] = {} # search by no keys, return all checkpoints
query_4 = {"source": "update", "step": 1} # no match
search_results_1 = [c async for c in saver.alist(None, filter=query_1)]
assert len(search_results_1) == 1
assert search_results_1[0].metadata == {
**_exclude_keys(configs[0]["configurable"]),
**metadata[0],
}
search_results_2 = [c async for c in saver.alist(None, filter=query_2)]
assert len(search_results_2) == 1
assert search_results_2[0].metadata == {
**_exclude_keys(configs[1]["configurable"]),
**metadata[1],
}
search_results_3 = [c async for c in saver.alist(None, filter=query_3)]
assert len(search_results_3) == 3
search_results_4 = [c async for c in saver.alist(None, filter=query_4)]
assert len(search_results_4) == 0
# search by config (defaults to checkpoints across all namespaces)
search_results_5 = [
c async for c in saver.alist({"configurable": {"thread_id": "thread-2"}})
]
assert len(search_results_5) == 2
assert {
search_results_5[0].config["configurable"]["checkpoint_ns"],
search_results_5[1].config["configurable"]["checkpoint_ns"],
} == {"", "inner"}
@pytest.mark.parametrize("saver_name", ["base", "pool", "pipe", "shallow"])
async def test_null_chars(saver_name: str, test_data) -> None:
async with _saver(saver_name) as saver:
config = await saver.aput(
test_data["configs"][0],
test_data["checkpoints"][0],
{"my_key": "\x00abc"},
{},
)
assert (await saver.aget_tuple(config)).metadata["my_key"] == "abc" # type: ignore
assert [c async for c in saver.alist(None, filter={"my_key": "abc"})][
0
].metadata["my_key"] == "abc"
@pytest.mark.parametrize("saver_name", ["base", "pool", "pipe"])
async def test_pending_sends_migration(saver_name: str) -> None:
async with _saver(saver_name) as saver:
config = {
"configurable": {
"thread_id": "thread-1",
"checkpoint_ns": "",
}
}
# create the first checkpoint
# and put some pending sends
checkpoint_0 = empty_checkpoint()
config = await saver.aput(config, checkpoint_0, {}, {})
await saver.aput_writes(
config, [(TASKS, "send-1"), (TASKS, "send-2")], task_id="task-1"
)
await saver.aput_writes(config, [(TASKS, "send-3")], task_id="task-2")
# check that fetching checkpoint_0 doesn't attach pending sends
# (they should be attached to the next checkpoint)
tuple_0 = await saver.aget_tuple(config)
assert tuple_0.checkpoint["channel_values"] == {}
assert tuple_0.checkpoint["channel_versions"] == {}
# create the second checkpoint
checkpoint_1 = create_checkpoint(checkpoint_0, {}, 1)
config = await saver.aput(config, checkpoint_1, {}, {})
# check that pending sends are attached to checkpoint_1
tuple_1 = await saver.aget_tuple(config)
assert tuple_1.checkpoint["channel_values"] == {
TASKS: ["send-1", "send-2", "send-3"]
}
assert TASKS in tuple_1.checkpoint["channel_versions"]
# check that list also applies the migration
search_results = [
c async for c in saver.alist({"configurable": {"thread_id": "thread-1"}})
]
assert len(search_results) == 2
assert search_results[-1].checkpoint["channel_values"] == {}
assert search_results[-1].checkpoint["channel_versions"] == {}
assert search_results[0].checkpoint["channel_values"] == {
TASKS: ["send-1", "send-2", "send-3"]
}
assert TASKS in search_results[0].checkpoint["channel_versions"]
@pytest.mark.parametrize("saver_name", ["base", "pool", "pipe"])
async def test_get_checkpoint_no_channel_values(
monkeypatch, saver_name: str, test_data
) -> None:
"""Backwards compatibility test that verifies a checkpoint with no channel_values key can be retrieved without throwing an error."""
async with _saver(saver_name) as saver:
config = {
"configurable": {
"thread_id": "thread-2",
"checkpoint_ns": "",
"__super_private_key": "super_private_value",
},
"metadata": {"run_id": "my_run_id"},
}
chkpnt: Checkpoint = create_checkpoint(empty_checkpoint(), {}, 1)
await saver.aput(config, chkpnt, {}, {})
load_checkpoint_tuple = saver._load_checkpoint_tuple
def patched_load_checkpoint_tuple(value):
value["checkpoint"].pop("channel_values", None)
return load_checkpoint_tuple(value)
monkeypatch.setattr(
saver, "_load_checkpoint_tuple", patched_load_checkpoint_tuple
)
checkpoint = await saver.aget_tuple(config)
assert checkpoint.checkpoint["channel_values"] == {}
@@ -1,687 +0,0 @@
# type: ignore
from __future__ import annotations
import asyncio
import itertools
import uuid
from collections.abc import AsyncIterator
from concurrent.futures import ThreadPoolExecutor
from contextlib import asynccontextmanager
from typing import Any
import pytest
from langchain_core.embeddings import Embeddings
from langgraph.store.base import (
GetOp,
Item,
ListNamespacesOp,
PutOp,
SearchOp,
)
from psycopg import AsyncConnection
from langgraph.store.postgres import AsyncPostgresStore
from tests.conftest import (
DEFAULT_URI,
VECTOR_TYPES,
CharacterEmbeddings,
)
TTL_SECONDS = 6
TTL_MINUTES = TTL_SECONDS / 60
@pytest.fixture(scope="function", params=["default", "pipe", "pool"])
async def store(request) -> AsyncIterator[AsyncPostgresStore]:
database = f"test_{uuid.uuid4().hex[:16]}"
uri_parts = DEFAULT_URI.split("/")
uri_base = "/".join(uri_parts[:-1])
query_params = ""
if "?" in uri_parts[-1]:
db_name, query_params = uri_parts[-1].split("?", 1)
query_params = "?" + query_params
conn_string = f"{uri_base}/{database}{query_params}"
admin_conn_string = DEFAULT_URI
ttl_config = {
"default_ttl": TTL_MINUTES,
"refresh_on_read": True,
"sweep_interval_minutes": TTL_MINUTES / 2,
}
async with await AsyncConnection.connect(
admin_conn_string, autocommit=True
) as conn:
await conn.execute(f"CREATE DATABASE {database}")
try:
async with AsyncPostgresStore.from_conn_string(
conn_string, ttl=ttl_config
) as store:
store.MIGRATIONS = [
(
mig.replace("ttl_minutes INT;", "ttl_minutes FLOAT;")
if isinstance(mig, str)
else mig
)
for mig in store.MIGRATIONS
]
await store.setup()
async with store._cursor() as cur:
# drop the migration index
await cur.execute("DROP TABLE IF EXISTS store_migrations")
await store.setup() # Will fail if migrations aren't idempotent
if request.param == "pipe":
async with AsyncPostgresStore.from_conn_string(
conn_string, pipeline=True, ttl=ttl_config
) as store:
await store.start_ttl_sweeper()
yield store
await store.stop_ttl_sweeper()
elif request.param == "pool":
async with AsyncPostgresStore.from_conn_string(
conn_string, pool_config={"min_size": 1, "max_size": 10}, ttl=ttl_config
) as store:
await store.start_ttl_sweeper()
yield store
await store.stop_ttl_sweeper()
else: # default
async with AsyncPostgresStore.from_conn_string(
conn_string, ttl=ttl_config
) as store:
await store.start_ttl_sweeper()
yield store
await store.stop_ttl_sweeper()
finally:
async with await AsyncConnection.connect(
admin_conn_string, autocommit=True
) as conn:
await conn.execute(f"DROP DATABASE {database}")
async def test_no_running_loop(store: AsyncPostgresStore) -> None:
with pytest.raises(asyncio.InvalidStateError):
store.put(("foo", "bar"), "baz", {"val": "baz"})
with pytest.raises(asyncio.InvalidStateError):
store.get(("foo", "bar"), "baz")
with pytest.raises(asyncio.InvalidStateError):
store.delete(("foo", "bar"), "baz")
with pytest.raises(asyncio.InvalidStateError):
store.search(("foo", "bar"))
with pytest.raises(asyncio.InvalidStateError):
store.list_namespaces(prefix=("foo",))
with pytest.raises(asyncio.InvalidStateError):
store.batch([PutOp(namespace=("foo", "bar"), key="baz", value={"val": "baz"})])
with ThreadPoolExecutor(max_workers=1) as executor:
future = executor.submit(store.put, ("foo", "bar"), "baz", {"val": "baz"})
result = await asyncio.wrap_future(future)
assert result is None
future = executor.submit(store.get, ("foo", "bar"), "baz")
result = await asyncio.wrap_future(future)
assert result.value == {"val": "baz"}
result = await asyncio.wrap_future(
executor.submit(store.list_namespaces, prefix=("foo",))
)
async def test_large_batches(request: Any, store: AsyncPostgresStore) -> None:
N = 100 # less important that we are performant here
M = 10
with ThreadPoolExecutor(max_workers=10) as executor:
futures = []
for m in range(M):
for i in range(N):
futures += [
executor.submit(
store.put,
("test", "foo", "bar", "baz", str(m % 2)),
f"key{i}",
value={"foo": "bar" + str(i)},
),
executor.submit(
store.get,
("test", "foo", "bar", "baz", str(m % 2)),
f"key{i}",
),
executor.submit(
store.list_namespaces,
prefix=None,
max_depth=m + 1,
),
executor.submit(
store.search,
("test",),
),
executor.submit(
store.put,
("test", "foo", "bar", "baz", str(m % 2)),
f"key{i}",
value={"foo": "bar" + str(i)},
),
executor.submit(
store.put,
("test", "foo", "bar", "baz", str(m % 2)),
f"key{i}",
None,
),
]
results = await asyncio.gather(
*(asyncio.wrap_future(future) for future in futures)
)
assert len(results) == M * N * 6
async def test_large_batches_async(store: AsyncPostgresStore) -> None:
N = 1000
M = 10
coros = []
for m in range(M):
for i in range(N):
coros.append(
store.aput(
("test", "foo", "bar", "baz", str(m % 2)),
f"key{i}",
value={"foo": "bar" + str(i)},
)
)
coros.append(
store.aget(
("test", "foo", "bar", "baz", str(m % 2)),
f"key{i}",
)
)
coros.append(
store.alist_namespaces(
prefix=None,
max_depth=m + 1,
)
)
coros.append(
store.asearch(
("test",),
)
)
coros.append(
store.aput(
("test", "foo", "bar", "baz", str(m % 2)),
f"key{i}",
value={"foo": "bar" + str(i)},
)
)
coros.append(
store.adelete(
("test", "foo", "bar", "baz", str(m % 2)),
f"key{i}",
)
)
results = await asyncio.gather(*coros)
assert len(results) == M * N * 6
async def test_abatch_order(store: AsyncPostgresStore) -> None:
# Setup test data
await store.aput(("test", "foo"), "key1", {"data": "value1"})
await store.aput(("test", "bar"), "key2", {"data": "value2"})
ops = [
GetOp(namespace=("test", "foo"), key="key1"),
PutOp(namespace=("test", "bar"), key="key2", value={"data": "value2"}),
SearchOp(
namespace_prefix=("test",), filter={"data": "value1"}, limit=10, offset=0
),
ListNamespacesOp(match_conditions=None, max_depth=None, limit=10, offset=0),
GetOp(namespace=("test",), key="key3"),
]
results = await store.abatch(ops)
assert len(results) == 5
assert isinstance(results[0], Item)
assert isinstance(results[0].value, dict)
assert results[0].value == {"data": "value1"}
assert results[0].key == "key1"
assert results[1] is None
assert isinstance(results[2], list)
assert len(results[2]) == 1
assert isinstance(results[3], list)
assert ("test", "foo") in results[3] and ("test", "bar") in results[3]
assert results[4] is None
ops_reordered = [
SearchOp(namespace_prefix=("test",), filter=None, limit=5, offset=0),
GetOp(namespace=("test", "bar"), key="key2"),
ListNamespacesOp(match_conditions=None, max_depth=None, limit=5, offset=0),
PutOp(namespace=("test",), key="key3", value={"data": "value3"}),
GetOp(namespace=("test", "foo"), key="key1"),
]
results_reordered = await store.abatch(ops_reordered)
assert len(results_reordered) == 5
assert isinstance(results_reordered[0], list)
assert len(results_reordered[0]) == 2
assert isinstance(results_reordered[1], Item)
assert results_reordered[1].value == {"data": "value2"}
assert results_reordered[1].key == "key2"
assert isinstance(results_reordered[2], list)
assert ("test", "foo") in results_reordered[2] and (
"test",
"bar",
) in results_reordered[2]
assert results_reordered[3] is None
assert isinstance(results_reordered[4], Item)
assert results_reordered[4].value == {"data": "value1"}
assert results_reordered[4].key == "key1"
async def test_batch_get_ops(store: AsyncPostgresStore) -> None:
# Setup test data
await store.aput(("test",), "key1", {"data": "value1"})
await store.aput(("test",), "key2", {"data": "value2"})
ops = [
GetOp(namespace=("test",), key="key1"),
GetOp(namespace=("test",), key="key2"),
GetOp(namespace=("test",), key="key3"),
]
results = await store.abatch(ops)
assert len(results) == 3
assert results[0] is not None
assert results[1] is not None
assert results[2] is None
assert results[0].key == "key1"
assert results[1].key == "key2"
async def test_batch_put_ops(store: AsyncPostgresStore) -> None:
ops = [
PutOp(namespace=("test",), key="key1", value={"data": "value1"}),
PutOp(namespace=("test",), key="key2", value={"data": "value2"}),
PutOp(namespace=("test",), key="key3", value=None),
]
results = await store.abatch(ops)
assert len(results) == 3
assert all(result is None for result in results)
# Verify the puts worked
items = await store.asearch(["test"], limit=10)
assert len(items) == 2 # key3 had None value so wasn't stored
async def test_batch_search_ops(store: AsyncPostgresStore) -> None:
# Setup test data
await store.aput(("test", "foo"), "key1", {"data": "value1"})
await store.aput(("test", "bar"), "key2", {"data": "value2"})
ops = [
SearchOp(
namespace_prefix=("test",), filter={"data": "value1"}, limit=10, offset=0
),
SearchOp(namespace_prefix=("test",), filter=None, limit=5, offset=0),
]
results = await store.abatch(ops)
assert len(results) == 2
assert len(results[0]) == 1 # Filtered results
assert len(results[1]) == 2 # All results
async def test_batch_list_namespaces_ops(store: AsyncPostgresStore) -> None:
# Setup test data
await store.aput(("test", "namespace1"), "key1", {"data": "value1"})
await store.aput(("test", "namespace2"), "key2", {"data": "value2"})
ops = [ListNamespacesOp(match_conditions=None, max_depth=None, limit=10, offset=0)]
results = await store.abatch(ops)
assert len(results) == 1
assert len(results[0]) == 2
assert ("test", "namespace1") in results[0]
assert ("test", "namespace2") in results[0]
@asynccontextmanager
async def _create_vector_store(
vector_type: str,
distance_type: str,
fake_embeddings: CharacterEmbeddings,
text_fields: list[str] | None = None,
) -> AsyncIterator[AsyncPostgresStore]:
"""Create a store with vector search enabled."""
database = f"test_{uuid.uuid4().hex[:16]}"
uri_parts = DEFAULT_URI.split("/")
uri_base = "/".join(uri_parts[:-1])
query_params = ""
if "?" in uri_parts[-1]:
db_name, query_params = uri_parts[-1].split("?", 1)
query_params = "?" + query_params
conn_string = f"{uri_base}/{database}{query_params}"
admin_conn_string = DEFAULT_URI
index_config = {
"dims": fake_embeddings.dims,
"embed": fake_embeddings,
"ann_index_config": {
"vector_type": vector_type,
},
"distance_type": distance_type,
"fields": text_fields,
}
async with await AsyncConnection.connect(
admin_conn_string, autocommit=True
) as conn:
await conn.execute(f"CREATE DATABASE {database}")
try:
async with AsyncPostgresStore.from_conn_string(
conn_string,
index=index_config,
) as store:
await store.setup()
yield store
finally:
async with await AsyncConnection.connect(
admin_conn_string, autocommit=True
) as conn:
await conn.execute(f"DROP DATABASE {database}")
@pytest.fixture(
scope="function",
params=[
(vector_type, distance_type)
for vector_type in VECTOR_TYPES
for distance_type in (
["hamming"] if vector_type == "bit" else ["l2", "inner_product", "cosine"]
)
],
ids=lambda p: f"{p[0]}_{p[1]}",
)
async def vector_store(
request,
fake_embeddings: CharacterEmbeddings,
) -> AsyncIterator[AsyncPostgresStore]:
"""Create a store with vector search enabled."""
vector_type, distance_type = request.param
async with _create_vector_store(
vector_type, distance_type, fake_embeddings
) as store:
yield store
async def test_vector_store_initialization(
vector_store: AsyncPostgresStore, fake_embeddings: CharacterEmbeddings
) -> None:
"""Test store initialization with embedding config."""
assert vector_store.index_config is not None
assert vector_store.index_config["dims"] == fake_embeddings.dims
if isinstance(vector_store.index_config["embed"], Embeddings):
assert vector_store.index_config["embed"] == fake_embeddings
async def test_vector_insert_with_auto_embedding(
vector_store: AsyncPostgresStore,
) -> None:
"""Test inserting items that get auto-embedded."""
docs = [
("doc1", {"text": "short text"}),
("doc2", {"text": "longer text document"}),
("doc3", {"text": "longest text document here"}),
("doc4", {"description": "text in description field"}),
("doc5", {"content": "text in content field"}),
("doc6", {"body": "text in body field"}),
]
for key, value in docs:
await vector_store.aput(("test",), key, value)
results = await vector_store.asearch(("test",), query="long text")
assert len(results) > 0
doc_order = [r.key for r in results]
assert "doc2" in doc_order
assert "doc3" in doc_order
async def test_vector_update_with_embedding(vector_store: AsyncPostgresStore) -> None:
"""Test that updating items properly updates their embeddings."""
await vector_store.aput(("test",), "doc1", {"text": "zany zebra Xerxes"})
await vector_store.aput(("test",), "doc2", {"text": "something about dogs"})
await vector_store.aput(("test",), "doc3", {"text": "text about birds"})
results_initial = await vector_store.asearch(("test",), query="Zany Xerxes")
assert len(results_initial) > 0
assert results_initial[0].key == "doc1"
initial_score = results_initial[0].score
await vector_store.aput(("test",), "doc1", {"text": "new text about dogs"})
results_after = await vector_store.asearch(("test",), query="Zany Xerxes")
after_score = next((r.score for r in results_after if r.key == "doc1"), 0.0)
assert after_score < initial_score
results_new = await vector_store.asearch(("test",), query="new text about dogs")
for r in results_new:
if r.key == "doc1":
assert r.score > after_score
# Don't index this one
await vector_store.aput(
("test",), "doc4", {"text": "new text about dogs"}, index=False
)
results_new = await vector_store.asearch(
("test",), query="new text about dogs", limit=3
)
assert not any(r.key == "doc4" for r in results_new)
async def test_vector_search_with_filters(vector_store: AsyncPostgresStore) -> None:
"""Test combining vector search with filters."""
docs = [
("doc1", {"text": "red apple", "color": "red", "score": 4.5}),
("doc2", {"text": "red car", "color": "red", "score": 3.0}),
("doc3", {"text": "green apple", "color": "green", "score": 4.0}),
("doc4", {"text": "blue car", "color": "blue", "score": 3.5}),
]
for key, value in docs:
await vector_store.aput(("test",), key, value)
results = await vector_store.asearch(
("test",), query="apple", filter={"color": "red"}
)
assert len(results) == 2
assert results[0].key == "doc1"
results = await vector_store.asearch(
("test",), query="car", filter={"color": "red"}
)
assert len(results) == 2
assert results[0].key == "doc2"
results = await vector_store.asearch(
("test",), query="bbbbluuu", filter={"score": {"$gt": 3.2}}
)
assert len(results) == 3
assert results[0].key == "doc4"
results = await vector_store.asearch(
("test",), query="apple", filter={"score": {"$gte": 4.0}, "color": "green"}
)
assert len(results) == 1
assert results[0].key == "doc3"
async def test_vector_search_pagination(vector_store: AsyncPostgresStore) -> None:
"""Test pagination with vector search."""
for i in range(5):
await vector_store.aput(
("test",), f"doc{i}", {"text": f"test document number {i}"}
)
results_page1 = await vector_store.asearch(("test",), query="test", limit=2)
results_page2 = await vector_store.asearch(
("test",), query="test", limit=2, offset=2
)
assert len(results_page1) == 2
assert len(results_page2) == 2
assert results_page1[0].key != results_page2[0].key
all_results = await vector_store.asearch(("test",), query="test", limit=10)
assert len(all_results) == 5
async def test_vector_search_edge_cases(vector_store: AsyncPostgresStore) -> None:
"""Test edge cases in vector search."""
await vector_store.aput(("test",), "doc1", {"text": "test document"})
perfect_match = await vector_store.asearch(("test",), query="text test document")
perfect_score = perfect_match[0].score
results = await vector_store.asearch(("test",), query="")
assert len(results) == 1
assert results[0].score is None
results = await vector_store.asearch(("test",), query=None)
assert len(results) == 1
assert results[0].score is None
long_query = "foo " * 100
results = await vector_store.asearch(("test",), query=long_query)
assert len(results) == 1
assert results[0].score < perfect_score
special_query = "test!@#$%^&*()"
results = await vector_store.asearch(("test",), query=special_query)
assert len(results) == 1
assert results[0].score < perfect_score
@pytest.mark.parametrize(
"vector_type,distance_type",
[
*itertools.product(["vector", "halfvec"], ["cosine", "inner_product", "l2"]),
],
)
async def test_embed_with_path(
request: Any,
fake_embeddings: CharacterEmbeddings,
vector_type: str,
distance_type: str,
) -> None:
"""Test vector search with specific text fields in Postgres store."""
async with _create_vector_store(
vector_type,
distance_type,
fake_embeddings,
text_fields=["key0", "key1", "key3"],
) as store:
# This will have 2 vectors representing it
doc1 = {
# Omit key0 - check it doesn't raise an error
"key1": "xxx",
"key2": "yyy",
"key3": "zzz",
}
# This will have 3 vectors representing it
doc2 = {
"key0": "uuu",
"key1": "vvv",
"key2": "www",
"key3": "xxx",
}
await store.aput(("test",), "doc1", doc1)
await store.aput(("test",), "doc2", doc2)
# doc2.key3 and doc1.key1 both would have the highest score
results = await store.asearch(("test",), query="xxx")
assert len(results) == 2
assert results[0].key != results[1].key
ascore = results[0].score
bscore = results[1].score
assert ascore == pytest.approx(bscore, abs=1e-3)
results = await store.asearch(("test",), query="uuu")
assert len(results) == 2
assert results[0].key != results[1].key
assert results[0].key == "doc2"
assert results[0].score > results[1].score
assert ascore == pytest.approx(results[0].score, abs=1e-3)
# Un-indexed - will have low results for both. Not zero (because we're projecting)
# but less than the above.
results = await store.asearch(("test",), query="www")
assert len(results) == 2
assert results[0].score < ascore
assert results[1].score < ascore
@pytest.mark.parametrize(
"vector_type,distance_type",
[
*itertools.product(["vector", "halfvec"], ["cosine", "inner_product", "l2"]),
],
)
async def test_search_sorting(
request: Any,
fake_embeddings: CharacterEmbeddings,
vector_type: str,
distance_type: str,
) -> None:
"""Test operation-level field configuration for vector search."""
async with _create_vector_store(
vector_type,
distance_type,
fake_embeddings,
text_fields=["key1"], # Default fields that won't match our test data
) as store:
amatch = {
"key1": "mmm",
}
await store.aput(("test", "M"), "M", amatch)
N = 100
for i in range(N):
await store.aput(("test", "A"), f"A{i}", {"key1": "no"})
for i in range(N):
await store.aput(("test", "Z"), f"Z{i}", {"key1": "no"})
results = await store.asearch(("test",), query="mmm", limit=10)
assert len(results) == 10
assert len(set(r.key for r in results)) == 10
assert results[0].key == "M"
assert results[0].score > results[1].score
async def test_store_ttl(store):
# Assumes a TTL of 1 minute = 60 seconds
ns = ("foo",)
await store.start_ttl_sweeper()
await store.aput(
ns,
key="item1",
value={"foo": "bar"},
ttl=TTL_MINUTES, # type: ignore
)
await asyncio.sleep(TTL_SECONDS - 2)
res = await store.aget(ns, key="item1", refresh_ttl=True)
assert res is not None
await asyncio.sleep(TTL_SECONDS - 2)
results = await store.asearch(ns, query="foo", refresh_ttl=True)
assert len(results) == 1
await asyncio.sleep(TTL_SECONDS - 2)
res = await store.aget(ns, key="item1", refresh_ttl=False)
assert res is not None
await asyncio.sleep(TTL_SECONDS - 1)
# Now has been (TTL_SECONDS-2)*2 > TTL_SECONDS + TTL_SECONDS/2
results = await store.asearch(ns, query="bar", refresh_ttl=False)
assert len(results) == 0
+133 -242
View File
@@ -1,14 +1,14 @@
# type: ignore
from __future__ import annotations
import re
import time
from contextlib import contextmanager
from typing import Any
from typing import Any, Optional
from uuid import uuid4
import pytest
from langchain_core.embeddings import Embeddings
from psycopg import Connection
from langgraph.store.base import (
GetOp,
Item,
@@ -17,8 +17,6 @@ from langgraph.store.base import (
PutOp,
SearchOp,
)
from psycopg import Connection
from langgraph.store.postgres import PostgresStore
from tests.conftest import (
DEFAULT_URI,
@@ -26,9 +24,6 @@ from tests.conftest import (
CharacterEmbeddings,
)
TTL_SECONDS = 6
TTL_MINUTES = TTL_SECONDS / 60
@pytest.fixture(scope="function", params=["default", "pipe", "pool"])
def store(request) -> PostgresStore:
@@ -37,56 +32,29 @@ def store(request) -> PostgresStore:
uri_base = "/".join(uri_parts[:-1])
query_params = ""
if "?" in uri_parts[-1]:
_, query_params = uri_parts[-1].split("?", 1)
db_name, query_params = uri_parts[-1].split("?", 1)
query_params = "?" + query_params
conn_string = f"{uri_base}/{database}{query_params}"
admin_conn_string = DEFAULT_URI
ttl_config = {
"default_ttl": TTL_MINUTES,
"refresh_on_read": True,
"sweep_interval_minutes": TTL_MINUTES / 2,
}
with Connection.connect(admin_conn_string, autocommit=True) as conn:
conn.execute(f"CREATE DATABASE {database}")
try:
with PostgresStore.from_conn_string(conn_string, ttl=ttl_config) as store:
store.MIGRATIONS = [
(
mig.replace("ttl_minutes INT;", "ttl_minutes FLOAT;")
if isinstance(mig, str)
else mig
)
for mig in store.MIGRATIONS
]
with PostgresStore.from_conn_string(conn_string) as store:
store.setup()
if request.param == "pipe":
with PostgresStore.from_conn_string(
conn_string,
pipeline=True,
ttl=ttl_config,
) as store:
store.start_ttl_sweeper()
with PostgresStore.from_conn_string(conn_string, pipeline=True) as store:
yield store
store.stop_ttl_sweeper()
elif request.param == "pool":
with PostgresStore.from_conn_string(
conn_string,
pool_config={"min_size": 1, "max_size": 10},
ttl=ttl_config,
conn_string, pool_config={"min_size": 1, "max_size": 10}
) as store:
store.start_ttl_sweeper()
yield store
store.stop_ttl_sweeper()
else: # default
with PostgresStore.from_conn_string(conn_string, ttl=ttl_config) as store:
store.start_ttl_sweeper()
with PostgresStore.from_conn_string(conn_string) as store:
yield store
store.stop_ttl_sweeper()
finally:
with Connection.connect(admin_conn_string, autocommit=True) as conn:
conn.execute(f"DROP DATABASE {database}")
@@ -252,127 +220,134 @@ def test_batch_list_namespaces_ops(store: PostgresStore) -> None:
assert all(ns[-1] == "public" for ns in results[2])
def test_basic_store_ops(store) -> None:
namespace = ("test", "documents")
item_id = "doc1"
item_value = {"title": "Test Document", "content": "Hello, World!"}
class TestPostgresStore:
@pytest.fixture(autouse=True)
def setup(self) -> None:
with PostgresStore.from_conn_string(DEFAULT_URI) as store:
store.setup()
store.put(namespace, item_id, item_value)
item = store.get(namespace, item_id)
def test_basic_store_ops(self) -> None:
with PostgresStore.from_conn_string(DEFAULT_URI) as store:
namespace = ("test", "documents")
item_id = "doc1"
item_value = {"title": "Test Document", "content": "Hello, World!"}
assert item
assert item.namespace == namespace
assert item.key == item_id
assert item.value == item_value
store.put(namespace, item_id, item_value)
item = store.get(namespace, item_id)
# Test update
updated_value = {"title": "Updated Document", "content": "Hello, Updated!"}
store.put(namespace, item_id, updated_value)
updated_item = store.get(namespace, item_id)
assert item
assert item.namespace == namespace
assert item.key == item_id
assert item.value == item_value
assert updated_item.value == updated_value
assert updated_item.updated_at > item.updated_at
# Test update
updated_value = {"title": "Updated Document", "content": "Hello, Updated!"}
store.put(namespace, item_id, updated_value)
updated_item = store.get(namespace, item_id)
# Test get from non-existent namespace
different_namespace = ("test", "other_documents")
item_in_different_namespace = store.get(different_namespace, item_id)
assert item_in_different_namespace is None
assert updated_item.value == updated_value
assert updated_item.updated_at > item.updated_at
# Test delete
store.delete(namespace, item_id)
deleted_item = store.get(namespace, item_id)
assert deleted_item is None
# Test get from non-existent namespace
different_namespace = ("test", "other_documents")
item_in_different_namespace = store.get(different_namespace, item_id)
assert item_in_different_namespace is None
# Test delete
store.delete(namespace, item_id)
deleted_item = store.get(namespace, item_id)
assert deleted_item is None
def test_list_namespaces(store) -> None:
# Create test data with various namespaces
test_namespaces = [
("test", "documents", "public"),
("test", "documents", "private"),
("test", "images", "public"),
("test", "images", "private"),
("prod", "documents", "public"),
("prod", "documents", "private"),
]
def test_list_namespaces(self) -> None:
with PostgresStore.from_conn_string(DEFAULT_URI) as store:
# Create test data with various namespaces
test_namespaces = [
("test", "documents", "public"),
("test", "documents", "private"),
("test", "images", "public"),
("test", "images", "private"),
("prod", "documents", "public"),
("prod", "documents", "private"),
]
# Insert test data
for namespace in test_namespaces:
store.put(namespace, "dummy", {"content": "dummy"})
# Insert test data
for namespace in test_namespaces:
store.put(namespace, "dummy", {"content": "dummy"})
# Test listing with various filters
all_namespaces = store.list_namespaces()
assert len(all_namespaces) == len(test_namespaces)
# Test listing with various filters
all_namespaces = store.list_namespaces()
assert len(all_namespaces) == len(test_namespaces)
# Test prefix filtering
test_prefix_namespaces = store.list_namespaces(prefix=["test"])
assert len(test_prefix_namespaces) == 4
assert all(ns[0] == "test" for ns in test_prefix_namespaces)
# Test prefix filtering
test_prefix_namespaces = store.list_namespaces(prefix=["test"])
assert len(test_prefix_namespaces) == 4
assert all(ns[0] == "test" for ns in test_prefix_namespaces)
# Test suffix filtering
public_namespaces = store.list_namespaces(suffix=["public"])
assert len(public_namespaces) == 3
assert all(ns[-1] == "public" for ns in public_namespaces)
# Test suffix filtering
public_namespaces = store.list_namespaces(suffix=["public"])
assert len(public_namespaces) == 3
assert all(ns[-1] == "public" for ns in public_namespaces)
# Test max depth
depth_2_namespaces = store.list_namespaces(max_depth=2)
assert all(len(ns) <= 2 for ns in depth_2_namespaces)
# Test max depth
depth_2_namespaces = store.list_namespaces(max_depth=2)
assert all(len(ns) <= 2 for ns in depth_2_namespaces)
# Test pagination
paginated_namespaces = store.list_namespaces(limit=3)
assert len(paginated_namespaces) == 3
# Test pagination
paginated_namespaces = store.list_namespaces(limit=3)
assert len(paginated_namespaces) == 3
# Cleanup
for namespace in test_namespaces:
store.delete(namespace, "dummy")
# Cleanup
for namespace in test_namespaces:
store.delete(namespace, "dummy")
def test_search(self) -> None:
with PostgresStore.from_conn_string(DEFAULT_URI) as store:
# Create test data
test_data = [
(
("test", "docs"),
"doc1",
{"title": "First Doc", "author": "Alice", "tags": ["important"]},
),
(
("test", "docs"),
"doc2",
{"title": "Second Doc", "author": "Bob", "tags": ["draft"]},
),
(
("test", "images"),
"img1",
{"title": "Image 1", "author": "Alice", "tags": ["final"]},
),
]
def test_search(store) -> None:
# Create test data
test_data = [
(
("test", "docs"),
"doc1",
{"title": "First Doc", "author": "Alice", "tags": ["important"]},
),
(
("test", "docs"),
"doc2",
{"title": "Second Doc", "author": "Bob", "tags": ["draft"]},
),
(
("test", "images"),
"img1",
{"title": "Image 1", "author": "Alice", "tags": ["final"]},
),
]
for namespace, key, value in test_data:
store.put(namespace, key, value)
for namespace, key, value in test_data:
store.put(namespace, key, value)
# Test basic search
all_items = store.search(["test"])
assert len(all_items) == 3
# Test basic search
all_items = store.search(["test"])
assert len(all_items) == 3
# Test namespace filtering
docs_items = store.search(["test", "docs"])
assert len(docs_items) == 2
assert all(item.namespace == ("test", "docs") for item in docs_items)
# Test namespace filtering
docs_items = store.search(["test", "docs"])
assert len(docs_items) == 2
assert all(item.namespace == ("test", "docs") for item in docs_items)
# Test value filtering
alice_items = store.search(["test"], filter={"author": "Alice"})
assert len(alice_items) == 2
assert all(item.value["author"] == "Alice" for item in alice_items)
# Test value filtering
alice_items = store.search(["test"], filter={"author": "Alice"})
assert len(alice_items) == 2
assert all(item.value["author"] == "Alice" for item in alice_items)
# Test pagination
paginated_items = store.search(["test"], limit=2)
assert len(paginated_items) == 2
# Test pagination
paginated_items = store.search(["test"], limit=2)
assert len(paginated_items) == 2
offset_items = store.search(["test"], offset=2)
assert len(offset_items) == 1
offset_items = store.search(["test"], offset=2)
assert len(offset_items) == 1
# Cleanup
for namespace, key, _ in test_data:
store.delete(namespace, key)
# Cleanup
for namespace, key, _ in test_data:
store.delete(namespace, key)
@contextmanager
@@ -380,8 +355,7 @@ def _create_vector_store(
vector_type: str,
distance_type: str,
fake_embeddings: Embeddings,
text_fields: list[str] | None = None,
enable_ttl: bool = True,
text_fields: Optional[list[str]] = None,
) -> PostgresStore:
"""Create a store with vector search enabled."""
database = f"test_{uuid4().hex[:16]}"
@@ -402,7 +376,7 @@ def _create_vector_store(
"vector_type": vector_type,
},
"distance_type": distance_type,
"fields": text_fields,
"text_fields": text_fields,
}
with Connection.connect(admin_conn_string, autocommit=True) as conn:
@@ -411,32 +385,23 @@ def _create_vector_store(
with PostgresStore.from_conn_string(
conn_string,
index=index_config,
ttl={"default_ttl": 2, "refresh_on_read": True} if enable_ttl else None,
) as store:
store.setup()
with store._cursor() as cur:
# drop the migration index
cur.execute("DROP TABLE IF EXISTS store_migrations")
store.setup() # Will fail if migrations aren't idempotent
yield store
finally:
with Connection.connect(admin_conn_string, autocommit=True) as conn:
conn.execute(f"DROP DATABASE {database}")
_vector_params = [
(vector_type, distance_type, True)
for vector_type in VECTOR_TYPES
for distance_type in (
["hamming"] if vector_type == "bit" else ["l2", "inner_product", "cosine"]
)
]
_vector_params += [(*_vector_params[-1][:2], False)]
@pytest.fixture(
scope="function",
params=_vector_params,
params=[
(vector_type, distance_type)
for vector_type in VECTOR_TYPES
for distance_type in (
["hamming"] if vector_type == "bit" else ["l2", "inner_product", "cosine"]
)
],
ids=lambda p: f"{p[0]}_{p[1]}",
)
def vector_store(
@@ -444,10 +409,8 @@ def vector_store(
fake_embeddings: Embeddings,
) -> PostgresStore:
"""Create a store with vector search enabled."""
vector_type, distance_type, enable_ttl = request.param
with _create_vector_store(
vector_type, distance_type, fake_embeddings, enable_ttl=enable_ttl
) as store:
vector_type, distance_type = request.param
with _create_vector_store(vector_type, distance_type, fake_embeddings) as store:
yield store
@@ -511,10 +474,7 @@ def test_vector_update_with_embedding(vector_store: PostgresStore) -> None:
assert not any(r.key == "doc4" for r in results_new)
@pytest.mark.parametrize("refresh_ttl", [True, False])
def test_vector_search_with_filters(
vector_store: PostgresStore, refresh_ttl: bool
) -> None:
def test_vector_search_with_filters(vector_store: PostgresStore) -> None:
"""Test combining vector search with filters."""
# Insert test documents
docs = [
@@ -527,23 +487,16 @@ def test_vector_search_with_filters(
for key, value in docs:
vector_store.put(("test",), key, value)
results = vector_store.search(
("test",), query="apple", filter={"color": "red"}, refresh_ttl=refresh_ttl
)
results = vector_store.search(("test",), query="apple", filter={"color": "red"})
assert len(results) == 2
assert results[0].key == "doc1"
results = vector_store.search(
("test",), query="car", filter={"color": "red"}, refresh_ttl=refresh_ttl
)
results = vector_store.search(("test",), query="car", filter={"color": "red"})
assert len(results) == 2
assert results[0].key == "doc2"
results = vector_store.search(
("test",),
query="bbbbluuu",
filter={"score": {"$gt": 3.2}},
refresh_ttl=refresh_ttl,
("test",), query="bbbbluuu", filter={"score": {"$gt": 3.2}}
)
assert len(results) == 3
assert results[0].key == "doc4"
@@ -735,7 +688,7 @@ def test_embed_with_path_operation_config(
store.put(("test",), "doc5", doc5, index=False)
results = store.search(("test",))
assert len(results) == 3
assert all(r.score is None for r in results), f"{results}"
assert all(r.score is None for r in results)
assert any(r.key == "doc5" for r in results)
results = store.search(("test",), query="hhh")
@@ -754,7 +707,7 @@ def _cosine_similarity(X: list[float], Y: list[list[float]]) -> list[float]:
similarities = []
for y in Y:
dot_product = sum(a * b for a, b in zip(X, y, strict=False))
dot_product = sum(a * b for a, b in zip(X, y))
norm1 = sum(a * a for a in X) ** 0.5
norm2 = sum(a * a for a in y) ** 0.5
similarity = dot_product / (norm1 * norm2) if norm1 > 0 and norm2 > 0 else 0.0
@@ -771,7 +724,7 @@ def _inner_product(X: list[float], Y: list[list[float]]) -> list[float]:
similarities = []
for y in Y:
similarity = sum(a * b for a, b in zip(X, y, strict=False))
similarity = sum(a * b for a, b in zip(X, y))
similarities.append(similarity)
return similarities
@@ -785,7 +738,7 @@ def _neg_l2_distance(X: list[float], Y: list[list[float]]) -> list[float]:
similarities = []
for y in Y:
similarity = sum((a - b) ** 2 for a, b in zip(X, y, strict=False)) ** 0.5
similarity = sum((a - b) ** 2 for a, b in zip(X, y)) ** 0.5
similarities.append(-similarity)
return similarities
@@ -837,65 +790,3 @@ def test_nonnull_migrations() -> None:
for migration in PostgresStore.MIGRATIONS:
statement = _leading_comment_remover.sub("", migration).split()[0]
assert statement.strip()
def test_store_ttl(store):
# Assumes a TTL of 1 minute = 60 seconds
ns = ("foo",)
store.put(
ns,
key="item1",
value={"foo": "bar"},
ttl=TTL_MINUTES, # type: ignore
)
time.sleep(TTL_SECONDS - 2)
res = store.get(ns, key="item1", refresh_ttl=True)
assert res is not None
time.sleep(TTL_SECONDS - 2)
results = store.search(ns, query="foo", refresh_ttl=True)
assert len(results) == 1
time.sleep(TTL_SECONDS - 2)
res = store.get(ns, key="item1", refresh_ttl=False)
assert res is not None
time.sleep(TTL_SECONDS - 1)
# Now has been (TTL_SECONDS-2)*2 > TTL_SECONDS + TTL_SECONDS/2
res = store.search(ns, query="bar", refresh_ttl=False)
assert len(res) == 0
@pytest.mark.parametrize(
"vector_type,distance_type",
[
("vector", "cosine"),
("vector", "inner_product"),
("halfvec", "cosine"),
("halfvec", "inner_product"),
],
)
def test_non_ascii(
request: Any,
fake_embeddings: CharacterEmbeddings,
vector_type: str,
distance_type: str,
) -> None:
"""Test support for non-ascii characters"""
with _create_vector_store(vector_type, distance_type, fake_embeddings) as store:
store.put(("user_123", "memories"), "1", {"text": "这是中文"}) # Chinese
store.put(
("user_123", "memories"), "2", {"text": "これは日本語です"}
) # Japanese
store.put(("user_123", "memories"), "3", {"text": "이건 한국어야"}) # Korean
store.put(("user_123", "memories"), "4", {"text": "Это русский"}) # Russian
store.put(("user_123", "memories"), "5", {"text": "यह रूसी है"}) # Hindi
result1 = store.search(("user_123", "memories"), query="这是中文")
result2 = store.search(("user_123", "memories"), query="これは日本語です")
result3 = store.search(("user_123", "memories"), query="이건 한국어야")
result4 = store.search(("user_123", "memories"), query="Это русский")
result5 = store.search(("user_123", "memories"), query="यह रूसी है")
assert result1[0].key == "1"
assert result2[0].key == "2"
assert result3[0].key == "3"
assert result4[0].key == "4"
assert result5[0].key == "5"
+11 -85
View File
@@ -7,6 +7,10 @@ from uuid import uuid4
import pytest
from langchain_core.runnables import RunnableConfig
from psycopg import Connection
from psycopg.rows import dict_row
from psycopg_pool import ConnectionPool
from langgraph.checkpoint.base import (
EXCLUDED_METADATA_KEYS,
Checkpoint,
@@ -14,11 +18,6 @@ from langgraph.checkpoint.base import (
create_checkpoint,
empty_checkpoint,
)
from langgraph.checkpoint.serde.types import TASKS
from psycopg import Connection
from psycopg.rows import dict_row
from psycopg_pool import ConnectionPool
from langgraph.checkpoint.postgres import PostgresSaver, ShallowPostgresSaver
from tests.conftest import DEFAULT_POSTGRES_URI
@@ -143,7 +142,8 @@ def test_data():
config_1: RunnableConfig = {
"configurable": {
"thread_id": "thread-1",
"checkpoint_id": "1",
# for backwards compatibility testing
"thread_ts": "1",
"checkpoint_ns": "",
}
}
@@ -169,11 +169,13 @@ def test_data():
metadata_1: CheckpointMetadata = {
"source": "input",
"step": 2,
"writes": {},
"score": 1,
}
metadata_2: CheckpointMetadata = {
"source": "loop",
"step": 1,
"writes": {"foo": "bar"},
"score": None,
}
metadata_3: CheckpointMetadata = {}
@@ -200,12 +202,14 @@ def test_combined_metadata(saver_name: str, test_data) -> None:
metadata: CheckpointMetadata = {
"source": "loop",
"step": 1,
"writes": {"foo": "bar"},
"score": None,
}
saver.put(config, chkpnt, metadata, {})
checkpoint = saver.get_tuple(config)
assert checkpoint.metadata == {
**metadata,
"thread_id": "thread-2",
"run_id": "my_run_id",
}
@@ -225,6 +229,7 @@ def test_search(saver_name: str, test_data) -> None:
query_1 = {"source": "input"} # search by 1 key
query_2 = {
"step": 1,
"writes": {"foo": "bar"},
} # search by multiple keys
query_3: dict[str, Any] = {} # search by no keys, return all checkpoints
query_4 = {"source": "update", "step": 1} # no match
@@ -279,82 +284,3 @@ def test_nonnull_migrations() -> None:
for migration in PostgresSaver.MIGRATIONS:
statement = _leading_comment_remover.sub("", migration).split()[0]
assert statement.strip()
@pytest.mark.parametrize("saver_name", ["base", "pool", "pipe"])
def test_pending_sends_migration(saver_name: str) -> None:
with _saver(saver_name) as saver:
config = {
"configurable": {
"thread_id": "thread-1",
"checkpoint_ns": "",
}
}
# create the first checkpoint
# and put some pending sends
checkpoint_0 = empty_checkpoint()
config = saver.put(config, checkpoint_0, {}, {})
saver.put_writes(
config, [(TASKS, "send-1"), (TASKS, "send-2")], task_id="task-1"
)
saver.put_writes(config, [(TASKS, "send-3")], task_id="task-2")
# check that fetching checkpoint_0 doesn't attach pending sends
# (they should be attached to the next checkpoint)
tuple_0 = saver.get_tuple(config)
assert tuple_0.checkpoint["channel_values"] == {}
assert tuple_0.checkpoint["channel_versions"] == {}
# create the second checkpoint
checkpoint_1 = create_checkpoint(checkpoint_0, {}, 1)
config = saver.put(config, checkpoint_1, {}, {})
# check that pending sends are attached to checkpoint_1
checkpoint_1 = saver.get_tuple(config)
assert checkpoint_1.checkpoint["channel_values"] == {
TASKS: ["send-1", "send-2", "send-3"]
}
assert TASKS in checkpoint_1.checkpoint["channel_versions"]
# check that list also applies the migration
search_results = [
c for c in saver.list({"configurable": {"thread_id": "thread-1"}})
]
assert len(search_results) == 2
assert search_results[-1].checkpoint["channel_values"] == {}
assert search_results[-1].checkpoint["channel_versions"] == {}
assert search_results[0].checkpoint["channel_values"] == {
TASKS: ["send-1", "send-2", "send-3"]
}
assert TASKS in search_results[0].checkpoint["channel_versions"]
@pytest.mark.parametrize("saver_name", ["base", "pool", "pipe"])
def test_get_checkpoint_no_channel_values(
monkeypatch, saver_name: str, test_data
) -> None:
"""Backwards compatibility test that verifies a checkpoint with no channel_values key can be retrieved without throwing an error."""
with _saver(saver_name) as saver:
config = {
"configurable": {
"thread_id": "thread-2",
"checkpoint_ns": "",
"__super_private_key": "super_private_value",
},
}
chkpnt: Checkpoint = create_checkpoint(empty_checkpoint(), {}, 1)
saver.put(config, chkpnt, {}, {})
load_checkpoint_tuple = saver._load_checkpoint_tuple
def patched_load_checkpoint_tuple(value):
value["checkpoint"].pop("channel_values", None)
return load_checkpoint_tuple(value)
monkeypatch.setattr(
saver, "_load_checkpoint_tuple", patched_load_checkpoint_tuple
)
checkpoint = saver.get_tuple(config)
assert checkpoint.checkpoint["channel_values"] == {}
-1346
View File
File diff suppressed because it is too large. Load diff
-21
View File
@@ -1,21 +0,0 @@
MIT License
Copyright (c) 2024 LangChain, Inc.
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.
-37
View File
@@ -1,37 +0,0 @@
.PHONY: test test_watch lint format
######################
# TESTING AND COVERAGE
######################
TEST ?= .
test:
uv run pytest $(TEST)
test_watch:
uv run ptw $(TEST)
######################
# LINTING AND FORMATTING
######################
# Define a variable for Python and notebook files.
PYTHON_FILES=.
MYPY_CACHE=.mypy_cache
lint format: PYTHON_FILES=.
lint_diff format_diff: PYTHON_FILES=$(shell git diff --name-only --relative --diff-filter=d main . | grep -E '\.py$$|\.ipynb$$')
lint_package: PYTHON_FILES=langgraph
lint_tests: PYTHON_FILES=tests
lint_tests: MYPY_CACHE=.mypy_cache_test
lint lint_diff lint_package lint_tests:
uv run ruff check .
[ "$(PYTHON_FILES)" = "" ] || uv run ruff format $(PYTHON_FILES) --diff
[ "$(PYTHON_FILES)" = "" ] || uv run ruff check --select I $(PYTHON_FILES)
[ "$(PYTHON_FILES)" = "" ] || mkdir -p $(MYPY_CACHE)
[ "$(PYTHON_FILES)" = "" ] || uv run mypy $(PYTHON_FILES) --cache-dir $(MYPY_CACHE)
format format_diff:
uv run ruff format $(PYTHON_FILES)
uv run ruff check --select I --fix $(PYTHON_FILES)
-88
View File
@@ -1,88 +0,0 @@
# LangGraph SQLite Checkpoint
Implementation of LangGraph CheckpointSaver that uses SQLite DB (both sync and async, via `aiosqlite`)
## Usage
```python
from langgraph.checkpoint.sqlite import SqliteSaver
write_config = {"configurable": {"thread_id": "1", "checkpoint_ns": ""}}
read_config = {"configurable": {"thread_id": "1"}}
with SqliteSaver.from_conn_string(":memory:") as checkpointer:
checkpoint = {
"v": 4,
"ts": "2024-07-31T20:14:19.804150+00:00",
"id": "1ef4f797-8335-6428-8001-8a1503f9b875",
"channel_values": {
"my_key": "meow",
"node": "node"
},
"channel_versions": {
"__start__": 2,
"my_key": 3,
"start:node": 3,
"node": 3
},
"versions_seen": {
"__input__": {},
"__start__": {
"__start__": 1
},
"node": {
"start:node": 2
}
},
}
# store checkpoint
checkpointer.put(write_config, checkpoint, {}, {})
# load checkpoint
checkpointer.get(read_config)
# list checkpoints
list(checkpointer.list(read_config))
```
### Async
```python
from langgraph.checkpoint.sqlite.aio import AsyncSqliteSaver
async with AsyncSqliteSaver.from_conn_string(":memory:") as checkpointer:
checkpoint = {
"v": 4,
"ts": "2024-07-31T20:14:19.804150+00:00",
"id": "1ef4f797-8335-6428-8001-8a1503f9b875",
"channel_values": {
"my_key": "meow",
"node": "node"
},
"channel_versions": {
"__start__": 2,
"my_key": 3,
"start:node": 3,
"node": 3
},
"versions_seen": {
"__input__": {},
"__start__": {
"__start__": 1
},
"node": {
"start:node": 2
}
},
}
# store checkpoint
await checkpointer.aput(write_config, checkpoint, {}, {})
# load checkpoint
await checkpointer.aget(read_config)
# list checkpoints
[c async for c in checkpointer.alist(read_config)]
```
@@ -1,120 +0,0 @@
from __future__ import annotations
import asyncio
import datetime
import sqlite3
import threading
from collections.abc import Mapping, Sequence
from langgraph.cache.base import BaseCache, FullKey, Namespace, ValueT
from langgraph.checkpoint.serde.base import SerializerProtocol
class SqliteCache(BaseCache[ValueT]):
"""File-based cache using SQLite."""
def __init__(
self,
*,
path: str,
serde: SerializerProtocol | None = None,
) -> None:
"""Initialize the cache with a file path."""
super().__init__(serde=serde)
# SQLite backing store
self._conn = sqlite3.connect(
path,
check_same_thread=False,
)
# Serialize access to the shared connection across threads
self._lock = threading.RLock()
# Better concurrency & atomicity
self._conn.execute("PRAGMA journal_mode=WAL;")
# Schema: key -> (expiry, encoding, value)
self._conn.execute(
"""CREATE TABLE IF NOT EXISTS cache (
ns TEXT,
key TEXT,
expiry REAL,
encoding TEXT NOT NULL,
val BLOB NOT NULL,
PRIMARY KEY (ns, key)
)"""
)
self._conn.commit()
def get(self, keys: Sequence[FullKey]) -> dict[FullKey, ValueT]:
"""Get the cached values for the given keys."""
with self._lock, self._conn:
now = datetime.datetime.now(datetime.timezone.utc).timestamp()
if not keys:
return {}
placeholders = ",".join("(?, ?)" for _ in keys)
params: list[str] = []
for ns_tuple, key in keys:
params.extend((",".join(ns_tuple), key))
cursor = self._conn.execute(
f"SELECT ns, key, expiry, encoding, val FROM cache WHERE (ns, key) IN ({placeholders})",
tuple(params),
)
values: dict[FullKey, ValueT] = {}
rows = cursor.fetchall()
for ns, key, expiry, encoding, raw in rows:
if expiry is not None and now > expiry:
# purge expired entry
self._conn.execute(
"DELETE FROM cache WHERE (ns, key) = (?, ?)", (ns, key)
)
continue
values[(tuple(ns.split(",")), key)] = self.serde.loads_typed(
(encoding, raw)
)
return values
async def aget(self, keys: Sequence[FullKey]) -> dict[FullKey, ValueT]:
"""Asynchronously get the cached values for the given keys."""
return await asyncio.to_thread(self.get, keys)
def set(self, mapping: Mapping[FullKey, tuple[ValueT, int | None]]) -> None:
"""Set the cached values for the given keys and TTLs."""
with self._lock, self._conn:
now = datetime.datetime.now(datetime.timezone.utc)
for key, (value, ttl) in mapping.items():
if ttl is not None:
delta = datetime.timedelta(seconds=ttl)
expiry: float | None = (now + delta).timestamp()
else:
expiry = None
encoding, raw = self.serde.dumps_typed(value)
self._conn.execute(
"INSERT OR REPLACE INTO cache (ns, key, expiry, encoding, val) VALUES (?, ?, ?, ?, ?)",
(",".join(key[0]), key[1], expiry, encoding, raw),
)
async def aset(self, mapping: Mapping[FullKey, tuple[ValueT, int | None]]) -> None:
"""Asynchronously set the cached values for the given keys and TTLs."""
await asyncio.to_thread(self.set, mapping)
def clear(self, namespaces: Sequence[Namespace] | None = None) -> None:
"""Delete the cached values for the given namespaces.
If no namespaces are provided, clear all cached values."""
with self._lock, self._conn:
if namespaces is None:
self._conn.execute("DELETE FROM cache")
else:
placeholders = ",".join("?" for _ in namespaces)
self._conn.execute(
f"DELETE FROM cache WHERE (ns) IN ({placeholders})",
tuple(",".join(key) for key in namespaces),
)
async def aclear(self, namespaces: Sequence[Namespace] | None = None) -> None:
"""Asynchronously delete the cached values for the given namespaces.
If no namespaces are provided, clear all cached values."""
await asyncio.to_thread(self.clear, namespaces)
def __del__(self) -> None:
try:
self._conn.close()
except Exception:
pass
@@ -1,556 +0,0 @@
from __future__ import annotations
import json
import random
import sqlite3
import threading
from collections.abc import AsyncIterator, Iterator, Sequence
from contextlib import closing, contextmanager
from typing import Any, cast
from langchain_core.runnables import RunnableConfig
from langgraph.checkpoint.base import (
WRITES_IDX_MAP,
BaseCheckpointSaver,
ChannelVersions,
Checkpoint,
CheckpointMetadata,
CheckpointTuple,
SerializerProtocol,
get_checkpoint_id,
get_checkpoint_metadata,
)
from langgraph.checkpoint.serde.jsonplus import JsonPlusSerializer
from langgraph.checkpoint.sqlite.utils import search_where
_AIO_ERROR_MSG = (
"The SqliteSaver does not support async methods. "
"Consider using AsyncSqliteSaver instead.\n"
"from langgraph.checkpoint.sqlite.aio import AsyncSqliteSaver\n"
"Note: AsyncSqliteSaver requires the aiosqlite package to use.\n"
"Install with:\n`pip install aiosqlite`\n"
"See https://langchain-ai.github.io/langgraph/reference/checkpoints/#langgraph.checkpoint.sqlite.aio.AsyncSqliteSaver"
"for more information."
)
class SqliteSaver(BaseCheckpointSaver[str]):
"""A checkpoint saver that stores checkpoints in a SQLite database.
Note:
This class is meant for lightweight, synchronous use cases
(demos and small projects) and does not
scale to multiple threads.
For a similar sqlite saver with `async` support,
consider using [AsyncSqliteSaver][langgraph.checkpoint.sqlite.aio.AsyncSqliteSaver].
Args:
conn (sqlite3.Connection): The SQLite database connection.
serde (Optional[SerializerProtocol]): The serializer to use for serializing and deserializing checkpoints. Defaults to JsonPlusSerializerCompat.
Examples:
>>> import sqlite3
>>> from langgraph.checkpoint.sqlite import SqliteSaver
>>> from langgraph.graph import StateGraph
>>>
>>> builder = StateGraph(int)
>>> builder.add_node("add_one", lambda x: x + 1)
>>> builder.set_entry_point("add_one")
>>> builder.set_finish_point("add_one")
>>> # Create a new SqliteSaver instance
>>> # Note: check_same_thread=False is OK as the implementation uses a lock
>>> # to ensure thread safety.
>>> conn = sqlite3.connect("checkpoints.sqlite", check_same_thread=False)
>>> memory = SqliteSaver(conn)
>>> graph = builder.compile(checkpointer=memory)
>>> config = {"configurable": {"thread_id": "1"}}
>>> graph.get_state(config)
>>> result = graph.invoke(3, config)
>>> graph.get_state(config)
StateSnapshot(values=4, next=(), config={'configurable': {'thread_id': '1', 'checkpoint_ns': '', 'checkpoint_id': '0c62ca34-ac19-445d-bbb0-5b4984975b2a'}}, parent_config=None)
""" # noqa
conn: sqlite3.Connection
is_setup: bool
def __init__(
self,
conn: sqlite3.Connection,
*,
serde: SerializerProtocol | None = None,
) -> None:
super().__init__(serde=serde)
self.jsonplus_serde = JsonPlusSerializer()
self.conn = conn
self.is_setup = False
self.lock = threading.Lock()
@classmethod
@contextmanager
def from_conn_string(cls, conn_string: str) -> Iterator[SqliteSaver]:
"""Create a new SqliteSaver instance from a connection string.
Args:
conn_string: The SQLite connection string.
Yields:
SqliteSaver: A new SqliteSaver instance.
Examples:
In memory:
with SqliteSaver.from_conn_string(":memory:") as memory:
...
To disk:
with SqliteSaver.from_conn_string("checkpoints.sqlite") as memory:
...
"""
with closing(
sqlite3.connect(
conn_string,
# https://ricardoanderegg.com/posts/python-sqlite-thread-safety/
check_same_thread=False,
)
) as conn:
yield cls(conn)
def setup(self) -> None:
"""Set up the checkpoint database.
This method creates the necessary tables in the SQLite database if they don't
already exist. It is called automatically when needed and should not be called
directly by the user.
"""
if self.is_setup:
return
self.conn.executescript(
"""
PRAGMA journal_mode=WAL;
CREATE TABLE IF NOT EXISTS checkpoints (
thread_id TEXT NOT NULL,
checkpoint_ns TEXT NOT NULL DEFAULT '',
checkpoint_id TEXT NOT NULL,
parent_checkpoint_id TEXT,
type TEXT,
checkpoint BLOB,
metadata BLOB,
PRIMARY KEY (thread_id, checkpoint_ns, checkpoint_id)
);
CREATE TABLE IF NOT EXISTS writes (
thread_id TEXT NOT NULL,
checkpoint_ns TEXT NOT NULL DEFAULT '',
checkpoint_id TEXT NOT NULL,
task_id TEXT NOT NULL,
idx INTEGER NOT NULL,
channel TEXT NOT NULL,
type TEXT,
value BLOB,
PRIMARY KEY (thread_id, checkpoint_ns, checkpoint_id, task_id, idx)
);
"""
)
self.is_setup = True
@contextmanager
def cursor(self, transaction: bool = True) -> Iterator[sqlite3.Cursor]:
"""Get a cursor for the SQLite database.
This method returns a cursor for the SQLite database. It is used internally
by the SqliteSaver and should not be called directly by the user.
Args:
transaction (bool): Whether to commit the transaction when the cursor is closed. Defaults to True.
Yields:
sqlite3.Cursor: A cursor for the SQLite database.
"""
with self.lock:
self.setup()
cur = self.conn.cursor()
try:
yield cur
finally:
if transaction:
self.conn.commit()
cur.close()
def get_tuple(self, config: RunnableConfig) -> CheckpointTuple | None:
"""Get a checkpoint tuple from the database.
This method retrieves a checkpoint tuple from the SQLite database based on the
provided config. If the config contains a `checkpoint_id` key, the checkpoint with
the matching thread ID and checkpoint ID is retrieved. Otherwise, the latest checkpoint
for the given thread ID is retrieved.
Args:
config: The config to use for retrieving the checkpoint.
Returns:
The retrieved checkpoint tuple, or None if no matching checkpoint was found.
Examples:
Basic:
>>> config = {"configurable": {"thread_id": "1"}}
>>> checkpoint_tuple = memory.get_tuple(config)
>>> print(checkpoint_tuple)
CheckpointTuple(...)
With checkpoint ID:
>>> config = {
... "configurable": {
... "thread_id": "1",
... "checkpoint_ns": "",
... "checkpoint_id": "1ef4f797-8335-6428-8001-8a1503f9b875",
... }
... }
>>> checkpoint_tuple = memory.get_tuple(config)
>>> print(checkpoint_tuple)
CheckpointTuple(...)
""" # noqa
checkpoint_ns = config["configurable"].get("checkpoint_ns", "")
with self.cursor(transaction=False) as cur:
# find the latest checkpoint for the thread_id
if checkpoint_id := get_checkpoint_id(config):
cur.execute(
"SELECT thread_id, checkpoint_id, parent_checkpoint_id, type, checkpoint, metadata FROM checkpoints WHERE thread_id = ? AND checkpoint_ns = ? AND checkpoint_id = ?",
(
str(config["configurable"]["thread_id"]),
checkpoint_ns,
checkpoint_id,
),
)
else:
cur.execute(
"SELECT thread_id, checkpoint_id, parent_checkpoint_id, type, checkpoint, metadata FROM checkpoints WHERE thread_id = ? AND checkpoint_ns = ? ORDER BY checkpoint_id DESC LIMIT 1",
(str(config["configurable"]["thread_id"]), checkpoint_ns),
)
# if a checkpoint is found, return it
if value := cur.fetchone():
(
thread_id,
checkpoint_id,
parent_checkpoint_id,
type,
checkpoint,
metadata,
) = value
if not get_checkpoint_id(config):
config = {
"configurable": {
"thread_id": thread_id,
"checkpoint_ns": checkpoint_ns,
"checkpoint_id": checkpoint_id,
}
}
# find any pending writes
cur.execute(
"SELECT task_id, channel, type, value FROM writes WHERE thread_id = ? AND checkpoint_ns = ? AND checkpoint_id = ? ORDER BY task_id, idx",
(
str(config["configurable"]["thread_id"]),
checkpoint_ns,
str(config["configurable"]["checkpoint_id"]),
),
)
# deserialize the checkpoint and metadata
return CheckpointTuple(
config,
self.serde.loads_typed((type, checkpoint)),
cast(
CheckpointMetadata,
json.loads(metadata) if metadata is not None else {},
),
(
{
"configurable": {
"thread_id": thread_id,
"checkpoint_ns": checkpoint_ns,
"checkpoint_id": parent_checkpoint_id,
}
}
if parent_checkpoint_id
else None
),
[
(task_id, channel, self.serde.loads_typed((type, value)))
for task_id, channel, type, value in cur
],
)
def list(
self,
config: RunnableConfig | None,
*,
filter: dict[str, Any] | None = None,
before: RunnableConfig | None = None,
limit: int | None = None,
) -> Iterator[CheckpointTuple]:
"""List checkpoints from the database.
This method retrieves a list of checkpoint tuples from the SQLite database based
on the provided config. The checkpoints are ordered by checkpoint ID in descending order (newest first).
Args:
config: The config to use for listing the checkpoints.
filter: Additional filtering criteria for metadata.
before: If provided, only checkpoints before the specified checkpoint ID are returned.
limit: The maximum number of checkpoints to return.
Yields:
An iterator of checkpoint tuples.
Examples:
>>> from langgraph.checkpoint.sqlite import SqliteSaver
>>> with SqliteSaver.from_conn_string(":memory:") as memory:
... # Run a graph, then list the checkpoints
>>> config = {"configurable": {"thread_id": "1"}}
>>> checkpoints = list(memory.list(config, limit=2))
>>> print(checkpoints)
[CheckpointTuple(...), CheckpointTuple(...)]
>>> config = {"configurable": {"thread_id": "1"}}
>>> before = {"configurable": {"checkpoint_id": "1ef4f797-8335-6428-8001-8a1503f9b875"}}
>>> with SqliteSaver.from_conn_string(":memory:") as memory:
... # Run a graph, then list the checkpoints
>>> checkpoints = list(memory.list(config, before=before))
>>> print(checkpoints)
[CheckpointTuple(...), ...]
"""
where, param_values = search_where(config, filter, before)
query = f"""SELECT thread_id, checkpoint_ns, checkpoint_id, parent_checkpoint_id, type, checkpoint, metadata
FROM checkpoints
{where}
ORDER BY checkpoint_id DESC"""
if limit is not None:
query += " LIMIT ?"
param_values = (*param_values, limit)
with self.cursor(transaction=False) as cur, closing(self.conn.cursor()) as wcur:
cur.execute(query, param_values)
for (
thread_id,
checkpoint_ns,
checkpoint_id,
parent_checkpoint_id,
type,
checkpoint,
metadata,
) in cur:
wcur.execute(
"SELECT task_id, channel, type, value FROM writes WHERE thread_id = ? AND checkpoint_ns = ? AND checkpoint_id = ? ORDER BY task_id, idx",
(thread_id, checkpoint_ns, checkpoint_id),
)
yield CheckpointTuple(
{
"configurable": {
"thread_id": thread_id,
"checkpoint_ns": checkpoint_ns,
"checkpoint_id": checkpoint_id,
}
},
self.serde.loads_typed((type, checkpoint)),
cast(
CheckpointMetadata,
json.loads(metadata) if metadata is not None else {},
),
(
{
"configurable": {
"thread_id": thread_id,
"checkpoint_ns": checkpoint_ns,
"checkpoint_id": parent_checkpoint_id,
}
}
if parent_checkpoint_id
else None
),
[
(task_id, channel, self.serde.loads_typed((type, value)))
for task_id, channel, type, value in wcur
],
)
def put(
self,
config: RunnableConfig,
checkpoint: Checkpoint,
metadata: CheckpointMetadata,
new_versions: ChannelVersions,
) -> RunnableConfig:
"""Save a checkpoint to the database.
This method saves a checkpoint to the SQLite database. The checkpoint is associated
with the provided config and its parent config (if any).
Args:
config: The config to associate with the checkpoint.
checkpoint: The checkpoint to save.
metadata: Additional metadata to save with the checkpoint.
new_versions: New channel versions as of this write.
Returns:
RunnableConfig: Updated configuration after storing the checkpoint.
Examples:
>>> from langgraph.checkpoint.sqlite import SqliteSaver
>>> with SqliteSaver.from_conn_string(":memory:") as memory:
>>> config = {"configurable": {"thread_id": "1", "checkpoint_ns": ""}}
>>> checkpoint = {"ts": "2024-05-04T06:32:42.235444+00:00", "id": "1ef4f797-8335-6428-8001-8a1503f9b875", "channel_values": {"key": "value"}}
>>> saved_config = memory.put(config, checkpoint, {"source": "input", "step": 1, "writes": {"key": "value"}}, {})
>>> print(saved_config)
{'configurable': {'thread_id': '1', 'checkpoint_ns': '', 'checkpoint_id': '1ef4f797-8335-6428-8001-8a1503f9b875'}}
"""
thread_id = config["configurable"]["thread_id"]
checkpoint_ns = config["configurable"]["checkpoint_ns"]
type_, serialized_checkpoint = self.serde.dumps_typed(checkpoint)
serialized_metadata = json.dumps(
get_checkpoint_metadata(config, metadata), ensure_ascii=False
).encode("utf-8", "ignore")
with self.cursor() as cur:
cur.execute(
"INSERT OR REPLACE INTO checkpoints (thread_id, checkpoint_ns, checkpoint_id, parent_checkpoint_id, type, checkpoint, metadata) VALUES (?, ?, ?, ?, ?, ?, ?)",
(
str(config["configurable"]["thread_id"]),
checkpoint_ns,
checkpoint["id"],
config["configurable"].get("checkpoint_id"),
type_,
serialized_checkpoint,
serialized_metadata,
),
)
return {
"configurable": {
"thread_id": thread_id,
"checkpoint_ns": checkpoint_ns,
"checkpoint_id": checkpoint["id"],
}
}
def put_writes(
self,
config: RunnableConfig,
writes: Sequence[tuple[str, Any]],
task_id: str,
task_path: str = "",
) -> None:
"""Store intermediate writes linked to a checkpoint.
This method saves intermediate writes associated with a checkpoint to the SQLite database.
Args:
config: Configuration of the related checkpoint.
writes: List of writes to store, each as (channel, value) pair.
task_id: Identifier for the task creating the writes.
task_path: Path of the task creating the writes.
"""
query = (
"INSERT OR REPLACE INTO writes (thread_id, checkpoint_ns, checkpoint_id, task_id, idx, channel, type, value) VALUES (?, ?, ?, ?, ?, ?, ?, ?)"
if all(w[0] in WRITES_IDX_MAP for w in writes)
else "INSERT OR IGNORE INTO writes (thread_id, checkpoint_ns, checkpoint_id, task_id, idx, channel, type, value) VALUES (?, ?, ?, ?, ?, ?, ?, ?)"
)
with self.cursor() as cur:
cur.executemany(
query,
[
(
str(config["configurable"]["thread_id"]),
str(config["configurable"]["checkpoint_ns"]),
str(config["configurable"]["checkpoint_id"]),
task_id,
WRITES_IDX_MAP.get(channel, idx),
channel,
*self.serde.dumps_typed(value),
)
for idx, (channel, value) in enumerate(writes)
],
)
def delete_thread(self, thread_id: str) -> None:
"""Delete all checkpoints and writes associated with a thread ID.
Args:
thread_id: The thread ID to delete.
Returns:
None
"""
with self.cursor() as cur:
cur.execute(
"DELETE FROM checkpoints WHERE thread_id = ?",
(str(thread_id),),
)
cur.execute(
"DELETE FROM writes WHERE thread_id = ?",
(str(thread_id),),
)
async def aget_tuple(self, config: RunnableConfig) -> CheckpointTuple | None:
"""Get a checkpoint tuple from the database asynchronously.
Note:
This async method is not supported by the SqliteSaver class.
Use get_tuple() instead, or consider using [AsyncSqliteSaver][langgraph.checkpoint.sqlite.aio.AsyncSqliteSaver].
"""
raise NotImplementedError(_AIO_ERROR_MSG)
async def alist(
self,
config: RunnableConfig | None,
*,
filter: dict[str, Any] | None = None,
before: RunnableConfig | None = None,
limit: int | None = None,
) -> AsyncIterator[CheckpointTuple]:
"""List checkpoints from the database asynchronously.
Note:
This async method is not supported by the SqliteSaver class.
Use list() instead, or consider using [AsyncSqliteSaver][langgraph.checkpoint.sqlite.aio.AsyncSqliteSaver].
"""
raise NotImplementedError(_AIO_ERROR_MSG)
yield
async def aput(
self,
config: RunnableConfig,
checkpoint: Checkpoint,
metadata: CheckpointMetadata,
new_versions: ChannelVersions,
) -> RunnableConfig:
"""Save a checkpoint to the database asynchronously.
Note:
This async method is not supported by the SqliteSaver class.
Use put() instead, or consider using [AsyncSqliteSaver][langgraph.checkpoint.sqlite.aio.AsyncSqliteSaver].
"""
raise NotImplementedError(_AIO_ERROR_MSG)
def get_next_version(self, current: str | None, channel: None) -> str:
"""Generate the next version ID for a channel.
This method creates a new version identifier for a channel based on its current version.
Args:
current (Optional[str]): The current version identifier of the channel.
Returns:
str: The next version identifier, which is guaranteed to be monotonically increasing.
"""
if current is None:
current_v = 0
elif isinstance(current, int):
current_v = current
else:
current_v = int(current.split(".")[0])
next_v = current_v + 1
next_h = random.random()
return f"{next_v:032}.{next_h:016}"
@@ -1,611 +0,0 @@
from __future__ import annotations
import asyncio
import json
import random
from collections.abc import AsyncIterator, Callable, Iterator, Sequence
from contextlib import asynccontextmanager
from typing import Any, TypeVar, cast
import aiosqlite
from langchain_core.runnables import RunnableConfig
from langgraph.checkpoint.base import (
WRITES_IDX_MAP,
BaseCheckpointSaver,
ChannelVersions,
Checkpoint,
CheckpointMetadata,
CheckpointTuple,
SerializerProtocol,
get_checkpoint_id,
get_checkpoint_metadata,
)
from langgraph.checkpoint.serde.jsonplus import JsonPlusSerializer
from langgraph.checkpoint.sqlite.utils import search_where
T = TypeVar("T", bound=Callable)
class AsyncSqliteSaver(BaseCheckpointSaver[str]):
"""An asynchronous checkpoint saver that stores checkpoints in a SQLite database.
This class provides an asynchronous interface for saving and retrieving checkpoints
using a SQLite database. It's designed for use in asynchronous environments and
offers better performance for I/O-bound operations compared to synchronous alternatives.
Attributes:
conn (aiosqlite.Connection): The asynchronous SQLite database connection.
serde (SerializerProtocol): The serializer used for encoding/decoding checkpoints.
Tip:
Requires the [aiosqlite](https://pypi.org/project/aiosqlite/) package.
Install it with `pip install aiosqlite`.
Warning:
While this class supports asynchronous checkpointing, it is not recommended
for production workloads due to limitations in SQLite's write performance.
For production use, consider a more robust database like PostgreSQL.
Tip:
Remember to **close the database connection** after executing your code,
otherwise, you may see the graph "hang" after execution (since the program
will not exit until the connection is closed).
The easiest way is to use the `async with` statement as shown in the examples.
```python
async with AsyncSqliteSaver.from_conn_string("checkpoints.sqlite") as saver:
# Your code here
graph = builder.compile(checkpointer=saver)
config = {"configurable": {"thread_id": "thread-1"}}
async for event in graph.astream_events(..., config, version="v1"):
print(event)
```
Examples:
Usage within StateGraph:
```pycon
>>> import asyncio
>>>
>>> from langgraph.checkpoint.sqlite.aio import AsyncSqliteSaver
>>> from langgraph.graph import StateGraph
>>>
>>> async def main():
>>> builder = StateGraph(int)
>>> builder.add_node("add_one", lambda x: x + 1)
>>> builder.set_entry_point("add_one")
>>> builder.set_finish_point("add_one")
>>> async with AsyncSqliteSaver.from_conn_string("checkpoints.db") as memory:
>>> graph = builder.compile(checkpointer=memory)
>>> coro = graph.ainvoke(1, {"configurable": {"thread_id": "thread-1"}})
>>> print(await asyncio.gather(coro))
>>>
>>> asyncio.run(main())
Output: [2]
```
Raw usage:
```pycon
>>> import asyncio
>>> import aiosqlite
>>> from langgraph.checkpoint.sqlite.aio import AsyncSqliteSaver
>>>
>>> async def main():
>>> async with aiosqlite.connect("checkpoints.db") as conn:
... saver = AsyncSqliteSaver(conn)
... config = {"configurable": {"thread_id": "1", "checkpoint_ns": ""}}
... checkpoint = {"ts": "2023-05-03T10:00:00Z", "data": {"key": "value"}, "id": "0c62ca34-ac19-445d-bbb0-5b4984975b2a"}
... saved_config = await saver.aput(config, checkpoint, {}, {})
... print(saved_config)
>>> asyncio.run(main())
{'configurable': {'thread_id': '1', 'checkpoint_ns': '', 'checkpoint_id': '0c62ca34-ac19-445d-bbb0-5b4984975b2a'}}
```
"""
lock: asyncio.Lock
is_setup: bool
def __init__(
self,
conn: aiosqlite.Connection,
*,
serde: SerializerProtocol | None = None,
):
super().__init__(serde=serde)
self.jsonplus_serde = JsonPlusSerializer()
self.conn = conn
self.lock = asyncio.Lock()
self.loop = asyncio.get_running_loop()
self.is_setup = False
@classmethod
@asynccontextmanager
async def from_conn_string(
cls, conn_string: str
) -> AsyncIterator[AsyncSqliteSaver]:
"""Create a new AsyncSqliteSaver instance from a connection string.
Args:
conn_string: The SQLite connection string.
Yields:
AsyncSqliteSaver: A new AsyncSqliteSaver instance.
"""
async with aiosqlite.connect(conn_string) as conn:
yield cls(conn)
def get_tuple(self, config: RunnableConfig) -> CheckpointTuple | None:
"""Get a checkpoint tuple from the database.
This method retrieves a checkpoint tuple from the SQLite database based on the
provided config. If the config contains a `checkpoint_id` key, the checkpoint with
the matching thread ID and checkpoint ID is retrieved. Otherwise, the latest checkpoint
for the given thread ID is retrieved.
Args:
config: The config to use for retrieving the checkpoint.
Returns:
The retrieved checkpoint tuple, or None if no matching checkpoint was found.
"""
try:
# check if we are in the main thread, only bg threads can block
# we don't check in other methods to avoid the overhead
if asyncio.get_running_loop() is self.loop:
raise asyncio.InvalidStateError(
"Synchronous calls to AsyncSqliteSaver are only allowed from a "
"different thread. From the main thread, use the async interface. "
"For example, use `await checkpointer.aget_tuple(...)` or `await "
"graph.ainvoke(...)`."
)
except RuntimeError:
pass
return asyncio.run_coroutine_threadsafe(
self.aget_tuple(config), self.loop
).result()
def list(
self,
config: RunnableConfig | None,
*,
filter: dict[str, Any] | None = None,
before: RunnableConfig | None = None,
limit: int | None = None,
) -> Iterator[CheckpointTuple]:
"""List checkpoints from the database asynchronously.
This method retrieves a list of checkpoint tuples from the SQLite database based
on the provided config. The checkpoints are ordered by checkpoint ID in descending order (newest first).
Args:
config: Base configuration for filtering checkpoints.
filter: Additional filtering criteria for metadata.
before: If provided, only checkpoints before the specified checkpoint ID are returned.
limit: Maximum number of checkpoints to return.
Yields:
An iterator of matching checkpoint tuples.
"""
try:
# check if we are in the main thread, only bg threads can block
# we don't check in other methods to avoid the overhead
if asyncio.get_running_loop() is self.loop:
raise asyncio.InvalidStateError(
"Synchronous calls to AsyncSqliteSaver are only allowed from a "
"different thread. From the main thread, use the async interface. "
"For example, use `checkpointer.alist(...)` or `await "
"graph.ainvoke(...)`."
)
except RuntimeError:
pass
aiter_ = self.alist(config, filter=filter, before=before, limit=limit)
while True:
try:
yield asyncio.run_coroutine_threadsafe(
anext(aiter_), # type: ignore[arg-type] # noqa: F821
self.loop,
).result()
except StopAsyncIteration:
break
def put(
self,
config: RunnableConfig,
checkpoint: Checkpoint,
metadata: CheckpointMetadata,
new_versions: ChannelVersions,
) -> RunnableConfig:
"""Save a checkpoint to the database.
This method saves a checkpoint to the SQLite database. The checkpoint is associated
with the provided config and its parent config (if any).
Args:
config: The config to associate with the checkpoint.
checkpoint: The checkpoint to save.
metadata: Additional metadata to save with the checkpoint.
new_versions: New channel versions as of this write.
Returns:
RunnableConfig: Updated configuration after storing the checkpoint.
"""
return asyncio.run_coroutine_threadsafe(
self.aput(config, checkpoint, metadata, new_versions), self.loop
).result()
def put_writes(
self,
config: RunnableConfig,
writes: Sequence[tuple[str, Any]],
task_id: str,
task_path: str = "",
) -> None:
return asyncio.run_coroutine_threadsafe(
self.aput_writes(config, writes, task_id, task_path), self.loop
).result()
def delete_thread(self, thread_id: str) -> None:
"""Delete all checkpoints and writes associated with a thread ID.
Args:
thread_id: The thread ID to delete.
Returns:
None
"""
try:
# check if we are in the main thread, only bg threads can block
# we don't check in other methods to avoid the overhead
if asyncio.get_running_loop() is self.loop:
raise asyncio.InvalidStateError(
"Synchronous calls to AsyncSqliteSaver are only allowed from a "
"different thread. From the main thread, use the async interface. "
"For example, use `checkpointer.alist(...)` or `await "
"graph.ainvoke(...)`."
)
except RuntimeError:
pass
return asyncio.run_coroutine_threadsafe(
self.adelete_thread(thread_id), self.loop
).result()
async def setup(self) -> None:
"""Set up the checkpoint database asynchronously.
This method creates the necessary tables in the SQLite database if they don't
already exist. It is called automatically when needed and should not be called
directly by the user.
"""
async with self.lock:
if self.is_setup:
return
if not self.conn.is_alive():
await self.conn
async with self.conn.executescript(
"""
PRAGMA journal_mode=WAL;
CREATE TABLE IF NOT EXISTS checkpoints (
thread_id TEXT NOT NULL,
checkpoint_ns TEXT NOT NULL DEFAULT '',
checkpoint_id TEXT NOT NULL,
parent_checkpoint_id TEXT,
type TEXT,
checkpoint BLOB,
metadata BLOB,
PRIMARY KEY (thread_id, checkpoint_ns, checkpoint_id)
);
CREATE TABLE IF NOT EXISTS writes (
thread_id TEXT NOT NULL,
checkpoint_ns TEXT NOT NULL DEFAULT '',
checkpoint_id TEXT NOT NULL,
task_id TEXT NOT NULL,
idx INTEGER NOT NULL,
channel TEXT NOT NULL,
type TEXT,
value BLOB,
PRIMARY KEY (thread_id, checkpoint_ns, checkpoint_id, task_id, idx)
);
"""
):
await self.conn.commit()
self.is_setup = True
async def aget_tuple(self, config: RunnableConfig) -> CheckpointTuple | None:
"""Get a checkpoint tuple from the database asynchronously.
This method retrieves a checkpoint tuple from the SQLite database based on the
provided config. If the config contains a `checkpoint_id` key, the checkpoint with
the matching thread ID and checkpoint ID is retrieved. Otherwise, the latest checkpoint
for the given thread ID is retrieved.
Args:
config: The config to use for retrieving the checkpoint.
Returns:
The retrieved checkpoint tuple, or None if no matching checkpoint was found.
"""
await self.setup()
checkpoint_ns = config["configurable"].get("checkpoint_ns", "")
async with self.lock, self.conn.cursor() as cur:
# find the latest checkpoint for the thread_id
if checkpoint_id := get_checkpoint_id(config):
await cur.execute(
"SELECT thread_id, checkpoint_id, parent_checkpoint_id, type, checkpoint, metadata FROM checkpoints WHERE thread_id = ? AND checkpoint_ns = ? AND checkpoint_id = ?",
(
str(config["configurable"]["thread_id"]),
checkpoint_ns,
checkpoint_id,
),
)
else:
await cur.execute(
"SELECT thread_id, checkpoint_id, parent_checkpoint_id, type, checkpoint, metadata FROM checkpoints WHERE thread_id = ? AND checkpoint_ns = ? ORDER BY checkpoint_id DESC LIMIT 1",
(str(config["configurable"]["thread_id"]), checkpoint_ns),
)
# if a checkpoint is found, return it
if value := await cur.fetchone():
(
thread_id,
checkpoint_id,
parent_checkpoint_id,
type,
checkpoint,
metadata,
) = value
if not get_checkpoint_id(config):
config = {
"configurable": {
"thread_id": thread_id,
"checkpoint_ns": checkpoint_ns,
"checkpoint_id": checkpoint_id,
}
}
# find any pending writes
await cur.execute(
"SELECT task_id, channel, type, value FROM writes WHERE thread_id = ? AND checkpoint_ns = ? AND checkpoint_id = ? ORDER BY task_id, idx",
(
str(config["configurable"]["thread_id"]),
checkpoint_ns,
str(config["configurable"]["checkpoint_id"]),
),
)
# deserialize the checkpoint and metadata
return CheckpointTuple(
config,
self.serde.loads_typed((type, checkpoint)),
cast(
CheckpointMetadata,
(json.loads(metadata) if metadata is not None else {}),
),
(
{
"configurable": {
"thread_id": thread_id,
"checkpoint_ns": checkpoint_ns,
"checkpoint_id": parent_checkpoint_id,
}
}
if parent_checkpoint_id
else None
),
[
(task_id, channel, self.serde.loads_typed((type, value)))
async for task_id, channel, type, value in cur
],
)
async def alist(
self,
config: RunnableConfig | None,
*,
filter: dict[str, Any] | None = None,
before: RunnableConfig | None = None,
limit: int | None = None,
) -> AsyncIterator[CheckpointTuple]:
"""List checkpoints from the database asynchronously.
This method retrieves a list of checkpoint tuples from the SQLite database based
on the provided config. The checkpoints are ordered by checkpoint ID in descending order (newest first).
Args:
config: Base configuration for filtering checkpoints.
filter: Additional filtering criteria for metadata.
before: If provided, only checkpoints before the specified checkpoint ID are returned.
limit: Maximum number of checkpoints to return.
Yields:
An asynchronous iterator of matching checkpoint tuples.
"""
await self.setup()
where, params = search_where(config, filter, before)
query = f"""SELECT thread_id, checkpoint_ns, checkpoint_id, parent_checkpoint_id, type, checkpoint, metadata
FROM checkpoints
{where}
ORDER BY checkpoint_id DESC"""
if limit is not None:
query += " LIMIT ?"
params = (*params, limit)
async with (
self.lock,
self.conn.execute(query, params) as cur,
self.conn.cursor() as wcur,
):
async for (
thread_id,
checkpoint_ns,
checkpoint_id,
parent_checkpoint_id,
type,
checkpoint,
metadata,
) in cur:
await wcur.execute(
"SELECT task_id, channel, type, value FROM writes WHERE thread_id = ? AND checkpoint_ns = ? AND checkpoint_id = ? ORDER BY task_id, idx",
(thread_id, checkpoint_ns, checkpoint_id),
)
yield CheckpointTuple(
{
"configurable": {
"thread_id": thread_id,
"checkpoint_ns": checkpoint_ns,
"checkpoint_id": checkpoint_id,
}
},
self.serde.loads_typed((type, checkpoint)),
cast(
CheckpointMetadata,
(json.loads(metadata) if metadata is not None else {}),
),
(
{
"configurable": {
"thread_id": thread_id,
"checkpoint_ns": checkpoint_ns,
"checkpoint_id": parent_checkpoint_id,
}
}
if parent_checkpoint_id
else None
),
[
(task_id, channel, self.serde.loads_typed((type, value)))
async for task_id, channel, type, value in wcur
],
)
async def aput(
self,
config: RunnableConfig,
checkpoint: Checkpoint,
metadata: CheckpointMetadata,
new_versions: ChannelVersions,
) -> RunnableConfig:
"""Save a checkpoint to the database asynchronously.
This method saves a checkpoint to the SQLite database. The checkpoint is associated
with the provided config and its parent config (if any).
Args:
config: The config to associate with the checkpoint.
checkpoint: The checkpoint to save.
metadata: Additional metadata to save with the checkpoint.
new_versions: New channel versions as of this write.
Returns:
RunnableConfig: Updated configuration after storing the checkpoint.
"""
await self.setup()
thread_id = config["configurable"]["thread_id"]
checkpoint_ns = config["configurable"]["checkpoint_ns"]
type_, serialized_checkpoint = self.serde.dumps_typed(checkpoint)
serialized_metadata = json.dumps(
get_checkpoint_metadata(config, metadata), ensure_ascii=False
).encode("utf-8", "ignore")
async with (
self.lock,
self.conn.execute(
"INSERT OR REPLACE INTO checkpoints (thread_id, checkpoint_ns, checkpoint_id, parent_checkpoint_id, type, checkpoint, metadata) VALUES (?, ?, ?, ?, ?, ?, ?)",
(
str(config["configurable"]["thread_id"]),
checkpoint_ns,
checkpoint["id"],
config["configurable"].get("checkpoint_id"),
type_,
serialized_checkpoint,
serialized_metadata,
),
),
):
await self.conn.commit()
return {
"configurable": {
"thread_id": thread_id,
"checkpoint_ns": checkpoint_ns,
"checkpoint_id": checkpoint["id"],
}
}
async def aput_writes(
self,
config: RunnableConfig,
writes: Sequence[tuple[str, Any]],
task_id: str,
task_path: str = "",
) -> None:
"""Store intermediate writes linked to a checkpoint asynchronously.
This method saves intermediate writes associated with a checkpoint to the database.
Args:
config: Configuration of the related checkpoint.
writes: List of writes to store, each as (channel, value) pair.
task_id: Identifier for the task creating the writes.
task_path: Path of the task creating the writes.
"""
query = (
"INSERT OR REPLACE INTO writes (thread_id, checkpoint_ns, checkpoint_id, task_id, idx, channel, type, value) VALUES (?, ?, ?, ?, ?, ?, ?, ?)"
if all(w[0] in WRITES_IDX_MAP for w in writes)
else "INSERT OR IGNORE INTO writes (thread_id, checkpoint_ns, checkpoint_id, task_id, idx, channel, type, value) VALUES (?, ?, ?, ?, ?, ?, ?, ?)"
)
await self.setup()
async with self.lock, self.conn.cursor() as cur:
await cur.executemany(
query,
[
(
str(config["configurable"]["thread_id"]),
str(config["configurable"]["checkpoint_ns"]),
str(config["configurable"]["checkpoint_id"]),
task_id,
WRITES_IDX_MAP.get(channel, idx),
channel,
*self.serde.dumps_typed(value),
)
for idx, (channel, value) in enumerate(writes)
],
)
await self.conn.commit()
async def adelete_thread(self, thread_id: str) -> None:
"""Delete all checkpoints and writes associated with a thread ID.
Args:
thread_id: The thread ID to delete.
Returns:
None
"""
async with self.lock, self.conn.cursor() as cur:
await cur.execute(
"DELETE FROM checkpoints WHERE thread_id = ?",
(str(thread_id),),
)
await cur.execute(
"DELETE FROM writes WHERE thread_id = ?",
(str(thread_id),),
)
await self.conn.commit()
def get_next_version(self, current: str | None, channel: None) -> str:
"""Generate the next version ID for a channel.
This method creates a new version identifier for a channel based on its current version.
Args:
current (Optional[str]): The current version identifier of the channel.
Returns:
str: The next version identifier, which is guaranteed to be monotonically increasing.
"""
if current is None:
current_v = 0
elif isinstance(current, int):
current_v = current
else:
current_v = int(current.split(".")[0])
next_v = current_v + 1
next_h = random.random()
return f"{next_v:032}.{next_h:016}"
@@ -1,116 +0,0 @@
from __future__ import annotations
import json
import re
from collections.abc import Sequence
from typing import Any
from langchain_core.runnables import RunnableConfig
from langgraph.checkpoint.base import get_checkpoint_id
_FILTER_PATTERN = re.compile(r"^[a-zA-Z0-9_.-]+$")
def _validate_filter_key(key: str) -> None:
"""Validate that a filter key is safe for use in SQL queries.
Args:
key: The filter key to validate
Raises:
ValueError: If the key contains invalid characters that could enable SQL injection
"""
# Allow alphanumeric characters, underscores, dots, and hyphens
# This covers typical JSON property names while preventing SQL injection
if not _FILTER_PATTERN.match(key):
raise ValueError(
f"Invalid filter key: '{key}'. Filter keys must contain only alphanumeric characters, underscores, dots, and hyphens."
)
def _metadata_predicate(
metadata_filter: dict[str, Any],
) -> tuple[Sequence[str], Sequence[Any]]:
"""Return WHERE clause predicates for (a)search() given metadata filter.
This method returns a tuple of a string and a tuple of values. The string
is the parametered WHERE clause predicate (excluding the WHERE keyword):
"column1 = ? AND column2 IS ?". The tuple of values contains the values
for each of the corresponding parameters.
"""
def _where_value(query_value: Any) -> tuple[str, Any]:
"""Return tuple of operator and value for WHERE clause predicate."""
if query_value is None:
return ("IS ?", None)
elif (
isinstance(query_value, str)
or isinstance(query_value, int)
or isinstance(query_value, float)
):
return ("= ?", query_value)
elif isinstance(query_value, bool):
return ("= ?", 1 if query_value else 0)
elif isinstance(query_value, dict) or isinstance(query_value, list):
# query value for JSON object cannot have trailing space after separators (, :)
# SQLite json_extract() returns JSON string without whitespace
return ("= ?", json.dumps(query_value, separators=(",", ":")))
else:
return ("= ?", str(query_value))
predicates = []
param_values = []
# process metadata query
for query_key, query_value in metadata_filter.items():
_validate_filter_key(query_key)
operator, param_value = _where_value(query_value)
predicates.append(
f"json_extract(CAST(metadata AS TEXT), '$.{query_key}') {operator}"
)
param_values.append(param_value)
return (predicates, param_values)
def search_where(
config: RunnableConfig | None,
filter: dict[str, Any] | None,
before: RunnableConfig | None = None,
) -> tuple[str, Sequence[Any]]:
"""Return WHERE clause predicates for (a)search() given metadata filter
and `before` config.
This method returns a tuple of a string and a tuple of values. The string
is the parametered WHERE clause predicate (including the WHERE keyword):
"WHERE column1 = ? AND column2 IS ?". The tuple of values contains the
values for each of the corresponding parameters.
"""
wheres = []
param_values = []
# construct predicate for config filter
if config is not None:
wheres.append("thread_id = ?")
param_values.append(config["configurable"]["thread_id"])
checkpoint_ns = config["configurable"].get("checkpoint_ns")
if checkpoint_ns is not None:
wheres.append("checkpoint_ns = ?")
param_values.append(checkpoint_ns)
if checkpoint_id := get_checkpoint_id(config):
wheres.append("checkpoint_id = ?")
param_values.append(checkpoint_id)
# construct predicate for metadata filter
if filter:
metadata_predicates, metadata_values = _metadata_predicate(filter)
wheres.extend(metadata_predicates)
param_values.extend(metadata_values)
# construct predicate for `before`
if before is not None:
wheres.append("checkpoint_id < ?")
param_values.append(get_checkpoint_id(before))
return ("WHERE " + " AND ".join(wheres) if wheres else "", param_values)
@@ -1,4 +0,0 @@
from langgraph.store.sqlite.aio import AsyncSqliteStore
from langgraph.store.sqlite.base import SqliteStore
__all__ = ["AsyncSqliteStore", "SqliteStore"]
@@ -1,623 +0,0 @@
from __future__ import annotations
import asyncio
import logging
from collections import defaultdict
from collections.abc import AsyncIterator, Callable, Iterable, Sequence
from contextlib import asynccontextmanager
from types import TracebackType
from typing import Any, cast
import aiosqlite
import orjson
import sqlite_vec # type: ignore[import-untyped]
from langgraph.store.base import (
GetOp,
ListNamespacesOp,
Op,
PutOp,
Result,
SearchOp,
TTLConfig,
)
from langgraph.store.base.batch import AsyncBatchedBaseStore
from langgraph.store.sqlite.base import (
_PLACEHOLDER,
BaseSqliteStore,
SqliteIndexConfig,
_decode_ns_text,
_ensure_index_config,
_group_ops,
_row_to_item,
_row_to_search_item,
)
logger = logging.getLogger(__name__)
class AsyncSqliteStore(AsyncBatchedBaseStore, BaseSqliteStore):
"""Asynchronous SQLite-backed store with optional vector search.
This class provides an asynchronous interface for storing and retrieving data
using a SQLite database with support for vector search capabilities.
Examples:
Basic setup and usage:
```python
from langgraph.store.sqlite import AsyncSqliteStore
async with AsyncSqliteStore.from_conn_string(":memory:") as store:
await store.setup() # Run migrations
# Store and retrieve data
await store.aput(("users", "123"), "prefs", {"theme": "dark"})
item = await store.aget(("users", "123"), "prefs")
```
Vector search using LangChain embeddings:
```python
from langchain_openai import OpenAIEmbeddings
from langgraph.store.sqlite import AsyncSqliteStore
async with AsyncSqliteStore.from_conn_string(
":memory:",
index={
"dims": 1536,
"embed": OpenAIEmbeddings(),
"fields": ["text"] # specify which fields to embed
}
) as store:
await store.setup() # Run migrations once
# Store documents
await store.aput(("docs",), "doc1", {"text": "Python tutorial"})
await store.aput(("docs",), "doc2", {"text": "TypeScript guide"})
await store.aput(("docs",), "doc3", {"text": "Other guide"}, index=False) # don't index
# Search by similarity
results = await store.asearch(("docs",), query="programming guides", limit=2)
```
Warning:
Make sure to call `setup()` before first use to create necessary tables and indexes.
Note:
This class requires the aiosqlite package. Install with `pip install aiosqlite`.
"""
def __init__(
self,
conn: aiosqlite.Connection,
*,
deserializer: Callable[[bytes | str | orjson.Fragment], dict[str, Any]]
| None = None,
index: SqliteIndexConfig | None = None,
ttl: TTLConfig | None = None,
):
"""Initialize the async SQLite store.
Args:
conn: The SQLite database connection.
deserializer: Optional custom deserializer function for values.
index: Optional vector search configuration.
ttl: Optional time-to-live configuration.
"""
super().__init__()
self._deserializer = deserializer
self.conn = conn
self.lock = asyncio.Lock()
self.loop = asyncio.get_running_loop()
self.is_setup = False
self.index_config = index
if self.index_config:
self.embeddings, self.index_config = _ensure_index_config(self.index_config)
else:
self.embeddings = None
self.ttl_config = ttl
self._ttl_sweeper_task: asyncio.Task[None] | None = None
self._ttl_stop_event = asyncio.Event()
@classmethod
@asynccontextmanager
async def from_conn_string(
cls,
conn_string: str,
*,
index: SqliteIndexConfig | None = None,
ttl: TTLConfig | None = None,
) -> AsyncIterator[AsyncSqliteStore]:
"""Create a new AsyncSqliteStore instance from a connection string.
Args:
conn_string: The SQLite connection string.
index: Optional vector search configuration.
ttl: Optional time-to-live configuration.
Returns:
An AsyncSqliteStore instance wrapped in an async context manager.
"""
async with aiosqlite.connect(conn_string, isolation_level=None) as conn:
yield cls(conn, index=index, ttl=ttl)
async def setup(self) -> None:
"""Set up the store database.
This method creates the necessary tables in the SQLite database if they don't
already exist and runs database migrations. It should be called before first use.
"""
async with self.lock:
if self.is_setup:
return
# Create migrations table if it doesn't exist
await self.conn.execute(
"""
CREATE TABLE IF NOT EXISTS store_migrations (
v INTEGER PRIMARY KEY
)
"""
)
# Check current migration version
async with self.conn.execute(
"SELECT v FROM store_migrations ORDER BY v DESC LIMIT 1"
) as cur:
row = await cur.fetchone()
if row is None:
version = -1
else:
version = row[0]
# Apply migrations
for v, sql in enumerate(self.MIGRATIONS[version + 1 :], start=version + 1):
await self.conn.executescript(sql)
await self.conn.execute(
"INSERT INTO store_migrations (v) VALUES (?)", (v,)
)
# Apply vector migrations if index config is provided
if self.index_config:
# Create vector migrations table if it doesn't exist
await self.conn.enable_load_extension(True)
await self.conn.load_extension(sqlite_vec.loadable_path())
await self.conn.enable_load_extension(False)
await self.conn.execute(
"""
CREATE TABLE IF NOT EXISTS vector_migrations (
v INTEGER PRIMARY KEY
)
"""
)
# Check current vector migration version
async with self.conn.execute(
"SELECT v FROM vector_migrations ORDER BY v DESC LIMIT 1"
) as cur:
row = await cur.fetchone()
if row is None:
version = -1
else:
version = row[0]
# Apply vector migrations
for v, sql in enumerate(
self.VECTOR_MIGRATIONS[version + 1 :], start=version + 1
):
await self.conn.executescript(sql)
await self.conn.execute(
"INSERT INTO vector_migrations (v) VALUES (?)", (v,)
)
self.is_setup = True
@asynccontextmanager
async def _cursor(
self, *, transaction: bool = True
) -> AsyncIterator[aiosqlite.Cursor]:
"""Get a cursor for the SQLite database.
Args:
transaction: Whether to use a transaction for database operations.
Yields:
An SQLite cursor object.
"""
if not self.is_setup:
await self.setup()
async with self.lock:
if transaction:
await self.conn.execute("BEGIN")
async with self.conn.cursor() as cur:
try:
yield cur
finally:
if transaction:
await self.conn.execute("COMMIT")
async def sweep_ttl(self) -> int:
"""Delete expired store items based on TTL.
Returns:
int: The number of deleted items.
"""
async with self._cursor() as cur:
await cur.execute(
"""
DELETE FROM store
WHERE expires_at IS NOT NULL AND expires_at < CURRENT_TIMESTAMP
"""
)
deleted_count = cur.rowcount
return deleted_count
async def start_ttl_sweeper(
self, sweep_interval_minutes: int | None = None
) -> asyncio.Task[None]:
"""Periodically delete expired store items based on TTL.
Returns:
Task that can be awaited or cancelled.
"""
if not self.ttl_config:
return asyncio.create_task(asyncio.sleep(0))
if self._ttl_sweeper_task is not None and not self._ttl_sweeper_task.done():
return self._ttl_sweeper_task
self._ttl_stop_event.clear()
interval = float(
sweep_interval_minutes or self.ttl_config.get("sweep_interval_minutes") or 5
)
logger.info(f"Starting store TTL sweeper with interval {interval} minutes")
async def _sweep_loop() -> None:
while not self._ttl_stop_event.is_set():
try:
try:
await asyncio.wait_for(
self._ttl_stop_event.wait(),
timeout=interval * 60,
)
break
except asyncio.TimeoutError:
pass
expired_items = await self.sweep_ttl()
if expired_items > 0:
logger.info(f"Store swept {expired_items} expired items")
except asyncio.CancelledError:
break
except Exception as exc:
logger.exception("Store TTL sweep iteration failed", exc_info=exc)
task = asyncio.create_task(_sweep_loop())
task.set_name("ttl_sweeper")
self._ttl_sweeper_task = task
return task
async def stop_ttl_sweeper(self, timeout: float | None = None) -> bool:
"""Stop the TTL sweeper task if it's running.
Args:
timeout: Maximum time to wait for the task to stop, in seconds.
If `None`, wait indefinitely.
Returns:
bool: True if the task was successfully stopped or wasn't running,
False if the timeout was reached before the task stopped.
"""
if self._ttl_sweeper_task is None or self._ttl_sweeper_task.done():
return True
logger.info("Stopping TTL sweeper task")
self._ttl_stop_event.set()
if timeout is not None:
try:
await asyncio.wait_for(self._ttl_sweeper_task, timeout=timeout)
success = True
except asyncio.TimeoutError:
success = False
else:
await self._ttl_sweeper_task
success = True
if success:
self._ttl_sweeper_task = None
logger.info("TTL sweeper task stopped")
else:
logger.warning("Timed out waiting for TTL sweeper task to stop")
return success
async def __aenter__(self) -> AsyncSqliteStore:
return self
async def __aexit__(
self,
exc_type: type[BaseException] | None,
exc_val: BaseException | None,
exc_tb: TracebackType | None,
) -> None:
# Ensure the TTL sweeper task is stopped when exiting the context
if hasattr(self, "_ttl_sweeper_task") and self._ttl_sweeper_task is not None:
# Set the event to signal the task to stop
self._ttl_stop_event.set()
# We don't wait for the task to complete here to avoid blocking
# The task will clean up itself gracefully
async def abatch(self, ops: Iterable[Op]) -> list[Result]:
"""Execute a batch of operations asynchronously.
Args:
ops: Iterable of operations to execute.
Returns:
List of operation results.
"""
grouped_ops, num_ops = _group_ops(ops)
results: list[Result] = [None] * num_ops
async with self._cursor(transaction=True) as cur:
if GetOp in grouped_ops:
await self._batch_get_ops(
cast(Sequence[tuple[int, GetOp]], grouped_ops[GetOp]), results, cur
)
if SearchOp in grouped_ops:
await self._batch_search_ops(
cast(Sequence[tuple[int, SearchOp]], grouped_ops[SearchOp]),
results,
cur,
)
if ListNamespacesOp in grouped_ops:
await self._batch_list_namespaces_ops(
cast(
Sequence[tuple[int, ListNamespacesOp]],
grouped_ops[ListNamespacesOp],
),
results,
cur,
)
if PutOp in grouped_ops:
await self._batch_put_ops(
cast(Sequence[tuple[int, PutOp]], grouped_ops[PutOp]), cur
)
return results
async def _batch_get_ops(
self,
get_ops: Sequence[tuple[int, GetOp]],
results: list[Result],
cur: aiosqlite.Cursor,
) -> None:
"""Process batch GET operations.
Args:
get_ops: Sequence of GET operations.
results: List to store results in.
cur: Database cursor.
"""
# Group all queries by namespace to execute all operations for each namespace together
namespace_queries = defaultdict(list)
for prepared_query in self._get_batch_GET_ops_queries(get_ops):
namespace_queries[prepared_query.namespace].append(prepared_query)
# Process each namespace's operations
for namespace, queries in namespace_queries.items():
# Execute TTL refresh queries first
for query in queries:
if query.kind == "refresh":
try:
await cur.execute(query.query, query.params)
except Exception as e:
raise ValueError(
f"Error executing TTL refresh: \n{query.query}\n{query.params}\n{e}"
) from e
# Then execute GET queries and process results
for query in queries:
if query.kind == "get":
try:
await cur.execute(query.query, query.params)
except Exception as e:
raise ValueError(
f"Error executing GET query: \n{query.query}\n{query.params}\n{e}"
) from e
rows = await cur.fetchall()
key_to_row = {
row[0]: {
"key": row[0],
"value": row[1],
"created_at": row[2],
"updated_at": row[3],
"expires_at": row[4] if len(row) > 4 else None,
"ttl_minutes": row[5] if len(row) > 5 else None,
}
for row in rows
}
# Process results for this query
for idx, key in query.items:
row = key_to_row.get(key)
if row:
results[idx] = _row_to_item(
namespace, row, loader=self._deserializer
)
else:
results[idx] = None
async def _batch_put_ops(
self,
put_ops: Sequence[tuple[int, PutOp]],
cur: aiosqlite.Cursor,
) -> None:
"""Process batch PUT operations.
Args:
put_ops: Sequence of PUT operations.
cur: Database cursor.
"""
queries, embedding_request = self._prepare_batch_PUT_queries(put_ops)
if embedding_request:
if self.embeddings is None:
# Should not get here since the embedding config is required
# to return an embedding_request above
raise ValueError(
"Embedding configuration is required for vector operations "
f"(for semantic search). "
f"Please provide an Embeddings when initializing the {self.__class__.__name__}."
)
query, txt_params = embedding_request
# Update the params to replace the raw text with the vectors
vectors = await self.embeddings.aembed_documents(
[param[-1] for param in txt_params]
)
# Convert vectors to SQLite-friendly format
vector_params = []
for (ns, k, pathname, _), vector in zip(txt_params, vectors, strict=False):
vector_params.extend(
[ns, k, pathname, sqlite_vec.serialize_float32(vector)]
)
queries.append((query, vector_params))
for query, params in queries:
await cur.execute(query, params)
async def _batch_search_ops(
self,
search_ops: Sequence[tuple[int, SearchOp]],
results: list[Result],
cur: aiosqlite.Cursor,
) -> None:
"""Process batch SEARCH operations.
Args:
search_ops: Sequence of SEARCH operations.
results: List to store results in.
cur: Database cursor.
"""
prepared_queries, embedding_requests = self._prepare_batch_search_queries(
search_ops
)
# Setup dot_product function if it doesn't exist
if embedding_requests and self.embeddings:
vectors = await self.embeddings.aembed_documents(
[query for _, query in embedding_requests]
)
for (embed_req_idx, _), embedding in zip(
embedding_requests, vectors, strict=False
):
# Find the corresponding query in prepared_queries
# The embed_req_idx is the original index in search_ops, which should map to prepared_queries
if embed_req_idx < len(prepared_queries):
_params_list: list = prepared_queries[embed_req_idx][1]
for i, param in enumerate(_params_list):
if param is _PLACEHOLDER:
_params_list[i] = sqlite_vec.serialize_float32(embedding)
else:
logger.warning(
f"Embedding request index {embed_req_idx} out of bounds for prepared_queries."
)
for (original_op_idx, _), (query, params, needs_refresh) in zip(
search_ops, prepared_queries, strict=False
):
await cur.execute(query, params)
rows = await cur.fetchall()
if needs_refresh and rows and self.ttl_config:
keys_to_refresh = []
for row_data in rows:
# Assuming row_data[0] is prefix (text), row_data[1] is key (text)
# These are raw text values directly from the DB.
keys_to_refresh.append((row_data[0], row_data[1]))
if keys_to_refresh:
updates_by_prefix = defaultdict(list)
for prefix_text, key_text in keys_to_refresh:
updates_by_prefix[prefix_text].append(key_text)
for prefix_text, key_list in updates_by_prefix.items():
placeholders = ",".join(["?"] * len(key_list))
update_query = f"""
UPDATE store
SET expires_at = DATETIME(CURRENT_TIMESTAMP, '+' || ttl_minutes || ' minutes')
WHERE prefix = ? AND key IN ({placeholders}) AND ttl_minutes IS NOT NULL
"""
update_params = (prefix_text, *key_list)
try:
await cur.execute(update_query, update_params)
except Exception as e:
logger.error(
f"Error during TTL refresh update for search: {e}"
)
# Process rows into items
if "score" in query: # Vector search query
items = [
_row_to_search_item(
_decode_ns_text(row[0]), # prefix
{
"key": row[1], # key
"value": row[2], # value
"created_at": row[3],
"updated_at": row[4],
"expires_at": row[5] if len(row) > 5 else None,
"ttl_minutes": row[6] if len(row) > 6 else None,
"score": row[7] if len(row) > 7 else None,
},
loader=self._deserializer,
)
for row in rows
]
else: # Regular search query
items = [
_row_to_search_item(
_decode_ns_text(row[0]), # prefix
{
"key": row[1], # key
"value": row[2], # value
"created_at": row[3],
"updated_at": row[4],
"expires_at": row[5] if len(row) > 5 else None,
"ttl_minutes": row[6] if len(row) > 6 else None,
},
loader=self._deserializer,
)
for row in rows
]
results[original_op_idx] = items
async def _batch_list_namespaces_ops(
self,
list_ops: Sequence[tuple[int, ListNamespacesOp]],
results: list[Result],
cur: aiosqlite.Cursor,
) -> None:
"""Process batch LIST NAMESPACES operations.
Args:
list_ops: Sequence of LIST NAMESPACES operations.
results: List to store results in.
cur: Database cursor.
"""
queries = self._get_batch_list_namespaces_queries(list_ops)
for (query, params), (idx, _) in zip(queries, list_ops, strict=False):
await cur.execute(query, params)
rows = await cur.fetchall()
results[idx] = [_decode_ns_text(row[0]) for row in rows]
File diff suppressed because it is too large. Load diff
-84
View File
@@ -1,84 +0,0 @@
[build-system]
requires = ["hatchling"]
build-backend = "hatchling.build"
[project]
name = "langgraph-checkpoint-sqlite"
version = "3.0.2"
description = "Library with a SQLite implementation of LangGraph checkpoint saver."
authors = []
requires-python = ">=3.10"
readme = "README.md"
license = "MIT"
license-files = ['LICENSE']
dependencies = [
"langgraph-checkpoint>=3,<5.0.0",
"aiosqlite>=0.20",
"sqlite-vec>=0.1.6",
]
[project.urls]
Source = "https://github.com/langchain-ai/langgraph/tree/main/libs/checkpoint-sqlite"
Twitter = "https://x.com/LangChainAI"
Slack = "https://www.langchain.com/join-community"
Reddit = "https://www.reddit.com/r/LangChain/"
[dependency-groups]
test = [
"pytest",
"pytest-asyncio",
"pytest-mock",
"pytest-watcher",
"langgraph-checkpoint",
"pytest-retry>=1.7.0",
]
lint = [
"ruff",
"codespell",
"mypy",
]
dev = [
{include-group = "test"},
{include-group = "lint"},
]
[tool.uv]
default-groups = ['dev']
[tool.uv.sources]
langgraph-checkpoint = { path = "../checkpoint", editable = true }
[tool.hatch.build.targets.wheel]
include = ["langgraph"]
[tool.pytest.ini_options]
addopts = "--strict-markers --strict-config --durations=5 -vv"
asyncio_mode = "auto"
[tool.ruff]
lint.select = [
"E", # pycodestyle
"F", # Pyflakes
"UP", # pyupgrade
"B", # flake8-bugbear
"I", # isort
"UP", # pyupgrade
]
lint.ignore = ["E501", "B008"]
target-version = "py310"
[tool.pytest-watcher]
now = true
delay = 0.1
runner_args = ["--ff", "-v", "--tb", "short"]
patterns = ["*.py"]
[tool.mypy]
# https://mypy.readthedocs.io/en/stable/config_file.html
disallow_untyped_defs = "True"
explicit_package_bases = "True"
warn_no_return = "False"
warn_unused_ignores = "True"
warn_redundant_casts = "True"
allow_redefinition = "True"
disable_error_code = "typeddict-item, return-value"
@@ -1,190 +0,0 @@
from typing import Any
import pytest
from langchain_core.runnables import RunnableConfig
from langgraph.checkpoint.base import (
Checkpoint,
CheckpointMetadata,
create_checkpoint,
empty_checkpoint,
)
from langgraph.checkpoint.sqlite.aio import AsyncSqliteSaver
class TestAsyncSqliteSaver:
@pytest.fixture(autouse=True)
def setup(self) -> None:
# objects for test setup
self.config_1: RunnableConfig = {
"configurable": {
"thread_id": "thread-1",
"checkpoint_id": "1",
"checkpoint_ns": "",
}
}
self.config_2: RunnableConfig = {
"configurable": {
"thread_id": "thread-2",
"checkpoint_id": "2",
"checkpoint_ns": "",
}
}
self.config_3: RunnableConfig = {
"configurable": {
"thread_id": "thread-2",
"checkpoint_id": "2-inner",
"checkpoint_ns": "inner",
}
}
self.chkpnt_1: Checkpoint = empty_checkpoint()
self.chkpnt_2: Checkpoint = create_checkpoint(self.chkpnt_1, {}, 1)
self.chkpnt_3: Checkpoint = empty_checkpoint()
self.metadata_1: CheckpointMetadata = {
"source": "input",
"step": 2,
"writes": {},
"score": 1,
}
self.metadata_2: CheckpointMetadata = {
"source": "loop",
"step": 1,
"writes": {"foo": "bar"},
"score": None,
}
self.metadata_3: CheckpointMetadata = {}
async def test_combined_metadata(self) -> None:
async with AsyncSqliteSaver.from_conn_string(":memory:") as saver:
config: RunnableConfig = {
"configurable": {
"thread_id": "thread-2",
"checkpoint_ns": "",
"__super_private_key": "super_private_value",
},
"metadata": {"run_id": "my_run_id"},
}
await saver.aput(config, self.chkpnt_2, self.metadata_2, {})
checkpoint = await saver.aget_tuple(config)
assert checkpoint is not None and checkpoint.metadata == {
**self.metadata_2,
"run_id": "my_run_id",
}
async def test_asearch(self) -> None:
async with AsyncSqliteSaver.from_conn_string(":memory:") as saver:
await saver.aput(self.config_1, self.chkpnt_1, self.metadata_1, {})
await saver.aput(self.config_2, self.chkpnt_2, self.metadata_2, {})
await saver.aput(self.config_3, self.chkpnt_3, self.metadata_3, {})
# call method / assertions
query_1 = {"source": "input"} # search by 1 key
query_2 = {
"step": 1,
"writes": {"foo": "bar"},
} # search by multiple keys
query_3: dict[str, Any] = {} # search by no keys, return all checkpoints
query_4 = {"source": "update", "step": 1} # no match
search_results_1 = [c async for c in saver.alist(None, filter=query_1)]
assert len(search_results_1) == 1
assert search_results_1[0].metadata == self.metadata_1
search_results_2 = [c async for c in saver.alist(None, filter=query_2)]
assert len(search_results_2) == 1
assert search_results_2[0].metadata == self.metadata_2
search_results_3 = [c async for c in saver.alist(None, filter=query_3)]
assert len(search_results_3) == 3
search_results_4 = [c async for c in saver.alist(None, filter=query_4)]
assert len(search_results_4) == 0
# search by config (defaults to checkpoints across all namespaces)
search_results_5 = [
c
async for c in saver.alist({"configurable": {"thread_id": "thread-2"}})
]
assert len(search_results_5) == 2
assert {
search_results_5[0].config["configurable"]["checkpoint_ns"],
search_results_5[1].config["configurable"]["checkpoint_ns"],
} == {"", "inner"}
# Test limit param
search_results_6 = [
c
async for c in saver.alist(
{"configurable": {"thread_id": "thread-2"}}, limit=1
)
]
assert len(search_results_6) == 1
assert search_results_6[0].config["configurable"]["thread_id"] == "thread-2"
# Test before param
search_results_7 = [
c async for c in saver.alist(None, before=search_results_5[1].config)
]
assert len(search_results_7) == 1
assert search_results_7[0].config["configurable"]["thread_id"] == "thread-1"
async def test_limit_parameter_sql_injection_prevention(self) -> None:
"""Test that the limit parameter properly uses parameterized queries to prevent SQL injection."""
async with AsyncSqliteSaver.from_conn_string(":memory:") as saver:
# Setup: Create multiple checkpoints
for i in range(5):
config: RunnableConfig = {
"configurable": {
"thread_id": f"thread-{i}",
"checkpoint_ns": "",
}
}
checkpoint = empty_checkpoint()
metadata: CheckpointMetadata = {"index": i}
await saver.aput(config, checkpoint, metadata, {})
# Test that limit works correctly with valid integer
results = [c async for c in saver.alist(None, limit=2)]
assert len(results) == 2
# Test that limit=0 returns no results
results = [c async for c in saver.alist(None, limit=0)]
assert len(results) == 0
# Test that limit=None returns all results
results = [c async for c in saver.alist(None, limit=None)]
assert len(results) == 5
# Test explicit SQL injection attempt via limit parameter
# Even if type checking is bypassed and a malicious string is passed,
# the parameterized query will treat it as a value, not SQL code
# This would cause an error (can't convert string to int for LIMIT),
# which is the correct secure behavior
malicious_limits = [
"1; DROP TABLE checkpoints; --",
"1 OR 1=1",
"999999 UNION SELECT * FROM checkpoints",
]
for malicious_limit in malicious_limits:
# The parameterized query should safely reject non-integer limits
# or convert them in a way that prevents SQL injection
try:
# Bypass type checking by casting
results = [
c
async for c in saver.alist(None, limit=malicious_limit) # type: ignore
]
# If it doesn't raise an error, it should at least not execute the injection
# SQLite's parameter binding will try to convert the string to an integer
# which will either fail or treat it as 0
except Exception:
# Expected: SQLite should reject invalid limit values
pass
# Verify the checkpoints table still exists and has all data
# (would have been dropped if injection succeeded)
results = [c async for c in saver.alist(None, limit=None)]
assert len(results) == 5
@@ -1,719 +0,0 @@
# mypy: disable-error-code="union-attr,arg-type,index,operator"
import asyncio
import os
import tempfile
import uuid
from collections.abc import AsyncIterator, Generator, Iterable
from contextlib import asynccontextmanager
from typing import cast
import pytest
from langgraph.store.base import (
GetOp,
Item,
ListNamespacesOp,
PutOp,
SearchOp,
)
from langgraph.store.sqlite import AsyncSqliteStore
from langgraph.store.sqlite.base import SqliteIndexConfig
from tests.test_store import CharacterEmbeddings
@pytest.fixture(scope="function", params=["memory", "file"])
async def store(request: pytest.FixtureRequest) -> AsyncIterator[AsyncSqliteStore]:
"""Create an AsyncSqliteStore for testing."""
if request.param == "memory":
# In-memory store
async with AsyncSqliteStore.from_conn_string(":memory:") as store:
await store.setup()
yield store
else:
# Temporary file store
temp_file = tempfile.NamedTemporaryFile(delete=False)
temp_file.close()
try:
async with AsyncSqliteStore.from_conn_string(temp_file.name) as store:
await store.setup()
yield store
finally:
os.unlink(temp_file.name)
@pytest.fixture(scope="function")
def fake_embeddings() -> CharacterEmbeddings:
"""Create fake embeddings for testing."""
return CharacterEmbeddings(dims=500)
@asynccontextmanager
async def create_vector_store(
fake_embeddings: CharacterEmbeddings,
conn_string: str = ":memory:",
text_fields: list[str] | None = None,
) -> AsyncIterator[AsyncSqliteStore]:
"""Create an AsyncSqliteStore with vector search capabilities."""
index_config: SqliteIndexConfig = {
"dims": fake_embeddings.dims,
"embed": fake_embeddings,
"text_fields": text_fields,
}
async with AsyncSqliteStore.from_conn_string(
conn_string, index=index_config
) as store:
await store.setup()
yield store
@pytest.fixture(scope="function", params=["memory", "file"])
def conn_string(request: pytest.FixtureRequest) -> Generator[str, None, None]:
if request.param == "memory":
yield ":memory:"
else:
temp_file = tempfile.NamedTemporaryFile(delete=False)
temp_file.close()
try:
yield temp_file.name
finally:
os.unlink(temp_file.name)
async def test_no_running_loop(store: AsyncSqliteStore) -> None:
"""Test that sync methods raise proper errors in the main thread."""
with pytest.raises(asyncio.InvalidStateError):
store.put(("foo", "bar"), "baz", {"val": "baz"})
with pytest.raises(asyncio.InvalidStateError):
store.get(("foo", "bar"), "baz")
with pytest.raises(asyncio.InvalidStateError):
store.delete(("foo", "bar"), "baz")
with pytest.raises(asyncio.InvalidStateError):
store.search(("foo", "bar"))
with pytest.raises(asyncio.InvalidStateError):
store.list_namespaces(prefix=("foo",))
with pytest.raises(asyncio.InvalidStateError):
store.batch([PutOp(namespace=("foo", "bar"), key="baz", value={"val": "baz"})])
async def test_large_batches_async(store: AsyncSqliteStore) -> None:
"""Test processing large batch operations asynchronously."""
N = 100
M = 10
coros = []
for m in range(M):
for i in range(N):
coros.append(
store.aput(
("test", "foo", "bar", "baz", str(m % 2)),
f"key{i}",
value={"foo": "bar" + str(i)},
)
)
coros.append(
asyncio.create_task(
store.aget(
("test", "foo", "bar", "baz", str(m % 2)),
f"key{i}",
)
)
)
coros.append(
asyncio.create_task(
store.alist_namespaces(
prefix=None,
max_depth=m + 1,
)
)
)
coros.append(
asyncio.create_task(
store.asearch(
("test",),
)
)
)
coros.append(
store.aput(
("test", "foo", "bar", "baz", str(m % 2)),
f"key{i}",
value={"foo": "bar" + str(i)},
)
)
coros.append(
store.adelete(
("test", "foo", "bar", "baz", str(m % 2)),
f"key{i}",
)
)
results = await asyncio.gather(*coros)
assert len(results) == M * N * 6
async def test_abatch_order(store: AsyncSqliteStore) -> None:
"""Test ordering of batch operations in async context."""
# Setup test data
await store.aput(("test", "foo"), "key1", {"data": "value1"})
await store.aput(("test", "bar"), "key2", {"data": "value2"})
ops = [
GetOp(namespace=("test", "foo"), key="key1"),
PutOp(namespace=("test", "bar"), key="key2", value={"data": "value2"}),
SearchOp(
namespace_prefix=("test",), filter={"data": "value1"}, limit=10, offset=0
),
ListNamespacesOp(match_conditions=None, max_depth=None, limit=10, offset=0),
GetOp(namespace=("test",), key="key3"),
]
results = await store.abatch(
cast(Iterable[GetOp | PutOp | SearchOp | ListNamespacesOp], ops)
)
assert len(results) == 5
assert isinstance(results[0], Item)
assert isinstance(results[0].value, dict)
assert results[0].value == {"data": "value1"}
assert results[0].key == "key1"
assert results[1] is None # Put operation returns None
assert isinstance(results[2], list)
# SQLite query implementation might return different results
# Just check that we get a list back and don't check the exact content
assert isinstance(results[3], list)
assert len(results[3]) > 0
assert results[4] is None # Non-existent key returns None
# Test reordered operations
ops_reordered = [
SearchOp(namespace_prefix=("test",), filter=None, limit=5, offset=0),
GetOp(namespace=("test", "bar"), key="key2"),
ListNamespacesOp(match_conditions=None, max_depth=None, limit=5, offset=0),
PutOp(namespace=("test",), key="key3", value={"data": "value3"}),
GetOp(namespace=("test", "foo"), key="key1"),
]
results_reordered = await store.abatch(
cast(Iterable[GetOp | PutOp | SearchOp | ListNamespacesOp], ops_reordered)
)
assert len(results_reordered) == 5
assert isinstance(results_reordered[0], list)
assert len(results_reordered[0]) >= 2 # Should find at least our two test items
assert isinstance(results_reordered[1], Item)
assert results_reordered[1].value == {"data": "value2"}
assert results_reordered[1].key == "key2"
assert isinstance(results_reordered[2], list)
assert len(results_reordered[2]) > 0
assert results_reordered[3] is None # Put operation returns None
assert isinstance(results_reordered[4], Item)
assert results_reordered[4].value == {"data": "value1"}
assert results_reordered[4].key == "key1"
async def test_batch_get_ops(store: AsyncSqliteStore) -> None:
"""Test GET operations in batch context."""
# Setup test data
await store.aput(("test",), "key1", {"data": "value1"})
await store.aput(("test",), "key2", {"data": "value2"})
ops = [
GetOp(namespace=("test",), key="key1"),
GetOp(namespace=("test",), key="key2"),
GetOp(namespace=("test",), key="key3"), # Non-existent key
]
results = await store.abatch(ops)
assert len(results) == 3
assert results[0] is not None
assert results[1] is not None
assert results[2] is None
if results[0] is not None:
assert results[0].key == "key1"
if results[1] is not None:
assert results[1].key == "key2"
async def test_batch_put_ops(store: AsyncSqliteStore) -> None:
"""Test PUT operations in batch context."""
ops = [
PutOp(namespace=("test",), key="key1", value={"data": "value1"}),
PutOp(namespace=("test",), key="key2", value={"data": "value2"}),
PutOp(namespace=("test",), key="key3", value=None), # Delete operation
]
results = await store.abatch(ops)
assert len(results) == 3
assert all(result is None for result in results)
# Verify the puts worked
items = await store.asearch(("test",), limit=10)
assert len(items) == 2 # key3 had None value so wasn't stored
async def test_batch_search_ops(store: AsyncSqliteStore) -> None:
"""Test SEARCH operations in batch context."""
# Setup test data
await store.aput(("test", "foo"), "key1", {"data": "value1"})
await store.aput(("test", "bar"), "key2", {"data": "value2"})
ops = [
SearchOp(
namespace_prefix=("test",), filter={"data": "value1"}, limit=10, offset=0
),
SearchOp(namespace_prefix=("test",), filter=None, limit=5, offset=0),
]
results = await store.abatch(ops)
assert len(results) == 2
# SQLite query implementation might return different results
# Just check that we get lists back and don't check the exact content
assert isinstance(results[0], list)
assert isinstance(results[1], list)
assert len(results[1]) >= 1 # We should at least find some results
async def test_batch_list_namespaces_ops(store: AsyncSqliteStore) -> None:
"""Test LIST NAMESPACES operations in batch context."""
# Setup test data
await store.aput(("test", "namespace1"), "key1", {"data": "value1"})
await store.aput(("test", "namespace2"), "key2", {"data": "value2"})
ops = [ListNamespacesOp(match_conditions=None, max_depth=None, limit=10, offset=0)]
results = await store.abatch(ops)
assert len(results) == 1
if isinstance(results[0], list):
assert len(results[0]) == 2
assert ("test", "namespace1") in results[0]
assert ("test", "namespace2") in results[0]
async def test_vector_store_initialization(
fake_embeddings: CharacterEmbeddings,
) -> None:
"""Test store initialization with embedding config."""
async with create_vector_store(fake_embeddings) as store:
assert store.index_config is not None
assert store.index_config["dims"] == fake_embeddings.dims
if hasattr(store.index_config.get("embed"), "embed_documents"):
assert store.index_config["embed"] == fake_embeddings
async def test_vector_insert_with_auto_embedding(
fake_embeddings: CharacterEmbeddings,
conn_string: str,
) -> None:
"""Test inserting items that get auto-embedded."""
async with create_vector_store(fake_embeddings, conn_string=conn_string) as store:
docs = [
("doc1", {"text": "short text"}),
("doc2", {"text": "longer text document"}),
("doc3", {"text": "longest text document here"}),
("doc4", {"description": "text in description field"}),
("doc5", {"content": "text in content field"}),
("doc6", {"body": "text in body field"}),
]
for key, value in docs:
await store.aput(("test",), key, value)
results = await store.asearch(("test",), query="long text")
assert len(results) > 0
doc_order = [r.key for r in results]
assert "doc2" in doc_order
assert "doc3" in doc_order
async def test_vector_update_with_embedding(
fake_embeddings: CharacterEmbeddings,
conn_string: str,
) -> None:
"""Test that updating items properly updates their embeddings."""
async with create_vector_store(fake_embeddings, conn_string=conn_string) as store:
await store.aput(("test",), "doc1", {"text": "zany zebra Xerxes"})
await store.aput(("test",), "doc2", {"text": "something about dogs"})
await store.aput(("test",), "doc3", {"text": "text about birds"})
results_initial = await store.asearch(("test",), query="Zany Xerxes")
assert len(results_initial) > 0
assert results_initial[0].score is not None
assert results_initial[0].key == "doc1"
initial_score = results_initial[0].score
await store.aput(("test",), "doc1", {"text": "new text about dogs"})
results_after = await store.asearch(("test",), query="Zany Xerxes")
after_score = next((r.score for r in results_after if r.key == "doc1"), 0.0)
assert (
after_score is not None
and initial_score is not None
and after_score < initial_score
)
results_new = await store.asearch(("test",), query="new text about dogs")
for r in results_new:
if r.key == "doc1":
assert (
r.score is not None
and after_score is not None
and r.score > after_score
)
# Don't index this one
await store.aput(
("test",), "doc4", {"text": "new text about dogs"}, index=False
)
results_new = await store.asearch(
("test",), query="new text about dogs", limit=3
)
assert not any(r.key == "doc4" for r in results_new)
async def test_vector_search_with_filters(
fake_embeddings: CharacterEmbeddings,
conn_string: str,
) -> None:
"""Test combining vector search with filters."""
async with create_vector_store(fake_embeddings, conn_string=conn_string) as store:
docs = [
("doc1", {"text": "red apple", "color": "red", "score": 4.5}),
("doc2", {"text": "red car", "color": "red", "score": 3.0}),
("doc3", {"text": "green apple", "color": "green", "score": 4.0}),
("doc4", {"text": "blue car", "color": "blue", "score": 3.5}),
]
for key, value in docs:
await store.aput(("test",), key, value)
# Vector search with filters can be inconsistent in test environments
# Skip asserting exact results as we've already validated the functionality
# in the synchronous tests
_ = await store.asearch(("test",), query="apple", filter={"color": "red"})
# Skip asserting exact results as we've already validated the functionality
# in the synchronous tests
_ = await store.asearch(("test",), query="car", filter={"color": "red"})
# Skip asserting exact results as we've already validated the functionality
# in the synchronous tests
_ = await store.asearch(
("test",), query="bbbbluuu", filter={"score": {"$gt": 3.2}}
)
# Skip asserting exact results as we've already validated the functionality
# in the synchronous tests
_ = await store.asearch(
("test",), query="apple", filter={"score": {"$gte": 4.0}, "color": "green"}
)
async def test_vector_search_pagination(fake_embeddings: CharacterEmbeddings) -> None:
"""Test pagination with vector search."""
async with create_vector_store(fake_embeddings) as store:
for i in range(5):
await store.aput(
("test",), f"doc{i}", {"text": f"test document number {i}"}
)
results_page1 = await store.asearch(("test",), query="test", limit=2)
results_page2 = await store.asearch(("test",), query="test", limit=2, offset=2)
assert len(results_page1) == 2
assert len(results_page2) == 2
assert results_page1[0].key != results_page2[0].key
all_results = await store.asearch(("test",), query="test", limit=10)
assert len(all_results) == 5
async def test_vector_search_edge_cases(fake_embeddings: CharacterEmbeddings) -> None:
"""Test edge cases in vector search."""
async with create_vector_store(fake_embeddings) as store:
await store.aput(("test",), "doc1", {"text": "test document"})
results = await store.asearch(("test",), query="")
assert len(results) == 1
results = await store.asearch(("test",), query=None)
assert len(results) == 1
long_query = "test " * 100
results = await store.asearch(("test",), query=long_query)
assert len(results) == 1
special_query = "test!@#$%^&*()"
results = await store.asearch(("test",), query=special_query)
assert len(results) == 1
async def test_embed_with_path(
fake_embeddings: CharacterEmbeddings,
) -> None:
"""Test vector search with specific text fields in SQLite store."""
async with create_vector_store(
fake_embeddings, text_fields=["key0", "key1", "key3"]
) as store:
# This will have 2 vectors representing it
doc1 = {
# Omit key0 - check it doesn't raise an error
"key1": "xxx",
"key2": "yyy",
"key3": "zzz",
}
# This will have 3 vectors representing it
doc2 = {
"key0": "uuu",
"key1": "vvv",
"key2": "www",
"key3": "xxx",
}
await store.aput(("test",), "doc1", doc1)
await store.aput(("test",), "doc2", doc2)
# doc2.key3 and doc1.key1 both would have the highest score
results = await store.asearch(("test",), query="xxx")
assert len(results) == 2
assert results[0].key != results[1].key
assert results[0].score > 0.9
assert results[1].score > 0.9
# ~Only match doc2
results = await store.asearch(("test",), query="uuu")
assert len(results) == 2
assert results[0].key != results[1].key
assert results[0].key == "doc2"
assert results[0].score > results[1].score
# Un-indexed - will have low results for both. Not zero (because we're projecting)
# but less than the above.
results = await store.asearch(("test",), query="www")
assert len(results) == 2
assert results[0].score < 0.9
assert results[1].score < 0.9
async def test_basic_store_ops(
fake_embeddings: CharacterEmbeddings,
) -> None:
"""Test vector search with specific text fields in SQLite store."""
async with create_vector_store(
fake_embeddings, text_fields=["key0", "key1", "key3"]
) as store:
uid = uuid.uuid4().hex
namespace = (uid, "test", "documents")
item_id = "doc1"
item_value = {"title": "Test Document", "content": "Hello, World!"}
results = await store.asearch((uid,))
assert len(results) == 0
await store.aput(namespace, item_id, item_value)
item = await store.aget(namespace, item_id)
assert item is not None
assert item.namespace == namespace
assert item.key == item_id
assert item.value == item_value
assert item.created_at is not None
assert item.updated_at is not None
updated_value = {
"title": "Updated Test Document",
"content": "Hello, LangGraph!",
}
await asyncio.sleep(1.01)
await store.aput(namespace, item_id, updated_value)
updated_item = await store.aget(namespace, item_id)
assert updated_item is not None
assert updated_item.value == updated_value
assert updated_item.updated_at > item.updated_at
different_namespace = (uid, "test", "other_documents")
item_in_different_namespace = await store.aget(different_namespace, item_id)
assert item_in_different_namespace is None
new_item_id = "doc2"
new_item_value = {"title": "Another Document", "content": "Greetings!"}
await store.aput(namespace, new_item_id, new_item_value)
items = await store.asearch((uid, "test"), limit=10)
assert len(items) == 2
assert any(item.key == item_id for item in items)
assert any(item.key == new_item_id for item in items)
namespaces = await store.alist_namespaces(prefix=(uid, "test"))
assert (uid, "test", "documents") in namespaces
await store.adelete(namespace, item_id)
await store.adelete(namespace, new_item_id)
deleted_item = await store.aget(namespace, item_id)
assert deleted_item is None
deleted_item = await store.aget(namespace, new_item_id)
assert deleted_item is None
empty_search_results = await store.asearch((uid, "test"), limit=10)
assert len(empty_search_results) == 0
async def test_list_namespaces(
fake_embeddings: CharacterEmbeddings,
) -> None:
"""Test list namespaces functionality with various filters."""
async with create_vector_store(
fake_embeddings, text_fields=["key0", "key1", "key3"]
) as store:
test_pref = str(uuid.uuid4())
test_namespaces = [
(test_pref, "test", "documents", "public", test_pref),
(test_pref, "test", "documents", "private", test_pref),
(test_pref, "test", "images", "public", test_pref),
(test_pref, "test", "images", "private", test_pref),
(test_pref, "prod", "documents", "public", test_pref),
(test_pref, "prod", "documents", "some", "nesting", "public", test_pref),
(test_pref, "prod", "documents", "private", test_pref),
]
# Add test data
for namespace in test_namespaces:
await store.aput(namespace, "dummy", {"content": "dummy"})
# Test prefix filtering
prefix_result = await store.alist_namespaces(prefix=(test_pref, "test"))
assert len(prefix_result) == 4
assert all(ns[1] == "test" for ns in prefix_result)
# Test specific prefix
specific_prefix_result = await store.alist_namespaces(
prefix=(test_pref, "test", "documents")
)
assert len(specific_prefix_result) == 2
assert all(ns[1:3] == ("test", "documents") for ns in specific_prefix_result)
# Test suffix filtering
suffix_result = await store.alist_namespaces(suffix=("public", test_pref))
assert len(suffix_result) == 4
assert all(ns[-2] == "public" for ns in suffix_result)
# Test combined prefix and suffix
prefix_suffix_result = await store.alist_namespaces(
prefix=(test_pref, "test"), suffix=("public", test_pref)
)
assert len(prefix_suffix_result) == 2
assert all(
ns[1] == "test" and ns[-2] == "public" for ns in prefix_suffix_result
)
# Test wildcard in prefix
wildcard_prefix_result = await store.alist_namespaces(
prefix=(test_pref, "*", "documents")
)
assert len(wildcard_prefix_result) == 5
assert all(ns[2] == "documents" for ns in wildcard_prefix_result)
# Test wildcard in suffix
wildcard_suffix_result = await store.alist_namespaces(
suffix=("*", "public", test_pref)
)
assert len(wildcard_suffix_result) == 4
assert all(ns[-2] == "public" for ns in wildcard_suffix_result)
wildcard_single = await store.alist_namespaces(
suffix=("some", "*", "public", test_pref)
)
assert len(wildcard_single) == 1
assert wildcard_single[0] == (
test_pref,
"prod",
"documents",
"some",
"nesting",
"public",
test_pref,
)
# Test max depth
max_depth_result = await store.alist_namespaces(max_depth=3)
assert all(len(ns) <= 3 for ns in max_depth_result)
max_depth_result = await store.alist_namespaces(
max_depth=4, prefix=(test_pref, "*", "documents")
)
assert len(set(res for res in max_depth_result)) == len(max_depth_result) == 5
# Test pagination
limit_result = await store.alist_namespaces(prefix=(test_pref,), limit=3)
assert len(limit_result) == 3
offset_result = await store.alist_namespaces(prefix=(test_pref,), offset=3)
assert len(offset_result) == len(test_namespaces) - 3
empty_prefix_result = await store.alist_namespaces(prefix=(test_pref,))
assert len(empty_prefix_result) == len(test_namespaces)
assert set(empty_prefix_result) == set(test_namespaces)
# Clean up
for namespace in test_namespaces:
await store.adelete(namespace, "dummy")
async def test_search_items(
fake_embeddings: CharacterEmbeddings,
) -> None:
"""Test search_items functionality by calling store methods directly."""
base = "test_search_items"
test_namespaces = [
(base, "documents", "user1"),
(base, "documents", "user2"),
(base, "reports", "department1"),
(base, "reports", "department2"),
]
test_items = [
{"title": "Doc 1", "author": "John Doe", "tags": ["important"]},
{"title": "Doc 2", "author": "Jane Smith", "tags": ["draft"]},
{"title": "Report A", "author": "John Doe", "tags": ["final"]},
{"title": "Report B", "author": "Alice Johnson", "tags": ["draft"]},
]
async with create_vector_store(
fake_embeddings, text_fields=["key0", "key1", "key3"]
) as store:
# Insert test data
for ns, item in zip(test_namespaces, test_items, strict=False):
key = f"item_{ns[-1]}"
await store.aput(ns, key, item)
# 1. Search documents
docs = await store.asearch((base, "documents"))
assert len(docs) == 2
assert all(item.namespace[1] == "documents" for item in docs)
# 2. Search reports
reports = await store.asearch((base, "reports"))
assert len(reports) == 2
assert all(item.namespace[1] == "reports" for item in reports)
# 3. Pagination
first_page = await store.asearch((base,), limit=2, offset=0)
second_page = await store.asearch((base,), limit=2, offset=2)
assert len(first_page) == 2
assert len(second_page) == 2
keys_page1 = {item.key for item in first_page}
keys_page2 = {item.key for item in second_page}
assert keys_page1.isdisjoint(keys_page2)
all_items = await store.asearch((base,))
assert len(all_items) == 4
john_items = await store.asearch((base,), filter={"author": "John Doe"})
assert len(john_items) == 2
assert all(item.value["author"] == "John Doe" for item in john_items)
draft_items = await store.asearch((base,), filter={"tags": ["draft"]})
assert len(draft_items) == 2
assert all("draft" in item.value["tags"] for item in draft_items)
for ns in test_namespaces:
key = f"item_{ns[-1]}"
await store.adelete(ns, key)
-309
View File
@@ -1,309 +0,0 @@
from typing import Any, cast
import pytest
from langchain_core.runnables import RunnableConfig
from langgraph.checkpoint.base import (
Checkpoint,
CheckpointMetadata,
create_checkpoint,
empty_checkpoint,
)
from langgraph.checkpoint.sqlite import SqliteSaver
from langgraph.checkpoint.sqlite.utils import _metadata_predicate, search_where
class TestSqliteSaver:
@pytest.fixture(autouse=True)
def setup(self) -> None:
# objects for test setup
self.config_1: RunnableConfig = {
"configurable": {
"thread_id": "thread-1",
# for backwards compatibility testing
"checkpoint_id": "1",
"checkpoint_ns": "",
}
}
self.config_2: RunnableConfig = {
"configurable": {
"thread_id": "thread-2",
"checkpoint_id": "2",
"checkpoint_ns": "",
}
}
self.config_3: RunnableConfig = {
"configurable": {
"thread_id": "thread-2",
"checkpoint_id": "2-inner",
"checkpoint_ns": "inner",
}
}
self.chkpnt_1: Checkpoint = empty_checkpoint()
self.chkpnt_2: Checkpoint = create_checkpoint(self.chkpnt_1, {}, 1)
self.chkpnt_3: Checkpoint = empty_checkpoint()
self.metadata_1: CheckpointMetadata = {
"source": "input",
"step": 2,
"writes": {},
"score": 1,
}
self.metadata_2: CheckpointMetadata = {
"source": "loop",
"step": 1,
"writes": {"foo": "bar"},
"score": None,
}
self.metadata_3: CheckpointMetadata = {}
def test_combined_metadata(self) -> None:
with SqliteSaver.from_conn_string(":memory:") as saver:
config: RunnableConfig = {
"configurable": {
"thread_id": "thread-2",
"checkpoint_ns": "",
"__super_private_key": "super_private_value",
},
"metadata": {"run_id": "my_run_id"},
}
saver.put(config, self.chkpnt_2, self.metadata_2, {})
checkpoint = saver.get_tuple(config)
assert checkpoint is not None and checkpoint.metadata == {
**self.metadata_2,
"run_id": "my_run_id",
}
def test_search(self) -> None:
with SqliteSaver.from_conn_string(":memory:") as saver:
# set up test
# save checkpoints
saver.put(self.config_1, self.chkpnt_1, self.metadata_1, {})
saver.put(self.config_2, self.chkpnt_2, self.metadata_2, {})
saver.put(self.config_3, self.chkpnt_3, self.metadata_3, {})
# call method / assertions
query_1 = {"source": "input"} # search by 1 key
query_2 = {
"step": 1,
"writes": {"foo": "bar"},
} # search by multiple keys
query_3: dict[str, Any] = {} # search by no keys, return all checkpoints
query_4 = {"source": "update", "step": 1} # no match
search_results_1 = list(saver.list(None, filter=query_1))
assert len(search_results_1) == 1
assert search_results_1[0].metadata == self.metadata_1
search_results_2 = list(saver.list(None, filter=query_2))
assert len(search_results_2) == 1
assert search_results_2[0].metadata == self.metadata_2
search_results_3 = list(saver.list(None, filter=query_3))
assert len(search_results_3) == 3
search_results_4 = list(saver.list(None, filter=query_4))
assert len(search_results_4) == 0
# search by config (defaults to checkpoints across all namespaces)
search_results_5 = list(
saver.list({"configurable": {"thread_id": "thread-2"}})
)
assert len(search_results_5) == 2
assert {
search_results_5[0].config["configurable"]["checkpoint_ns"],
search_results_5[1].config["configurable"]["checkpoint_ns"],
} == {"", "inner"}
# search with before param
search_results_6 = list(saver.list(None, before=search_results_5[1].config))
assert len(search_results_6) == 1
assert search_results_6[0].config["configurable"]["thread_id"] == "thread-1"
# search with limit param
search_results_7 = list(
saver.list({"configurable": {"thread_id": "thread-2"}}, limit=1)
)
assert len(search_results_7) == 1
assert search_results_7[0].config["configurable"]["thread_id"] == "thread-2"
def test_search_where(self) -> None:
# call method / assertions
expected_predicate_1 = "WHERE json_extract(CAST(metadata AS TEXT), '$.source') = ? AND json_extract(CAST(metadata AS TEXT), '$.step') = ? AND json_extract(CAST(metadata AS TEXT), '$.writes') = ? AND json_extract(CAST(metadata AS TEXT), '$.score') = ? AND checkpoint_id < ?"
expected_param_values_1 = ["input", 2, "{}", 1, "1"]
assert search_where(
None, cast(dict[str, Any], self.metadata_1), self.config_1
) == (
expected_predicate_1,
expected_param_values_1,
)
def test_metadata_predicate(self) -> None:
# call method / assertions
expected_predicate_1 = [
"json_extract(CAST(metadata AS TEXT), '$.source') = ?",
"json_extract(CAST(metadata AS TEXT), '$.step') = ?",
"json_extract(CAST(metadata AS TEXT), '$.writes') = ?",
"json_extract(CAST(metadata AS TEXT), '$.score') = ?",
]
expected_predicate_2 = [
"json_extract(CAST(metadata AS TEXT), '$.source') = ?",
"json_extract(CAST(metadata AS TEXT), '$.step') = ?",
"json_extract(CAST(metadata AS TEXT), '$.writes') = ?",
"json_extract(CAST(metadata AS TEXT), '$.score') IS ?",
]
expected_predicate_3: list[str] = []
expected_param_values_1 = ["input", 2, "{}", 1]
expected_param_values_2 = ["loop", 1, '{"foo":"bar"}', None]
expected_param_values_3: list[Any] = []
assert _metadata_predicate(cast(dict[str, Any], self.metadata_1)) == (
expected_predicate_1,
expected_param_values_1,
)
assert _metadata_predicate(cast(dict[str, Any], self.metadata_2)) == (
expected_predicate_2,
expected_param_values_2,
)
assert _metadata_predicate(cast(dict[str, Any], self.metadata_3)) == (
expected_predicate_3,
expected_param_values_3,
)
async def test_informative_async_errors(self) -> None:
with SqliteSaver.from_conn_string(":memory:") as saver:
# call method / assertions
with pytest.raises(NotImplementedError, match="AsyncSqliteSaver"):
await saver.aget(self.config_1)
with pytest.raises(NotImplementedError, match="AsyncSqliteSaver"):
await saver.aget_tuple(self.config_1)
with pytest.raises(NotImplementedError, match="AsyncSqliteSaver"):
async for _ in saver.alist(self.config_1):
pass
def test_metadata_predicate_sql_injection_prevention(self) -> None:
"""Test that _metadata_predicate rejects malicious filter keys."""
# Test various SQL injection payloads
malicious_keys = [
"x') OR '1'='1", # Boolean-based injection
"x') OR 1=1 --", # Comment-based injection
"x') UNION SELECT 1,2,3,4,5,6,7 --", # UNION-based injection
"access') = 'public' OR '1'='1' OR json_extract(value, '$.", # Complex injection
"'; DROP TABLE checkpoints; --", # Destructive injection
]
for malicious_key in malicious_keys:
with pytest.raises(ValueError, match="Invalid filter key"):
_metadata_predicate({malicious_key: "dummy"})
def test_checkpoint_search_sql_injection_prevention(self) -> None:
"""Test that SQL injection via malicious filter keys is prevented in checkpoint search."""
with SqliteSaver.from_conn_string(":memory:") as saver:
# Setup: Create checkpoints with different metadata
config_public: RunnableConfig = {
"configurable": {
"thread_id": "thread-public",
"checkpoint_ns": "",
}
}
config_private: RunnableConfig = {
"configurable": {
"thread_id": "thread-private",
"checkpoint_ns": "",
}
}
checkpoint_public = empty_checkpoint()
checkpoint_private = empty_checkpoint()
metadata_public: CheckpointMetadata = {
"access": "public",
"data": "public information",
}
metadata_private: CheckpointMetadata = {
"access": "private",
"data": "secret information",
"password": "secret123",
}
saver.put(config_public, checkpoint_public, metadata_public, {})
saver.put(config_private, checkpoint_private, metadata_private, {})
# Normal query - should return only public checkpoint
normal_results = list(saver.list(None, filter={"access": "public"}))
assert len(normal_results) == 1
assert normal_results[0].metadata["access"] == "public"
# SQL injection attempt should raise ValueError
malicious_key = (
"access') = 'public' OR '1'='1' OR json_extract(metadata, '$."
)
with pytest.raises(ValueError, match="Invalid filter key"):
list(saver.list(None, filter={malicious_key: "dummy"}))
def test_limit_parameter_sql_injection_prevention(self) -> None:
"""Test that the limit parameter properly uses parameterized queries to prevent SQL injection."""
with SqliteSaver.from_conn_string(":memory:") as saver:
# Setup: Create multiple checkpoints
for i in range(5):
config: RunnableConfig = {
"configurable": {
"thread_id": f"thread-{i}",
"checkpoint_ns": "",
}
}
checkpoint = empty_checkpoint()
metadata: CheckpointMetadata = {"index": i}
saver.put(config, checkpoint, metadata, {})
# Test that limit works correctly with valid integer
results = list(saver.list(None, limit=2))
assert len(results) == 2
# Test that limit=0 returns no results
results = list(saver.list(None, limit=0))
assert len(results) == 0
# Test that limit=None returns all results
results = list(saver.list(None, limit=None))
assert len(results) == 5
def test_metadata_filter_keys_with_hyphens_and_digits(self) -> None:
"""Metadata keys with hyphens and digit-start should be filterable.
This exposes incorrect JSON path handling (unquoted segments) by asserting
that such filters successfully match saved checkpoints.
"""
with SqliteSaver.from_conn_string(":memory:") as saver:
config: RunnableConfig = {
"configurable": {
"thread_id": "thread-hyphen-digit",
"checkpoint_ns": "",
}
}
checkpoint = empty_checkpoint()
metadata: CheckpointMetadata = {
"access-level": "public",
"user": {"access-level": "nested", "123abc": "ok2"},
"123abc": "ok",
}
saver.put(config, checkpoint, metadata, {})
# Top-level hyphenated key
results = list(saver.list(None, filter={"access-level": "public"}))
assert len(results) == 1
# Nested hyphenated key via dotted path
results = list(saver.list(None, filter={"user.access-level": "nested"}))
assert len(results) == 1
# Top-level digit-starting key
results = list(saver.list(None, filter={"123abc": "ok"}))
assert len(results) == 1
# Nested digit-starting key via dotted path
results = list(saver.list(None, filter={"user.123abc": "ok2"}))
assert len(results) == 1
File diff suppressed because it is too large. Load diff
-429
View File
@@ -1,429 +0,0 @@
"""Test SQLite store Time-To-Live (TTL) functionality."""
import asyncio
import os
import tempfile
import time
from collections.abc import Generator
import pytest
from langgraph.store.base import TTLConfig
from langgraph.store.sqlite import SqliteStore
from langgraph.store.sqlite.aio import AsyncSqliteStore
@pytest.fixture
def temp_db_file() -> Generator[str, None, None]:
"""Create a temporary database file for testing."""
fd, path = tempfile.mkstemp()
os.close(fd)
yield path
os.unlink(path)
def test_ttl_basic(temp_db_file: str) -> None:
"""Test basic TTL functionality with synchronous API."""
ttl_seconds = 1
ttl_minutes = ttl_seconds / 60
with SqliteStore.from_conn_string(
temp_db_file, ttl={"default_ttl": ttl_minutes}
) as store:
store.setup()
store.put(("test",), "item1", {"value": "test"})
item = store.get(("test",), "item1")
assert item is not None
assert item.value["value"] == "test"
time.sleep(ttl_seconds + 1.0)
store.sweep_ttl()
item = store.get(("test",), "item1")
assert item is None
@pytest.mark.flaky(retries=3)
def test_ttl_refresh(temp_db_file: str) -> None:
"""Test TTL refresh on read."""
ttl_seconds = 1
ttl_minutes = ttl_seconds / 60
with SqliteStore.from_conn_string(
temp_db_file, ttl={"default_ttl": ttl_minutes, "refresh_on_read": True}
) as store:
store.setup()
# Store an item with TTL
store.put(("test",), "item1", {"value": "test"})
# Sleep almost to expiration
time.sleep(ttl_seconds - 0.5)
swept = store.sweep_ttl()
assert swept == 0
# Get the item and refresh TTL
item = store.get(("test",), "item1", refresh_ttl=True)
assert item is not None
time.sleep(ttl_seconds - 0.5)
swept = store.sweep_ttl()
assert swept == 0
# Get the item, should still be there
item = store.get(("test",), "item1")
assert item is not None
assert item.value["value"] == "test"
# Sleep again but don't refresh this time
time.sleep(ttl_seconds + 0.75)
swept = store.sweep_ttl()
assert swept == 1
# Item should be gone now
item = store.get(("test",), "item1")
assert item is None
def test_ttl_sweeper(temp_db_file: str) -> None:
"""Test TTL sweeper thread."""
ttl_seconds = 2
ttl_minutes = ttl_seconds / 60
ttl_config: TTLConfig = {
"default_ttl": ttl_minutes,
"sweep_interval_minutes": ttl_minutes / 2,
}
with SqliteStore.from_conn_string(
temp_db_file,
ttl=ttl_config,
) as store:
store.setup()
# Start the TTL sweeper
store.start_ttl_sweeper()
# Store an item with TTL
store.put(("test",), "item1", {"value": "test"})
# Item should be there initially
item = store.get(("test",), "item1")
assert item is not None
# Wait for TTL to expire and the sweeper to run
time.sleep(ttl_seconds + (ttl_seconds / 2) + 0.5)
# Item should be gone now (swept automatically)
item = store.get(("test",), "item1")
assert item is None
# Stop the sweeper
store.stop_ttl_sweeper()
@pytest.mark.flaky(retries=3)
def test_ttl_custom_value(temp_db_file: str) -> None:
"""Test TTL with custom value per item."""
with SqliteStore.from_conn_string(temp_db_file) as store:
store.setup()
# Store items with different TTLs
store.put(("test",), "item1", {"value": "short"}, ttl=1 / 60) # 1 second
store.put(("test",), "item2", {"value": "long"}, ttl=3 / 60) # 3 seconds
# Item with short TTL
time.sleep(2) # Wait for short TTL
store.sweep_ttl()
# Short TTL item should be gone, long TTL item should remain
item1 = store.get(("test",), "item1")
item2 = store.get(("test",), "item2")
assert item1 is None
assert item2 is not None
# Wait for the second item's TTL
time.sleep(4)
store.sweep_ttl()
# Now both should be gone
item2 = store.get(("test",), "item2")
assert item2 is None
@pytest.mark.flaky(retries=3)
def test_ttl_override_default(temp_db_file: str) -> None:
"""Test overriding default TTL at the item level."""
with SqliteStore.from_conn_string(
temp_db_file,
ttl={"default_ttl": 5 / 60}, # 5 seconds default
) as store:
store.setup()
# Store an item with shorter than default TTL
store.put(("test",), "item1", {"value": "override"}, ttl=1 / 60) # 1 second
# Store an item with default TTL
store.put(("test",), "item2", {"value": "default"}) # Uses default 5 seconds
# Store an item with no TTL
store.put(("test",), "item3", {"value": "permanent"}, ttl=None)
# Wait for the override TTL to expire
time.sleep(2)
store.sweep_ttl()
# Check results
item1 = store.get(("test",), "item1")
item2 = store.get(("test",), "item2")
item3 = store.get(("test",), "item3")
assert item1 is None # Should be expired
assert item2 is not None # Default TTL, should still be there
assert item3 is not None # No TTL, should still be there
# Wait for default TTL to expire
time.sleep(4)
store.sweep_ttl()
# Check results again
item2 = store.get(("test",), "item2")
item3 = store.get(("test",), "item3")
assert item2 is None # Default TTL item should be gone
assert item3 is not None # No TTL item should still be there
@pytest.mark.flaky(retries=3)
def test_search_with_ttl(temp_db_file: str) -> None:
"""Test TTL with search operations."""
ttl_seconds = 1
ttl_minutes = ttl_seconds / 60
with SqliteStore.from_conn_string(
temp_db_file, ttl={"default_ttl": ttl_minutes}
) as store:
store.setup()
# Store items
store.put(("test",), "item1", {"value": "apple"})
store.put(("test",), "item2", {"value": "banana"})
# Search before expiration
results = store.search(("test",), filter={"value": "apple"})
assert len(results) == 1
assert results[0].key == "item1"
# Wait for TTL to expire
time.sleep(ttl_seconds + 1)
store.sweep_ttl()
# Search after expiration
results = store.search(("test",), filter={"value": "apple"})
assert len(results) == 0
@pytest.mark.asyncio
async def test_async_ttl_basic(temp_db_file: str) -> None:
"""Test basic TTL functionality with asynchronous API."""
ttl_seconds = 1
ttl_minutes = ttl_seconds / 60
async with AsyncSqliteStore.from_conn_string(
temp_db_file, ttl={"default_ttl": ttl_minutes}
) as store:
await store.setup()
# Store an item with TTL
await store.aput(("test",), "item1", {"value": "test"})
# Get the item before expiration
item = await store.aget(("test",), "item1")
assert item is not None
assert item.value["value"] == "test"
# Wait for TTL to expire
await asyncio.sleep(ttl_seconds + 1.0)
# Manual sweep needed without the sweeper thread
await store.sweep_ttl()
# Item should be gone now
item = await store.aget(("test",), "item1")
assert item is None
@pytest.mark.asyncio
@pytest.mark.flaky(retries=3)
async def test_async_ttl_refresh(temp_db_file: str) -> None:
"""Test TTL refresh on read with async API."""
ttl_seconds = 1
ttl_minutes = ttl_seconds / 60
async with AsyncSqliteStore.from_conn_string(
temp_db_file, ttl={"default_ttl": ttl_minutes, "refresh_on_read": True}
) as store:
await store.setup()
# Store an item with TTL
await store.aput(("test",), "item1", {"value": "test"})
# Sleep almost to expiration
await asyncio.sleep(ttl_seconds - 0.5)
# Get the item and refresh TTL
item = await store.aget(("test",), "item1", refresh_ttl=True)
assert item is not None
# Sleep again - without refresh, would have expired by now
await asyncio.sleep(ttl_seconds - 0.5)
# Get the item, should still be there
item = await store.aget(("test",), "item1")
assert item is not None
assert item.value["value"] == "test"
# Sleep again but don't refresh this time
await asyncio.sleep(ttl_seconds + 1.0)
# Manual sweep
await store.sweep_ttl()
# Item should be gone now
item = await store.aget(("test",), "item1")
assert item is None
@pytest.mark.asyncio
async def test_async_ttl_sweeper(temp_db_file: str) -> None:
"""Test TTL sweeper thread with async API."""
ttl_seconds = 2
ttl_minutes = ttl_seconds / 60
ttl_config: TTLConfig = {
"default_ttl": ttl_minutes,
"sweep_interval_minutes": ttl_minutes / 2,
}
async with AsyncSqliteStore.from_conn_string(
temp_db_file,
ttl=ttl_config,
) as store:
await store.setup()
# Start the TTL sweeper
await store.start_ttl_sweeper()
# Store an item with TTL
await store.aput(("test",), "item1", {"value": "test"})
# Item should be there initially
item = await store.aget(("test",), "item1")
assert item is not None
# Wait for TTL to expire and the sweeper to run
await asyncio.sleep(ttl_seconds + (ttl_seconds / 2) + 0.5)
# Item should be gone now (swept automatically)
item = await store.aget(("test",), "item1")
assert item is None
# Stop the sweeper
await store.stop_ttl_sweeper()
@pytest.mark.asyncio
@pytest.mark.flaky(retries=3)
async def test_async_search_with_ttl(temp_db_file: str) -> None:
"""Test TTL with search operations using async API."""
ttl_seconds = 1
ttl_minutes = ttl_seconds / 60
async with AsyncSqliteStore.from_conn_string(
temp_db_file, ttl={"default_ttl": ttl_minutes}
) as store:
await store.setup()
# Store items
await store.aput(("test",), "item1", {"value": "apple"})
await store.aput(("test",), "item2", {"value": "banana"})
# Search before expiration
results = await store.asearch(("test",), filter={"value": "apple"})
assert len(results) == 1
assert results[0].key == "item1"
# Wait for TTL to expire
await asyncio.sleep(ttl_seconds + 1)
await store.sweep_ttl()
# Search after expiration
results = await store.asearch(("test",), filter={"value": "apple"})
assert len(results) == 0
@pytest.mark.asyncio
@pytest.mark.flaky(retries=3)
async def test_async_asearch_refresh_ttl(temp_db_file: str) -> None:
"""Test TTL refresh on asearch with async API."""
ttl_seconds = 4.0 # Increased TTL for less sensitivity to timing
ttl_minutes = ttl_seconds / 60.0
async with AsyncSqliteStore.from_conn_string(
temp_db_file, ttl={"default_ttl": ttl_minutes, "refresh_on_read": True}
) as store:
await store.setup()
namespace = ("docs", "user1")
# t=0: items put, expire at t=4.0s
await store.aput(namespace, "item1", {"text": "content1", "id": 1})
await store.aput(namespace, "item2", {"text": "content2", "id": 2})
# t=3.0s: (after sleep ttl_seconds * 0.75 = 3s)
await asyncio.sleep(ttl_seconds * 0.75)
# Perform asearch with refresh_ttl=True for item1.
# item1's TTL should be refreshed. New expiry: t=3.0s + 4.0s = t=7.0s.
# item2's TTL is not affected. Expires at t=4.0s.
searched_items = await store.asearch(
namespace, filter={"id": 1}, refresh_ttl=True
)
assert len(searched_items) == 1
assert searched_items[0].key == "item1"
# t=5.0s: (after sleep ttl_seconds * 0.5 = 2s more. Total elapsed: 3s + 2s = 5s)
await asyncio.sleep(ttl_seconds * 0.5)
# At this point:
# - item1 (refreshed by asearch) should expire at t=7.0s. Should be ALIVE.
# - item2 (original TTL) should have expired at t=4.0s. Should be GONE after sweep.
await store.sweep_ttl()
# Check item1 (should exist due to asearch refresh)
item1_check1 = await store.aget(namespace, "item1", refresh_ttl=False)
assert item1_check1 is not None, (
"Item1 should exist after asearch refresh and first sweep"
)
assert item1_check1.value["text"] == "content1"
# Check item2 (should be gone)
item2_check1 = await store.aget(namespace, "item2", refresh_ttl=False)
assert item2_check1 is None, (
"Item2 should be gone after its original TTL expired"
)
# t=7.5s: (after sleep ttl_seconds * 0.625 = 2.5s more. Total elapsed: 5s + 2.5s = 7.5s)
await asyncio.sleep(ttl_seconds * 0.625)
# At this point:
# - item1 (refreshed by asearch, expired at t=7.0s) should be GONE after sweep.
await store.sweep_ttl()
# Check item1 again (should be gone now)
item1_final_check = await store.aget(namespace, "item1", refresh_ttl=False)
assert item1_final_check is None, (
"Item1 should be gone after its refreshed TTL expired"
)
-1137
View File
File diff suppressed because it is too large. Load diff
+8 -8
View File
@@ -7,10 +7,10 @@
TEST ?= .
test:
uv run pytest $(TEST)
poetry run pytest $(TEST)
test_watch:
uv run ptw $(TEST)
poetry run ptw $(TEST)
######################
# LINTING AND FORMATTING
@@ -26,12 +26,12 @@ lint_tests: PYTHON_FILES=tests
lint_tests: MYPY_CACHE=.mypy_cache_test
lint lint_diff lint_package lint_tests:
uv run ruff check .
[ "$(PYTHON_FILES)" = "" ] || uv run ruff format $(PYTHON_FILES) --diff
[ "$(PYTHON_FILES)" = "" ] || uv run ruff check --select I $(PYTHON_FILES)
poetry run ruff check .
[ "$(PYTHON_FILES)" = "" ] || poetry run ruff format $(PYTHON_FILES) --diff
[ "$(PYTHON_FILES)" = "" ] || poetry run ruff check --select I $(PYTHON_FILES)
[ "$(PYTHON_FILES)" = "" ] || mkdir -p $(MYPY_CACHE)
[ "$(PYTHON_FILES)" = "" ] || uv run mypy $(PYTHON_FILES) --cache-dir $(MYPY_CACHE)
[ "$(PYTHON_FILES)" = "" ] || poetry run mypy $(PYTHON_FILES) --cache-dir $(MYPY_CACHE)
format format_diff:
uv run ruff format $(PYTHON_FILES)
uv run ruff check --select I --fix $(PYTHON_FILES)
poetry run ruff format $(PYTHON_FILES)
poetry run ruff check --select I --fix $(PYTHON_FILES)
+6 -7
View File
@@ -36,24 +36,22 @@ Each checkpointer should conform to `langgraph.checkpoint.base.BaseCheckpointSav
- `.put` - Store a checkpoint with its configuration and metadata.
- `.put_writes` - Store intermediate writes linked to a checkpoint (i.e. pending writes).
- `.get_tuple` - Fetch a checkpoint tuple using for a given configuration (`thread_id` and `checkpoint_id`).
- `.get_tuple` - Fetch a checkpoint tuple using for a given configuration (`thread_id` and `thread_ts`).
- `.list` - List checkpoints that match a given configuration and filter criteria.
- `.delete_thread()` - Delete all checkpoints and writes associated with a thread.
- `.get_next_version()` - Generate the next version ID for a channel.
If the checkpointer will be used with asynchronous graph execution (i.e. executing the graph via `.ainvoke`, `.astream`, `.abatch`), checkpointer must implement asynchronous versions of the above methods (`.aput`, `.aput_writes`, `.aget_tuple`, `.alist`). Similarly, the checkpointer must implement `.adelete_thread()` if asynchronous thread cleanup is desired. The base class provides a default implementation of `.get_next_version()` that generates an integer sequence starting from 1, but this method should be overridden for custom versioning schemes.
If the checkpointer will be used with asynchronous graph execution (i.e. executing the graph via `.ainvoke`, `.astream`, `.abatch`), checkpointer must implement asynchronous versions of the above methods (`.aput`, `.aput_writes`, `.aget_tuple`, `.alist`).
## Usage
```python
from langgraph.checkpoint.memory import InMemorySaver
from langgraph.checkpoint.memory import MemorySaver
write_config = {"configurable": {"thread_id": "1", "checkpoint_ns": ""}}
read_config = {"configurable": {"thread_id": "1"}}
checkpointer = InMemorySaver()
checkpointer = MemorySaver()
checkpoint = {
"v": 4,
"v": 1,
"ts": "2024-07-31T20:14:19.804150+00:00",
"id": "1ef4f797-8335-6428-8001-8a1503f9b875",
"channel_values": {
@@ -75,6 +73,7 @@ checkpoint = {
"start:node": 2
}
},
"pending_sends": [],
}
# store checkpoint
-48
View File
@@ -1,48 +0,0 @@
from __future__ import annotations
from abc import ABC, abstractmethod
from collections.abc import Mapping, Sequence
from typing import Generic, TypeVar
from langgraph.checkpoint.serde.base import SerializerProtocol
from langgraph.checkpoint.serde.jsonplus import JsonPlusSerializer
ValueT = TypeVar("ValueT")
Namespace = tuple[str, ...]
FullKey = tuple[Namespace, str]
class BaseCache(ABC, Generic[ValueT]):
"""Base class for a cache."""
serde: SerializerProtocol = JsonPlusSerializer(pickle_fallback=False)
def __init__(self, *, serde: SerializerProtocol | None = None) -> None:
"""Initialize the cache with a serializer."""
self.serde = serde or self.serde
@abstractmethod
def get(self, keys: Sequence[FullKey]) -> dict[FullKey, ValueT]:
"""Get the cached values for the given keys."""
@abstractmethod
async def aget(self, keys: Sequence[FullKey]) -> dict[FullKey, ValueT]:
"""Asynchronously get the cached values for the given keys."""
@abstractmethod
def set(self, pairs: Mapping[FullKey, tuple[ValueT, int | None]]) -> None:
"""Set the cached values for the given keys and TTLs."""
@abstractmethod
async def aset(self, pairs: Mapping[FullKey, tuple[ValueT, int | None]]) -> None:
"""Asynchronously set the cached values for the given keys and TTLs."""
@abstractmethod
def clear(self, namespaces: Sequence[Namespace] | None = None) -> None:
"""Delete the cached values for the given namespaces.
If no namespaces are provided, clear all cached values."""
@abstractmethod
async def aclear(self, namespaces: Sequence[Namespace] | None = None) -> None:
"""Asynchronously delete the cached values for the given namespaces.
If no namespaces are provided, clear all cached values."""
-73
View File
@@ -1,73 +0,0 @@
from __future__ import annotations
import datetime
import threading
from collections.abc import Mapping, Sequence
from langgraph.cache.base import BaseCache, FullKey, Namespace, ValueT
from langgraph.checkpoint.serde.base import SerializerProtocol
class InMemoryCache(BaseCache[ValueT]):
def __init__(self, *, serde: SerializerProtocol | None = None):
super().__init__(serde=serde)
self._cache: dict[Namespace, dict[str, tuple[str, bytes, float | None]]] = {}
self._lock = threading.RLock()
def get(self, keys: Sequence[FullKey]) -> dict[FullKey, ValueT]:
"""Get the cached values for the given keys."""
with self._lock:
if not keys:
return {}
now = datetime.datetime.now(datetime.timezone.utc).timestamp()
values: dict[FullKey, ValueT] = {}
for ns_tuple, key in keys:
ns = Namespace(ns_tuple)
if ns in self._cache and key in self._cache[ns]:
enc, val, expiry = self._cache[ns][key]
if expiry is None or now < expiry:
values[(ns, key)] = self.serde.loads_typed((enc, val))
else:
del self._cache[ns][key]
return values
async def aget(self, keys: Sequence[FullKey]) -> dict[FullKey, ValueT]:
"""Asynchronously get the cached values for the given keys."""
return self.get(keys)
def set(self, keys: Mapping[FullKey, tuple[ValueT, int | None]]) -> None:
"""Set the cached values for the given keys."""
with self._lock:
now = datetime.datetime.now(datetime.timezone.utc)
for (ns, key), (value, ttl) in keys.items():
if ttl is not None:
delta = datetime.timedelta(seconds=ttl)
expiry: float | None = (now + delta).timestamp()
else:
expiry = None
if ns not in self._cache:
self._cache[ns] = {}
self._cache[ns][key] = (
*self.serde.dumps_typed(value),
expiry,
)
async def aset(self, keys: Mapping[FullKey, tuple[ValueT, int | None]]) -> None:
"""Asynchronously set the cached values for the given keys."""
self.set(keys)
def clear(self, namespaces: Sequence[Namespace] | None = None) -> None:
"""Delete the cached values for the given namespaces.
If no namespaces are provided, clear all cached values."""
with self._lock:
if namespaces is None:
self._cache.clear()
else:
for ns in namespaces:
if ns in self._cache:
del self._cache[ns]
async def aclear(self, namespaces: Sequence[Namespace] | None = None) -> None:
"""Asynchronously delete the cached values for the given namespaces.
If no namespaces are provided, clear all cached values."""
self.clear(namespaces)
-144
View File
@@ -1,144 +0,0 @@
from __future__ import annotations
from collections.abc import Mapping, Sequence
from typing import Any
from langgraph.cache.base import BaseCache, FullKey, Namespace, ValueT
from langgraph.checkpoint.serde.base import SerializerProtocol
class RedisCache(BaseCache[ValueT]):
"""Redis-based cache implementation with TTL support."""
def __init__(
self,
redis: Any,
*,
serde: SerializerProtocol | None = None,
prefix: str = "langgraph:cache:",
) -> None:
"""Initialize the cache with a Redis client.
Args:
redis: Redis client instance (sync or async)
serde: Serializer to use for values
prefix: Key prefix for all cached values
"""
super().__init__(serde=serde)
self.redis = redis
self.prefix = prefix
def _make_key(self, ns: Namespace, key: str) -> str:
"""Create a Redis key from namespace and key."""
ns_str = ":".join(ns) if ns else ""
return f"{self.prefix}{ns_str}:{key}" if ns_str else f"{self.prefix}{key}"
def _parse_key(self, redis_key: str) -> tuple[Namespace, str]:
"""Parse a Redis key back to namespace and key."""
if not redis_key.startswith(self.prefix):
raise ValueError(
f"Key {redis_key} does not start with prefix {self.prefix}"
)
remaining = redis_key[len(self.prefix) :]
if ":" in remaining:
parts = remaining.split(":")
key = parts[-1]
ns_parts = parts[:-1]
return (tuple(ns_parts), key)
else:
return (tuple(), remaining)
def get(self, keys: Sequence[FullKey]) -> dict[FullKey, ValueT]:
"""Get the cached values for the given keys."""
if not keys:
return {}
# Build Redis keys
redis_keys = [self._make_key(ns, key) for ns, key in keys]
# Get values from Redis using MGET
try:
raw_values = self.redis.mget(redis_keys)
except Exception:
# If Redis is unavailable, return empty dict
return {}
values: dict[FullKey, ValueT] = {}
for i, raw_value in enumerate(raw_values):
if raw_value is not None:
try:
# Deserialize the value
encoding, data = raw_value.split(b":", 1)
values[keys[i]] = self.serde.loads_typed((encoding.decode(), data))
except Exception:
# Skip corrupted entries
continue
return values
async def aget(self, keys: Sequence[FullKey]) -> dict[FullKey, ValueT]:
"""Asynchronously get the cached values for the given keys."""
return self.get(keys)
def set(self, mapping: Mapping[FullKey, tuple[ValueT, int | None]]) -> None:
"""Set the cached values for the given keys and TTLs."""
if not mapping:
return
# Use pipeline for efficient batch operations
pipe = self.redis.pipeline()
for (ns, key), (value, ttl) in mapping.items():
redis_key = self._make_key(ns, key)
encoding, data = self.serde.dumps_typed(value)
# Store as "encoding:data" format
serialized_value = f"{encoding}:".encode() + data
if ttl is not None:
pipe.setex(redis_key, ttl, serialized_value)
else:
pipe.set(redis_key, serialized_value)
try:
pipe.execute()
except Exception:
# Silently fail if Redis is unavailable
pass
async def aset(self, mapping: Mapping[FullKey, tuple[ValueT, int | None]]) -> None:
"""Asynchronously set the cached values for the given keys and TTLs."""
self.set(mapping)
def clear(self, namespaces: Sequence[Namespace] | None = None) -> None:
"""Delete the cached values for the given namespaces.
If no namespaces are provided, clear all cached values."""
try:
if namespaces is None:
# Clear all keys with our prefix
pattern = f"{self.prefix}*"
keys = self.redis.keys(pattern)
if keys:
self.redis.delete(*keys)
else:
# Clear specific namespaces
keys_to_delete = []
for ns in namespaces:
ns_str = ":".join(ns) if ns else ""
pattern = (
f"{self.prefix}{ns_str}:*" if ns_str else f"{self.prefix}*"
)
keys = self.redis.keys(pattern)
keys_to_delete.extend(keys)
if keys_to_delete:
self.redis.delete(*keys_to_delete)
except Exception:
# Silently fail if Redis is unavailable
pass
async def aclear(self, namespaces: Sequence[Namespace] | None = None) -> None:
"""Asynchronously delete the cached values for the given namespaces.
If no namespaces are provided, clear all cached values."""
self.clear(namespaces)
@@ -1,17 +1,19 @@
from __future__ import annotations
from collections.abc import AsyncIterator, Iterator, Mapping, Sequence
from collections.abc import Iterator, Mapping, Sequence
from datetime import datetime, timezone
from typing import ( # noqa: UP035
Any,
Dict,
Generic,
List,
Literal,
NamedTuple,
Optional,
Tuple,
TypedDict,
TypeVar,
Union,
)
from langchain_core.runnables import RunnableConfig
from langgraph.checkpoint.base.id import uuid6
from langgraph.checkpoint.serde.base import SerializerProtocol, maybe_add_typed_methods
from langgraph.checkpoint.serde.jsonplus import JsonPlusSerializer
@@ -21,10 +23,29 @@ from langgraph.checkpoint.serde.types import (
RESUME,
SCHEDULED,
ChannelProtocol,
SendProtocol,
)
V = TypeVar("V", int, float, str)
PendingWrite = tuple[str, str, Any]
PendingWrite = Tuple[str, str, Any]
class CheckpointConfig(TypedDict, total=False):
"""Configuration for a Runnable."""
metadata: dict[str, Any]
"""
Metadata for this call and any sub-calls (eg. a Chain calling an LLM).
Keys should be strings, values should be JSON-serializable.
"""
configurable: dict[str, Any]
"""
Runtime values for attributes previously made configurable on this Runnable,
or sub-Runnables, through .configurable_fields() or .configurable_alternatives().
Check .output_schema() for a description of the attributes that have been made
configurable.
"""
# Marked as total=False to allow for future expansion.
@@ -34,17 +55,22 @@ class CheckpointMetadata(TypedDict, total=False):
source: Literal["input", "loop", "update", "fork"]
"""The source of the checkpoint.
- `"input"`: The checkpoint was created from an input to invoke/stream/batch.
- `"loop"`: The checkpoint was created from inside the pregel loop.
- `"update"`: The checkpoint was created from a manual state update.
- `"fork"`: The checkpoint was created as a copy of another checkpoint.
- "input": The checkpoint was created from an input to invoke/stream/batch.
- "loop": The checkpoint was created from inside the pregel loop.
- "update": The checkpoint was created from a manual state update.
- "fork": The checkpoint was created as a copy of another checkpoint.
"""
step: int
"""The step number of the checkpoint.
`-1` for the first `"input"` checkpoint.
`0` for the first `"loop"` checkpoint.
`...` for the `nth` checkpoint afterwards.
-1 for the first "input" checkpoint.
0 for the first "loop" checkpoint.
... for the nth checkpoint afterwards.
"""
writes: dict[str, Any]
"""The writes that were made between the previous checkpoint and this one.
Mapping from node name to writes emitted by that node.
"""
parents: dict[str, str]
"""The IDs of the parent checkpoints.
@@ -53,41 +79,52 @@ class CheckpointMetadata(TypedDict, total=False):
"""
ChannelVersions = dict[str, str | int | float]
class TaskInfo(TypedDict):
status: Literal["scheduled", "success", "error"]
ChannelVersions = dict[str, Union[str, int, float]]
class Checkpoint(TypedDict):
"""State snapshot at a given point in time."""
v: int
"""The version of the checkpoint format. Currently `1`."""
"""The version of the checkpoint format. Currently 1."""
id: str
"""The ID of the checkpoint.
This is both unique and monotonically increasing, so can be used for sorting
checkpoints from first to last."""
"""The ID of the checkpoint. This is both unique and monotonically
increasing, so can be used for sorting checkpoints from first to last."""
ts: str
"""The timestamp of the checkpoint in ISO 8601 format."""
channel_values: dict[str, Any]
"""The values of the channels at the time of the checkpoint.
Mapping from channel name to deserialized channel snapshot value.
"""
channel_versions: ChannelVersions
"""The versions of the channels at the time of the checkpoint.
The keys are channel names and the values are monotonically increasing
version strings for each channel.
"""
versions_seen: dict[str, ChannelVersions]
"""Map from node ID to map from channel name to version seen.
This keeps track of the versions of the channels that each node has seen.
Used to determine which nodes to execute next.
"""
updated_channels: list[str] | None
"""The channels that were updated in this checkpoint.
"""
pending_sends: List[SendProtocol]
"""List of inputs pushed to nodes but not yet processed.
Cleared by the next checkpoint."""
def empty_checkpoint() -> Checkpoint:
return Checkpoint(
v=1,
id=str(uuid6(clock_seq=-2)),
ts=datetime.now(timezone.utc).isoformat(),
channel_values={},
channel_versions={},
versions_seen={},
pending_sends=[],
)
def copy_checkpoint(checkpoint: Checkpoint) -> Checkpoint:
@@ -99,18 +136,48 @@ def copy_checkpoint(checkpoint: Checkpoint) -> Checkpoint:
channel_versions=checkpoint["channel_versions"].copy(),
versions_seen={k: v.copy() for k, v in checkpoint["versions_seen"].items()},
pending_sends=checkpoint.get("pending_sends", []).copy(),
updated_channels=checkpoint.get("updated_channels", None),
)
def create_checkpoint(
checkpoint: Checkpoint,
channels: Optional[Mapping[str, ChannelProtocol]],
step: int,
*,
id: Optional[str] = None,
) -> Checkpoint:
"""Create a checkpoint for the given channels."""
ts = datetime.now(timezone.utc).isoformat()
if channels is None:
values = checkpoint["channel_values"]
else:
values = {}
for k, v in channels.items():
if k not in checkpoint["channel_versions"]:
continue
try:
values[k] = v.checkpoint()
except EmptyChannelError:
pass
return Checkpoint(
v=1,
ts=ts,
id=id or str(uuid6(clock_seq=step)),
channel_values=values,
channel_versions=checkpoint["channel_versions"],
versions_seen=checkpoint["versions_seen"],
pending_sends=checkpoint.get("pending_sends", []),
)
class CheckpointTuple(NamedTuple):
"""A tuple containing a checkpoint and its associated data."""
config: RunnableConfig
config: CheckpointConfig
checkpoint: Checkpoint
metadata: CheckpointMetadata
parent_config: RunnableConfig | None = None
pending_writes: list[PendingWrite] | None = None
parent_config: Optional[CheckpointConfig] = None
pending_writes: Optional[List[PendingWrite]] = None
class BaseCheckpointSaver(Generic[V]):
@@ -132,39 +199,30 @@ class BaseCheckpointSaver(Generic[V]):
def __init__(
self,
*,
serde: SerializerProtocol | None = None,
serde: Optional[SerializerProtocol] = None,
) -> None:
self.serde = maybe_add_typed_methods(serde or self.serde)
@property
def config_specs(self) -> list:
"""Define the configuration options for the checkpoint saver.
Returns:
list: List of configuration field specs.
"""
return []
def get(self, config: RunnableConfig) -> Checkpoint | None:
def get(self, config: CheckpointConfig) -> Optional[Checkpoint]:
"""Fetch a checkpoint using the given configuration.
Args:
config: Configuration specifying which checkpoint to retrieve.
config (CheckpointConfig): Configuration specifying which checkpoint to retrieve.
Returns:
The requested checkpoint, or `None` if not found.
Optional[Checkpoint]: The requested checkpoint, or None if not found.
"""
if value := self.get_tuple(config):
return value.checkpoint
def get_tuple(self, config: RunnableConfig) -> CheckpointTuple | None:
def get_tuple(self, config: CheckpointConfig) -> Optional[CheckpointTuple]:
"""Fetch a checkpoint tuple using the given configuration.
Args:
config: Configuration specifying which checkpoint to retrieve.
config (CheckpointConfig): Configuration specifying which checkpoint to retrieve.
Returns:
The requested checkpoint tuple, or `None` if not found.
Optional[CheckpointTuple]: The requested checkpoint tuple, or None if not found.
Raises:
NotImplementedError: Implement this method in your custom checkpoint saver.
@@ -173,22 +231,22 @@ class BaseCheckpointSaver(Generic[V]):
def list(
self,
config: RunnableConfig | None,
config: Optional[CheckpointConfig],
*,
filter: dict[str, Any] | None = None,
before: RunnableConfig | None = None,
limit: int | None = None,
filter: Optional[Dict[str, Any]] = None,
before: Optional[CheckpointConfig] = None,
limit: Optional[int] = None,
) -> Iterator[CheckpointTuple]:
"""List checkpoints that match the given criteria.
Args:
config: Base configuration for filtering checkpoints.
filter: Additional filtering criteria.
before: List checkpoints created before this configuration.
limit: Maximum number of checkpoints to return.
config (Optional[CheckpointConfig]): Base configuration for filtering checkpoints.
filter (Optional[Dict[str, Any]]): Additional filtering criteria.
before (Optional[CheckpointConfig]): List checkpoints created before this configuration.
limit (Optional[int]): Maximum number of checkpoints to return.
Returns:
Iterator of matching checkpoint tuples.
Iterator[CheckpointTuple]: Iterator of matching checkpoint tuples.
Raises:
NotImplementedError: Implement this method in your custom checkpoint saver.
@@ -197,21 +255,21 @@ class BaseCheckpointSaver(Generic[V]):
def put(
self,
config: RunnableConfig,
config: CheckpointConfig,
checkpoint: Checkpoint,
metadata: CheckpointMetadata,
new_versions: ChannelVersions,
) -> RunnableConfig:
) -> CheckpointConfig:
"""Store a checkpoint with its configuration and metadata.
Args:
config: Configuration for the checkpoint.
checkpoint: The checkpoint to store.
metadata: Additional metadata for the checkpoint.
new_versions: New channel versions as of this write.
config (CheckpointConfig): Configuration for the checkpoint.
checkpoint (Checkpoint): The checkpoint to store.
metadata (CheckpointMetadata): Additional metadata for the checkpoint.
new_versions (ChannelVersions): New channel versions as of this write.
Returns:
RunnableConfig: Updated configuration after storing the checkpoint.
CheckpointConfig: Updated configuration after storing the checkpoint.
Raises:
NotImplementedError: Implement this method in your custom checkpoint saver.
@@ -220,150 +278,33 @@ class BaseCheckpointSaver(Generic[V]):
def put_writes(
self,
config: RunnableConfig,
writes: Sequence[tuple[str, Any]],
config: CheckpointConfig,
writes: Sequence[Tuple[str, Any]],
task_id: str,
task_path: str = "",
) -> None:
"""Store intermediate writes linked to a checkpoint.
Args:
config: Configuration of the related checkpoint.
writes: List of writes to store.
task_id: Identifier for the task creating the writes.
task_path: Path of the task creating the writes.
config (CheckpointConfig): Configuration of the related checkpoint.
writes (List[Tuple[str, Any]]): List of writes to store.
task_id (str): Identifier for the task creating the writes.
task_path (str): Path of the task creating the writes.
Raises:
NotImplementedError: Implement this method in your custom checkpoint saver.
"""
raise NotImplementedError
def delete_thread(
self,
thread_id: str,
) -> None:
"""Delete all checkpoints and writes associated with a specific thread ID.
Args:
thread_id: The thread ID whose checkpoints should be deleted.
"""
raise NotImplementedError
async def aget(self, config: RunnableConfig) -> Checkpoint | None:
"""Asynchronously fetch a checkpoint using the given configuration.
Args:
config: Configuration specifying which checkpoint to retrieve.
Returns:
The requested checkpoint, or `None` if not found.
"""
if value := await self.aget_tuple(config):
return value.checkpoint
async def aget_tuple(self, config: RunnableConfig) -> CheckpointTuple | None:
"""Asynchronously fetch a checkpoint tuple using the given configuration.
Args:
config: Configuration specifying which checkpoint to retrieve.
Returns:
The requested checkpoint tuple, or `None` if not found.
Raises:
NotImplementedError: Implement this method in your custom checkpoint saver.
"""
raise NotImplementedError
async def alist(
self,
config: RunnableConfig | None,
*,
filter: dict[str, Any] | None = None,
before: RunnableConfig | None = None,
limit: int | None = None,
) -> AsyncIterator[CheckpointTuple]:
"""Asynchronously list checkpoints that match the given criteria.
Args:
config: Base configuration for filtering checkpoints.
filter: Additional filtering criteria for metadata.
before: List checkpoints created before this configuration.
limit: Maximum number of checkpoints to return.
Returns:
Async iterator of matching checkpoint tuples.
Raises:
NotImplementedError: Implement this method in your custom checkpoint saver.
"""
raise NotImplementedError
yield
async def aput(
self,
config: RunnableConfig,
checkpoint: Checkpoint,
metadata: CheckpointMetadata,
new_versions: ChannelVersions,
) -> RunnableConfig:
"""Asynchronously store a checkpoint with its configuration and metadata.
Args:
config: Configuration for the checkpoint.
checkpoint: The checkpoint to store.
metadata: Additional metadata for the checkpoint.
new_versions: New channel versions as of this write.
Returns:
RunnableConfig: Updated configuration after storing the checkpoint.
Raises:
NotImplementedError: Implement this method in your custom checkpoint saver.
"""
raise NotImplementedError
async def aput_writes(
self,
config: RunnableConfig,
writes: Sequence[tuple[str, Any]],
task_id: str,
task_path: str = "",
) -> None:
"""Asynchronously store intermediate writes linked to a checkpoint.
Args:
config: Configuration of the related checkpoint.
writes: List of writes to store.
task_id: Identifier for the task creating the writes.
task_path: Path of the task creating the writes.
Raises:
NotImplementedError: Implement this method in your custom checkpoint saver.
"""
raise NotImplementedError
async def adelete_thread(
self,
thread_id: str,
) -> None:
"""Delete all checkpoints and writes associated with a specific thread ID.
Args:
thread_id: The thread ID whose checkpoints should be deleted.
"""
raise NotImplementedError
def get_next_version(self, current: V | None, channel: None) -> V:
def get_next_version(self, current: Optional[V], channel: ChannelProtocol) -> V:
"""Generate the next version ID for a channel.
Default is to use integer versions, incrementing by `1`.
If you override, you can use `str`/`int`/`float` versions, as long as they are monotonically increasing.
Default is to use integer versions, incrementing by 1. If you override, you can use str/int/float versions,
as long as they are monotonically increasing.
Args:
current: The current version identifier (`int`, `float`, or `str`).
channel: Deprecated argument, kept for backwards compatibility.
current (Optional[V]): The current version identifier (int, float, or str).
channel (BaseChannel): The channel being versioned.
Returns:
V: The next version identifier, which must be increasing.
@@ -383,42 +324,30 @@ class EmptyChannelError(Exception):
pass
def get_checkpoint_id(config: RunnableConfig) -> str | None:
"""Get checkpoint ID."""
return config["configurable"].get("checkpoint_id")
def get_checkpoint_id(config: CheckpointConfig) -> Optional[str]:
"""Get checkpoint ID in a backwards-compatible manner (fallback on thread_ts)."""
return config["configurable"].get(
"checkpoint_id", config["configurable"].get("thread_ts")
)
def get_checkpoint_metadata(
config: RunnableConfig, metadata: CheckpointMetadata
config: CheckpointConfig, metadata: CheckpointMetadata
) -> CheckpointMetadata:
"""Get checkpoint metadata in a backwards-compatible manner."""
metadata = {
k: v.replace("\u0000", "") if isinstance(v, str) else v
for k, v in metadata.items()
}
metadata = metadata.copy()
for obj in (config.get("metadata"), config.get("configurable")):
if not obj:
continue
for key, v in obj.items():
for key in obj:
if key in metadata or key in EXCLUDED_METADATA_KEYS or key.startswith("__"):
continue
elif isinstance(v, str):
metadata[key] = v.replace("\u0000", "")
elif isinstance(v, (int, bool, float)):
metadata[key] = v
v = obj[key]
if isinstance(v, (str, int, bool, float)):
metadata[key] = v # type: ignore[literal-required]
return metadata
def get_serializable_checkpoint_metadata(
config: RunnableConfig, metadata: CheckpointMetadata
) -> CheckpointMetadata:
"""Get checkpoint metadata in a backwards-compatible manner."""
checkpoint_metadata = get_checkpoint_metadata(config, metadata)
if "writes" in checkpoint_metadata:
checkpoint_metadata.pop("writes")
return checkpoint_metadata
"""
Mapping from error type to error index.
Regular writes just map to their index in the list of writes being saved.
@@ -429,66 +358,7 @@ Each Checkpointer implementation should use this mapping in put_writes.
WRITES_IDX_MAP = {ERROR: -1, SCHEDULED: -2, INTERRUPT: -3, RESUME: -4}
EXCLUDED_METADATA_KEYS = {
"thread_id",
"checkpoint_id",
"checkpoint_ns",
"checkpoint_map",
"langgraph_step",
"langgraph_node",
"langgraph_triggers",
"langgraph_path",
"langgraph_checkpoint_ns",
}
# --- below are deprecated utilities used by past versions of LangGraph ---
LATEST_VERSION = 2
def empty_checkpoint() -> Checkpoint:
from datetime import datetime, timezone
return Checkpoint(
v=LATEST_VERSION,
id=str(uuid6(clock_seq=-2)),
ts=datetime.now(timezone.utc).isoformat(),
channel_values={},
channel_versions={},
versions_seen={},
pending_sends=[],
updated_channels=None,
)
def create_checkpoint(
checkpoint: Checkpoint,
channels: Mapping[str, ChannelProtocol] | None,
step: int,
*,
id: str | None = None,
) -> Checkpoint:
"""Create a checkpoint for the given channels."""
from datetime import datetime, timezone
ts = datetime.now(timezone.utc).isoformat()
if channels is None:
values = checkpoint["channel_values"]
else:
values = {}
for k, v in channels.items():
if k not in checkpoint["channel_versions"]:
continue
try:
values[k] = v.checkpoint()
except EmptyChannelError:
pass
return Checkpoint(
v=LATEST_VERSION,
ts=ts,
id=id or str(uuid6(clock_seq=step)),
channel_values=values,
channel_versions=checkpoint["channel_versions"],
versions_seen=checkpoint["versions_seen"],
pending_sends=checkpoint.get("pending_sends", []),
updated_channels=None,
)
@@ -3,11 +3,10 @@ https://github.com/oittaa/uuid6-python/blob/main/src/uuid6/__init__.py#L95
Bundled in to avoid install issues with uuid6 package
"""
from __future__ import annotations
import random
import time
import uuid
from typing import Optional, Tuple
_last_v6_timestamp = None
@@ -19,12 +18,12 @@ class UUID(uuid.UUID):
def __init__(
self,
hex: str | None = None,
bytes: bytes | None = None,
bytes_le: bytes | None = None,
fields: tuple[int, int, int, int, int, int] | None = None,
int: int | None = None,
version: int | None = None,
hex: Optional[str] = None,
bytes: Optional[bytes] = None,
bytes_le: Optional[bytes] = None,
fields: Optional[Tuple[int, int, int, int, int, int]] = None,
int: Optional[int] = None,
version: Optional[int] = None,
*,
is_safe: uuid.SafeUUID = uuid.SafeUUID.unknown,
) -> None:
@@ -76,7 +75,7 @@ def _subsec_decode(value: int) -> int:
return -(-value * 10**6 // 2**20)
def uuid6(node: int | None = None, clock_seq: int | None = None) -> UUID:
def uuid6(node: Optional[int] = None, clock_seq: Optional[int] = None) -> UUID:
r"""UUID version 6 is a field-compatible version of UUIDv1, reordered for
improved DB locality. It is expected that UUIDv6 will primarily be
used in contexts where there are existing v1 UUIDs. Systems that do
@@ -1,165 +1,136 @@
from __future__ import annotations
import logging
import os
import pickle
import random
import shutil
from collections import defaultdict
from collections.abc import AsyncIterator, Iterator, Sequence
from contextlib import AbstractAsyncContextManager, AbstractContextManager, ExitStack
from collections.abc import Iterator, Sequence
from contextlib import AbstractContextManager, ExitStack
from types import TracebackType
from typing import Any
from langchain_core.runnables import RunnableConfig
from typing import Any, Optional
from langgraph.checkpoint.base import (
WRITES_IDX_MAP,
BaseCheckpointSaver,
ChannelVersions,
Checkpoint,
CheckpointConfig,
CheckpointMetadata,
CheckpointTuple,
SerializerProtocol,
get_checkpoint_id,
get_checkpoint_metadata,
)
from langgraph.checkpoint.serde.types import TASKS, ChannelProtocol
logger = logging.getLogger(__name__)
class InMemorySaver(
BaseCheckpointSaver[str], AbstractContextManager, AbstractAsyncContextManager
):
class InMemorySaver(BaseCheckpointSaver[str], AbstractContextManager):
"""An in-memory checkpoint saver.
This checkpoint saver stores checkpoints in memory using a `defaultdict`.
This checkpoint saver stores checkpoints in memory using a defaultdict.
Note:
Only use `InMemorySaver` for debugging or testing purposes.
For production use cases we recommend installing [langgraph-checkpoint-postgres](https://pypi.org/project/langgraph-checkpoint-postgres/) and using `PostgresSaver` / `AsyncPostgresSaver`.
If you are using LangSmith Deployment, no checkpointer needs to be specified. The correct managed checkpointer will be used automatically.
Args:
serde: The serializer to use for serializing and deserializing checkpoints.
serde (Optional[SerializerProtocol]): The serializer to use for serializing and deserializing checkpoints. Defaults to None.
Example:
```python
import asyncio
Examples:
from langgraph.checkpoint.memory import InMemorySaver
from langgraph.graph import StateGraph
import asyncio
builder = StateGraph(int)
builder.add_node("add_one", lambda x: x + 1)
builder.set_entry_point("add_one")
builder.set_finish_point("add_one")
from langgraph.checkpoint.memory import InMemorySaver
from langgraph.graph import StateGraph
memory = InMemorySaver()
graph = builder.compile(checkpointer=memory)
coro = graph.ainvoke(1, {"configurable": {"thread_id": "thread-1"}})
asyncio.run(coro) # Output: 2
```
builder = StateGraph(int)
builder.add_node("add_one", lambda x: x + 1)
builder.set_entry_point("add_one")
builder.set_finish_point("add_one")
memory = InMemorySaver()
graph = builder.compile(checkpointer=memory)
coro = graph.ainvoke(1, {"configurable": {"thread_id": "thread-1"}})
asyncio.run(coro) # Output: 2
"""
# thread ID -> checkpoint NS -> checkpoint ID -> checkpoint mapping
storage: defaultdict[
str,
dict[str, dict[str, tuple[tuple[str, bytes], tuple[str, bytes], str | None]]],
dict[
str, dict[str, tuple[tuple[str, bytes], tuple[str, bytes], Optional[str]]]
],
]
# (thread ID, checkpoint NS, checkpoint ID) -> (task ID, write idx)
writes: defaultdict[
tuple[str, str, str],
dict[tuple[str, int], tuple[str, str, tuple[str, bytes], str]],
]
blobs: dict[
tuple[
str, str, str, str | int | float
], # thread id, checkpoint ns, channel, version
tuple[str, bytes],
]
def __init__(
self,
*,
serde: SerializerProtocol | None = None,
serde: Optional[SerializerProtocol] = None,
factory: type[defaultdict] = defaultdict,
) -> None:
super().__init__(serde=serde)
self.storage = factory(lambda: defaultdict(dict))
self.writes = factory(dict)
self.blobs = factory()
self.stack = ExitStack()
if factory is not defaultdict:
self.stack.enter_context(self.storage) # type: ignore[arg-type]
self.stack.enter_context(self.writes) # type: ignore[arg-type]
self.stack.enter_context(self.blobs) # type: ignore[arg-type]
def __enter__(self) -> InMemorySaver:
self.stack.__enter__()
return self
def __enter__(self) -> "InMemorySaver":
return self.stack.__enter__()
def __exit__(
self,
exc_type: type[BaseException] | None,
exc_value: BaseException | None,
traceback: TracebackType | None,
) -> bool | None:
exc_type: Optional[type[BaseException]],
exc_value: Optional[BaseException],
traceback: Optional[TracebackType],
) -> Optional[bool]:
return self.stack.__exit__(exc_type, exc_value, traceback)
async def __aenter__(self) -> InMemorySaver:
self.stack.__enter__()
return self
async def __aexit__(
self,
__exc_type: type[BaseException] | None,
__exc_value: BaseException | None,
__traceback: TracebackType | None,
) -> bool | None:
return self.stack.__exit__(__exc_type, __exc_value, __traceback)
def _load_blobs(
self, thread_id: str, checkpoint_ns: str, versions: ChannelVersions
) -> dict[str, Any]:
channel_values: dict[str, Any] = {}
for k, v in versions.items():
kk = (thread_id, checkpoint_ns, k, v)
if kk in self.blobs:
vv = self.blobs[kk]
if vv[0] != "empty":
channel_values[k] = self.serde.loads_typed(vv)
return channel_values
def get_tuple(self, config: RunnableConfig) -> CheckpointTuple | None:
def get_tuple(self, config: CheckpointConfig) -> Optional[CheckpointTuple]:
"""Get a checkpoint tuple from the in-memory storage.
This method retrieves a checkpoint tuple from the in-memory storage based on the
provided config. If the config contains a `checkpoint_id` key, the checkpoint with
provided config. If the config contains a "checkpoint_id" key, the checkpoint with
the matching thread ID and timestamp is retrieved. Otherwise, the latest checkpoint
for the given thread ID is retrieved.
Args:
config: The config to use for retrieving the checkpoint.
config (CheckpointConfig): The config to use for retrieving the checkpoint.
Returns:
The retrieved checkpoint tuple, or None if no matching checkpoint was found.
Optional[CheckpointTuple]: The retrieved checkpoint tuple, or None if no matching checkpoint was found.
"""
thread_id: str = config["configurable"]["thread_id"]
checkpoint_ns: str = config["configurable"].get("checkpoint_ns", "")
thread_id = config["configurable"]["thread_id"]
checkpoint_ns = config["configurable"].get("checkpoint_ns", "")
if checkpoint_id := get_checkpoint_id(config):
if saved := self.storage[thread_id][checkpoint_ns].get(checkpoint_id):
checkpoint, metadata, parent_checkpoint_id = saved
writes = self.writes[(thread_id, checkpoint_ns, checkpoint_id)].values()
checkpoint_: Checkpoint = self.serde.loads_typed(checkpoint)
if parent_checkpoint_id:
sends = sorted(
(
(*w, k[1])
for k, w in self.writes[
(thread_id, checkpoint_ns, parent_checkpoint_id)
].items()
if w[1] == TASKS
),
key=lambda w: (w[3], w[0], w[4]),
)
else:
sends = []
return CheckpointTuple(
config=config,
checkpoint={
**checkpoint_,
"channel_values": self._load_blobs(
thread_id, checkpoint_ns, checkpoint_["channel_versions"]
),
**self.serde.loads_typed(checkpoint),
"pending_sends": [self.serde.loads_typed(s[2]) for s in sends],
},
metadata=self.serde.loads_typed(metadata),
pending_writes=[
@@ -182,7 +153,19 @@ class InMemorySaver(
checkpoint_id = max(checkpoints.keys())
checkpoint, metadata, parent_checkpoint_id = checkpoints[checkpoint_id]
writes = self.writes[(thread_id, checkpoint_ns, checkpoint_id)].values()
checkpoint_ = self.serde.loads_typed(checkpoint)
if parent_checkpoint_id:
sends = sorted(
(
(*w, k[1])
for k, w in self.writes[
(thread_id, checkpoint_ns, parent_checkpoint_id)
].items()
if w[1] == TASKS
),
key=lambda w: (w[3], w[0], w[4]),
)
else:
sends = []
return CheckpointTuple(
config={
"configurable": {
@@ -192,10 +175,8 @@ class InMemorySaver(
}
},
checkpoint={
**checkpoint_,
"channel_values": self._load_blobs(
thread_id, checkpoint_ns, checkpoint_["channel_versions"]
),
**self.serde.loads_typed(checkpoint),
"pending_sends": [self.serde.loads_typed(s[2]) for s in sends],
},
metadata=self.serde.loads_typed(metadata),
pending_writes=[
@@ -216,11 +197,11 @@ class InMemorySaver(
def list(
self,
config: RunnableConfig | None,
config: Optional[CheckpointConfig],
*,
filter: dict[str, Any] | None = None,
before: RunnableConfig | None = None,
limit: int | None = None,
filter: Optional[dict[str, Any]] = None,
before: Optional[CheckpointConfig] = None,
limit: Optional[int] = None,
) -> Iterator[CheckpointTuple]:
"""List checkpoints from the in-memory storage.
@@ -228,13 +209,13 @@ class InMemorySaver(
on the provided criteria.
Args:
config: Base configuration for filtering checkpoints.
filter: Additional filtering criteria for metadata.
before: List checkpoints created before this configuration.
limit: Maximum number of checkpoints to return.
config (Optional[CheckpointConfig]): Base configuration for filtering checkpoints.
filter (Optional[Dict[str, Any]]): Additional filtering criteria for metadata.
before (Optional[CheckpointConfig]): List checkpoints created before this configuration.
limit (Optional[int]): Maximum number of checkpoints to return.
Yields:
An iterator of matching checkpoint tuples.
Iterator[CheckpointTuple]: An iterator of matching checkpoint tuples.
"""
thread_ids = (config["configurable"]["thread_id"],) if config else self.storage
config_checkpoint_ns = (
@@ -288,7 +269,19 @@ class InMemorySaver(
(thread_id, checkpoint_ns, checkpoint_id)
].values()
checkpoint_: Checkpoint = self.serde.loads_typed(checkpoint)
if parent_checkpoint_id:
sends = sorted(
(
(*w, k[1])
for k, w in self.writes[
(thread_id, checkpoint_ns, parent_checkpoint_id)
].items()
if w[1] == TASKS
),
key=lambda w: (w[3], w[0], w[4]),
)
else:
sends = []
yield CheckpointTuple(
config={
@@ -299,12 +292,10 @@ class InMemorySaver(
}
},
checkpoint={
**checkpoint_,
"channel_values": self._load_blobs(
thread_id,
checkpoint_ns,
checkpoint_["channel_versions"],
),
**self.serde.loads_typed(checkpoint),
"pending_sends": [
self.serde.loads_typed(s[2]) for s in sends
],
},
metadata=metadata,
parent_config=(
@@ -325,33 +316,29 @@ class InMemorySaver(
def put(
self,
config: RunnableConfig,
config: CheckpointConfig,
checkpoint: Checkpoint,
metadata: CheckpointMetadata,
new_versions: ChannelVersions,
) -> RunnableConfig:
) -> CheckpointConfig:
"""Save a checkpoint to the in-memory storage.
This method saves a checkpoint to the in-memory storage. The checkpoint is associated
with the provided config.
Args:
config: The config to associate with the checkpoint.
checkpoint: The checkpoint to save.
metadata: Additional metadata to save with the checkpoint.
new_versions: New versions as of this write
config (CheckpointConfig): The config to associate with the checkpoint.
checkpoint (Checkpoint): The checkpoint to save.
metadata (CheckpointMetadata): Additional metadata to save with the checkpoint.
new_versions (dict): New versions as of this write
Returns:
RunnableConfig: The updated config containing the saved checkpoint's timestamp.
CheckpointConfig: The updated config containing the saved checkpoint's timestamp.
"""
c = checkpoint.copy()
c.pop("pending_sends") # type: ignore[misc]
thread_id = config["configurable"]["thread_id"]
checkpoint_ns = config["configurable"]["checkpoint_ns"]
values: dict[str, Any] = c.pop("channel_values") # type: ignore[misc]
for k, v in new_versions.items():
self.blobs[(thread_id, checkpoint_ns, k, v)] = (
self.serde.dumps_typed(values[k]) if k in values else ("empty", b"")
)
self.storage[thread_id][checkpoint_ns].update(
{
checkpoint["id"]: (
@@ -371,7 +358,7 @@ class InMemorySaver(
def put_writes(
self,
config: RunnableConfig,
config: CheckpointConfig,
writes: Sequence[tuple[str, Any]],
task_id: str,
task_path: str = "",
@@ -382,13 +369,13 @@ class InMemorySaver(
with the provided config.
Args:
config: The config to associate with the writes.
writes: The writes to save.
task_id: Identifier for the task creating the writes.
task_path: Path of the task creating the writes.
config (CheckpointConfig): The config to associate with the writes.
writes (list[tuple[str, Any]]): The writes to save.
task_id (str): Identifier for the task creating the writes.
task_path (str): Path of the task creating the writes.
Returns:
RunnableConfig: The updated config containing the saved writes' timestamp.
CheckpointConfig: The updated config containing the saved writes' timestamp.
"""
thread_id = config["configurable"]["thread_id"]
checkpoint_ns = config["configurable"].get("checkpoint_ns", "")
@@ -407,115 +394,7 @@ class InMemorySaver(
task_path,
)
def delete_thread(self, thread_id: str) -> None:
"""Delete all checkpoints and writes associated with a thread ID.
Args:
thread_id: The thread ID to delete.
Returns:
None
"""
if thread_id in self.storage:
del self.storage[thread_id]
for k in list(self.writes.keys()):
if k[0] == thread_id:
del self.writes[k]
for k in list(self.blobs.keys()):
if k[0] == thread_id:
del self.blobs[k]
async def aget_tuple(self, config: RunnableConfig) -> CheckpointTuple | None:
"""Asynchronous version of `get_tuple`.
This method is an asynchronous wrapper around `get_tuple` that runs the synchronous
method in a separate thread using asyncio.
Args:
config: The config to use for retrieving the checkpoint.
Returns:
The retrieved checkpoint tuple, or None if no matching checkpoint was found.
"""
return self.get_tuple(config)
async def alist(
self,
config: RunnableConfig | None,
*,
filter: dict[str, Any] | None = None,
before: RunnableConfig | None = None,
limit: int | None = None,
) -> AsyncIterator[CheckpointTuple]:
"""Asynchronous version of `list`.
This method is an asynchronous wrapper around `list` that runs the synchronous
method in a separate thread using asyncio.
Args:
config: The config to use for listing the checkpoints.
Yields:
An asynchronous iterator of checkpoint tuples.
"""
for item in self.list(config, filter=filter, before=before, limit=limit):
yield item
async def aput(
self,
config: RunnableConfig,
checkpoint: Checkpoint,
metadata: CheckpointMetadata,
new_versions: ChannelVersions,
) -> RunnableConfig:
"""Asynchronous version of `put`.
Args:
config: The config to associate with the checkpoint.
checkpoint: The checkpoint to save.
metadata: Additional metadata to save with the checkpoint.
new_versions: New versions as of this write
Returns:
RunnableConfig: The updated config containing the saved checkpoint's timestamp.
"""
return self.put(config, checkpoint, metadata, new_versions)
async def aput_writes(
self,
config: RunnableConfig,
writes: Sequence[tuple[str, Any]],
task_id: str,
task_path: str = "",
) -> None:
"""Asynchronous version of `put_writes`.
This method is an asynchronous wrapper around `put_writes` that runs the synchronous
method in a separate thread using asyncio.
Args:
config: The config to associate with the writes.
writes: The writes to save, each as a (channel, value) pair.
task_id: Identifier for the task creating the writes.
task_path: Path of the task creating the writes.
Returns:
None
"""
return self.put_writes(config, writes, task_id, task_path)
async def adelete_thread(self, thread_id: str) -> None:
"""Delete all checkpoints and writes associated with a thread ID.
Args:
thread_id: The thread ID to delete.
Returns:
None
"""
return self.delete_thread(thread_id)
def get_next_version(self, current: str | None, channel: None) -> str:
def get_next_version(self, current: Optional[str], channel: ChannelProtocol) -> str:
if current is None:
current_v = 0
elif isinstance(current, int):
@@ -574,7 +453,7 @@ class PersistentDict(defaultdict):
self.sync()
self.clear()
def __enter__(self) -> PersistentDict:
def __enter__(self) -> "PersistentDict":
return self
def __exit__(self, *exc_info: Any) -> None:
@@ -598,6 +477,6 @@ class PersistentDict(defaultdict):
except EOFError:
return
except Exception:
logger.error(f"Failed to load file: {fileobj.name}")
logging.error(f"Failed to load file: {fileobj.name}")
raise
raise ValueError("File not in a supported format")
@@ -1,35 +1,36 @@
from __future__ import annotations
from typing import Any, Protocol, runtime_checkable
from typing import Any, Protocol
class UntypedSerializerProtocol(Protocol):
"""Protocol for serialization and deserialization of objects."""
def dumps(self, obj: Any) -> bytes: ...
def loads(self, data: bytes) -> Any: ...
@runtime_checkable
class SerializerProtocol(Protocol):
"""Protocol for serialization and deserialization of objects.
- `dumps_typed`: Serialize an object to a tuple `(type, bytes)`.
- `loads_typed`: Deserialize an object from a tuple `(type, bytes)`.
- `dumps`: Serialize an object to bytes.
- `dumps_typed`: Serialize an object to a tuple (type, bytes).
- `loads`: Deserialize an object from bytes.
- `loads_typed`: Deserialize an object from a tuple (type, bytes).
Valid implementations include the `pickle`, `json` and `orjson` modules.
"""
def dumps(self, obj: Any) -> bytes: ...
def dumps_typed(self, obj: Any) -> tuple[str, bytes]: ...
def loads(self, data: bytes) -> Any: ...
def loads_typed(self, data: tuple[str, bytes]) -> Any: ...
class SerializerCompat(SerializerProtocol):
def __init__(self, serde: UntypedSerializerProtocol) -> None:
def __init__(self, serde: SerializerProtocol) -> None:
self.serde = serde
def dumps(self, obj: Any) -> bytes:
return self.serde.dumps(obj)
def loads(self, data: bytes) -> Any:
return self.serde.loads(data)
def dumps_typed(self, obj: Any) -> tuple[str, bytes]:
return type(obj).__name__, self.serde.dumps(obj)
@@ -37,28 +38,10 @@ class SerializerCompat(SerializerProtocol):
return self.serde.loads(data[1])
def maybe_add_typed_methods(
serde: SerializerProtocol | UntypedSerializerProtocol,
) -> SerializerProtocol:
def maybe_add_typed_methods(serde: SerializerProtocol) -> SerializerProtocol:
"""Wrap serde old serde implementations in a class with loads_typed and dumps_typed for backwards compatibility."""
if not isinstance(serde, SerializerProtocol):
if not hasattr(serde, "loads_typed") or not hasattr(serde, "dumps_typed"):
return SerializerCompat(serde)
return serde
class CipherProtocol(Protocol):
"""Protocol for encryption and decryption of data.
- `encrypt`: Encrypt plaintext.
- `decrypt`: Decrypt ciphertext.
"""
def encrypt(self, plaintext: bytes) -> tuple[str, bytes]:
"""Encrypt plaintext. Returns a tuple `(cipher name, ciphertext)`."""
...
def decrypt(self, ciphername: str, ciphertext: bytes) -> bytes:
"""Decrypt ciphertext. Returns the plaintext."""
...
@@ -1,80 +0,0 @@
import os
from typing import Any
from langgraph.checkpoint.serde.base import CipherProtocol, SerializerProtocol
from langgraph.checkpoint.serde.jsonplus import JsonPlusSerializer
class EncryptedSerializer(SerializerProtocol):
"""Serializer that encrypts and decrypts data using an encryption protocol."""
def __init__(
self, cipher: CipherProtocol, serde: SerializerProtocol = JsonPlusSerializer()
) -> None:
self.cipher = cipher
self.serde = serde
def dumps_typed(self, obj: Any) -> tuple[str, bytes]:
"""Serialize an object to a tuple `(type, bytes)` and encrypt the bytes."""
# serialize data
typ, data = self.serde.dumps_typed(obj)
# encrypt data
ciphername, ciphertext = self.cipher.encrypt(data)
# add cipher name to type
return f"{typ}+{ciphername}", ciphertext
def loads_typed(self, data: tuple[str, bytes]) -> Any:
enc_cipher, ciphertext = data
# unencrypted data
if "+" not in enc_cipher:
return self.serde.loads_typed(data)
# extract cipher name
typ, ciphername = enc_cipher.split("+", 1)
# decrypt data
decrypted_data = self.cipher.decrypt(ciphername, ciphertext)
# deserialize data
return self.serde.loads_typed((typ, decrypted_data))
@classmethod
def from_pycryptodome_aes(
cls, serde: SerializerProtocol = JsonPlusSerializer(), **kwargs: Any
) -> "EncryptedSerializer":
"""Create an `EncryptedSerializer` using AES encryption."""
try:
from Crypto.Cipher import AES # type: ignore
except ImportError:
raise ImportError(
"Pycryptodome is not installed. Please install it with `pip install pycryptodome`."
) from None
# check if AES key is provided
if "key" in kwargs:
key: bytes = kwargs.pop("key")
else:
key_str = os.getenv("LANGGRAPH_AES_KEY")
if key_str is None:
raise ValueError("LANGGRAPH_AES_KEY environment variable is not set.")
key = key_str.encode()
if len(key) not in (16, 24, 32):
raise ValueError("LANGGRAPH_AES_KEY must be 16, 24, or 32 bytes long.")
# set default mode to EAX if not provided
if kwargs.get("mode") is None:
kwargs["mode"] = AES.MODE_EAX
class PycryptodomeAesCipher(CipherProtocol):
def encrypt(self, plaintext: bytes) -> tuple[str, bytes]:
cipher = AES.new(key, **kwargs)
ciphertext, tag = cipher.encrypt_and_digest(plaintext)
return "aes", cipher.nonce + tag + ciphertext
def decrypt(self, ciphername: str, ciphertext: bytes) -> bytes:
assert ciphername == "aes", f"Unsupported cipher: {ciphername}"
nonce = ciphertext[:16]
tag = ciphertext[16:32]
actual_ciphertext = ciphertext[32:]
cipher = AES.new(key, **kwargs, nonce=nonce)
return cipher.decrypt_and_verify(actual_ciphertext, tag)
return cls(PycryptodomeAesCipher(), serde)
@@ -1,16 +1,11 @@
from __future__ import annotations
import dataclasses
import decimal
import importlib
import json
import logging
import pathlib
import pickle
import re
import sys
from collections import deque
from collections.abc import Callable, Sequence
from collections.abc import Sequence
from datetime import date, datetime, time, timedelta, timezone
from enum import Enum
from inspect import isclass
@@ -22,59 +17,25 @@ from ipaddress import (
IPv6Interface,
IPv6Network,
)
from typing import Any, Literal
from typing import Any, Callable, Optional, Union
from uuid import UUID
from zoneinfo import ZoneInfo
import ormsgpack
from langchain_core.load.load import Reviver
import msgpack # type: ignore[import-untyped]
from zoneinfo import ZoneInfo
from langgraph.checkpoint.serde.base import SerializerProtocol
from langgraph.checkpoint.serde.types import SendProtocol
from langgraph.store.base import Item
LC_REVIVER = Reviver()
EMPTY_BYTES = b""
logger = logging.getLogger(__name__)
class JsonPlusSerializer(SerializerProtocol):
"""Serializer that uses ormsgpack, with optional fallbacks.
!!! warning
Security note: This serializer is intended for use within the `BaseCheckpointSaver`
class and called within the Pregel loop. It should not be used on untrusted
python objects. If an attacker can write directly to your checkpoint database,
they may be able to trigger code execution when data is deserialized.
"""
def __init__(
self,
*,
pickle_fallback: bool = False,
allowed_json_modules: Sequence[tuple[str, ...]] | Literal[True] | None = None,
__unpack_ext_hook__: Callable[[int, bytes], Any] | None = None,
) -> None:
self.pickle_fallback = pickle_fallback
self._allowed_modules = (
{mod_and_name for mod_and_name in allowed_json_modules}
if allowed_json_modules and allowed_json_modules is not True
else (allowed_json_modules if allowed_json_modules is True else None)
)
self._unpack_ext_hook = (
__unpack_ext_hook__
if __unpack_ext_hook__ is not None
else _msgpack_ext_hook
)
def _encode_constructor_args(
self,
constructor: Callable | type[Any],
constructor: Union[Callable, type[Any]],
*,
method: None | str | Sequence[None | str] = None,
args: Sequence[Any] | None = None,
kwargs: dict[str, Any] | None = None,
method: Union[None, str, Sequence[Union[None, str]]] = None,
args: Optional[Sequence[Any]] = None,
kwargs: Optional[dict[str, Any]] = None,
) -> dict[str, Any]:
out = {
"lc": 2,
@@ -89,123 +50,160 @@ class JsonPlusSerializer(SerializerProtocol):
out["kwargs"] = kwargs
return out
def _default(self, obj: Any) -> Union[str, dict[str, Any]]:
if hasattr(obj, "model_dump") and callable(obj.model_dump):
return self._encode_constructor_args(
obj.__class__, method=(None, "model_construct"), kwargs=obj.model_dump()
)
elif hasattr(obj, "dict") and callable(obj.dict):
return self._encode_constructor_args(
obj.__class__, method=(None, "construct"), kwargs=obj.dict()
)
elif hasattr(obj, "_asdict") and callable(obj._asdict):
return self._encode_constructor_args(obj.__class__, kwargs=obj._asdict())
elif isinstance(obj, pathlib.Path):
return self._encode_constructor_args(pathlib.Path, args=obj.parts)
elif isinstance(obj, re.Pattern):
return self._encode_constructor_args(
re.compile, args=(obj.pattern, obj.flags)
)
elif isinstance(obj, UUID):
return self._encode_constructor_args(UUID, args=(obj.hex,))
elif isinstance(obj, decimal.Decimal):
return self._encode_constructor_args(decimal.Decimal, args=(str(obj),))
elif isinstance(obj, (set, frozenset, deque)):
return self._encode_constructor_args(type(obj), args=(tuple(obj),))
elif isinstance(obj, (IPv4Address, IPv4Interface, IPv4Network)):
return self._encode_constructor_args(obj.__class__, args=(str(obj),))
elif isinstance(obj, (IPv6Address, IPv6Interface, IPv6Network)):
return self._encode_constructor_args(obj.__class__, args=(str(obj),))
elif isinstance(obj, datetime):
return self._encode_constructor_args(
datetime, method="fromisoformat", args=(obj.isoformat(),)
)
elif isinstance(obj, timezone):
return self._encode_constructor_args(
timezone,
args=obj.__getinitargs__(), # type: ignore[attr-defined]
)
elif isinstance(obj, ZoneInfo):
return self._encode_constructor_args(ZoneInfo, args=(obj.key,))
elif isinstance(obj, timedelta):
return self._encode_constructor_args(
timedelta, args=(obj.days, obj.seconds, obj.microseconds)
)
elif isinstance(obj, date):
return self._encode_constructor_args(
date, args=(obj.year, obj.month, obj.day)
)
elif isinstance(obj, time):
return self._encode_constructor_args(
time,
args=(obj.hour, obj.minute, obj.second, obj.microsecond, obj.tzinfo),
kwargs={"fold": obj.fold},
)
elif dataclasses.is_dataclass(obj):
return self._encode_constructor_args(
obj.__class__,
kwargs={
field.name: getattr(obj, field.name)
for field in dataclasses.fields(obj)
},
)
elif isinstance(obj, Enum):
return self._encode_constructor_args(obj.__class__, args=(obj.value,))
elif isinstance(obj, SendProtocol):
return self._encode_constructor_args(
obj.__class__, kwargs={"node": obj.node, "arg": obj.arg}
)
elif isinstance(obj, (bytes, bytearray)):
return self._encode_constructor_args(
obj.__class__, method="fromhex", args=(obj.hex(),)
)
elif isinstance(obj, BaseException):
return repr(obj)
else:
raise TypeError(
f"Object of type {obj.__class__.__name__} is not JSON serializable"
)
def _reviver(self, value: dict[str, Any]) -> Any:
if self._allowed_modules and (
if (
value.get("lc", None) == 2
and value.get("type", None) == "constructor"
and value.get("id", None) is not None
):
try:
return self._revive_lc2(value)
except InvalidModuleError as e:
logger.warning(
"Object %s is not in the deserialization allowlist.\n%s",
value["id"],
e.message,
)
# Get module and class name
[*module, name] = value["id"]
# Import module
mod = importlib.import_module(".".join(module))
# Import class
cls = getattr(mod, name)
# Instantiate class
method = value.get("method")
if isinstance(method, str):
methods = [getattr(cls, method)]
elif isinstance(method, list):
methods = [
cls if method is None else getattr(cls, method)
for method in method
]
else:
methods = [cls]
args = value.get("args")
kwargs = value.get("kwargs")
for method in methods:
try:
if isclass(method) and issubclass(method, BaseException):
return None
if args and kwargs:
return method(*args, **kwargs)
elif args:
return method(*args)
elif kwargs:
return method(**kwargs)
else:
return method()
except Exception:
continue
except Exception:
return None
return LC_REVIVER(value)
return value
def _revive_lc2(self, value: dict[str, Any]) -> Any:
self._check_allowed_modules(value)
[*module, name] = value["id"]
try:
mod = importlib.import_module(".".join(module))
cls = getattr(mod, name)
method = value.get("method")
if isinstance(method, str):
methods = [getattr(cls, method)]
elif isinstance(method, list):
methods = [cls if m is None else getattr(cls, m) for m in method]
else:
methods = [cls]
args = value.get("args")
kwargs = value.get("kwargs")
for method in methods:
try:
if isclass(method) and issubclass(method, BaseException):
return None
if args and kwargs:
return method(*args, **kwargs)
elif args:
return method(*args)
elif kwargs:
return method(**kwargs)
else:
return method()
except Exception:
continue
except Exception:
return None
def _check_allowed_modules(self, value: dict[str, Any]) -> None:
needed = tuple(value["id"])
method = value.get("method")
if isinstance(method, list):
method_display = ",".join(m or "<init>" for m in method)
elif isinstance(method, str):
method_display = method
else:
method_display = "<init>"
dotted = ".".join(needed)
if not self._allowed_modules:
raise InvalidModuleError(
f"Refused to deserialize JSON constructor: {dotted} (method: {method_display}). "
"No allowed_json_modules configured.\n\n"
"Unblock with ONE of:\n"
f" • JsonPlusSerializer(allowed_json_modules=[{needed!r}, ...])\n"
" • (DANGEROUS) JsonPlusSerializer(allowed_json_modules=True)\n\n"
"Note: Prefix allowlists are intentionally unsupported; prefer exact symbols "
"or plain-JSON representations revived without import-time side effects."
)
if self._allowed_modules is True:
return
if needed in self._allowed_modules:
return
raise InvalidModuleError(
f"Refused to deserialize JSON constructor: {dotted} (method: {method_display}). "
"Symbol is not in the deserialization allowlist.\n\n"
"Add exactly this symbol to unblock:\n"
f" JsonPlusSerializer(allowed_json_modules=[{needed!r}, ...])\n"
"Or, as a last resort (DANGEROUS):\n"
" JsonPlusSerializer(allowed_json_modules=True)"
def dumps(self, obj: Any) -> bytes:
return json.dumps(obj, default=self._default, ensure_ascii=False).encode(
"utf-8", "ignore"
)
def dumps_typed(self, obj: Any) -> tuple[str, bytes]:
if obj is None:
return "null", EMPTY_BYTES
elif isinstance(obj, bytes):
if isinstance(obj, bytes):
return "bytes", obj
elif isinstance(obj, bytearray):
return "bytearray", obj
else:
try:
return "msgpack", _msgpack_enc(obj)
except ormsgpack.MsgpackEncodeError as exc:
if self.pickle_fallback:
return "pickle", pickle.dumps(obj)
raise exc
except UnicodeEncodeError:
return "json", self.dumps(obj)
def loads(self, data: bytes) -> Any:
return json.loads(data, object_hook=self._reviver)
def loads_typed(self, data: tuple[str, bytes]) -> Any:
type_, data_ = data
if type_ == "null":
return None
elif type_ == "bytes":
if type_ == "bytes":
return data_
elif type_ == "bytearray":
return bytearray(data_)
elif type_ == "json":
return json.loads(data_, object_hook=self._reviver)
return self.loads(data_)
elif type_ == "msgpack":
return ormsgpack.unpackb(
data_, ext_hook=self._unpack_ext_hook, option=ormsgpack.OPT_NON_STR_KEYS
return msgpack.unpackb(
data_, ext_hook=_msgpack_ext_hook, strict_map_key=False
)
elif self.pickle_fallback and type_ == "pickle":
return pickle.loads(data_)
else:
raise NotImplementedError(f"Unknown serialization type: {type_}")
@@ -218,12 +216,11 @@ EXT_CONSTRUCTOR_KW_ARGS = 2
EXT_METHOD_SINGLE_ARG = 3
EXT_PYDANTIC_V1 = 4
EXT_PYDANTIC_V2 = 5
EXT_NUMPY_ARRAY = 6
def _msgpack_default(obj: Any) -> str | ormsgpack.Ext:
def _msgpack_default(obj: Any) -> Union[str, msgpack.ExtType]:
if hasattr(obj, "model_dump") and callable(obj.model_dump): # pydantic v2
return ormsgpack.Ext(
return msgpack.ExtType(
EXT_PYDANTIC_V2,
_msgpack_enc(
(
@@ -235,7 +232,7 @@ def _msgpack_default(obj: Any) -> str | ormsgpack.Ext:
),
)
elif hasattr(obj, "get_secret_value") and callable(obj.get_secret_value):
return ormsgpack.Ext(
return msgpack.ExtType(
EXT_CONSTRUCTOR_SINGLE_ARG,
_msgpack_enc(
(
@@ -246,7 +243,7 @@ def _msgpack_default(obj: Any) -> str | ormsgpack.Ext:
),
)
elif hasattr(obj, "dict") and callable(obj.dict): # pydantic v1
return ormsgpack.Ext(
return msgpack.ExtType(
EXT_PYDANTIC_V1,
_msgpack_enc(
(
@@ -257,7 +254,7 @@ def _msgpack_default(obj: Any) -> str | ormsgpack.Ext:
),
)
elif hasattr(obj, "_asdict") and callable(obj._asdict): # namedtuple
return ormsgpack.Ext(
return msgpack.ExtType(
EXT_CONSTRUCTOR_KW_ARGS,
_msgpack_enc(
(
@@ -268,56 +265,56 @@ def _msgpack_default(obj: Any) -> str | ormsgpack.Ext:
),
)
elif isinstance(obj, pathlib.Path):
return ormsgpack.Ext(
return msgpack.ExtType(
EXT_CONSTRUCTOR_POS_ARGS,
_msgpack_enc(
(obj.__class__.__module__, obj.__class__.__name__, obj.parts),
),
)
elif isinstance(obj, re.Pattern):
return ormsgpack.Ext(
return msgpack.ExtType(
EXT_CONSTRUCTOR_POS_ARGS,
_msgpack_enc(
("re", "compile", (obj.pattern, obj.flags)),
),
)
elif isinstance(obj, UUID):
return ormsgpack.Ext(
return msgpack.ExtType(
EXT_CONSTRUCTOR_SINGLE_ARG,
_msgpack_enc(
(obj.__class__.__module__, obj.__class__.__name__, obj.hex),
),
)
elif isinstance(obj, decimal.Decimal):
return ormsgpack.Ext(
return msgpack.ExtType(
EXT_CONSTRUCTOR_SINGLE_ARG,
_msgpack_enc(
(obj.__class__.__module__, obj.__class__.__name__, str(obj)),
),
)
elif isinstance(obj, (set, frozenset, deque)):
return ormsgpack.Ext(
return msgpack.ExtType(
EXT_CONSTRUCTOR_SINGLE_ARG,
_msgpack_enc(
(obj.__class__.__module__, obj.__class__.__name__, tuple(obj)),
),
)
elif isinstance(obj, (IPv4Address, IPv4Interface, IPv4Network)):
return ormsgpack.Ext(
return msgpack.ExtType(
EXT_CONSTRUCTOR_SINGLE_ARG,
_msgpack_enc(
(obj.__class__.__module__, obj.__class__.__name__, str(obj)),
),
)
elif isinstance(obj, (IPv6Address, IPv6Interface, IPv6Network)):
return ormsgpack.Ext(
return msgpack.ExtType(
EXT_CONSTRUCTOR_SINGLE_ARG,
_msgpack_enc(
(obj.__class__.__module__, obj.__class__.__name__, str(obj)),
),
)
elif isinstance(obj, datetime):
return ormsgpack.Ext(
return msgpack.ExtType(
EXT_METHOD_SINGLE_ARG,
_msgpack_enc(
(
@@ -329,7 +326,7 @@ def _msgpack_default(obj: Any) -> str | ormsgpack.Ext:
),
)
elif isinstance(obj, timedelta):
return ormsgpack.Ext(
return msgpack.ExtType(
EXT_CONSTRUCTOR_POS_ARGS,
_msgpack_enc(
(
@@ -340,7 +337,7 @@ def _msgpack_default(obj: Any) -> str | ormsgpack.Ext:
),
)
elif isinstance(obj, date):
return ormsgpack.Ext(
return msgpack.ExtType(
EXT_CONSTRUCTOR_POS_ARGS,
_msgpack_enc(
(
@@ -351,7 +348,7 @@ def _msgpack_default(obj: Any) -> str | ormsgpack.Ext:
),
)
elif isinstance(obj, time):
return ormsgpack.Ext(
return msgpack.ExtType(
EXT_CONSTRUCTOR_KW_ARGS,
_msgpack_enc(
(
@@ -369,7 +366,7 @@ def _msgpack_default(obj: Any) -> str | ormsgpack.Ext:
),
)
elif isinstance(obj, timezone):
return ormsgpack.Ext(
return msgpack.ExtType(
EXT_CONSTRUCTOR_POS_ARGS,
_msgpack_enc(
(
@@ -380,21 +377,21 @@ def _msgpack_default(obj: Any) -> str | ormsgpack.Ext:
),
)
elif isinstance(obj, ZoneInfo):
return ormsgpack.Ext(
return msgpack.ExtType(
EXT_CONSTRUCTOR_SINGLE_ARG,
_msgpack_enc(
(obj.__class__.__module__, obj.__class__.__name__, obj.key),
),
)
elif isinstance(obj, Enum):
return ormsgpack.Ext(
return msgpack.ExtType(
EXT_CONSTRUCTOR_SINGLE_ARG,
_msgpack_enc(
(obj.__class__.__module__, obj.__class__.__name__, obj.value),
),
)
elif isinstance(obj, SendProtocol):
return ormsgpack.Ext(
return msgpack.ExtType(
EXT_CONSTRUCTOR_POS_ARGS,
_msgpack_enc(
(obj.__class__.__module__, obj.__class__.__name__, (obj.node, obj.arg)),
@@ -402,7 +399,7 @@ def _msgpack_default(obj: Any) -> str | ormsgpack.Ext:
)
elif dataclasses.is_dataclass(obj):
# doesn't use dataclasses.asdict to avoid deepcopy and recursion
return ormsgpack.Ext(
return msgpack.ExtType(
EXT_CONSTRUCTOR_KW_ARGS,
_msgpack_enc(
(
@@ -416,7 +413,7 @@ def _msgpack_default(obj: Any) -> str | ormsgpack.Ext:
),
)
elif isinstance(obj, Item):
return ormsgpack.Ext(
return msgpack.ExtType(
EXT_CONSTRUCTOR_KW_ARGS,
_msgpack_enc(
(
@@ -426,21 +423,6 @@ def _msgpack_default(obj: Any) -> str | ormsgpack.Ext:
),
),
)
elif (np_mod := sys.modules.get("numpy")) is not None and isinstance(
obj, np_mod.ndarray
):
order = "F" if obj.flags.f_contiguous and not obj.flags.c_contiguous else "C"
if obj.flags.c_contiguous:
mv = memoryview(obj)
try:
meta = (obj.dtype.str, obj.shape, order, mv)
return ormsgpack.Ext(EXT_NUMPY_ARRAY, _msgpack_enc(meta))
finally:
mv.release()
else:
buf = obj.tobytes(order="A")
meta = (obj.dtype.str, obj.shape, order, buf)
return ormsgpack.Ext(EXT_NUMPY_ARRAY, _msgpack_enc(meta))
elif isinstance(obj, BaseException):
return repr(obj)
@@ -451,8 +433,8 @@ def _msgpack_default(obj: Any) -> str | ormsgpack.Ext:
def _msgpack_ext_hook(code: int, data: bytes) -> Any:
if code == EXT_CONSTRUCTOR_SINGLE_ARG:
try:
tup = ormsgpack.unpackb(
data, ext_hook=_msgpack_ext_hook, option=ormsgpack.OPT_NON_STR_KEYS
tup = msgpack.unpackb(
data, ext_hook=_msgpack_ext_hook, strict_map_key=False
)
# module, name, arg
return getattr(importlib.import_module(tup[0]), tup[1])(tup[2])
@@ -460,8 +442,8 @@ def _msgpack_ext_hook(code: int, data: bytes) -> Any:
return
elif code == EXT_CONSTRUCTOR_POS_ARGS:
try:
tup = ormsgpack.unpackb(
data, ext_hook=_msgpack_ext_hook, option=ormsgpack.OPT_NON_STR_KEYS
tup = msgpack.unpackb(
data, ext_hook=_msgpack_ext_hook, strict_map_key=False
)
# module, name, args
return getattr(importlib.import_module(tup[0]), tup[1])(*tup[2])
@@ -469,8 +451,8 @@ def _msgpack_ext_hook(code: int, data: bytes) -> Any:
return
elif code == EXT_CONSTRUCTOR_KW_ARGS:
try:
tup = ormsgpack.unpackb(
data, ext_hook=_msgpack_ext_hook, option=ormsgpack.OPT_NON_STR_KEYS
tup = msgpack.unpackb(
data, ext_hook=_msgpack_ext_hook, strict_map_key=False
)
# module, name, args
return getattr(importlib.import_module(tup[0]), tup[1])(**tup[2])
@@ -478,8 +460,8 @@ def _msgpack_ext_hook(code: int, data: bytes) -> Any:
return
elif code == EXT_METHOD_SINGLE_ARG:
try:
tup = ormsgpack.unpackb(
data, ext_hook=_msgpack_ext_hook, option=ormsgpack.OPT_NON_STR_KEYS
tup = msgpack.unpackb(
data, ext_hook=_msgpack_ext_hook, strict_map_key=False
)
# module, name, arg, method
return getattr(getattr(importlib.import_module(tup[0]), tup[1]), tup[3])(
@@ -489,8 +471,8 @@ def _msgpack_ext_hook(code: int, data: bytes) -> Any:
return
elif code == EXT_PYDANTIC_V1:
try:
tup = ormsgpack.unpackb(
data, ext_hook=_msgpack_ext_hook, option=ormsgpack.OPT_NON_STR_KEYS
tup = msgpack.unpackb(
data, ext_hook=_msgpack_ext_hook, strict_map_key=False
)
# module, name, kwargs
cls = getattr(importlib.import_module(tup[0]), tup[1])
@@ -499,16 +481,11 @@ def _msgpack_ext_hook(code: int, data: bytes) -> Any:
except Exception:
return cls.construct(**tup[2])
except Exception:
# for pydantic objects we can't find/reconstruct
# let's return the kwargs dict instead
try:
return tup[2]
except NameError:
return
return
elif code == EXT_PYDANTIC_V2:
try:
tup = ormsgpack.unpackb(
data, ext_hook=_msgpack_ext_hook, option=ormsgpack.OPT_NON_STR_KEYS
tup = msgpack.unpackb(
data, ext_hook=_msgpack_ext_hook, strict_map_key=False
)
# module, name, kwargs, method
cls = getattr(importlib.import_module(tup[0]), tup[1])
@@ -516,135 +493,9 @@ def _msgpack_ext_hook(code: int, data: bytes) -> Any:
return cls(**tup[2])
except Exception:
return cls.model_construct(**tup[2])
except Exception:
# for pydantic objects we can't find/reconstruct
# let's return the kwargs dict instead
try:
return tup[2]
except NameError:
return
elif code == EXT_NUMPY_ARRAY:
try:
import numpy as _np
dtype_str, shape, order, buf = ormsgpack.unpackb(
data, ext_hook=_msgpack_ext_hook, option=ormsgpack.OPT_NON_STR_KEYS
)
arr = _np.frombuffer(buf, dtype=_np.dtype(dtype_str))
return arr.reshape(shape, order=order)
except Exception:
return
def _msgpack_ext_hook_to_json(code: int, data: bytes) -> Any:
if code == EXT_CONSTRUCTOR_SINGLE_ARG:
try:
tup = ormsgpack.unpackb(
data,
ext_hook=_msgpack_ext_hook_to_json,
option=ormsgpack.OPT_NON_STR_KEYS,
)
if tup[0] == "uuid" and tup[1] == "UUID":
hex_ = tup[2]
return (
f"{hex_[:8]}-{hex_[8:12]}-{hex_[12:16]}-{hex_[16:20]}-{hex_[20:]}"
)
# module, name, arg
return tup[2]
except Exception:
return
elif code == EXT_CONSTRUCTOR_POS_ARGS:
try:
tup = ormsgpack.unpackb(
data,
ext_hook=_msgpack_ext_hook_to_json,
option=ormsgpack.OPT_NON_STR_KEYS,
)
if tup[0] == "langgraph.types" and tup[1] == "Send":
from langgraph.types import Send # type: ignore
return Send(*tup[2])
# module, name, args
return tup[2]
except Exception:
return
elif code == EXT_CONSTRUCTOR_KW_ARGS:
try:
tup = ormsgpack.unpackb(
data,
ext_hook=_msgpack_ext_hook_to_json,
option=ormsgpack.OPT_NON_STR_KEYS,
)
# module, name, args
return tup[2]
except Exception:
return
elif code == EXT_METHOD_SINGLE_ARG:
try:
tup = ormsgpack.unpackb(
data,
ext_hook=_msgpack_ext_hook_to_json,
option=ormsgpack.OPT_NON_STR_KEYS,
)
# module, name, arg, method
return tup[2]
except Exception:
return
elif code == EXT_PYDANTIC_V1:
try:
tup = ormsgpack.unpackb(
data,
ext_hook=_msgpack_ext_hook_to_json,
option=ormsgpack.OPT_NON_STR_KEYS,
)
# module, name, kwargs
return tup[2]
except Exception:
# for pydantic objects we can't find/reconstruct
# let's return the kwargs dict instead
return
elif code == EXT_PYDANTIC_V2:
try:
tup = ormsgpack.unpackb(
data,
ext_hook=_msgpack_ext_hook_to_json,
option=ormsgpack.OPT_NON_STR_KEYS,
)
# module, name, kwargs, method
return tup[2]
except Exception:
return
elif code == EXT_NUMPY_ARRAY:
try:
import numpy as _np
dtype_str, shape, order, buf = ormsgpack.unpackb(
data,
ext_hook=_msgpack_ext_hook_to_json,
option=ormsgpack.OPT_NON_STR_KEYS,
)
arr = _np.frombuffer(buf, dtype=_np.dtype(dtype_str))
return arr.reshape(shape, order=order).tolist()
except Exception:
return
class InvalidModuleError(Exception):
"""Exception raised when a module is not in the allowlist."""
def __init__(self, message: str):
self.message = message
_option = (
ormsgpack.OPT_NON_STR_KEYS
| ormsgpack.OPT_PASSTHROUGH_DATACLASS
| ormsgpack.OPT_PASSTHROUGH_DATETIME
| ormsgpack.OPT_PASSTHROUGH_ENUM
| ormsgpack.OPT_PASSTHROUGH_UUID
| ormsgpack.OPT_REPLACE_SURROGATES
)
def _msgpack_enc(data: Any) -> bytes:
return ormsgpack.packb(data, default=_msgpack_default, option=_option)
return msgpack.packb(data, default=_msgpack_default)
@@ -1,7 +1,8 @@
from collections.abc import Sequence
from typing import (
Any,
Optional,
Protocol,
Sequence,
TypeVar,
runtime_checkable,
)
@@ -27,9 +28,9 @@ class ChannelProtocol(Protocol[Value, Update, C]):
@property
def UpdateType(self) -> Any: ...
def checkpoint(self) -> C | None: ...
def checkpoint(self) -> Optional[C]: ...
def from_checkpoint(self, checkpoint: C | None) -> Self: ...
def from_checkpoint(self, checkpoint: Optional[C]) -> Self: ...
def update(self, values: Sequence[Update]) -> bool: ...
File diff suppressed because it is too large. Load diff
@@ -1,365 +0,0 @@
"""Utilities for batching operations in a background task."""
from __future__ import annotations
import asyncio
import functools
import weakref
from collections.abc import Callable, Iterable
from typing import Any, Literal, TypeVar
from langgraph.store.base import (
NOT_PROVIDED,
BaseStore,
GetOp,
Item,
ListNamespacesOp,
MatchCondition,
NamespacePath,
NotProvided,
Op,
PutOp,
Result,
SearchItem,
SearchOp,
_ensure_refresh,
_ensure_ttl,
_validate_namespace,
)
F = TypeVar("F", bound=Callable)
def _check_loop(func: F) -> F:
@functools.wraps(func)
def wrapper(store: AsyncBatchedBaseStore, *args: Any, **kwargs: Any) -> Any:
method_name: str = func.__name__
try:
current_loop = asyncio.get_running_loop()
if current_loop is store._loop:
replacement_str = (
f"Specifically, replace `store.{method_name}(...)` with `await store.a{method_name}(...)"
if method_name
else "For example, replace `store.get(...)` with `await store.aget(...)`"
)
raise asyncio.InvalidStateError(
f"Synchronous calls to {store.__class__.__name__} detected in the main event loop. "
"This can lead to deadlocks or performance issues. "
"Please use the asynchronous interface for main thread operations. "
f"{replacement_str} "
)
except RuntimeError:
pass
return func(store, *args, **kwargs)
return wrapper
class AsyncBatchedBaseStore(BaseStore):
"""Efficiently batch operations in a background task."""
__slots__ = ("_loop", "_aqueue", "_task")
def __init__(self) -> None:
super().__init__()
self._loop = asyncio.get_running_loop()
self._aqueue: asyncio.Queue[tuple[asyncio.Future, Op]] = asyncio.Queue()
self._task: asyncio.Task | None = None
self._ensure_task()
def __del__(self) -> None:
try:
if self._task:
self._task.cancel()
except RuntimeError:
pass
def _ensure_task(self) -> None:
"""Ensure the background processing loop is running."""
if self._task is None or self._task.done():
self._task = self._loop.create_task(_run(self._aqueue, weakref.ref(self)))
async def aget(
self,
namespace: tuple[str, ...],
key: str,
*,
refresh_ttl: bool | None = None,
) -> Item | None:
self._ensure_task()
fut = self._loop.create_future()
self._aqueue.put_nowait(
(
fut,
GetOp(
namespace,
key,
refresh_ttl=_ensure_refresh(self.ttl_config, refresh_ttl),
),
)
)
return await fut
async def asearch(
self,
namespace_prefix: tuple[str, ...],
/,
*,
query: str | None = None,
filter: dict[str, Any] | None = None,
limit: int = 10,
offset: int = 0,
refresh_ttl: bool | None = None,
) -> list[SearchItem]:
self._ensure_task()
fut = self._loop.create_future()
self._aqueue.put_nowait(
(
fut,
SearchOp(
namespace_prefix,
filter,
limit,
offset,
query,
refresh_ttl=_ensure_refresh(self.ttl_config, refresh_ttl),
),
)
)
return await fut
async def aput(
self,
namespace: tuple[str, ...],
key: str,
value: dict[str, Any],
index: Literal[False] | list[str] | None = None,
*,
ttl: float | None | NotProvided = NOT_PROVIDED,
) -> None:
self._ensure_task()
_validate_namespace(namespace)
fut = self._loop.create_future()
self._aqueue.put_nowait(
(
fut,
PutOp(
namespace, key, value, index, ttl=_ensure_ttl(self.ttl_config, ttl)
),
)
)
return await fut
async def adelete(
self,
namespace: tuple[str, ...],
key: str,
) -> None:
self._ensure_task()
fut = self._loop.create_future()
self._aqueue.put_nowait((fut, PutOp(namespace, key, None)))
return await fut
async def alist_namespaces(
self,
*,
prefix: NamespacePath | None = None,
suffix: NamespacePath | None = None,
max_depth: int | None = None,
limit: int = 100,
offset: int = 0,
) -> list[tuple[str, ...]]:
self._ensure_task()
fut = self._loop.create_future()
match_conditions = []
if prefix:
match_conditions.append(MatchCondition(match_type="prefix", path=prefix))
if suffix:
match_conditions.append(MatchCondition(match_type="suffix", path=suffix))
op = ListNamespacesOp(
match_conditions=tuple(match_conditions),
max_depth=max_depth,
limit=limit,
offset=offset,
)
self._aqueue.put_nowait((fut, op))
return await fut
@_check_loop
def batch(self, ops: Iterable[Op]) -> list[Result]:
return asyncio.run_coroutine_threadsafe(self.abatch(ops), self._loop).result()
@_check_loop
def get(
self,
namespace: tuple[str, ...],
key: str,
*,
refresh_ttl: bool | None = None,
) -> Item | None:
return asyncio.run_coroutine_threadsafe(
self.aget(namespace, key=key, refresh_ttl=refresh_ttl), self._loop
).result()
@_check_loop
def search(
self,
namespace_prefix: tuple[str, ...],
/,
*,
query: str | None = None,
filter: dict[str, Any] | None = None,
limit: int = 10,
offset: int = 0,
refresh_ttl: bool | None = None,
) -> list[SearchItem]:
return asyncio.run_coroutine_threadsafe(
self.asearch(
namespace_prefix,
query=query,
filter=filter,
limit=limit,
offset=offset,
refresh_ttl=refresh_ttl,
),
self._loop,
).result()
@_check_loop
def put(
self,
namespace: tuple[str, ...],
key: str,
value: dict[str, Any],
index: Literal[False] | list[str] | None = None,
*,
ttl: float | None | NotProvided = NOT_PROVIDED,
) -> None:
_validate_namespace(namespace)
asyncio.run_coroutine_threadsafe(
self.aput(
namespace,
key=key,
value=value,
index=index,
ttl=_ensure_ttl(self.ttl_config, ttl),
),
self._loop,
).result()
@_check_loop
def delete(
self,
namespace: tuple[str, ...],
key: str,
) -> None:
asyncio.run_coroutine_threadsafe(
self.adelete(namespace, key=key), self._loop
).result()
@_check_loop
def list_namespaces(
self,
*,
prefix: NamespacePath | None = None,
suffix: NamespacePath | None = None,
max_depth: int | None = None,
limit: int = 100,
offset: int = 0,
) -> list[tuple[str, ...]]:
return asyncio.run_coroutine_threadsafe(
self.alist_namespaces(
prefix=prefix,
suffix=suffix,
max_depth=max_depth,
limit=limit,
offset=offset,
),
self._loop,
).result()
def _dedupe_ops(values: list[Op]) -> tuple[list[int] | None, list[Op]]:
"""Dedupe operations while preserving order for results.
Args:
values: List of operations to dedupe
Returns:
Tuple of (listen indices, deduped operations)
where listen indices map deduped operation results back to original positions
"""
if len(values) <= 1:
return None, list(values)
dedupped: list[Op] = []
listen: list[int] = []
puts: dict[tuple[tuple[str, ...], str], int] = {}
for op in values:
if isinstance(op, (GetOp, SearchOp, ListNamespacesOp)):
try:
listen.append(dedupped.index(op))
except ValueError:
listen.append(len(dedupped))
dedupped.append(op)
elif isinstance(op, PutOp):
putkey = (op.namespace, op.key)
if putkey in puts:
# Overwrite previous put
ix = puts[putkey]
dedupped[ix] = op
listen.append(ix)
else:
puts[putkey] = len(dedupped)
listen.append(len(dedupped))
dedupped.append(op)
else: # Any new ops will be treated regularly
listen.append(len(dedupped))
dedupped.append(op)
return listen, dedupped
async def _run(
aqueue: asyncio.Queue[tuple[asyncio.Future, Op]],
store: weakref.ReferenceType[BaseStore],
) -> None:
while item := await aqueue.get():
# check if store is still alive
if s := store():
try:
# accumulate operations scheduled in same tick
items = [item]
try:
while item := aqueue.get_nowait():
items.append(item)
except asyncio.QueueEmpty:
pass
# get the operations to run
futs = [item[0] for item in items]
values = [item[1] for item in items]
# action each operation
try:
listen, dedupped = _dedupe_ops(values)
results = await s.abatch(dedupped)
if listen is not None:
results = [results[ix] for ix in listen]
# set the results of each operation
for fut, result in zip(futs, results, strict=False):
# guard against future being done (e.g. cancelled)
if not fut.done():
fut.set_result(result)
except Exception as e:
for fut in futs:
# guard against future being done (e.g. cancelled)
if not fut.done():
fut.set_exception(e)
finally:
# remove strong ref to store
del s
else:
break
+14 -121
View File
@@ -6,15 +6,9 @@ with LangChain-compatible tools while maintaining support for both synchronous a
asynchronous operations.
"""
from __future__ import annotations
import asyncio
import functools
import json
from collections.abc import Awaitable, Callable, Sequence
from typing import Any
from langchain_core.embeddings import Embeddings
from typing import Any, Callable, Optional, Sequence, Union
EmbeddingsFunc = Callable[[Sequence[str]], list[list[float]]]
"""Type for synchronous embedding functions.
@@ -24,16 +18,10 @@ where each embedding is a list of floats. The dimensionality of the embeddings
should be consistent for all inputs.
"""
AEmbeddingsFunc = Callable[[Sequence[str]], Awaitable[list[list[float]]]]
"""Type for asynchronous embedding functions.
Similar to EmbeddingsFunc, but returns an awaitable that resolves to the embeddings.
"""
def ensure_embeddings(
embed: Embeddings | EmbeddingsFunc | AEmbeddingsFunc | str | None,
) -> Embeddings:
embed: Union[EmbeddingsFunc, str, None],
) -> "Embeddings":
"""Ensure that an embedding function conforms to LangChain's Embeddings interface.
This function wraps arbitrary embedding functions to make them compatible with
@@ -49,9 +37,7 @@ def ensure_embeddings(
An Embeddings instance that wraps the provided function(s).
??? example "Examples"
Wrap a synchronous embedding function:
```python
def my_embed_fn(texts):
return [[0.1, 0.2] for _ in texts]
@@ -60,18 +46,7 @@ def ensure_embeddings(
result = embeddings.embed_query("hello") # Returns [0.1, 0.2]
```
Wrap an asynchronous embedding function:
```python
async def my_async_fn(texts):
return [[0.1, 0.2] for _ in texts]
embeddings = ensure_embeddings(my_async_fn)
result = await embeddings.aembed_query("hello") # Returns [0.1, 0.2]
```
Initialize embeddings using a provider string:
```python
# Requires langchain>=0.3.9 and langgraph-checkpoint>=2.0.11
embeddings = ensure_embeddings("openai:text-embedding-3-small")
@@ -103,10 +78,10 @@ def ensure_embeddings(
if isinstance(embed, Embeddings):
return embed
return EmbeddingsLambda(embed)
return Embeddings(embed)
class EmbeddingsLambda(Embeddings):
class Embeddings:
"""Wrapper to convert embedding functions into LangChain's Embeddings interface.
This class allows arbitrary embedding functions to be used with LangChain-compatible
@@ -123,9 +98,7 @@ class EmbeddingsLambda(Embeddings):
will raise an error. If sync, it will be used for both sync and async operations.
??? example "Examples"
With a sync function:
```python
def my_embed_fn(texts):
# Return 2D embeddings for each text
@@ -135,27 +108,14 @@ class EmbeddingsLambda(Embeddings):
result = embeddings.embed_query("hello") # Returns [0.1, 0.2]
await embeddings.aembed_query("hello") # Also returns [0.1, 0.2]
```
With an async function:
```python
async def my_async_fn(texts):
return [[0.1, 0.2] for _ in texts]
embeddings = EmbeddingsLambda(my_async_fn)
await embeddings.aembed_query("hello") # Returns [0.1, 0.2]
# Note: embed_query() would raise an error
```
"""
def __init__(
self,
func: EmbeddingsFunc | AEmbeddingsFunc,
func: EmbeddingsFunc,
) -> None:
if func is None:
raise ValueError("func must be provided")
if _is_async_callable(func):
self.afunc = func
else:
self.func = func
@@ -168,16 +128,8 @@ class EmbeddingsLambda(Embeddings):
Returns:
list of embeddings, one per input text. Each embedding is a list of floats.
Raises:
ValueError: If the instance was initialized with only an async function.
"""
func = getattr(self, "func", None)
if func is None:
raise ValueError(
"EmbeddingsLambda was initialized with an async function but no sync function. "
"Use aembed_documents for async operation or provide a sync function."
)
return func(texts)
return self.func(texts)
def embed_query(self, text: str) -> list[float]:
"""Embed a single piece of text.
@@ -194,43 +146,8 @@ class EmbeddingsLambda(Embeddings):
"""
return self.embed_documents([text])[0]
async def aembed_documents(self, texts: list[str]) -> list[list[float]]:
"""Asynchronously embed a list of texts into vectors.
Args:
texts: list of texts to convert to embeddings.
Returns:
list of embeddings, one per input text. Each embedding is a list of floats.
Note:
If no async function was provided, this falls back to the sync implementation.
"""
afunc = getattr(self, "afunc", None)
if afunc is None:
return await super().aembed_documents(texts)
return await afunc(texts)
async def aembed_query(self, text: str) -> list[float]:
"""Asynchronously embed a single piece of text.
Args:
text: Text to convert to an embedding.
Returns:
Embedding vector as a list of floats.
Note:
This is equivalent to calling aembed_documents with a single text
and taking the first result.
"""
afunc = getattr(self, "afunc", None)
if afunc is None:
return await super().aembed_query(text)
return (await afunc([text]))[0]
def get_text_at_path(obj: Any, path: str | list[str]) -> list[str]:
def get_text_at_path(obj: Any, path: Union[str, list[str]]) -> list[str]:
"""Extract text from an object using a path expression or pre-tokenized path.
Args:
@@ -245,7 +162,7 @@ def get_text_at_path(obj: Any, path: str | list[str]) -> list[str]:
- Nested paths in multi-field: "{field1,nested.field2}"
"""
if not path or path == "$":
return [json.dumps(obj, sort_keys=True, ensure_ascii=False)]
return [json.dumps(obj, sort_keys=True)]
tokens = tokenize_path(path) if isinstance(path, str) else path
@@ -256,7 +173,7 @@ def get_text_at_path(obj: Any, path: str | list[str]) -> list[str]:
elif obj is None:
return []
elif isinstance(obj, (list, dict)):
return [json.dumps(obj, sort_keys=True, ensure_ascii=False)]
return [json.dumps(obj, sort_keys=True)]
return []
token = tokens[pos]
@@ -288,7 +205,7 @@ def get_text_at_path(obj: Any, path: str | list[str]) -> list[str]:
for field in fields:
nested_tokens = tokenize_path(field)
if nested_tokens:
current_obj: dict | None = obj
current_obj: Optional[dict] = obj
for nested_token in nested_tokens:
if (
isinstance(current_obj, dict)
@@ -302,11 +219,7 @@ def get_text_at_path(obj: Any, path: str | list[str]) -> list[str]:
if isinstance(current_obj, (str, int, float, bool)):
results.append(str(current_obj))
elif isinstance(current_obj, (list, dict)):
results.append(
json.dumps(
current_obj, sort_keys=True, ensure_ascii=False
)
)
results.append(json.dumps(current_obj, sort_keys=True))
# Handle wildcard
elif token == "*":
@@ -396,28 +309,8 @@ def tokenize_path(path: str) -> list[str]:
return tokens
def _is_async_callable(
func: Any,
) -> bool:
"""Check if a function is async.
This includes both async def functions and classes with async __call__ methods.
Args:
func: Function or callable object to check.
Returns:
True if the function is async, False otherwise.
"""
return (
asyncio.iscoroutinefunction(func)
or hasattr(func, "__call__") # noqa: B004
and asyncio.iscoroutinefunction(func.__call__)
)
@functools.lru_cache
def _get_init_embeddings() -> Callable[[str], Embeddings] | None:
def _get_init_embeddings() -> Optional[Callable[[str], Embeddings]]:
try:
from langchain.embeddings import init_embeddings # type: ignore
@@ -429,5 +322,5 @@ def _get_init_embeddings() -> Callable[[str], Embeddings] | None:
__all__ = [
"ensure_embeddings",
"EmbeddingsFunc",
"AEmbeddingsFunc",
"Embeddings",
]
@@ -59,35 +59,6 @@
results = store.search(("docs",), query="python programming")
```
Async vector search using OpenAI SDK:
```python
from openai import AsyncOpenAI
from langgraph.store.memory import InMemoryStore
client = AsyncOpenAI()
async def aembed_texts(texts: list[str]) -> list[list[float]]:
response = await client.embeddings.create(
model="text-embedding-3-small",
input=texts
)
return [e.embedding for e in response.data]
store = InMemoryStore(
index={
"dims": 1536,
"embed": aembed_texts
}
)
# Store documents
await store.aput(("docs",), "doc1", {"text": "Python tutorial"})
await store.aput(("docs",), "doc2", {"text": "TypeScript guide"})
# Search by similarity
results = await store.asearch(("docs",), query="python programming")
```
Warning:
This store keeps all data in memory. Data is lost when the process exits.
For persistence, use a database-backed store like PostgresStore.
@@ -99,22 +70,17 @@ Tip:
```
"""
from __future__ import annotations
import asyncio
import concurrent.futures as cf
import functools
import logging
from collections import defaultdict
from collections.abc import Iterable
from datetime import datetime, timezone
from importlib import util
from typing import Any
from langchain_core.embeddings import Embeddings
from typing import Any, Iterable, Optional
from langgraph.store.base import (
BaseStore,
Embeddings,
GetOp,
IndexConfig,
Item,
@@ -180,7 +146,7 @@ class InMemoryStore(BaseStore):
"embeddings",
)
def __init__(self, *, index: IndexConfig | None = None) -> None:
def __init__(self, *, index: Optional[IndexConfig] = None) -> None:
# Both _data and _vectors are wrapped in the In-memory API
# Do not change their names
self._data: dict[tuple[str, ...], dict[str, Item]] = defaultdict(dict)
@@ -191,7 +157,7 @@ class InMemoryStore(BaseStore):
self.index_config = index
if self.index_config:
self.index_config = self.index_config.copy()
self.embeddings: Embeddings | None = ensure_embeddings(
self.embeddings: Optional[Embeddings] = ensure_embeddings(
self.index_config.get("embed"),
)
self.index_config["__tokenized_fields"] = [
@@ -218,21 +184,6 @@ class InMemoryStore(BaseStore):
self._apply_put_ops(put_ops)
return results
async def abatch(self, ops: Iterable[Op]) -> list[Result]:
# The batch/abatch methods are treated as internal.
# Users should access via put/search/get/list_namespaces/etc.
results, put_ops, search_ops = self._prepare_ops(ops)
if search_ops:
queryinmem_store = await self._aembed_search_queries(search_ops)
self._batch_search(search_ops, queryinmem_store, results)
to_embed = self._extract_texts(put_ops)
if to_embed and self.index_config and self.embeddings:
embeddings = await self.embeddings.aembed_documents(list(to_embed))
self._insertinmem_store(to_embed, embeddings)
self._apply_put_ops(put_ops)
return results
# Helpers
def _filter_items(self, op: SearchOp) -> list[tuple[Item, list[list[float]]]]:
@@ -284,21 +235,6 @@ class InMemoryStore(BaseStore):
return queryinmem_store
async def _aembed_search_queries(
self,
search_ops: dict[int, tuple[SearchOp, list[tuple[Item, list[list[float]]]]]],
) -> dict[str, list[float]]:
queryinmem_store = {}
if self.index_config and self.embeddings and search_ops:
queries = {op.query for (op, _) in search_ops.values() if op.query}
if queries:
coros = [self.embeddings.aembed_query(q) for q in list(queries)]
results = await asyncio.gather(*coros)
queryinmem_store = dict(zip(queries, results, strict=False))
return queryinmem_store
def _batch_search(
self,
ops: dict[int, tuple[SearchOp, list[tuple[Item, list[list[float]]]]]],
@@ -323,13 +259,11 @@ class InMemoryStore(BaseStore):
scores = _cosine_similarity(query_embedding, flat_vectors)
sorted_results = sorted(
zip(scores, flat_items, strict=False),
key=lambda x: x[0],
reverse=True,
zip(scores, flat_items), key=lambda x: x[0], reverse=True
)
# max pooling
seen: set[tuple[tuple[str, ...], str]] = set()
kept: list[tuple[float | None, Item]] = []
kept: list[tuple[Optional[float], Item]] = []
for score, item in sorted_results:
key = (item.namespace, item.key)
if key in seen:
@@ -454,7 +388,7 @@ class InMemoryStore(BaseStore):
f"Number of embeddings ({len(embeddings)}) does not"
f" match number of indices ({len(indices)})"
)
for embedding, (ns, key, path) in zip(embeddings, indices, strict=False):
for embedding, (ns, key, path) in zip(embeddings, indices):
self._vectors[ns][key][path] = embedding
def _handle_list_namespaces(self, op: ListNamespacesOp) -> list[tuple[str, ...]]:
@@ -498,7 +432,7 @@ def _cosine_similarity(X: list[float], Y: list[list[float]]) -> list[float]:
if not Y:
return []
if _check_numpy():
import numpy as np
import numpy as np # type: ignore[import-not-found]
X_arr = np.array(X) if not isinstance(X, np.ndarray) else X
Y_arr = np.array(Y) if not isinstance(Y, np.ndarray) else Y
@@ -513,7 +447,7 @@ def _cosine_similarity(X: list[float], Y: list[list[float]]) -> list[float]:
similarities = []
for y in Y:
dot_product = sum(a * b for a, b in zip(X, y, strict=False))
dot_product = sum(a * b for a, b in zip(X, y))
norm1 = sum(a * a for a in X) ** 0.5
norm2 = sum(a * a for a in y) ** 0.5
similarity = dot_product / (norm1 * norm2) if norm1 > 0 and norm2 > 0 else 0.0
@@ -531,14 +465,14 @@ def _does_match(match_condition: MatchCondition, key: tuple[str, ...]) -> bool:
return False
if match_type == "prefix":
for k_elem, p_elem in zip(key, path, strict=False):
for k_elem, p_elem in zip(key, path):
if p_elem == "*":
continue # Wildcard matches any element
if k_elem != p_elem:
return False
return True
elif match_type == "suffix":
for k_elem, p_elem in zip(reversed(key), reversed(path), strict=False):
for k_elem, p_elem in zip(reversed(key), reversed(path)):
if p_elem == "*":
continue # Wildcard matches any element
if k_elem != p_elem:
@@ -565,10 +499,7 @@ def _compare_values(item_value: Any, filter_value: Any) -> bool:
return (
isinstance(item_value, (list, tuple))
and len(item_value) == len(filter_value)
and all(
_compare_values(iv, fv)
for iv, fv in zip(item_value, filter_value, strict=False)
)
and all(_compare_values(iv, fv) for iv, fv in zip(item_value, filter_value))
)
else:
return item_value == filter_value
+927
View File
@@ -0,0 +1,927 @@
# This file is automatically @generated by Poetry 1.8.2 and should not be changed by hand.
[[package]]
name = "annotated-types"
version = "0.7.0"
description = "Reusable constraint types to use with typing.Annotated"
optional = false
python-versions = ">=3.8"
files = [
{file = "annotated_types-0.7.0-py3-none-any.whl", hash = "sha256:1f02e8b43a8fbbc3f3e0d4f0f4bfc8131bcb4eebe8849b8e5c773f3a1c582a53"},
{file = "annotated_types-0.7.0.tar.gz", hash = "sha256:aff07c09a53a08bc8cfccb9c85b05f1aa9a2a6f23728d790723543408344ce89"},
]
[[package]]
name = "certifi"
version = "2024.7.4"
description = "Python package for providing Mozilla's CA Bundle."
optional = false
python-versions = ">=3.6"
files = [
{file = "certifi-2024.7.4-py3-none-any.whl", hash = "sha256:c198e21b1289c2ab85ee4e67bb4b4ef3ead0892059901a8d5b622f24a1101e90"},
{file = "certifi-2024.7.4.tar.gz", hash = "sha256:5a1e7645bc0ec61a09e26c36f6106dd4cf40c6db3a1fb6352b0244e7fb057c7b"},
]
[[package]]
name = "charset-normalizer"
version = "3.3.2"
description = "The Real First Universal Charset Detector. Open, modern and actively maintained alternative to Chardet."
optional = false
python-versions = ">=3.7.0"
files = [
{file = "charset-normalizer-3.3.2.tar.gz", hash = "sha256:f30c3cb33b24454a82faecaf01b19c18562b1e89558fb6c56de4d9118a032fd5"},
{file = "charset_normalizer-3.3.2-cp310-cp310-macosx_10_9_universal2.whl", hash = "sha256:25baf083bf6f6b341f4121c2f3c548875ee6f5339300e08be3f2b2ba1721cdd3"},
{file = "charset_normalizer-3.3.2-cp310-cp310-macosx_10_9_x86_64.whl", hash = "sha256:06435b539f889b1f6f4ac1758871aae42dc3a8c0e24ac9e60c2384973ad73027"},
{file = "charset_normalizer-3.3.2-cp310-cp310-macosx_11_0_arm64.whl", hash = "sha256:9063e24fdb1e498ab71cb7419e24622516c4a04476b17a2dab57e8baa30d6e03"},
{file = "charset_normalizer-3.3.2-cp310-cp310-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:6897af51655e3691ff853668779c7bad41579facacf5fd7253b0133308cf000d"},
{file = "charset_normalizer-3.3.2-cp310-cp310-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", hash = "sha256:1d3193f4a680c64b4b6a9115943538edb896edc190f0b222e73761716519268e"},
{file = "charset_normalizer-3.3.2-cp310-cp310-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:cd70574b12bb8a4d2aaa0094515df2463cb429d8536cfb6c7ce983246983e5a6"},
{file = "charset_normalizer-3.3.2-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:8465322196c8b4d7ab6d1e049e4c5cb460d0394da4a27d23cc242fbf0034b6b5"},
{file = "charset_normalizer-3.3.2-cp310-cp310-manylinux_2_5_i686.manylinux1_i686.manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:a9a8e9031d613fd2009c182b69c7b2c1ef8239a0efb1df3f7c8da66d5dd3d537"},
{file = "charset_normalizer-3.3.2-cp310-cp310-musllinux_1_1_aarch64.whl", hash = "sha256:beb58fe5cdb101e3a055192ac291b7a21e3b7ef4f67fa1d74e331a7f2124341c"},
{file = "charset_normalizer-3.3.2-cp310-cp310-musllinux_1_1_i686.whl", hash = "sha256:e06ed3eb3218bc64786f7db41917d4e686cc4856944f53d5bdf83a6884432e12"},
{file = "charset_normalizer-3.3.2-cp310-cp310-musllinux_1_1_ppc64le.whl", hash = "sha256:2e81c7b9c8979ce92ed306c249d46894776a909505d8f5a4ba55b14206e3222f"},
{file = "charset_normalizer-3.3.2-cp310-cp310-musllinux_1_1_s390x.whl", hash = "sha256:572c3763a264ba47b3cf708a44ce965d98555f618ca42c926a9c1616d8f34269"},
{file = "charset_normalizer-3.3.2-cp310-cp310-musllinux_1_1_x86_64.whl", hash = "sha256:fd1abc0d89e30cc4e02e4064dc67fcc51bd941eb395c502aac3ec19fab46b519"},
{file = "charset_normalizer-3.3.2-cp310-cp310-win32.whl", hash = "sha256:3d47fa203a7bd9c5b6cee4736ee84ca03b8ef23193c0d1ca99b5089f72645c73"},
{file = "charset_normalizer-3.3.2-cp310-cp310-win_amd64.whl", hash = "sha256:10955842570876604d404661fbccbc9c7e684caf432c09c715ec38fbae45ae09"},
{file = "charset_normalizer-3.3.2-cp311-cp311-macosx_10_9_universal2.whl", hash = "sha256:802fe99cca7457642125a8a88a084cef28ff0cf9407060f7b93dca5aa25480db"},
{file = "charset_normalizer-3.3.2-cp311-cp311-macosx_10_9_x86_64.whl", hash = "sha256:573f6eac48f4769d667c4442081b1794f52919e7edada77495aaed9236d13a96"},
{file = "charset_normalizer-3.3.2-cp311-cp311-macosx_11_0_arm64.whl", hash = "sha256:549a3a73da901d5bc3ce8d24e0600d1fa85524c10287f6004fbab87672bf3e1e"},
{file = "charset_normalizer-3.3.2-cp311-cp311-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:f27273b60488abe721a075bcca6d7f3964f9f6f067c8c4c605743023d7d3944f"},
{file = "charset_normalizer-3.3.2-cp311-cp311-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", hash = "sha256:1ceae2f17a9c33cb48e3263960dc5fc8005351ee19db217e9b1bb15d28c02574"},
{file = "charset_normalizer-3.3.2-cp311-cp311-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:65f6f63034100ead094b8744b3b97965785388f308a64cf8d7c34f2f2e5be0c4"},
{file = "charset_normalizer-3.3.2-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:753f10e867343b4511128c6ed8c82f7bec3bd026875576dfd88483c5c73b2fd8"},
{file = "charset_normalizer-3.3.2-cp311-cp311-manylinux_2_5_i686.manylinux1_i686.manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:4a78b2b446bd7c934f5dcedc588903fb2f5eec172f3d29e52a9096a43722adfc"},
{file = "charset_normalizer-3.3.2-cp311-cp311-musllinux_1_1_aarch64.whl", hash = "sha256:e537484df0d8f426ce2afb2d0f8e1c3d0b114b83f8850e5f2fbea0e797bd82ae"},
{file = "charset_normalizer-3.3.2-cp311-cp311-musllinux_1_1_i686.whl", hash = "sha256:eb6904c354526e758fda7167b33005998fb68c46fbc10e013ca97f21ca5c8887"},
{file = "charset_normalizer-3.3.2-cp311-cp311-musllinux_1_1_ppc64le.whl", hash = "sha256:deb6be0ac38ece9ba87dea880e438f25ca3eddfac8b002a2ec3d9183a454e8ae"},
{file = "charset_normalizer-3.3.2-cp311-cp311-musllinux_1_1_s390x.whl", hash = "sha256:4ab2fe47fae9e0f9dee8c04187ce5d09f48eabe611be8259444906793ab7cbce"},
{file = "charset_normalizer-3.3.2-cp311-cp311-musllinux_1_1_x86_64.whl", hash = "sha256:80402cd6ee291dcb72644d6eac93785fe2c8b9cb30893c1af5b8fdd753b9d40f"},
{file = "charset_normalizer-3.3.2-cp311-cp311-win32.whl", hash = "sha256:7cd13a2e3ddeed6913a65e66e94b51d80a041145a026c27e6bb76c31a853c6ab"},
{file = "charset_normalizer-3.3.2-cp311-cp311-win_amd64.whl", hash = "sha256:663946639d296df6a2bb2aa51b60a2454ca1cb29835324c640dafb5ff2131a77"},
{file = "charset_normalizer-3.3.2-cp312-cp312-macosx_10_9_universal2.whl", hash = "sha256:0b2b64d2bb6d3fb9112bafa732def486049e63de9618b5843bcdd081d8144cd8"},
{file = "charset_normalizer-3.3.2-cp312-cp312-macosx_10_9_x86_64.whl", hash = "sha256:ddbb2551d7e0102e7252db79ba445cdab71b26640817ab1e3e3648dad515003b"},
{file = "charset_normalizer-3.3.2-cp312-cp312-macosx_11_0_arm64.whl", hash = "sha256:55086ee1064215781fff39a1af09518bc9255b50d6333f2e4c74ca09fac6a8f6"},
{file = "charset_normalizer-3.3.2-cp312-cp312-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:8f4a014bc36d3c57402e2977dada34f9c12300af536839dc38c0beab8878f38a"},
{file = "charset_normalizer-3.3.2-cp312-cp312-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", hash = "sha256:a10af20b82360ab00827f916a6058451b723b4e65030c5a18577c8b2de5b3389"},
{file = "charset_normalizer-3.3.2-cp312-cp312-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:8d756e44e94489e49571086ef83b2bb8ce311e730092d2c34ca8f7d925cb20aa"},
{file = "charset_normalizer-3.3.2-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:90d558489962fd4918143277a773316e56c72da56ec7aa3dc3dbbe20fdfed15b"},
{file = "charset_normalizer-3.3.2-cp312-cp312-manylinux_2_5_i686.manylinux1_i686.manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:6ac7ffc7ad6d040517be39eb591cac5ff87416c2537df6ba3cba3bae290c0fed"},
{file = "charset_normalizer-3.3.2-cp312-cp312-musllinux_1_1_aarch64.whl", hash = "sha256:7ed9e526742851e8d5cc9e6cf41427dfc6068d4f5a3bb03659444b4cabf6bc26"},
{file = "charset_normalizer-3.3.2-cp312-cp312-musllinux_1_1_i686.whl", hash = "sha256:8bdb58ff7ba23002a4c5808d608e4e6c687175724f54a5dade5fa8c67b604e4d"},
{file = "charset_normalizer-3.3.2-cp312-cp312-musllinux_1_1_ppc64le.whl", hash = "sha256:6b3251890fff30ee142c44144871185dbe13b11bab478a88887a639655be1068"},
{file = "charset_normalizer-3.3.2-cp312-cp312-musllinux_1_1_s390x.whl", hash = "sha256:b4a23f61ce87adf89be746c8a8974fe1c823c891d8f86eb218bb957c924bb143"},
{file = "charset_normalizer-3.3.2-cp312-cp312-musllinux_1_1_x86_64.whl", hash = "sha256:efcb3f6676480691518c177e3b465bcddf57cea040302f9f4e6e191af91174d4"},
{file = "charset_normalizer-3.3.2-cp312-cp312-win32.whl", hash = "sha256:d965bba47ddeec8cd560687584e88cf699fd28f192ceb452d1d7ee807c5597b7"},
{file = "charset_normalizer-3.3.2-cp312-cp312-win_amd64.whl", hash = "sha256:96b02a3dc4381e5494fad39be677abcb5e6634bf7b4fa83a6dd3112607547001"},
{file = "charset_normalizer-3.3.2-cp37-cp37m-macosx_10_9_x86_64.whl", hash = "sha256:95f2a5796329323b8f0512e09dbb7a1860c46a39da62ecb2324f116fa8fdc85c"},
{file = "charset_normalizer-3.3.2-cp37-cp37m-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:c002b4ffc0be611f0d9da932eb0f704fe2602a9a949d1f738e4c34c75b0863d5"},
{file = "charset_normalizer-3.3.2-cp37-cp37m-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", hash = "sha256:a981a536974bbc7a512cf44ed14938cf01030a99e9b3a06dd59578882f06f985"},
{file = "charset_normalizer-3.3.2-cp37-cp37m-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:3287761bc4ee9e33561a7e058c72ac0938c4f57fe49a09eae428fd88aafe7bb6"},
{file = "charset_normalizer-3.3.2-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:42cb296636fcc8b0644486d15c12376cb9fa75443e00fb25de0b8602e64c1714"},
{file = "charset_normalizer-3.3.2-cp37-cp37m-manylinux_2_5_i686.manylinux1_i686.manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:0a55554a2fa0d408816b3b5cedf0045f4b8e1a6065aec45849de2d6f3f8e9786"},
{file = "charset_normalizer-3.3.2-cp37-cp37m-musllinux_1_1_aarch64.whl", hash = "sha256:c083af607d2515612056a31f0a8d9e0fcb5876b7bfc0abad3ecd275bc4ebc2d5"},
{file = "charset_normalizer-3.3.2-cp37-cp37m-musllinux_1_1_i686.whl", hash = "sha256:87d1351268731db79e0f8e745d92493ee2841c974128ef629dc518b937d9194c"},
{file = "charset_normalizer-3.3.2-cp37-cp37m-musllinux_1_1_ppc64le.whl", hash = "sha256:bd8f7df7d12c2db9fab40bdd87a7c09b1530128315d047a086fa3ae3435cb3a8"},
{file = "charset_normalizer-3.3.2-cp37-cp37m-musllinux_1_1_s390x.whl", hash = "sha256:c180f51afb394e165eafe4ac2936a14bee3eb10debc9d9e4db8958fe36afe711"},
{file = "charset_normalizer-3.3.2-cp37-cp37m-musllinux_1_1_x86_64.whl", hash = "sha256:8c622a5fe39a48f78944a87d4fb8a53ee07344641b0562c540d840748571b811"},
{file = "charset_normalizer-3.3.2-cp37-cp37m-win32.whl", hash = "sha256:db364eca23f876da6f9e16c9da0df51aa4f104a972735574842618b8c6d999d4"},
{file = "charset_normalizer-3.3.2-cp37-cp37m-win_amd64.whl", hash = "sha256:86216b5cee4b06df986d214f664305142d9c76df9b6512be2738aa72a2048f99"},
{file = "charset_normalizer-3.3.2-cp38-cp38-macosx_10_9_universal2.whl", hash = "sha256:6463effa3186ea09411d50efc7d85360b38d5f09b870c48e4600f63af490e56a"},
{file = "charset_normalizer-3.3.2-cp38-cp38-macosx_10_9_x86_64.whl", hash = "sha256:6c4caeef8fa63d06bd437cd4bdcf3ffefe6738fb1b25951440d80dc7df8c03ac"},
{file = "charset_normalizer-3.3.2-cp38-cp38-macosx_11_0_arm64.whl", hash = "sha256:37e55c8e51c236f95b033f6fb391d7d7970ba5fe7ff453dad675e88cf303377a"},
{file = "charset_normalizer-3.3.2-cp38-cp38-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:fb69256e180cb6c8a894fee62b3afebae785babc1ee98b81cdf68bbca1987f33"},
{file = "charset_normalizer-3.3.2-cp38-cp38-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", hash = "sha256:ae5f4161f18c61806f411a13b0310bea87f987c7d2ecdbdaad0e94eb2e404238"},
{file = "charset_normalizer-3.3.2-cp38-cp38-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:b2b0a0c0517616b6869869f8c581d4eb2dd83a4d79e0ebcb7d373ef9956aeb0a"},
{file = "charset_normalizer-3.3.2-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:45485e01ff4d3630ec0d9617310448a8702f70e9c01906b0d0118bdf9d124cf2"},
{file = "charset_normalizer-3.3.2-cp38-cp38-manylinux_2_5_i686.manylinux1_i686.manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:eb00ed941194665c332bf8e078baf037d6c35d7c4f3102ea2d4f16ca94a26dc8"},
{file = "charset_normalizer-3.3.2-cp38-cp38-musllinux_1_1_aarch64.whl", hash = "sha256:2127566c664442652f024c837091890cb1942c30937add288223dc895793f898"},
{file = "charset_normalizer-3.3.2-cp38-cp38-musllinux_1_1_i686.whl", hash = "sha256:a50aebfa173e157099939b17f18600f72f84eed3049e743b68ad15bd69b6bf99"},
{file = "charset_normalizer-3.3.2-cp38-cp38-musllinux_1_1_ppc64le.whl", hash = "sha256:4d0d1650369165a14e14e1e47b372cfcb31d6ab44e6e33cb2d4e57265290044d"},
{file = "charset_normalizer-3.3.2-cp38-cp38-musllinux_1_1_s390x.whl", hash = "sha256:923c0c831b7cfcb071580d3f46c4baf50f174be571576556269530f4bbd79d04"},
{file = "charset_normalizer-3.3.2-cp38-cp38-musllinux_1_1_x86_64.whl", hash = "sha256:06a81e93cd441c56a9b65d8e1d043daeb97a3d0856d177d5c90ba85acb3db087"},
{file = "charset_normalizer-3.3.2-cp38-cp38-win32.whl", hash = "sha256:6ef1d82a3af9d3eecdba2321dc1b3c238245d890843e040e41e470ffa64c3e25"},
{file = "charset_normalizer-3.3.2-cp38-cp38-win_amd64.whl", hash = "sha256:eb8821e09e916165e160797a6c17edda0679379a4be5c716c260e836e122f54b"},
{file = "charset_normalizer-3.3.2-cp39-cp39-macosx_10_9_universal2.whl", hash = "sha256:c235ebd9baae02f1b77bcea61bce332cb4331dc3617d254df3323aa01ab47bd4"},
{file = "charset_normalizer-3.3.2-cp39-cp39-macosx_10_9_x86_64.whl", hash = "sha256:5b4c145409bef602a690e7cfad0a15a55c13320ff7a3ad7ca59c13bb8ba4d45d"},
{file = "charset_normalizer-3.3.2-cp39-cp39-macosx_11_0_arm64.whl", hash = "sha256:68d1f8a9e9e37c1223b656399be5d6b448dea850bed7d0f87a8311f1ff3dabb0"},
{file = "charset_normalizer-3.3.2-cp39-cp39-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:22afcb9f253dac0696b5a4be4a1c0f8762f8239e21b99680099abd9b2b1b2269"},
{file = "charset_normalizer-3.3.2-cp39-cp39-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", hash = "sha256:e27ad930a842b4c5eb8ac0016b0a54f5aebbe679340c26101df33424142c143c"},
{file = "charset_normalizer-3.3.2-cp39-cp39-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:1f79682fbe303db92bc2b1136016a38a42e835d932bab5b3b1bfcfbf0640e519"},
{file = "charset_normalizer-3.3.2-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:b261ccdec7821281dade748d088bb6e9b69e6d15b30652b74cbbac25e280b796"},
{file = "charset_normalizer-3.3.2-cp39-cp39-manylinux_2_5_i686.manylinux1_i686.manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:122c7fa62b130ed55f8f285bfd56d5f4b4a5b503609d181f9ad85e55c89f4185"},
{file = "charset_normalizer-3.3.2-cp39-cp39-musllinux_1_1_aarch64.whl", hash = "sha256:d0eccceffcb53201b5bfebb52600a5fb483a20b61da9dbc885f8b103cbe7598c"},
{file = "charset_normalizer-3.3.2-cp39-cp39-musllinux_1_1_i686.whl", hash = "sha256:9f96df6923e21816da7e0ad3fd47dd8f94b2a5ce594e00677c0013018b813458"},
{file = "charset_normalizer-3.3.2-cp39-cp39-musllinux_1_1_ppc64le.whl", hash = "sha256:7f04c839ed0b6b98b1a7501a002144b76c18fb1c1850c8b98d458ac269e26ed2"},
{file = "charset_normalizer-3.3.2-cp39-cp39-musllinux_1_1_s390x.whl", hash = "sha256:34d1c8da1e78d2e001f363791c98a272bb734000fcef47a491c1e3b0505657a8"},
{file = "charset_normalizer-3.3.2-cp39-cp39-musllinux_1_1_x86_64.whl", hash = "sha256:ff8fa367d09b717b2a17a052544193ad76cd49979c805768879cb63d9ca50561"},
{file = "charset_normalizer-3.3.2-cp39-cp39-win32.whl", hash = "sha256:aed38f6e4fb3f5d6bf81bfa990a07806be9d83cf7bacef998ab1a9bd660a581f"},
{file = "charset_normalizer-3.3.2-cp39-cp39-win_amd64.whl", hash = "sha256:b01b88d45a6fcb69667cd6d2f7a9aeb4bf53760d7fc536bf679ec94fe9f3ff3d"},
{file = "charset_normalizer-3.3.2-py3-none-any.whl", hash = "sha256:3e4d1f6587322d2788836a99c69062fbb091331ec940e02d12d179c1d53e25fc"},
]
[[package]]
name = "codespell"
version = "2.3.0"
description = "Codespell"
optional = false
python-versions = ">=3.8"
files = [
{file = "codespell-2.3.0-py3-none-any.whl", hash = "sha256:a9c7cef2501c9cfede2110fd6d4e5e62296920efe9abfb84648df866e47f58d1"},
{file = "codespell-2.3.0.tar.gz", hash = "sha256:360c7d10f75e65f67bad720af7007e1060a5d395670ec11a7ed1fed9dd17471f"},
]
[package.extras]
dev = ["Pygments", "build", "chardet", "pre-commit", "pytest", "pytest-cov", "pytest-dependency", "ruff", "tomli", "twine"]
hard-encoding-detection = ["chardet"]
toml = ["tomli"]
types = ["chardet (>=5.1.0)", "mypy", "pytest", "pytest-cov", "pytest-dependency"]
[[package]]
name = "colorama"
version = "0.4.6"
description = "Cross-platform colored terminal text."
optional = false
python-versions = "!=3.0.*,!=3.1.*,!=3.2.*,!=3.3.*,!=3.4.*,!=3.5.*,!=3.6.*,>=2.7"
files = [
{file = "colorama-0.4.6-py2.py3-none-any.whl", hash = "sha256:4f1d9991f5acc0ca119f9d443620b77f9d6b33703e51011c16baf57afb285fc6"},
{file = "colorama-0.4.6.tar.gz", hash = "sha256:08695f5cb7ed6e0531a20572697297273c47b8cae5a63ffc6d6ed5c201be6e44"},
]
[[package]]
name = "dataclasses-json"
version = "0.6.7"
description = "Easily serialize dataclasses to and from JSON."
optional = false
python-versions = "<4.0,>=3.7"
files = [
{file = "dataclasses_json-0.6.7-py3-none-any.whl", hash = "sha256:0dbf33f26c8d5305befd61b39d2b3414e8a407bedc2834dea9b8d642666fb40a"},
{file = "dataclasses_json-0.6.7.tar.gz", hash = "sha256:b6b3e528266ea45b9535223bc53ca645f5208833c29229e847b3f26a1cc55fc0"},
]
[package.dependencies]
marshmallow = ">=3.18.0,<4.0.0"
typing-inspect = ">=0.4.0,<1"
[[package]]
name = "exceptiongroup"
version = "1.2.2"
description = "Backport of PEP 654 (exception groups)"
optional = false
python-versions = ">=3.7"
files = [
{file = "exceptiongroup-1.2.2-py3-none-any.whl", hash = "sha256:3111b9d131c238bec2f8f516e123e14ba243563fb135d3fe885990585aa7795b"},
{file = "exceptiongroup-1.2.2.tar.gz", hash = "sha256:47c2edf7c6738fafb49fd34290706d1a1a2f4d1c6df275526b62cbb4aa5393cc"},
]
[package.extras]
test = ["pytest (>=6)"]
[[package]]
name = "idna"
version = "3.7"
description = "Internationalized Domain Names in Applications (IDNA)"
optional = false
python-versions = ">=3.5"
files = [
{file = "idna-3.7-py3-none-any.whl", hash = "sha256:82fee1fc78add43492d3a1898bfa6d8a904cc97d8427f683ed8e798d07761aa0"},
{file = "idna-3.7.tar.gz", hash = "sha256:028ff3aadf0609c1fd278d8ea3089299412a7a8b9bd005dd08b9f8285bcb5cfc"},
]
[[package]]
name = "iniconfig"
version = "2.0.0"
description = "brain-dead simple config-ini parsing"
optional = false
python-versions = ">=3.7"
files = [
{file = "iniconfig-2.0.0-py3-none-any.whl", hash = "sha256:b6a85871a79d2e3b22d2d1b94ac2824226a63c6b741c88f7ae975f18b6778374"},
{file = "iniconfig-2.0.0.tar.gz", hash = "sha256:2d91e135bf72d31a410b17c16da610a82cb55f6b0477d1a902134b24a455b8b3"},
]
[[package]]
name = "jsonpatch"
version = "1.33"
description = "Apply JSON-Patches (RFC 6902)"
optional = false
python-versions = ">=2.7, !=3.0.*, !=3.1.*, !=3.2.*, !=3.3.*, !=3.4.*, !=3.5.*, !=3.6.*"
files = [
{file = "jsonpatch-1.33-py2.py3-none-any.whl", hash = "sha256:0ae28c0cd062bbd8b8ecc26d7d164fbbea9652a1a3693f3b956c1eae5145dade"},
{file = "jsonpatch-1.33.tar.gz", hash = "sha256:9fcd4009c41e6d12348b4a0ff2563ba56a2923a7dfee731d004e212e1ee5030c"},
]
[package.dependencies]
jsonpointer = ">=1.9"
[[package]]
name = "jsonpointer"
version = "3.0.0"
description = "Identify specific nodes in a JSON document (RFC 6901)"
optional = false
python-versions = ">=3.7"
files = [
{file = "jsonpointer-3.0.0-py2.py3-none-any.whl", hash = "sha256:13e088adc14fca8b6aa8177c044e12701e6ad4b28ff10e65f2267a90109c9942"},
{file = "jsonpointer-3.0.0.tar.gz", hash = "sha256:2b2d729f2091522d61c3b31f82e11870f60b68f43fbc705cb76bf4b832af59ef"},
]
[[package]]
name = "langchain-core"
version = "0.2.38"
description = "Building applications with LLMs through composability"
optional = false
python-versions = "<4.0,>=3.8.1"
files = [
{file = "langchain_core-0.2.38-py3-none-any.whl", hash = "sha256:8a5729bc7e68b4af089af20eff44fe4e7ca21d0e0c87ec21cef7621981fd1a4a"},
{file = "langchain_core-0.2.38.tar.gz", hash = "sha256:eb69dbedd344f2ee1f15bcea6c71a05884b867588fadc42d04632e727c1238f3"},
]
[package.dependencies]
jsonpatch = ">=1.33,<2.0"
langsmith = ">=0.1.75,<0.2.0"
packaging = ">=23.2,<25"
pydantic = [
{version = ">=1,<3", markers = "python_full_version < \"3.12.4\""},
{version = ">=2.7.4,<3.0.0", markers = "python_full_version >= \"3.12.4\""},
]
PyYAML = ">=5.3"
tenacity = ">=8.1.0,<8.4.0 || >8.4.0,<9.0.0"
typing-extensions = ">=4.7"
[[package]]
name = "langsmith"
version = "0.1.93"
description = "Client library to connect to the LangSmith LLM Tracing and Evaluation Platform."
optional = false
python-versions = "<4.0,>=3.8.1"
files = [
{file = "langsmith-0.1.93-py3-none-any.whl", hash = "sha256:811210b9d5f108f36431bd7b997eb9476a9ecf5a2abd7ddbb606c1cdcf0f43ce"},
{file = "langsmith-0.1.93.tar.gz", hash = "sha256:285b6ad3a54f50fa8eb97b5f600acc57d0e37e139dd8cf2111a117d0435ba9b4"},
]
[package.dependencies]
orjson = ">=3.9.14,<4.0.0"
pydantic = [
{version = ">=1,<3", markers = "python_full_version < \"3.12.4\""},
{version = ">=2.7.4,<3.0.0", markers = "python_full_version >= \"3.12.4\""},
]
requests = ">=2,<3"
[[package]]
name = "marshmallow"
version = "3.21.3"
description = "A lightweight library for converting complex datatypes to and from native Python datatypes."
optional = false
python-versions = ">=3.8"
files = [
{file = "marshmallow-3.21.3-py3-none-any.whl", hash = "sha256:86ce7fb914aa865001a4b2092c4c2872d13bc347f3d42673272cabfdbad386f1"},
{file = "marshmallow-3.21.3.tar.gz", hash = "sha256:4f57c5e050a54d66361e826f94fba213eb10b67b2fdb02c3e0343ce207ba1662"},
]
[package.dependencies]
packaging = ">=17.0"
[package.extras]
dev = ["marshmallow[tests]", "pre-commit (>=3.5,<4.0)", "tox"]
docs = ["alabaster (==0.7.16)", "autodocsumm (==0.2.12)", "sphinx (==7.3.7)", "sphinx-issues (==4.1.0)", "sphinx-version-warning (==1.1.2)"]
tests = ["pytest", "pytz", "simplejson"]
[[package]]
name = "msgpack"
version = "1.1.0"
description = "MessagePack serializer"
optional = false
python-versions = ">=3.8"
files = [
{file = "msgpack-1.1.0-cp310-cp310-macosx_10_9_universal2.whl", hash = "sha256:7ad442d527a7e358a469faf43fda45aaf4ac3249c8310a82f0ccff9164e5dccd"},
{file = "msgpack-1.1.0-cp310-cp310-macosx_10_9_x86_64.whl", hash = "sha256:74bed8f63f8f14d75eec75cf3d04ad581da6b914001b474a5d3cd3372c8cc27d"},
{file = "msgpack-1.1.0-cp310-cp310-macosx_11_0_arm64.whl", hash = "sha256:914571a2a5b4e7606997e169f64ce53a8b1e06f2cf2c3a7273aa106236d43dd5"},
{file = "msgpack-1.1.0-cp310-cp310-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:c921af52214dcbb75e6bdf6a661b23c3e6417f00c603dd2070bccb5c3ef499f5"},
{file = "msgpack-1.1.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:d8ce0b22b890be5d252de90d0e0d119f363012027cf256185fc3d474c44b1b9e"},
{file = "msgpack-1.1.0-cp310-cp310-manylinux_2_5_i686.manylinux1_i686.manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:73322a6cc57fcee3c0c57c4463d828e9428275fb85a27aa2aa1a92fdc42afd7b"},
{file = "msgpack-1.1.0-cp310-cp310-musllinux_1_2_aarch64.whl", hash = "sha256:e1f3c3d21f7cf67bcf2da8e494d30a75e4cf60041d98b3f79875afb5b96f3a3f"},
{file = "msgpack-1.1.0-cp310-cp310-musllinux_1_2_i686.whl", hash = "sha256:64fc9068d701233effd61b19efb1485587560b66fe57b3e50d29c5d78e7fef68"},
{file = "msgpack-1.1.0-cp310-cp310-musllinux_1_2_x86_64.whl", hash = "sha256:42f754515e0f683f9c79210a5d1cad631ec3d06cea5172214d2176a42e67e19b"},
{file = "msgpack-1.1.0-cp310-cp310-win32.whl", hash = "sha256:3df7e6b05571b3814361e8464f9304c42d2196808e0119f55d0d3e62cd5ea044"},
{file = "msgpack-1.1.0-cp310-cp310-win_amd64.whl", hash = "sha256:685ec345eefc757a7c8af44a3032734a739f8c45d1b0ac45efc5d8977aa4720f"},
{file = "msgpack-1.1.0-cp311-cp311-macosx_10_9_universal2.whl", hash = "sha256:3d364a55082fb2a7416f6c63ae383fbd903adb5a6cf78c5b96cc6316dc1cedc7"},
{file = "msgpack-1.1.0-cp311-cp311-macosx_10_9_x86_64.whl", hash = "sha256:79ec007767b9b56860e0372085f8504db5d06bd6a327a335449508bbee9648fa"},
{file = "msgpack-1.1.0-cp311-cp311-macosx_11_0_arm64.whl", hash = "sha256:6ad622bf7756d5a497d5b6836e7fc3752e2dd6f4c648e24b1803f6048596f701"},
{file = "msgpack-1.1.0-cp311-cp311-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:8e59bca908d9ca0de3dc8684f21ebf9a690fe47b6be93236eb40b99af28b6ea6"},
{file = "msgpack-1.1.0-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:5e1da8f11a3dd397f0a32c76165cf0c4eb95b31013a94f6ecc0b280c05c91b59"},
{file = "msgpack-1.1.0-cp311-cp311-manylinux_2_5_i686.manylinux1_i686.manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:452aff037287acb1d70a804ffd022b21fa2bb7c46bee884dbc864cc9024128a0"},
{file = "msgpack-1.1.0-cp311-cp311-musllinux_1_2_aarch64.whl", hash = "sha256:8da4bf6d54ceed70e8861f833f83ce0814a2b72102e890cbdfe4b34764cdd66e"},
{file = "msgpack-1.1.0-cp311-cp311-musllinux_1_2_i686.whl", hash = "sha256:41c991beebf175faf352fb940bf2af9ad1fb77fd25f38d9142053914947cdbf6"},
{file = "msgpack-1.1.0-cp311-cp311-musllinux_1_2_x86_64.whl", hash = "sha256:a52a1f3a5af7ba1c9ace055b659189f6c669cf3657095b50f9602af3a3ba0fe5"},
{file = "msgpack-1.1.0-cp311-cp311-win32.whl", hash = "sha256:58638690ebd0a06427c5fe1a227bb6b8b9fdc2bd07701bec13c2335c82131a88"},
{file = "msgpack-1.1.0-cp311-cp311-win_amd64.whl", hash = "sha256:fd2906780f25c8ed5d7b323379f6138524ba793428db5d0e9d226d3fa6aa1788"},
{file = "msgpack-1.1.0-cp312-cp312-macosx_10_9_universal2.whl", hash = "sha256:d46cf9e3705ea9485687aa4001a76e44748b609d260af21c4ceea7f2212a501d"},
{file = "msgpack-1.1.0-cp312-cp312-macosx_10_9_x86_64.whl", hash = "sha256:5dbad74103df937e1325cc4bfeaf57713be0b4f15e1c2da43ccdd836393e2ea2"},
{file = "msgpack-1.1.0-cp312-cp312-macosx_11_0_arm64.whl", hash = "sha256:58dfc47f8b102da61e8949708b3eafc3504509a5728f8b4ddef84bd9e16ad420"},
{file = "msgpack-1.1.0-cp312-cp312-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:4676e5be1b472909b2ee6356ff425ebedf5142427842aa06b4dfd5117d1ca8a2"},
{file = "msgpack-1.1.0-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:17fb65dd0bec285907f68b15734a993ad3fc94332b5bb21b0435846228de1f39"},
{file = "msgpack-1.1.0-cp312-cp312-manylinux_2_5_i686.manylinux1_i686.manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:a51abd48c6d8ac89e0cfd4fe177c61481aca2d5e7ba42044fd218cfd8ea9899f"},
{file = "msgpack-1.1.0-cp312-cp312-musllinux_1_2_aarch64.whl", hash = "sha256:2137773500afa5494a61b1208619e3871f75f27b03bcfca7b3a7023284140247"},
{file = "msgpack-1.1.0-cp312-cp312-musllinux_1_2_i686.whl", hash = "sha256:398b713459fea610861c8a7b62a6fec1882759f308ae0795b5413ff6a160cf3c"},
{file = "msgpack-1.1.0-cp312-cp312-musllinux_1_2_x86_64.whl", hash = "sha256:06f5fd2f6bb2a7914922d935d3b8bb4a7fff3a9a91cfce6d06c13bc42bec975b"},
{file = "msgpack-1.1.0-cp312-cp312-win32.whl", hash = "sha256:ad33e8400e4ec17ba782f7b9cf868977d867ed784a1f5f2ab46e7ba53b6e1e1b"},
{file = "msgpack-1.1.0-cp312-cp312-win_amd64.whl", hash = "sha256:115a7af8ee9e8cddc10f87636767857e7e3717b7a2e97379dc2054712693e90f"},
{file = "msgpack-1.1.0-cp313-cp313-macosx_10_13_universal2.whl", hash = "sha256:071603e2f0771c45ad9bc65719291c568d4edf120b44eb36324dcb02a13bfddf"},
{file = "msgpack-1.1.0-cp313-cp313-macosx_10_13_x86_64.whl", hash = "sha256:0f92a83b84e7c0749e3f12821949d79485971f087604178026085f60ce109330"},
{file = "msgpack-1.1.0-cp313-cp313-macosx_11_0_arm64.whl", hash = "sha256:4a1964df7b81285d00a84da4e70cb1383f2e665e0f1f2a7027e683956d04b734"},
{file = "msgpack-1.1.0-cp313-cp313-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:59caf6a4ed0d164055ccff8fe31eddc0ebc07cf7326a2aaa0dbf7a4001cd823e"},
{file = "msgpack-1.1.0-cp313-cp313-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:0907e1a7119b337971a689153665764adc34e89175f9a34793307d9def08e6ca"},
{file = "msgpack-1.1.0-cp313-cp313-manylinux_2_5_i686.manylinux1_i686.manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:65553c9b6da8166e819a6aa90ad15288599b340f91d18f60b2061f402b9a4915"},
{file = "msgpack-1.1.0-cp313-cp313-musllinux_1_2_aarch64.whl", hash = "sha256:7a946a8992941fea80ed4beae6bff74ffd7ee129a90b4dd5cf9c476a30e9708d"},
{file = "msgpack-1.1.0-cp313-cp313-musllinux_1_2_i686.whl", hash = "sha256:4b51405e36e075193bc051315dbf29168d6141ae2500ba8cd80a522964e31434"},
{file = "msgpack-1.1.0-cp313-cp313-musllinux_1_2_x86_64.whl", hash = "sha256:b4c01941fd2ff87c2a934ee6055bda4ed353a7846b8d4f341c428109e9fcde8c"},
{file = "msgpack-1.1.0-cp313-cp313-win32.whl", hash = "sha256:7c9a35ce2c2573bada929e0b7b3576de647b0defbd25f5139dcdaba0ae35a4cc"},
{file = "msgpack-1.1.0-cp313-cp313-win_amd64.whl", hash = "sha256:bce7d9e614a04d0883af0b3d4d501171fbfca038f12c77fa838d9f198147a23f"},
{file = "msgpack-1.1.0-cp38-cp38-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:c40ffa9a15d74e05ba1fe2681ea33b9caffd886675412612d93ab17b58ea2fec"},
{file = "msgpack-1.1.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:f1ba6136e650898082d9d5a5217d5906d1e138024f836ff48691784bbe1adf96"},
{file = "msgpack-1.1.0-cp38-cp38-manylinux_2_5_i686.manylinux1_i686.manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:e0856a2b7e8dcb874be44fea031d22e5b3a19121be92a1e098f46068a11b0870"},
{file = "msgpack-1.1.0-cp38-cp38-musllinux_1_2_aarch64.whl", hash = "sha256:471e27a5787a2e3f974ba023f9e265a8c7cfd373632247deb225617e3100a3c7"},
{file = "msgpack-1.1.0-cp38-cp38-musllinux_1_2_i686.whl", hash = "sha256:646afc8102935a388ffc3914b336d22d1c2d6209c773f3eb5dd4d6d3b6f8c1cb"},
{file = "msgpack-1.1.0-cp38-cp38-musllinux_1_2_x86_64.whl", hash = "sha256:13599f8829cfbe0158f6456374e9eea9f44eee08076291771d8ae93eda56607f"},
{file = "msgpack-1.1.0-cp38-cp38-win32.whl", hash = "sha256:8a84efb768fb968381e525eeeb3d92857e4985aacc39f3c47ffd00eb4509315b"},
{file = "msgpack-1.1.0-cp38-cp38-win_amd64.whl", hash = "sha256:879a7b7b0ad82481c52d3c7eb99bf6f0645dbdec5134a4bddbd16f3506947feb"},
{file = "msgpack-1.1.0-cp39-cp39-macosx_10_9_universal2.whl", hash = "sha256:53258eeb7a80fc46f62fd59c876957a2d0e15e6449a9e71842b6d24419d88ca1"},
{file = "msgpack-1.1.0-cp39-cp39-macosx_10_9_x86_64.whl", hash = "sha256:7e7b853bbc44fb03fbdba34feb4bd414322180135e2cb5164f20ce1c9795ee48"},
{file = "msgpack-1.1.0-cp39-cp39-macosx_11_0_arm64.whl", hash = "sha256:f3e9b4936df53b970513eac1758f3882c88658a220b58dcc1e39606dccaaf01c"},
{file = "msgpack-1.1.0-cp39-cp39-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:46c34e99110762a76e3911fc923222472c9d681f1094096ac4102c18319e6468"},
{file = "msgpack-1.1.0-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:8a706d1e74dd3dea05cb54580d9bd8b2880e9264856ce5068027eed09680aa74"},
{file = "msgpack-1.1.0-cp39-cp39-manylinux_2_5_i686.manylinux1_i686.manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:534480ee5690ab3cbed89d4c8971a5c631b69a8c0883ecfea96c19118510c846"},
{file = "msgpack-1.1.0-cp39-cp39-musllinux_1_2_aarch64.whl", hash = "sha256:8cf9e8c3a2153934a23ac160cc4cba0ec035f6867c8013cc6077a79823370346"},
{file = "msgpack-1.1.0-cp39-cp39-musllinux_1_2_i686.whl", hash = "sha256:3180065ec2abbe13a4ad37688b61b99d7f9e012a535b930e0e683ad6bc30155b"},
{file = "msgpack-1.1.0-cp39-cp39-musllinux_1_2_x86_64.whl", hash = "sha256:c5a91481a3cc573ac8c0d9aace09345d989dc4a0202b7fcb312c88c26d4e71a8"},
{file = "msgpack-1.1.0-cp39-cp39-win32.whl", hash = "sha256:f80bc7d47f76089633763f952e67f8214cb7b3ee6bfa489b3cb6a84cfac114cd"},
{file = "msgpack-1.1.0-cp39-cp39-win_amd64.whl", hash = "sha256:4d1b7ff2d6146e16e8bd665ac726a89c74163ef8cd39fa8c1087d4e52d3a2325"},
{file = "msgpack-1.1.0.tar.gz", hash = "sha256:dd432ccc2c72b914e4cb77afce64aab761c1137cc698be3984eee260bcb2896e"},
]
[[package]]
name = "mypy"
version = "1.11.0"
description = "Optional static typing for Python"
optional = false
python-versions = ">=3.8"
files = [
{file = "mypy-1.11.0-cp310-cp310-macosx_10_9_x86_64.whl", hash = "sha256:a3824187c99b893f90c845bab405a585d1ced4ff55421fdf5c84cb7710995229"},
{file = "mypy-1.11.0-cp310-cp310-macosx_11_0_arm64.whl", hash = "sha256:96f8dbc2c85046c81bcddc246232d500ad729cb720da4e20fce3b542cab91287"},
{file = "mypy-1.11.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:1a5d8d8dd8613a3e2be3eae829ee891b6b2de6302f24766ff06cb2875f5be9c6"},
{file = "mypy-1.11.0-cp310-cp310-musllinux_1_1_x86_64.whl", hash = "sha256:72596a79bbfb195fd41405cffa18210af3811beb91ff946dbcb7368240eed6be"},
{file = "mypy-1.11.0-cp310-cp310-win_amd64.whl", hash = "sha256:35ce88b8ed3a759634cb4eb646d002c4cef0a38f20565ee82b5023558eb90c00"},
{file = "mypy-1.11.0-cp311-cp311-macosx_10_9_x86_64.whl", hash = "sha256:98790025861cb2c3db8c2f5ad10fc8c336ed2a55f4daf1b8b3f877826b6ff2eb"},
{file = "mypy-1.11.0-cp311-cp311-macosx_11_0_arm64.whl", hash = "sha256:25bcfa75b9b5a5f8d67147a54ea97ed63a653995a82798221cca2a315c0238c1"},
{file = "mypy-1.11.0-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:0bea2a0e71c2a375c9fa0ede3d98324214d67b3cbbfcbd55ac8f750f85a414e3"},
{file = "mypy-1.11.0-cp311-cp311-musllinux_1_1_x86_64.whl", hash = "sha256:d2b3d36baac48e40e3064d2901f2fbd2a2d6880ec6ce6358825c85031d7c0d4d"},
{file = "mypy-1.11.0-cp311-cp311-win_amd64.whl", hash = "sha256:d8e2e43977f0e09f149ea69fd0556623919f816764e26d74da0c8a7b48f3e18a"},
{file = "mypy-1.11.0-cp312-cp312-macosx_10_9_x86_64.whl", hash = "sha256:1d44c1e44a8be986b54b09f15f2c1a66368eb43861b4e82573026e04c48a9e20"},
{file = "mypy-1.11.0-cp312-cp312-macosx_11_0_arm64.whl", hash = "sha256:cea3d0fb69637944dd321f41bc896e11d0fb0b0aa531d887a6da70f6e7473aba"},
{file = "mypy-1.11.0-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:a83ec98ae12d51c252be61521aa5731f5512231d0b738b4cb2498344f0b840cd"},
{file = "mypy-1.11.0-cp312-cp312-musllinux_1_1_x86_64.whl", hash = "sha256:c7b73a856522417beb78e0fb6d33ef89474e7a622db2653bc1285af36e2e3e3d"},
{file = "mypy-1.11.0-cp312-cp312-win_amd64.whl", hash = "sha256:f2268d9fcd9686b61ab64f077be7ffbc6fbcdfb4103e5dd0cc5eaab53a8886c2"},
{file = "mypy-1.11.0-cp38-cp38-macosx_10_9_x86_64.whl", hash = "sha256:940bfff7283c267ae6522ef926a7887305945f716a7704d3344d6d07f02df850"},
{file = "mypy-1.11.0-cp38-cp38-macosx_11_0_arm64.whl", hash = "sha256:14f9294528b5f5cf96c721f231c9f5b2733164e02c1c018ed1a0eff8a18005ac"},
{file = "mypy-1.11.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:d7b54c27783991399046837df5c7c9d325d921394757d09dbcbf96aee4649fe9"},
{file = "mypy-1.11.0-cp38-cp38-musllinux_1_1_x86_64.whl", hash = "sha256:65f190a6349dec29c8d1a1cd4aa71284177aee5949e0502e6379b42873eddbe7"},
{file = "mypy-1.11.0-cp38-cp38-win_amd64.whl", hash = "sha256:dbe286303241fea8c2ea5466f6e0e6a046a135a7e7609167b07fd4e7baf151bf"},
{file = "mypy-1.11.0-cp39-cp39-macosx_10_9_x86_64.whl", hash = "sha256:104e9c1620c2675420abd1f6c44bab7dd33cc85aea751c985006e83dcd001095"},
{file = "mypy-1.11.0-cp39-cp39-macosx_11_0_arm64.whl", hash = "sha256:f006e955718ecd8d159cee9932b64fba8f86ee6f7728ca3ac66c3a54b0062abe"},
{file = "mypy-1.11.0-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:becc9111ca572b04e7e77131bc708480cc88a911adf3d0239f974c034b78085c"},
{file = "mypy-1.11.0-cp39-cp39-musllinux_1_1_x86_64.whl", hash = "sha256:6801319fe76c3f3a3833f2b5af7bd2c17bb93c00026a2a1b924e6762f5b19e13"},
{file = "mypy-1.11.0-cp39-cp39-win_amd64.whl", hash = "sha256:c1a184c64521dc549324ec6ef7cbaa6b351912be9cb5edb803c2808a0d7e85ac"},
{file = "mypy-1.11.0-py3-none-any.whl", hash = "sha256:56913ec8c7638b0091ef4da6fcc9136896914a9d60d54670a75880c3e5b99ace"},
{file = "mypy-1.11.0.tar.gz", hash = "sha256:93743608c7348772fdc717af4aeee1997293a1ad04bc0ea6efa15bf65385c538"},
]
[package.dependencies]
mypy-extensions = ">=1.0.0"
tomli = {version = ">=1.1.0", markers = "python_version < \"3.11\""}
typing-extensions = ">=4.6.0"
[package.extras]
dmypy = ["psutil (>=4.0)"]
install-types = ["pip"]
mypyc = ["setuptools (>=50)"]
reports = ["lxml"]
[[package]]
name = "mypy-extensions"
version = "1.0.0"
description = "Type system extensions for programs checked with the mypy type checker."
optional = false
python-versions = ">=3.5"
files = [
{file = "mypy_extensions-1.0.0-py3-none-any.whl", hash = "sha256:4392f6c0eb8a5668a69e23d168ffa70f0be9ccfd32b5cc2d26a34ae5b844552d"},
{file = "mypy_extensions-1.0.0.tar.gz", hash = "sha256:75dbf8955dc00442a438fc4d0666508a9a97b6bd41aa2f0ffe9d2f2725af0782"},
]
[[package]]
name = "orjson"
version = "3.10.6"
description = "Fast, correct Python JSON library supporting dataclasses, datetimes, and numpy"
optional = false
python-versions = ">=3.8"
files = [
{file = "orjson-3.10.6-cp310-cp310-macosx_10_15_x86_64.macosx_11_0_arm64.macosx_10_15_universal2.whl", hash = "sha256:fb0ee33124db6eaa517d00890fc1a55c3bfe1cf78ba4a8899d71a06f2d6ff5c7"},
{file = "orjson-3.10.6-cp310-cp310-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:9c1c4b53b24a4c06547ce43e5fee6ec4e0d8fe2d597f4647fc033fd205707365"},
{file = "orjson-3.10.6-cp310-cp310-manylinux_2_17_armv7l.manylinux2014_armv7l.whl", hash = "sha256:eadc8fd310edb4bdbd333374f2c8fec6794bbbae99b592f448d8214a5e4050c0"},
{file = "orjson-3.10.6-cp310-cp310-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", hash = "sha256:61272a5aec2b2661f4fa2b37c907ce9701e821b2c1285d5c3ab0207ebd358d38"},
{file = "orjson-3.10.6-cp310-cp310-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:57985ee7e91d6214c837936dc1608f40f330a6b88bb13f5a57ce5257807da143"},
{file = "orjson-3.10.6-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:633a3b31d9d7c9f02d49c4ab4d0a86065c4a6f6adc297d63d272e043472acab5"},
{file = "orjson-3.10.6-cp310-cp310-musllinux_1_2_aarch64.whl", hash = "sha256:1c680b269d33ec444afe2bdc647c9eb73166fa47a16d9a75ee56a374f4a45f43"},
{file = "orjson-3.10.6-cp310-cp310-musllinux_1_2_x86_64.whl", hash = "sha256:f759503a97a6ace19e55461395ab0d618b5a117e8d0fbb20e70cfd68a47327f2"},
{file = "orjson-3.10.6-cp310-none-win32.whl", hash = "sha256:95a0cce17f969fb5391762e5719575217bd10ac5a189d1979442ee54456393f3"},
{file = "orjson-3.10.6-cp310-none-win_amd64.whl", hash = "sha256:df25d9271270ba2133cc88ee83c318372bdc0f2cd6f32e7a450809a111efc45c"},
{file = "orjson-3.10.6-cp311-cp311-macosx_10_15_x86_64.macosx_11_0_arm64.macosx_10_15_universal2.whl", hash = "sha256:b1ec490e10d2a77c345def52599311849fc063ae0e67cf4f84528073152bb2ba"},
{file = "orjson-3.10.6-cp311-cp311-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:55d43d3feb8f19d07e9f01e5b9be4f28801cf7c60d0fa0d279951b18fae1932b"},
{file = "orjson-3.10.6-cp311-cp311-manylinux_2_17_armv7l.manylinux2014_armv7l.whl", hash = "sha256:ac3045267e98fe749408eee1593a142e02357c5c99be0802185ef2170086a863"},
{file = "orjson-3.10.6-cp311-cp311-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", hash = "sha256:c27bc6a28ae95923350ab382c57113abd38f3928af3c80be6f2ba7eb8d8db0b0"},
{file = "orjson-3.10.6-cp311-cp311-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:d27456491ca79532d11e507cadca37fb8c9324a3976294f68fb1eff2dc6ced5a"},
{file = "orjson-3.10.6-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:05ac3d3916023745aa3b3b388e91b9166be1ca02b7c7e41045da6d12985685f0"},
{file = "orjson-3.10.6-cp311-cp311-musllinux_1_2_aarch64.whl", hash = "sha256:1335d4ef59ab85cab66fe73fd7a4e881c298ee7f63ede918b7faa1b27cbe5212"},
{file = "orjson-3.10.6-cp311-cp311-musllinux_1_2_x86_64.whl", hash = "sha256:4bbc6d0af24c1575edc79994c20e1b29e6fb3c6a570371306db0993ecf144dc5"},
{file = "orjson-3.10.6-cp311-none-win32.whl", hash = "sha256:450e39ab1f7694465060a0550b3f6d328d20297bf2e06aa947b97c21e5241fbd"},
{file = "orjson-3.10.6-cp311-none-win_amd64.whl", hash = "sha256:227df19441372610b20e05bdb906e1742ec2ad7a66ac8350dcfd29a63014a83b"},
{file = "orjson-3.10.6-cp312-cp312-macosx_10_15_x86_64.macosx_11_0_arm64.macosx_10_15_universal2.whl", hash = "sha256:ea2977b21f8d5d9b758bb3f344a75e55ca78e3ff85595d248eee813ae23ecdfb"},
{file = "orjson-3.10.6-cp312-cp312-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:b6f3d167d13a16ed263b52dbfedff52c962bfd3d270b46b7518365bcc2121eed"},
{file = "orjson-3.10.6-cp312-cp312-manylinux_2_17_armv7l.manylinux2014_armv7l.whl", hash = "sha256:f710f346e4c44a4e8bdf23daa974faede58f83334289df80bc9cd12fe82573c7"},
{file = "orjson-3.10.6-cp312-cp312-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", hash = "sha256:7275664f84e027dcb1ad5200b8b18373e9c669b2a9ec33d410c40f5ccf4b257e"},
{file = "orjson-3.10.6-cp312-cp312-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:0943e4c701196b23c240b3d10ed8ecd674f03089198cf503105b474a4f77f21f"},
{file = "orjson-3.10.6-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:446dee5a491b5bc7d8f825d80d9637e7af43f86a331207b9c9610e2f93fee22a"},
{file = "orjson-3.10.6-cp312-cp312-musllinux_1_2_aarch64.whl", hash = "sha256:64c81456d2a050d380786413786b057983892db105516639cb5d3ee3c7fd5148"},
{file = "orjson-3.10.6-cp312-cp312-musllinux_1_2_x86_64.whl", hash = "sha256:960db0e31c4e52fa0fc3ecbaea5b2d3b58f379e32a95ae6b0ebeaa25b93dfd34"},
{file = "orjson-3.10.6-cp312-none-win32.whl", hash = "sha256:a6ea7afb5b30b2317e0bee03c8d34c8181bc5a36f2afd4d0952f378972c4efd5"},
{file = "orjson-3.10.6-cp312-none-win_amd64.whl", hash = "sha256:874ce88264b7e655dde4aeaacdc8fd772a7962faadfb41abe63e2a4861abc3dc"},
{file = "orjson-3.10.6-cp313-none-win32.whl", hash = "sha256:efdf2c5cde290ae6b83095f03119bdc00303d7a03b42b16c54517baa3c4ca3d0"},
{file = "orjson-3.10.6-cp313-none-win_amd64.whl", hash = "sha256:8e190fe7888e2e4392f52cafb9626113ba135ef53aacc65cd13109eb9746c43e"},
{file = "orjson-3.10.6-cp38-cp38-macosx_10_15_x86_64.macosx_11_0_arm64.macosx_10_15_universal2.whl", hash = "sha256:66680eae4c4e7fc193d91cfc1353ad6d01b4801ae9b5314f17e11ba55e934183"},
{file = "orjson-3.10.6-cp38-cp38-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:caff75b425db5ef8e8f23af93c80f072f97b4fb3afd4af44482905c9f588da28"},
{file = "orjson-3.10.6-cp38-cp38-manylinux_2_17_armv7l.manylinux2014_armv7l.whl", hash = "sha256:3722fddb821b6036fd2a3c814f6bd9b57a89dc6337b9924ecd614ebce3271394"},
{file = "orjson-3.10.6-cp38-cp38-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", hash = "sha256:c2c116072a8533f2fec435fde4d134610f806bdac20188c7bd2081f3e9e0133f"},
{file = "orjson-3.10.6-cp38-cp38-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:6eeb13218c8cf34c61912e9df2de2853f1d009de0e46ea09ccdf3d757896af0a"},
{file = "orjson-3.10.6-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:965a916373382674e323c957d560b953d81d7a8603fbeee26f7b8248638bd48b"},
{file = "orjson-3.10.6-cp38-cp38-musllinux_1_2_aarch64.whl", hash = "sha256:03c95484d53ed8e479cade8628c9cea00fd9d67f5554764a1110e0d5aa2de96e"},
{file = "orjson-3.10.6-cp38-cp38-musllinux_1_2_x86_64.whl", hash = "sha256:e060748a04cccf1e0a6f2358dffea9c080b849a4a68c28b1b907f272b5127e9b"},
{file = "orjson-3.10.6-cp38-none-win32.whl", hash = "sha256:738dbe3ef909c4b019d69afc19caf6b5ed0e2f1c786b5d6215fbb7539246e4c6"},
{file = "orjson-3.10.6-cp38-none-win_amd64.whl", hash = "sha256:d40f839dddf6a7d77114fe6b8a70218556408c71d4d6e29413bb5f150a692ff7"},
{file = "orjson-3.10.6-cp39-cp39-macosx_10_15_x86_64.macosx_11_0_arm64.macosx_10_15_universal2.whl", hash = "sha256:697a35a083c4f834807a6232b3e62c8b280f7a44ad0b759fd4dce748951e70db"},
{file = "orjson-3.10.6-cp39-cp39-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:fd502f96bf5ea9a61cbc0b2b5900d0dd68aa0da197179042bdd2be67e51a1e4b"},
{file = "orjson-3.10.6-cp39-cp39-manylinux_2_17_armv7l.manylinux2014_armv7l.whl", hash = "sha256:f215789fb1667cdc874c1b8af6a84dc939fd802bf293a8334fce185c79cd359b"},
{file = "orjson-3.10.6-cp39-cp39-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", hash = "sha256:a2debd8ddce948a8c0938c8c93ade191d2f4ba4649a54302a7da905a81f00b56"},
{file = "orjson-3.10.6-cp39-cp39-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:5410111d7b6681d4b0d65e0f58a13be588d01b473822483f77f513c7f93bd3b2"},
{file = "orjson-3.10.6-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:bb1f28a137337fdc18384079fa5726810681055b32b92253fa15ae5656e1dddb"},
{file = "orjson-3.10.6-cp39-cp39-musllinux_1_2_aarch64.whl", hash = "sha256:bf2fbbce5fe7cd1aa177ea3eab2b8e6a6bc6e8592e4279ed3db2d62e57c0e1b2"},
{file = "orjson-3.10.6-cp39-cp39-musllinux_1_2_x86_64.whl", hash = "sha256:79b9b9e33bd4c517445a62b90ca0cc279b0f1f3970655c3df9e608bc3f91741a"},
{file = "orjson-3.10.6-cp39-none-win32.whl", hash = "sha256:30b0a09a2014e621b1adf66a4f705f0809358350a757508ee80209b2d8dae219"},
{file = "orjson-3.10.6-cp39-none-win_amd64.whl", hash = "sha256:49e3bc615652617d463069f91b867a4458114c5b104e13b7ae6872e5f79d0844"},
{file = "orjson-3.10.6.tar.gz", hash = "sha256:e54b63d0a7c6c54a5f5f726bc93a2078111ef060fec4ecbf34c5db800ca3b3a7"},
]
[[package]]
name = "packaging"
version = "24.1"
description = "Core utilities for Python packages"
optional = false
python-versions = ">=3.8"
files = [
{file = "packaging-24.1-py3-none-any.whl", hash = "sha256:5b8f2217dbdbd2f7f384c41c628544e6d52f2d0f53c6d0c3ea61aa5d1d7ff124"},
{file = "packaging-24.1.tar.gz", hash = "sha256:026ed72c8ed3fcce5bf8950572258698927fd1dbda10a5e981cdf0ac37f4f002"},
]
[[package]]
name = "pluggy"
version = "1.5.0"
description = "plugin and hook calling mechanisms for python"
optional = false
python-versions = ">=3.8"
files = [
{file = "pluggy-1.5.0-py3-none-any.whl", hash = "sha256:44e1ad92c8ca002de6377e165f3e0f1be63266ab4d554740532335b9d75ea669"},
{file = "pluggy-1.5.0.tar.gz", hash = "sha256:2cffa88e94fdc978c4c574f15f9e59b7f4201d439195c3715ca9e2486f1d0cf1"},
]
[package.extras]
dev = ["pre-commit", "tox"]
testing = ["pytest", "pytest-benchmark"]
[[package]]
name = "pydantic"
version = "2.8.2"
description = "Data validation using Python type hints"
optional = false
python-versions = ">=3.8"
files = [
{file = "pydantic-2.8.2-py3-none-any.whl", hash = "sha256:73ee9fddd406dc318b885c7a2eab8a6472b68b8fb5ba8150949fc3db939f23c8"},
{file = "pydantic-2.8.2.tar.gz", hash = "sha256:6f62c13d067b0755ad1c21a34bdd06c0c12625a22b0fc09c6b149816604f7c2a"},
]
[package.dependencies]
annotated-types = ">=0.4.0"
pydantic-core = "2.20.1"
typing-extensions = [
{version = ">=4.6.1", markers = "python_version < \"3.13\""},
{version = ">=4.12.2", markers = "python_version >= \"3.13\""},
]
[package.extras]
email = ["email-validator (>=2.0.0)"]
[[package]]
name = "pydantic-core"
version = "2.20.1"
description = "Core functionality for Pydantic validation and serialization"
optional = false
python-versions = ">=3.8"
files = [
{file = "pydantic_core-2.20.1-cp310-cp310-macosx_10_12_x86_64.whl", hash = "sha256:3acae97ffd19bf091c72df4d726d552c473f3576409b2a7ca36b2f535ffff4a3"},
{file = "pydantic_core-2.20.1-cp310-cp310-macosx_11_0_arm64.whl", hash = "sha256:41f4c96227a67a013e7de5ff8f20fb496ce573893b7f4f2707d065907bffdbd6"},
{file = "pydantic_core-2.20.1-cp310-cp310-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:5f239eb799a2081495ea659d8d4a43a8f42cd1fe9ff2e7e436295c38a10c286a"},
{file = "pydantic_core-2.20.1-cp310-cp310-manylinux_2_17_armv7l.manylinux2014_armv7l.whl", hash = "sha256:53e431da3fc53360db73eedf6f7124d1076e1b4ee4276b36fb25514544ceb4a3"},
{file = "pydantic_core-2.20.1-cp310-cp310-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", hash = "sha256:f1f62b2413c3a0e846c3b838b2ecd6c7a19ec6793b2a522745b0869e37ab5bc1"},
{file = "pydantic_core-2.20.1-cp310-cp310-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:5d41e6daee2813ecceea8eda38062d69e280b39df793f5a942fa515b8ed67953"},
{file = "pydantic_core-2.20.1-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:3d482efec8b7dc6bfaedc0f166b2ce349df0011f5d2f1f25537ced4cfc34fd98"},
{file = "pydantic_core-2.20.1-cp310-cp310-manylinux_2_5_i686.manylinux1_i686.whl", hash = "sha256:e93e1a4b4b33daed65d781a57a522ff153dcf748dee70b40c7258c5861e1768a"},
{file = "pydantic_core-2.20.1-cp310-cp310-musllinux_1_1_aarch64.whl", hash = "sha256:e7c4ea22b6739b162c9ecaaa41d718dfad48a244909fe7ef4b54c0b530effc5a"},
{file = "pydantic_core-2.20.1-cp310-cp310-musllinux_1_1_x86_64.whl", hash = "sha256:4f2790949cf385d985a31984907fecb3896999329103df4e4983a4a41e13e840"},
{file = "pydantic_core-2.20.1-cp310-none-win32.whl", hash = "sha256:5e999ba8dd90e93d57410c5e67ebb67ffcaadcea0ad973240fdfd3a135506250"},
{file = "pydantic_core-2.20.1-cp310-none-win_amd64.whl", hash = "sha256:512ecfbefef6dac7bc5eaaf46177b2de58cdf7acac8793fe033b24ece0b9566c"},
{file = "pydantic_core-2.20.1-cp311-cp311-macosx_10_12_x86_64.whl", hash = "sha256:d2a8fa9d6d6f891f3deec72f5cc668e6f66b188ab14bb1ab52422fe8e644f312"},
{file = "pydantic_core-2.20.1-cp311-cp311-macosx_11_0_arm64.whl", hash = "sha256:175873691124f3d0da55aeea1d90660a6ea7a3cfea137c38afa0a5ffabe37b88"},
{file = "pydantic_core-2.20.1-cp311-cp311-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:37eee5b638f0e0dcd18d21f59b679686bbd18917b87db0193ae36f9c23c355fc"},
{file = "pydantic_core-2.20.1-cp311-cp311-manylinux_2_17_armv7l.manylinux2014_armv7l.whl", hash = "sha256:25e9185e2d06c16ee438ed39bf62935ec436474a6ac4f9358524220f1b236e43"},
{file = "pydantic_core-2.20.1-cp311-cp311-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", hash = "sha256:150906b40ff188a3260cbee25380e7494ee85048584998c1e66df0c7a11c17a6"},
{file = "pydantic_core-2.20.1-cp311-cp311-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:8ad4aeb3e9a97286573c03df758fc7627aecdd02f1da04516a86dc159bf70121"},
{file = "pydantic_core-2.20.1-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:d3f3ed29cd9f978c604708511a1f9c2fdcb6c38b9aae36a51905b8811ee5cbf1"},
{file = "pydantic_core-2.20.1-cp311-cp311-manylinux_2_5_i686.manylinux1_i686.whl", hash = "sha256:b0dae11d8f5ded51699c74d9548dcc5938e0804cc8298ec0aa0da95c21fff57b"},
{file = "pydantic_core-2.20.1-cp311-cp311-musllinux_1_1_aarch64.whl", hash = "sha256:faa6b09ee09433b87992fb5a2859efd1c264ddc37280d2dd5db502126d0e7f27"},
{file = "pydantic_core-2.20.1-cp311-cp311-musllinux_1_1_x86_64.whl", hash = "sha256:9dc1b507c12eb0481d071f3c1808f0529ad41dc415d0ca11f7ebfc666e66a18b"},
{file = "pydantic_core-2.20.1-cp311-none-win32.whl", hash = "sha256:fa2fddcb7107e0d1808086ca306dcade7df60a13a6c347a7acf1ec139aa6789a"},
{file = "pydantic_core-2.20.1-cp311-none-win_amd64.whl", hash = "sha256:40a783fb7ee353c50bd3853e626f15677ea527ae556429453685ae32280c19c2"},
{file = "pydantic_core-2.20.1-cp312-cp312-macosx_10_12_x86_64.whl", hash = "sha256:595ba5be69b35777474fa07f80fc260ea71255656191adb22a8c53aba4479231"},
{file = "pydantic_core-2.20.1-cp312-cp312-macosx_11_0_arm64.whl", hash = "sha256:a4f55095ad087474999ee28d3398bae183a66be4823f753cd7d67dd0153427c9"},
{file = "pydantic_core-2.20.1-cp312-cp312-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:f9aa05d09ecf4c75157197f27cdc9cfaeb7c5f15021c6373932bf3e124af029f"},
{file = "pydantic_core-2.20.1-cp312-cp312-manylinux_2_17_armv7l.manylinux2014_armv7l.whl", hash = "sha256:e97fdf088d4b31ff4ba35db26d9cc472ac7ef4a2ff2badeabf8d727b3377fc52"},
{file = "pydantic_core-2.20.1-cp312-cp312-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", hash = "sha256:bc633a9fe1eb87e250b5c57d389cf28998e4292336926b0b6cdaee353f89a237"},
{file = "pydantic_core-2.20.1-cp312-cp312-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:d573faf8eb7e6b1cbbcb4f5b247c60ca8be39fe2c674495df0eb4318303137fe"},
{file = "pydantic_core-2.20.1-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:26dc97754b57d2fd00ac2b24dfa341abffc380b823211994c4efac7f13b9e90e"},
{file = "pydantic_core-2.20.1-cp312-cp312-manylinux_2_5_i686.manylinux1_i686.whl", hash = "sha256:33499e85e739a4b60c9dac710c20a08dc73cb3240c9a0e22325e671b27b70d24"},
{file = "pydantic_core-2.20.1-cp312-cp312-musllinux_1_1_aarch64.whl", hash = "sha256:bebb4d6715c814597f85297c332297c6ce81e29436125ca59d1159b07f423eb1"},
{file = "pydantic_core-2.20.1-cp312-cp312-musllinux_1_1_x86_64.whl", hash = "sha256:516d9227919612425c8ef1c9b869bbbee249bc91912c8aaffb66116c0b447ebd"},
{file = "pydantic_core-2.20.1-cp312-none-win32.whl", hash = "sha256:469f29f9093c9d834432034d33f5fe45699e664f12a13bf38c04967ce233d688"},
{file = "pydantic_core-2.20.1-cp312-none-win_amd64.whl", hash = "sha256:035ede2e16da7281041f0e626459bcae33ed998cca6a0a007a5ebb73414ac72d"},
{file = "pydantic_core-2.20.1-cp313-cp313-macosx_10_12_x86_64.whl", hash = "sha256:0827505a5c87e8aa285dc31e9ec7f4a17c81a813d45f70b1d9164e03a813a686"},
{file = "pydantic_core-2.20.1-cp313-cp313-macosx_11_0_arm64.whl", hash = "sha256:19c0fa39fa154e7e0b7f82f88ef85faa2a4c23cc65aae2f5aea625e3c13c735a"},
{file = "pydantic_core-2.20.1-cp313-cp313-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:4aa223cd1e36b642092c326d694d8bf59b71ddddc94cdb752bbbb1c5c91d833b"},
{file = "pydantic_core-2.20.1-cp313-cp313-manylinux_2_17_armv7l.manylinux2014_armv7l.whl", hash = "sha256:c336a6d235522a62fef872c6295a42ecb0c4e1d0f1a3e500fe949415761b8a19"},
{file = "pydantic_core-2.20.1-cp313-cp313-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", hash = "sha256:7eb6a0587eded33aeefea9f916899d42b1799b7b14b8f8ff2753c0ac1741edac"},
{file = "pydantic_core-2.20.1-cp313-cp313-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:70c8daf4faca8da5a6d655f9af86faf6ec2e1768f4b8b9d0226c02f3d6209703"},
{file = "pydantic_core-2.20.1-cp313-cp313-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:e9fa4c9bf273ca41f940bceb86922a7667cd5bf90e95dbb157cbb8441008482c"},
{file = "pydantic_core-2.20.1-cp313-cp313-manylinux_2_5_i686.manylinux1_i686.whl", hash = "sha256:11b71d67b4725e7e2a9f6e9c0ac1239bbc0c48cce3dc59f98635efc57d6dac83"},
{file = "pydantic_core-2.20.1-cp313-cp313-musllinux_1_1_aarch64.whl", hash = "sha256:270755f15174fb983890c49881e93f8f1b80f0b5e3a3cc1394a255706cabd203"},
{file = "pydantic_core-2.20.1-cp313-cp313-musllinux_1_1_x86_64.whl", hash = "sha256:c81131869240e3e568916ef4c307f8b99583efaa60a8112ef27a366eefba8ef0"},
{file = "pydantic_core-2.20.1-cp313-none-win32.whl", hash = "sha256:b91ced227c41aa29c672814f50dbb05ec93536abf8f43cd14ec9521ea09afe4e"},
{file = "pydantic_core-2.20.1-cp313-none-win_amd64.whl", hash = "sha256:65db0f2eefcaad1a3950f498aabb4875c8890438bc80b19362cf633b87a8ab20"},
{file = "pydantic_core-2.20.1-cp38-cp38-macosx_10_12_x86_64.whl", hash = "sha256:4745f4ac52cc6686390c40eaa01d48b18997cb130833154801a442323cc78f91"},
{file = "pydantic_core-2.20.1-cp38-cp38-macosx_11_0_arm64.whl", hash = "sha256:a8ad4c766d3f33ba8fd692f9aa297c9058970530a32c728a2c4bfd2616d3358b"},
{file = "pydantic_core-2.20.1-cp38-cp38-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:41e81317dd6a0127cabce83c0c9c3fbecceae981c8391e6f1dec88a77c8a569a"},
{file = "pydantic_core-2.20.1-cp38-cp38-manylinux_2_17_armv7l.manylinux2014_armv7l.whl", hash = "sha256:04024d270cf63f586ad41fff13fde4311c4fc13ea74676962c876d9577bcc78f"},
{file = "pydantic_core-2.20.1-cp38-cp38-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", hash = "sha256:eaad4ff2de1c3823fddf82f41121bdf453d922e9a238642b1dedb33c4e4f98ad"},
{file = "pydantic_core-2.20.1-cp38-cp38-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:26ab812fa0c845df815e506be30337e2df27e88399b985d0bb4e3ecfe72df31c"},
{file = "pydantic_core-2.20.1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:3c5ebac750d9d5f2706654c638c041635c385596caf68f81342011ddfa1e5598"},
{file = "pydantic_core-2.20.1-cp38-cp38-manylinux_2_5_i686.manylinux1_i686.whl", hash = "sha256:2aafc5a503855ea5885559eae883978c9b6d8c8993d67766ee73d82e841300dd"},
{file = "pydantic_core-2.20.1-cp38-cp38-musllinux_1_1_aarch64.whl", hash = "sha256:4868f6bd7c9d98904b748a2653031fc9c2f85b6237009d475b1008bfaeb0a5aa"},
{file = "pydantic_core-2.20.1-cp38-cp38-musllinux_1_1_x86_64.whl", hash = "sha256:aa2f457b4af386254372dfa78a2eda2563680d982422641a85f271c859df1987"},
{file = "pydantic_core-2.20.1-cp38-none-win32.whl", hash = "sha256:225b67a1f6d602de0ce7f6c1c3ae89a4aa25d3de9be857999e9124f15dab486a"},
{file = "pydantic_core-2.20.1-cp38-none-win_amd64.whl", hash = "sha256:6b507132dcfc0dea440cce23ee2182c0ce7aba7054576efc65634f080dbe9434"},
{file = "pydantic_core-2.20.1-cp39-cp39-macosx_10_12_x86_64.whl", hash = "sha256:b03f7941783b4c4a26051846dea594628b38f6940a2fdc0df00b221aed39314c"},
{file = "pydantic_core-2.20.1-cp39-cp39-macosx_11_0_arm64.whl", hash = "sha256:1eedfeb6089ed3fad42e81a67755846ad4dcc14d73698c120a82e4ccf0f1f9f6"},
{file = "pydantic_core-2.20.1-cp39-cp39-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:635fee4e041ab9c479e31edda27fcf966ea9614fff1317e280d99eb3e5ab6fe2"},
{file = "pydantic_core-2.20.1-cp39-cp39-manylinux_2_17_armv7l.manylinux2014_armv7l.whl", hash = "sha256:77bf3ac639c1ff567ae3b47f8d4cc3dc20f9966a2a6dd2311dcc055d3d04fb8a"},
{file = "pydantic_core-2.20.1-cp39-cp39-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", hash = "sha256:7ed1b0132f24beeec5a78b67d9388656d03e6a7c837394f99257e2d55b461611"},
{file = "pydantic_core-2.20.1-cp39-cp39-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:c6514f963b023aeee506678a1cf821fe31159b925c4b76fe2afa94cc70b3222b"},
{file = "pydantic_core-2.20.1-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:10d4204d8ca33146e761c79f83cc861df20e7ae9f6487ca290a97702daf56006"},
{file = "pydantic_core-2.20.1-cp39-cp39-manylinux_2_5_i686.manylinux1_i686.whl", hash = "sha256:2d036c7187b9422ae5b262badb87a20a49eb6c5238b2004e96d4da1231badef1"},
{file = "pydantic_core-2.20.1-cp39-cp39-musllinux_1_1_aarch64.whl", hash = "sha256:9ebfef07dbe1d93efb94b4700f2d278494e9162565a54f124c404a5656d7ff09"},
{file = "pydantic_core-2.20.1-cp39-cp39-musllinux_1_1_x86_64.whl", hash = "sha256:6b9d9bb600328a1ce523ab4f454859e9d439150abb0906c5a1983c146580ebab"},
{file = "pydantic_core-2.20.1-cp39-none-win32.whl", hash = "sha256:784c1214cb6dd1e3b15dd8b91b9a53852aed16671cc3fbe4786f4f1db07089e2"},
{file = "pydantic_core-2.20.1-cp39-none-win_amd64.whl", hash = "sha256:d2fe69c5434391727efa54b47a1e7986bb0186e72a41b203df8f5b0a19a4f669"},
{file = "pydantic_core-2.20.1-pp310-pypy310_pp73-macosx_10_12_x86_64.whl", hash = "sha256:a45f84b09ac9c3d35dfcf6a27fd0634d30d183205230a0ebe8373a0e8cfa0906"},
{file = "pydantic_core-2.20.1-pp310-pypy310_pp73-macosx_11_0_arm64.whl", hash = "sha256:d02a72df14dfdbaf228424573a07af10637bd490f0901cee872c4f434a735b94"},
{file = "pydantic_core-2.20.1-pp310-pypy310_pp73-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:d2b27e6af28f07e2f195552b37d7d66b150adbaa39a6d327766ffd695799780f"},
{file = "pydantic_core-2.20.1-pp310-pypy310_pp73-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:084659fac3c83fd674596612aeff6041a18402f1e1bc19ca39e417d554468482"},
{file = "pydantic_core-2.20.1-pp310-pypy310_pp73-manylinux_2_5_i686.manylinux1_i686.whl", hash = "sha256:242b8feb3c493ab78be289c034a1f659e8826e2233786e36f2893a950a719bb6"},
{file = "pydantic_core-2.20.1-pp310-pypy310_pp73-musllinux_1_1_aarch64.whl", hash = "sha256:38cf1c40a921d05c5edc61a785c0ddb4bed67827069f535d794ce6bcded919fc"},
{file = "pydantic_core-2.20.1-pp310-pypy310_pp73-musllinux_1_1_x86_64.whl", hash = "sha256:e0bbdd76ce9aa5d4209d65f2b27fc6e5ef1312ae6c5333c26db3f5ade53a1e99"},
{file = "pydantic_core-2.20.1-pp310-pypy310_pp73-win_amd64.whl", hash = "sha256:254ec27fdb5b1ee60684f91683be95e5133c994cc54e86a0b0963afa25c8f8a6"},
{file = "pydantic_core-2.20.1-pp39-pypy39_pp73-macosx_10_12_x86_64.whl", hash = "sha256:407653af5617f0757261ae249d3fba09504d7a71ab36ac057c938572d1bc9331"},
{file = "pydantic_core-2.20.1-pp39-pypy39_pp73-macosx_11_0_arm64.whl", hash = "sha256:c693e916709c2465b02ca0ad7b387c4f8423d1db7b4649c551f27a529181c5ad"},
{file = "pydantic_core-2.20.1-pp39-pypy39_pp73-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:5b5ff4911aea936a47d9376fd3ab17e970cc543d1b68921886e7f64bd28308d1"},
{file = "pydantic_core-2.20.1-pp39-pypy39_pp73-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:177f55a886d74f1808763976ac4efd29b7ed15c69f4d838bbd74d9d09cf6fa86"},
{file = "pydantic_core-2.20.1-pp39-pypy39_pp73-manylinux_2_5_i686.manylinux1_i686.whl", hash = "sha256:964faa8a861d2664f0c7ab0c181af0bea66098b1919439815ca8803ef136fc4e"},
{file = "pydantic_core-2.20.1-pp39-pypy39_pp73-musllinux_1_1_aarch64.whl", hash = "sha256:4dd484681c15e6b9a977c785a345d3e378d72678fd5f1f3c0509608da24f2ac0"},
{file = "pydantic_core-2.20.1-pp39-pypy39_pp73-musllinux_1_1_x86_64.whl", hash = "sha256:f6d6cff3538391e8486a431569b77921adfcdef14eb18fbf19b7c0a5294d4e6a"},
{file = "pydantic_core-2.20.1-pp39-pypy39_pp73-win_amd64.whl", hash = "sha256:a6d511cc297ff0883bc3708b465ff82d7560193169a8b93260f74ecb0a5e08a7"},
{file = "pydantic_core-2.20.1.tar.gz", hash = "sha256:26ca695eeee5f9f1aeeb211ffc12f10bcb6f71e2989988fda61dabd65db878d4"},
]
[package.dependencies]
typing-extensions = ">=4.6.0,<4.7.0 || >4.7.0"
[[package]]
name = "pytest"
version = "7.4.4"
description = "pytest: simple powerful testing with Python"
optional = false
python-versions = ">=3.7"
files = [
{file = "pytest-7.4.4-py3-none-any.whl", hash = "sha256:b090cdf5ed60bf4c45261be03239c2c1c22df034fbffe691abe93cd80cea01d8"},
{file = "pytest-7.4.4.tar.gz", hash = "sha256:2cf0005922c6ace4a3e2ec8b4080eb0d9753fdc93107415332f50ce9e7994280"},
]
[package.dependencies]
colorama = {version = "*", markers = "sys_platform == \"win32\""}
exceptiongroup = {version = ">=1.0.0rc8", markers = "python_version < \"3.11\""}
iniconfig = "*"
packaging = "*"
pluggy = ">=0.12,<2.0"
tomli = {version = ">=1.0.0", markers = "python_version < \"3.11\""}
[package.extras]
testing = ["argcomplete", "attrs (>=19.2.0)", "hypothesis (>=3.56)", "mock", "nose", "pygments (>=2.7.2)", "requests", "setuptools", "xmlschema"]
[[package]]
name = "pytest-asyncio"
version = "0.21.2"
description = "Pytest support for asyncio"
optional = false
python-versions = ">=3.7"
files = [
{file = "pytest_asyncio-0.21.2-py3-none-any.whl", hash = "sha256:ab664c88bb7998f711d8039cacd4884da6430886ae8bbd4eded552ed2004f16b"},
{file = "pytest_asyncio-0.21.2.tar.gz", hash = "sha256:d67738fc232b94b326b9d060750beb16e0074210b98dd8b58a5239fa2a154f45"},
]
[package.dependencies]
pytest = ">=7.0.0"
[package.extras]
docs = ["sphinx (>=5.3)", "sphinx-rtd-theme (>=1.0)"]
testing = ["coverage (>=6.2)", "flaky (>=3.5.0)", "hypothesis (>=5.7.1)", "mypy (>=0.931)", "pytest-trio (>=0.7.0)"]
[[package]]
name = "pytest-mock"
version = "3.14.0"
description = "Thin-wrapper around the mock package for easier use with pytest"
optional = false
python-versions = ">=3.8"
files = [
{file = "pytest-mock-3.14.0.tar.gz", hash = "sha256:2719255a1efeceadbc056d6bf3df3d1c5015530fb40cf347c0f9afac88410bd0"},
{file = "pytest_mock-3.14.0-py3-none-any.whl", hash = "sha256:0b72c38033392a5f4621342fe11e9219ac11ec9d375f8e2a0c164539e0d70f6f"},
]
[package.dependencies]
pytest = ">=6.2.5"
[package.extras]
dev = ["pre-commit", "pytest-asyncio", "tox"]
[[package]]
name = "pytest-watcher"
version = "0.4.2"
description = "Automatically rerun your tests on file modifications"
optional = false
python-versions = "<4.0.0,>=3.7.0"
files = [
{file = "pytest_watcher-0.4.2-py3-none-any.whl", hash = "sha256:a43949ba67dd8d7e1fd0de5eea44a999081f0aec9f93b4e744264b4c6a3d9bbe"},
{file = "pytest_watcher-0.4.2.tar.gz", hash = "sha256:7b292f025ca19617cd7567c228c6187b5087f2da9e4d2cf6e144e5764a0471b0"},
]
[package.dependencies]
tomli = {version = ">=2.0.1,<3.0.0", markers = "python_version < \"3.11\""}
watchdog = ">=2.0.0"
[[package]]
name = "pyyaml"
version = "6.0.1"
description = "YAML parser and emitter for Python"
optional = false
python-versions = ">=3.6"
files = [
{file = "PyYAML-6.0.1-cp310-cp310-macosx_10_9_x86_64.whl", hash = "sha256:d858aa552c999bc8a8d57426ed01e40bef403cd8ccdd0fc5f6f04a00414cac2a"},
{file = "PyYAML-6.0.1-cp310-cp310-macosx_11_0_arm64.whl", hash = "sha256:fd66fc5d0da6d9815ba2cebeb4205f95818ff4b79c3ebe268e75d961704af52f"},
{file = "PyYAML-6.0.1-cp310-cp310-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:69b023b2b4daa7548bcfbd4aa3da05b3a74b772db9e23b982788168117739938"},
{file = "PyYAML-6.0.1-cp310-cp310-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:81e0b275a9ecc9c0c0c07b4b90ba548307583c125f54d5b6946cfee6360c733d"},
{file = "PyYAML-6.0.1-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:ba336e390cd8e4d1739f42dfe9bb83a3cc2e80f567d8805e11b46f4a943f5515"},
{file = "PyYAML-6.0.1-cp310-cp310-musllinux_1_1_x86_64.whl", hash = "sha256:326c013efe8048858a6d312ddd31d56e468118ad4cdeda36c719bf5bb6192290"},
{file = "PyYAML-6.0.1-cp310-cp310-win32.whl", hash = "sha256:bd4af7373a854424dabd882decdc5579653d7868b8fb26dc7d0e99f823aa5924"},
{file = "PyYAML-6.0.1-cp310-cp310-win_amd64.whl", hash = "sha256:fd1592b3fdf65fff2ad0004b5e363300ef59ced41c2e6b3a99d4089fa8c5435d"},
{file = "PyYAML-6.0.1-cp311-cp311-macosx_10_9_x86_64.whl", hash = "sha256:6965a7bc3cf88e5a1c3bd2e0b5c22f8d677dc88a455344035f03399034eb3007"},
{file = "PyYAML-6.0.1-cp311-cp311-macosx_11_0_arm64.whl", hash = "sha256:f003ed9ad21d6a4713f0a9b5a7a0a79e08dd0f221aff4525a2be4c346ee60aab"},
{file = "PyYAML-6.0.1-cp311-cp311-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:42f8152b8dbc4fe7d96729ec2b99c7097d656dc1213a3229ca5383f973a5ed6d"},
{file = "PyYAML-6.0.1-cp311-cp311-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:062582fca9fabdd2c8b54a3ef1c978d786e0f6b3a1510e0ac93ef59e0ddae2bc"},
{file = "PyYAML-6.0.1-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:d2b04aac4d386b172d5b9692e2d2da8de7bfb6c387fa4f801fbf6fb2e6ba4673"},
{file = "PyYAML-6.0.1-cp311-cp311-musllinux_1_1_x86_64.whl", hash = "sha256:e7d73685e87afe9f3b36c799222440d6cf362062f78be1013661b00c5c6f678b"},
{file = "PyYAML-6.0.1-cp311-cp311-win32.whl", hash = "sha256:1635fd110e8d85d55237ab316b5b011de701ea0f29d07611174a1b42f1444741"},
{file = "PyYAML-6.0.1-cp311-cp311-win_amd64.whl", hash = "sha256:bf07ee2fef7014951eeb99f56f39c9bb4af143d8aa3c21b1677805985307da34"},
{file = "PyYAML-6.0.1-cp312-cp312-macosx_10_9_x86_64.whl", hash = "sha256:855fb52b0dc35af121542a76b9a84f8d1cd886ea97c84703eaa6d88e37a2ad28"},
{file = "PyYAML-6.0.1-cp312-cp312-macosx_11_0_arm64.whl", hash = "sha256:40df9b996c2b73138957fe23a16a4f0ba614f4c0efce1e9406a184b6d07fa3a9"},
{file = "PyYAML-6.0.1-cp312-cp312-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:a08c6f0fe150303c1c6b71ebcd7213c2858041a7e01975da3a99aed1e7a378ef"},
{file = "PyYAML-6.0.1-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:6c22bec3fbe2524cde73d7ada88f6566758a8f7227bfbf93a408a9d86bcc12a0"},
{file = "PyYAML-6.0.1-cp312-cp312-musllinux_1_1_x86_64.whl", hash = "sha256:8d4e9c88387b0f5c7d5f281e55304de64cf7f9c0021a3525bd3b1c542da3b0e4"},
{file = "PyYAML-6.0.1-cp312-cp312-win32.whl", hash = "sha256:d483d2cdf104e7c9fa60c544d92981f12ad66a457afae824d146093b8c294c54"},
{file = "PyYAML-6.0.1-cp312-cp312-win_amd64.whl", hash = "sha256:0d3304d8c0adc42be59c5f8a4d9e3d7379e6955ad754aa9d6ab7a398b59dd1df"},
{file = "PyYAML-6.0.1-cp36-cp36m-macosx_10_9_x86_64.whl", hash = "sha256:50550eb667afee136e9a77d6dc71ae76a44df8b3e51e41b77f6de2932bfe0f47"},
{file = "PyYAML-6.0.1-cp36-cp36m-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:1fe35611261b29bd1de0070f0b2f47cb6ff71fa6595c077e42bd0c419fa27b98"},
{file = "PyYAML-6.0.1-cp36-cp36m-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:704219a11b772aea0d8ecd7058d0082713c3562b4e271b849ad7dc4a5c90c13c"},
{file = "PyYAML-6.0.1-cp36-cp36m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:afd7e57eddb1a54f0f1a974bc4391af8bcce0b444685d936840f125cf046d5bd"},
{file = "PyYAML-6.0.1-cp36-cp36m-win32.whl", hash = "sha256:fca0e3a251908a499833aa292323f32437106001d436eca0e6e7833256674585"},
{file = "PyYAML-6.0.1-cp36-cp36m-win_amd64.whl", hash = "sha256:f22ac1c3cac4dbc50079e965eba2c1058622631e526bd9afd45fedd49ba781fa"},
{file = "PyYAML-6.0.1-cp37-cp37m-macosx_10_9_x86_64.whl", hash = "sha256:b1275ad35a5d18c62a7220633c913e1b42d44b46ee12554e5fd39c70a243d6a3"},
{file = "PyYAML-6.0.1-cp37-cp37m-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:18aeb1bf9a78867dc38b259769503436b7c72f7a1f1f4c93ff9a17de54319b27"},
{file = "PyYAML-6.0.1-cp37-cp37m-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:596106435fa6ad000c2991a98fa58eeb8656ef2325d7e158344fb33864ed87e3"},
{file = "PyYAML-6.0.1-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:baa90d3f661d43131ca170712d903e6295d1f7a0f595074f151c0aed377c9b9c"},
{file = "PyYAML-6.0.1-cp37-cp37m-win32.whl", hash = "sha256:9046c58c4395dff28dd494285c82ba00b546adfc7ef001486fbf0324bc174fba"},
{file = "PyYAML-6.0.1-cp37-cp37m-win_amd64.whl", hash = "sha256:4fb147e7a67ef577a588a0e2c17b6db51dda102c71de36f8549b6816a96e1867"},
{file = "PyYAML-6.0.1-cp38-cp38-macosx_10_9_x86_64.whl", hash = "sha256:1d4c7e777c441b20e32f52bd377e0c409713e8bb1386e1099c2415f26e479595"},
{file = "PyYAML-6.0.1-cp38-cp38-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:a0cd17c15d3bb3fa06978b4e8958dcdc6e0174ccea823003a106c7d4d7899ac5"},
{file = "PyYAML-6.0.1-cp38-cp38-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:28c119d996beec18c05208a8bd78cbe4007878c6dd15091efb73a30e90539696"},
{file = "PyYAML-6.0.1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:7e07cbde391ba96ab58e532ff4803f79c4129397514e1413a7dc761ccd755735"},
{file = "PyYAML-6.0.1-cp38-cp38-musllinux_1_1_x86_64.whl", hash = "sha256:49a183be227561de579b4a36efbb21b3eab9651dd81b1858589f796549873dd6"},
{file = "PyYAML-6.0.1-cp38-cp38-win32.whl", hash = "sha256:184c5108a2aca3c5b3d3bf9395d50893a7ab82a38004c8f61c258d4428e80206"},
{file = "PyYAML-6.0.1-cp38-cp38-win_amd64.whl", hash = "sha256:1e2722cc9fbb45d9b87631ac70924c11d3a401b2d7f410cc0e3bbf249f2dca62"},
{file = "PyYAML-6.0.1-cp39-cp39-macosx_10_9_x86_64.whl", hash = "sha256:9eb6caa9a297fc2c2fb8862bc5370d0303ddba53ba97e71f08023b6cd73d16a8"},
{file = "PyYAML-6.0.1-cp39-cp39-macosx_11_0_arm64.whl", hash = "sha256:c8098ddcc2a85b61647b2590f825f3db38891662cfc2fc776415143f599bb859"},
{file = "PyYAML-6.0.1-cp39-cp39-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:5773183b6446b2c99bb77e77595dd486303b4faab2b086e7b17bc6bef28865f6"},
{file = "PyYAML-6.0.1-cp39-cp39-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:b786eecbdf8499b9ca1d697215862083bd6d2a99965554781d0d8d1ad31e13a0"},
{file = "PyYAML-6.0.1-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:bc1bf2925a1ecd43da378f4db9e4f799775d6367bdb94671027b73b393a7c42c"},
{file = "PyYAML-6.0.1-cp39-cp39-musllinux_1_1_x86_64.whl", hash = "sha256:04ac92ad1925b2cff1db0cfebffb6ffc43457495c9b3c39d3fcae417d7125dc5"},
{file = "PyYAML-6.0.1-cp39-cp39-win32.whl", hash = "sha256:faca3bdcf85b2fc05d06ff3fbc1f83e1391b3e724afa3feba7d13eeab355484c"},
{file = "PyYAML-6.0.1-cp39-cp39-win_amd64.whl", hash = "sha256:510c9deebc5c0225e8c96813043e62b680ba2f9c50a08d3724c7f28a747d1486"},
{file = "PyYAML-6.0.1.tar.gz", hash = "sha256:bfdf460b1736c775f2ba9f6a92bca30bc2095067b8a9d77876d1fad6cc3b4a43"},
]
[[package]]
name = "requests"
version = "2.32.3"
description = "Python HTTP for Humans."
optional = false
python-versions = ">=3.8"
files = [
{file = "requests-2.32.3-py3-none-any.whl", hash = "sha256:70761cfe03c773ceb22aa2f671b4757976145175cdfca038c02654d061d6dcc6"},
{file = "requests-2.32.3.tar.gz", hash = "sha256:55365417734eb18255590a9ff9eb97e9e1da868d4ccd6402399eaf68af20a760"},
]
[package.dependencies]
certifi = ">=2017.4.17"
charset-normalizer = ">=2,<4"
idna = ">=2.5,<4"
urllib3 = ">=1.21.1,<3"
[package.extras]
socks = ["PySocks (>=1.5.6,!=1.5.7)"]
use-chardet-on-py3 = ["chardet (>=3.0.2,<6)"]
[[package]]
name = "ruff"
version = "0.6.2"
description = "An extremely fast Python linter and code formatter, written in Rust."
optional = false
python-versions = ">=3.7"
files = [
{file = "ruff-0.6.2-py3-none-linux_armv6l.whl", hash = "sha256:5c8cbc6252deb3ea840ad6a20b0f8583caab0c5ef4f9cca21adc5a92b8f79f3c"},
{file = "ruff-0.6.2-py3-none-macosx_10_12_x86_64.whl", hash = "sha256:17002fe241e76544448a8e1e6118abecbe8cd10cf68fde635dad480dba594570"},
{file = "ruff-0.6.2-py3-none-macosx_11_0_arm64.whl", hash = "sha256:3dbeac76ed13456f8158b8f4fe087bf87882e645c8e8b606dd17b0b66c2c1158"},
{file = "ruff-0.6.2-py3-none-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:094600ee88cda325988d3f54e3588c46de5c18dae09d683ace278b11f9d4d534"},
{file = "ruff-0.6.2-py3-none-manylinux_2_17_armv7l.manylinux2014_armv7l.whl", hash = "sha256:316d418fe258c036ba05fbf7dfc1f7d3d4096db63431546163b472285668132b"},
{file = "ruff-0.6.2-py3-none-manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:d72b8b3abf8a2d51b7b9944a41307d2f442558ccb3859bbd87e6ae9be1694a5d"},
{file = "ruff-0.6.2-py3-none-manylinux_2_17_ppc64.manylinux2014_ppc64.whl", hash = "sha256:2aed7e243be68487aa8982e91c6e260982d00da3f38955873aecd5a9204b1d66"},
{file = "ruff-0.6.2-py3-none-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", hash = "sha256:d371f7fc9cec83497fe7cf5eaf5b76e22a8efce463de5f775a1826197feb9df8"},
{file = "ruff-0.6.2-py3-none-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:a8f310d63af08f583363dfb844ba8f9417b558199c58a5999215082036d795a1"},
{file = "ruff-0.6.2-py3-none-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:7db6880c53c56addb8638fe444818183385ec85eeada1d48fc5abe045301b2f1"},
{file = "ruff-0.6.2-py3-none-musllinux_1_2_aarch64.whl", hash = "sha256:1175d39faadd9a50718f478d23bfc1d4da5743f1ab56af81a2b6caf0a2394f23"},
{file = "ruff-0.6.2-py3-none-musllinux_1_2_armv7l.whl", hash = "sha256:5b939f9c86d51635fe486585389f54582f0d65b8238e08c327c1534844b3bb9a"},
{file = "ruff-0.6.2-py3-none-musllinux_1_2_i686.whl", hash = "sha256:d0d62ca91219f906caf9b187dea50d17353f15ec9bb15aae4a606cd697b49b4c"},
{file = "ruff-0.6.2-py3-none-musllinux_1_2_x86_64.whl", hash = "sha256:7438a7288f9d67ed3c8ce4d059e67f7ed65e9fe3aa2ab6f5b4b3610e57e3cb56"},
{file = "ruff-0.6.2-py3-none-win32.whl", hash = "sha256:279d5f7d86696df5f9549b56b9b6a7f6c72961b619022b5b7999b15db392a4da"},
{file = "ruff-0.6.2-py3-none-win_amd64.whl", hash = "sha256:d9f3469c7dd43cd22eb1c3fc16926fb8258d50cb1b216658a07be95dd117b0f2"},
{file = "ruff-0.6.2-py3-none-win_arm64.whl", hash = "sha256:f28fcd2cd0e02bdf739297516d5643a945cc7caf09bd9bcb4d932540a5ea4fa9"},
{file = "ruff-0.6.2.tar.gz", hash = "sha256:239ee6beb9e91feb8e0ec384204a763f36cb53fb895a1a364618c6abb076b3be"},
]
[[package]]
name = "tenacity"
version = "8.5.0"
description = "Retry code until it succeeds"
optional = false
python-versions = ">=3.8"
files = [
{file = "tenacity-8.5.0-py3-none-any.whl", hash = "sha256:b594c2a5945830c267ce6b79a166228323ed52718f30302c1359836112346687"},
{file = "tenacity-8.5.0.tar.gz", hash = "sha256:8bc6c0c8a09b31e6cad13c47afbed1a567518250a9a171418582ed8d9c20ca78"},
]
[package.extras]
doc = ["reno", "sphinx"]
test = ["pytest", "tornado (>=4.5)", "typeguard"]
[[package]]
name = "tomli"
version = "2.0.1"
description = "A lil' TOML parser"
optional = false
python-versions = ">=3.7"
files = [
{file = "tomli-2.0.1-py3-none-any.whl", hash = "sha256:939de3e7a6161af0c887ef91b7d41a53e7c5a1ca976325f429cb46ea9bc30ecc"},
{file = "tomli-2.0.1.tar.gz", hash = "sha256:de526c12914f0c550d15924c62d72abc48d6fe7364aa87328337a31007fe8a4f"},
]
[[package]]
name = "typing-extensions"
version = "4.12.2"
description = "Backported and Experimental Type Hints for Python 3.8+"
optional = false
python-versions = ">=3.8"
files = [
{file = "typing_extensions-4.12.2-py3-none-any.whl", hash = "sha256:04e5ca0351e0f3f85c6853954072df659d0d13fac324d0072316b67d7794700d"},
{file = "typing_extensions-4.12.2.tar.gz", hash = "sha256:1a7ead55c7e559dd4dee8856e3a88b41225abfe1ce8df57b7c13915fe121ffb8"},
]
[[package]]
name = "typing-inspect"
version = "0.9.0"
description = "Runtime inspection utilities for typing module."
optional = false
python-versions = "*"
files = [
{file = "typing_inspect-0.9.0-py3-none-any.whl", hash = "sha256:9ee6fc59062311ef8547596ab6b955e1b8aa46242d854bfc78f4f6b0eff35f9f"},
{file = "typing_inspect-0.9.0.tar.gz", hash = "sha256:b23fc42ff6f6ef6954e4852c1fb512cdd18dbea03134f91f856a95ccc9461f78"},
]
[package.dependencies]
mypy-extensions = ">=0.3.0"
typing-extensions = ">=3.7.4"
[[package]]
name = "urllib3"
version = "2.2.2"
description = "HTTP library with thread-safe connection pooling, file post, and more."
optional = false
python-versions = ">=3.8"
files = [
{file = "urllib3-2.2.2-py3-none-any.whl", hash = "sha256:a448b2f64d686155468037e1ace9f2d2199776e17f0a46610480d311f73e3472"},
{file = "urllib3-2.2.2.tar.gz", hash = "sha256:dd505485549a7a552833da5e6063639d0d177c04f23bc3864e41e5dc5f612168"},
]
[package.extras]
brotli = ["brotli (>=1.0.9)", "brotlicffi (>=0.8.0)"]
h2 = ["h2 (>=4,<5)"]
socks = ["pysocks (>=1.5.6,!=1.5.7,<2.0)"]
zstd = ["zstandard (>=0.18.0)"]
[[package]]
name = "watchdog"
version = "4.0.1"
description = "Filesystem events monitoring"
optional = false
python-versions = ">=3.8"
files = [
{file = "watchdog-4.0.1-cp310-cp310-macosx_10_9_universal2.whl", hash = "sha256:da2dfdaa8006eb6a71051795856bedd97e5b03e57da96f98e375682c48850645"},
{file = "watchdog-4.0.1-cp310-cp310-macosx_10_9_x86_64.whl", hash = "sha256:e93f451f2dfa433d97765ca2634628b789b49ba8b504fdde5837cdcf25fdb53b"},
{file = "watchdog-4.0.1-cp310-cp310-macosx_11_0_arm64.whl", hash = "sha256:ef0107bbb6a55f5be727cfc2ef945d5676b97bffb8425650dadbb184be9f9a2b"},
{file = "watchdog-4.0.1-cp311-cp311-macosx_10_9_universal2.whl", hash = "sha256:17e32f147d8bf9657e0922c0940bcde863b894cd871dbb694beb6704cfbd2fb5"},
{file = "watchdog-4.0.1-cp311-cp311-macosx_10_9_x86_64.whl", hash = "sha256:03e70d2df2258fb6cb0e95bbdbe06c16e608af94a3ffbd2b90c3f1e83eb10767"},
{file = "watchdog-4.0.1-cp311-cp311-macosx_11_0_arm64.whl", hash = "sha256:123587af84260c991dc5f62a6e7ef3d1c57dfddc99faacee508c71d287248459"},
{file = "watchdog-4.0.1-cp312-cp312-macosx_10_9_universal2.whl", hash = "sha256:093b23e6906a8b97051191a4a0c73a77ecc958121d42346274c6af6520dec175"},
{file = "watchdog-4.0.1-cp312-cp312-macosx_10_9_x86_64.whl", hash = "sha256:611be3904f9843f0529c35a3ff3fd617449463cb4b73b1633950b3d97fa4bfb7"},
{file = "watchdog-4.0.1-cp312-cp312-macosx_11_0_arm64.whl", hash = "sha256:62c613ad689ddcb11707f030e722fa929f322ef7e4f18f5335d2b73c61a85c28"},
{file = "watchdog-4.0.1-cp38-cp38-macosx_10_9_universal2.whl", hash = "sha256:d4925e4bf7b9bddd1c3de13c9b8a2cdb89a468f640e66fbfabaf735bd85b3e35"},
{file = "watchdog-4.0.1-cp38-cp38-macosx_10_9_x86_64.whl", hash = "sha256:cad0bbd66cd59fc474b4a4376bc5ac3fc698723510cbb64091c2a793b18654db"},
{file = "watchdog-4.0.1-cp38-cp38-macosx_11_0_arm64.whl", hash = "sha256:a3c2c317a8fb53e5b3d25790553796105501a235343f5d2bf23bb8649c2c8709"},
{file = "watchdog-4.0.1-cp39-cp39-macosx_10_9_universal2.whl", hash = "sha256:c9904904b6564d4ee8a1ed820db76185a3c96e05560c776c79a6ce5ab71888ba"},
{file = "watchdog-4.0.1-cp39-cp39-macosx_10_9_x86_64.whl", hash = "sha256:667f3c579e813fcbad1b784db7a1aaa96524bed53437e119f6a2f5de4db04235"},
{file = "watchdog-4.0.1-cp39-cp39-macosx_11_0_arm64.whl", hash = "sha256:d10a681c9a1d5a77e75c48a3b8e1a9f2ae2928eda463e8d33660437705659682"},
{file = "watchdog-4.0.1-pp310-pypy310_pp73-macosx_10_9_x86_64.whl", hash = "sha256:0144c0ea9997b92615af1d94afc0c217e07ce2c14912c7b1a5731776329fcfc7"},
{file = "watchdog-4.0.1-pp310-pypy310_pp73-macosx_11_0_arm64.whl", hash = "sha256:998d2be6976a0ee3a81fb8e2777900c28641fb5bfbd0c84717d89bca0addcdc5"},
{file = "watchdog-4.0.1-pp38-pypy38_pp73-macosx_10_9_x86_64.whl", hash = "sha256:e7921319fe4430b11278d924ef66d4daa469fafb1da679a2e48c935fa27af193"},
{file = "watchdog-4.0.1-pp38-pypy38_pp73-macosx_11_0_arm64.whl", hash = "sha256:f0de0f284248ab40188f23380b03b59126d1479cd59940f2a34f8852db710625"},
{file = "watchdog-4.0.1-pp39-pypy39_pp73-macosx_10_9_x86_64.whl", hash = "sha256:bca36be5707e81b9e6ce3208d92d95540d4ca244c006b61511753583c81c70dd"},
{file = "watchdog-4.0.1-pp39-pypy39_pp73-macosx_11_0_arm64.whl", hash = "sha256:ab998f567ebdf6b1da7dc1e5accfaa7c6992244629c0fdaef062f43249bd8dee"},
{file = "watchdog-4.0.1-py3-none-manylinux2014_aarch64.whl", hash = "sha256:dddba7ca1c807045323b6af4ff80f5ddc4d654c8bce8317dde1bd96b128ed253"},
{file = "watchdog-4.0.1-py3-none-manylinux2014_armv7l.whl", hash = "sha256:4513ec234c68b14d4161440e07f995f231be21a09329051e67a2118a7a612d2d"},
{file = "watchdog-4.0.1-py3-none-manylinux2014_i686.whl", hash = "sha256:4107ac5ab936a63952dea2a46a734a23230aa2f6f9db1291bf171dac3ebd53c6"},
{file = "watchdog-4.0.1-py3-none-manylinux2014_ppc64.whl", hash = "sha256:6e8c70d2cd745daec2a08734d9f63092b793ad97612470a0ee4cbb8f5f705c57"},
{file = "watchdog-4.0.1-py3-none-manylinux2014_ppc64le.whl", hash = "sha256:f27279d060e2ab24c0aa98363ff906d2386aa6c4dc2f1a374655d4e02a6c5e5e"},
{file = "watchdog-4.0.1-py3-none-manylinux2014_s390x.whl", hash = "sha256:f8affdf3c0f0466e69f5b3917cdd042f89c8c63aebdb9f7c078996f607cdb0f5"},
{file = "watchdog-4.0.1-py3-none-manylinux2014_x86_64.whl", hash = "sha256:ac7041b385f04c047fcc2951dc001671dee1b7e0615cde772e84b01fbf68ee84"},
{file = "watchdog-4.0.1-py3-none-win32.whl", hash = "sha256:206afc3d964f9a233e6ad34618ec60b9837d0582b500b63687e34011e15bb429"},
{file = "watchdog-4.0.1-py3-none-win_amd64.whl", hash = "sha256:7577b3c43e5909623149f76b099ac49a1a01ca4e167d1785c76eb52fa585745a"},
{file = "watchdog-4.0.1-py3-none-win_ia64.whl", hash = "sha256:d7b9f5f3299e8dd230880b6c55504a1f69cf1e4316275d1b215ebdd8187ec88d"},
{file = "watchdog-4.0.1.tar.gz", hash = "sha256:eebaacf674fa25511e8867028d281e602ee6500045b57f43b08778082f7f8b44"},
]
[package.extras]
watchmedo = ["PyYAML (>=3.10)"]
[metadata]
lock-version = "2.0"
python-versions = "^3.9.0,<4.0"
content-hash = "8861f12053a7b4594cd8a218f31b78f861d7e5391017bc9209cc5bccbd6f769c"
+30 -45
View File
@@ -1,56 +1,43 @@
[build-system]
requires = ["hatchling"]
build-backend = "hatchling.build"
[project]
[tool.poetry]
name = "langgraph-checkpoint"
version = "4.0.0"
version = "2.0.16"
description = "Library with base interfaces for LangGraph checkpoint savers."
authors = []
requires-python = ">=3.10"
readme = "README.md"
license = "MIT"
license-files = ['LICENSE']
dependencies = [
"langchain-core>=0.2.38",
"ormsgpack>=1.12.0",
]
readme = "README.md"
repository = "https://www.github.com/langchain-ai/langgraph"
packages = [{ include = "langgraph" }]
[project.urls]
Source = "https://github.com/langchain-ai/langgraph/tree/main/libs/checkpoint"
Twitter = "https://x.com/LangChainAI"
Slack = "https://www.langchain.com/join-community"
Reddit = "https://www.reddit.com/r/LangChain/"
[tool.poetry.dependencies]
python = "^3.9.0,<4.0"
langchain-core = ">=0.2.38,<0.4"
msgpack = "^1.1.0"
[dependency-groups]
test = [
"pytest",
"pytest-asyncio",
"pytest-mock",
"pytest-watcher",
"dataclasses-json",
"numpy",
"pandas",
"pandas-stubs>=2.2.2.240807",
"redis",
]
lint = [
"ruff",
"codespell",
"mypy",
]
dev = [
{include-group = "test"},
{include-group = "lint"},
]
[tool.hatch.build.targets.wheel]
include = ["langgraph"]
[tool.poetry.group.dev.dependencies]
ruff = "^0.6.2"
codespell = "^2.2.0"
pytest = "^7.2.1"
pytest-asyncio = "^0.21.1"
pytest-mock = "^3.11.1"
pytest-watcher = "^0.4.1"
mypy = "^1.10.0"
dataclasses-json = "^0.6.7"
[tool.pytest.ini_options]
# --strict-markers will raise errors on unknown marks.
# https://docs.pytest.org/en/7.1.x/how-to/mark.html#raising-errors-on-unknown-marks
#
# https://docs.pytest.org/en/7.1.x/reference/reference.html
# --strict-config any warnings encountered while parsing the `pytest`
# section of the configuration file raise errors.
addopts = "--strict-markers --strict-config --durations=5 -vv"
asyncio_mode = "auto"
[build-system]
requires = ["poetry-core"]
build-backend = "poetry.core.masonry.api"
[tool.ruff]
lint.select = [
"E", # pycodestyle
@@ -58,10 +45,8 @@ lint.select = [
"UP", # pyupgrade
"B", # flake8-bugbear
"I", # isort
"UP", # pyupgrade
]
lint.ignore = ["E501", "B008"]
target-version = "py310"
lint.ignore = ["E501", "B008", "UP007", "UP006"]
[tool.pytest-watcher]
now = true
+1 -1
View File
@@ -5,7 +5,7 @@ import random
from collections import Counter, defaultdict
from typing import Any
from langchain_core.embeddings import Embeddings
from langgraph.store.base.embed import Embeddings
class CharacterEmbeddings(Embeddings):
+45 -359
View File
@@ -1,5 +1,4 @@
import dataclasses
import json
import pathlib
import re
import sys
@@ -9,21 +8,14 @@ from datetime import date, datetime, time, timezone
from decimal import Decimal
from enum import Enum
from ipaddress import IPv4Address
from zoneinfo import ZoneInfo
import dataclasses_json
import numpy as np
import pandas as pd
import pytest
from pydantic import BaseModel, SecretStr
from pydantic.v1 import BaseModel as BaseModelV1
from pydantic.v1 import SecretStr as SecretStrV1
from zoneinfo import ZoneInfo
from langgraph.checkpoint.serde.jsonplus import (
InvalidModuleError,
JsonPlusSerializer,
_msgpack_ext_hook_to_json,
)
from langgraph.checkpoint.serde.jsonplus import JsonPlusSerializer
from langgraph.store.base import Item
@@ -62,15 +54,22 @@ class MyDataclass:
pass
@dataclasses.dataclass(slots=True)
class MyDataclassWSlots:
foo: str
bar: int
inner: InnerDataclass
if sys.version_info < (3, 10):
def something(self) -> None:
class MyDataclassWSlots(MyDataclass):
pass
else:
@dataclasses.dataclass(slots=True)
class MyDataclassWSlots:
foo: str
bar: int
inner: InnerDataclass
def something(self) -> None:
pass
class MyEnum(Enum):
FOO = "foo"
@@ -105,12 +104,15 @@ def test_serde_jsonplus() -> None:
"time": current_time,
"uid": uid,
"timestamp": current_timestamp,
"my_rich_dict": {(1, 2, 3): 45},
"my_slotted_class": MyDataclassWSlots("bar", 2, InnerDataclass("hello")),
"my_dataclass": MyDataclass("foo", 1, InnerDataclass("hello")),
"my_enum": MyEnum.FOO,
"my_pydantic": MyPydantic(foo="foo", bar=1, inner=InnerPydantic(hello="hello")),
"my_pydantic_v1": MyPydanticV1(
foo="foo", bar=1, inner=InnerPydanticV1(hello="hello")
),
"my_secret_str": SecretStr("meow"),
"my_secret_str_v1": SecretStrV1("meow"),
"person": Person(name="foo"),
"a_bool": True,
"a_none": None,
@@ -128,16 +130,10 @@ def test_serde_jsonplus() -> None:
key="my-key",
namespace=("a", "name", " "),
created_at=datetime(2024, 9, 24, 17, 29, 10, 128397),
updated_at=datetime(2024, 9, 24, 17, 29, 11, 128397),
updated_at=datetime(2024, 9, 24, 17, 29, 10, 128397),
),
}
if sys.version_info < (3, 14):
to_serialize["my_pydantic_v1"] = MyPydanticV1(
foo="foo", bar=1, inner=InnerPydanticV1(hello="hello")
)
to_serialize["my_secret_str_v1"] = SecretStrV1("meow")
serde = JsonPlusSerializer()
dumped = serde.dumps_typed(to_serialize)
@@ -149,140 +145,23 @@ def test_serde_jsonplus() -> None:
assert serde.loads_typed(serde.dumps_typed(value)) == value
surrogates = [
"Hello??",
"Python??",
"Surrogate??",
"Example??",
"String??",
"With??",
"Surrogates??",
"Embedded??",
"In??",
"The??",
"Text??",
"Hello\ud83d\ude00",
"Python\ud83d\udc0d",
"Surrogate\ud834\udd1e",
"Example\ud83c\udf89",
"String\ud83c\udfa7",
"With\ud83c\udf08",
"Surrogates\ud83d\ude0e",
"Embedded\ud83d\udcbb",
"In\ud83c\udf0e",
"The\ud83d\udcd6",
"Text\ud83d\udcac",
"收花🙄·到",
]
serde = JsonPlusSerializer(pickle_fallback=False)
assert serde.loads_typed(serde.dumps_typed(surrogates)) == surrogates
def test_serde_jsonplus_json_mode() -> None:
uid = uuid.UUID(int=1)
deque_instance = deque([1, 2, 3])
tzn = ZoneInfo("America/New_York")
ip4 = IPv4Address("192.168.0.1")
current_date = date(2024, 4, 19)
current_time = time(23, 4, 57, 51022, timezone.max)
current_timestamp = datetime(2024, 4, 19, 23, 4, 57, 51022, timezone.max)
to_serialize = {
"path": pathlib.Path("foo", "bar"),
"re": re.compile(r"foo", re.DOTALL),
"decimal": Decimal("1.10101"),
"set": {1, 2, frozenset({1, 2})},
"frozen_set": frozenset({1, 2, 3}),
"ip4": ip4,
"deque": deque_instance,
"tzn": tzn,
"date": current_date,
"time": current_time,
"uid": uid,
"timestamp": current_timestamp,
"my_slotted_class": MyDataclassWSlots("bar", 2, InnerDataclass("hello")),
"my_dataclass": MyDataclass("foo", 1, InnerDataclass("hello")),
"my_enum": MyEnum.FOO,
"my_pydantic": MyPydantic(foo="foo", bar=1, inner=InnerPydantic(hello="hello")),
"my_secret_str": SecretStr("meow"),
"person": Person(name="foo"),
"a_bool": True,
"a_none": None,
"a_str": "foo",
"a_str_nuc": "foo\u0000",
"a_str_uc": "foo ⛰️",
"a_str_ucuc": "foo \u26f0\ufe0f\u0000",
"a_str_ucucuc": "foo \\u26f0\\ufe0f",
"an_int": 1,
"a_float": 1.1,
"a_bytes": b"my bytes",
"a_bytearray": bytearray([42]),
"my_item": Item(
value={},
key="my-key",
namespace=("a", "name", " "),
created_at=datetime(2024, 9, 24, 17, 29, 10, 128397),
updated_at=datetime(2024, 9, 24, 17, 29, 11, 128397),
),
}
if sys.version_info < (3, 14):
to_serialize["my_pydantic_v1"] = MyPydanticV1(
foo="foo", bar=1, inner=InnerPydanticV1(hello="hello")
)
to_serialize["my_secret_str_v1"] = SecretStrV1("meow")
serde = JsonPlusSerializer(__unpack_ext_hook__=_msgpack_ext_hook_to_json)
dumped = serde.dumps_typed(to_serialize)
assert dumped[0] == "msgpack"
result = serde.loads_typed(dumped)
expected_result = {
"path": ["foo", "bar"],
"re": ["foo", 48],
"decimal": "1.10101",
"set": [1, 2, [1, 2]],
"frozen_set": [1, 2, 3],
"ip4": "192.168.0.1",
"deque": [1, 2, 3],
"tzn": "America/New_York",
"date": [2024, 4, 19],
"time": {
"hour": 23,
"minute": 4,
"second": 57,
"microsecond": 51022,
"tzinfo": [[0, 86340, 0]],
"fold": 0,
},
"uid": "00000000-0000-0000-0000-000000000001",
"timestamp": "2024-04-19T23:04:57.051022+23:59",
"my_slotted_class": {"foo": "bar", "bar": 2, "inner": {"hello": "hello"}},
"my_dataclass": {"foo": "foo", "bar": 1, "inner": {"hello": "hello"}},
"my_enum": "foo",
"my_pydantic": {"foo": "foo", "bar": 1, "inner": {"hello": "hello"}},
"my_secret_str": "meow",
"person": {"name": "foo"},
"a_bool": True,
"a_none": None,
"a_str": "foo",
"a_str_nuc": "foo\x00",
"a_str_uc": "foo ⛰️",
"a_str_ucuc": "foo ⛰️\x00",
"a_str_ucucuc": "foo \\u26f0\\ufe0f",
"an_int": 1,
"a_float": 1.1,
"a_bytes": b"my bytes",
"a_bytearray": b"*",
"my_item": {
"namespace": ["a", "name", " "],
"key": "my-key",
"value": {},
"created_at": "2024-09-24T17:29:10.128397",
"updated_at": "2024-09-24T17:29:11.128397",
},
}
if sys.version_info < (3, 14):
expected_result["my_pydantic_v1"] = {
"foo": "foo",
"bar": 1,
"inner": {"hello": "hello"},
}
expected_result["my_secret_str_v1"] = "meow"
assert result == expected_result
assert serde.loads_typed(serde.dumps_typed(surrogates)) == [
v.encode("utf-8", "ignore").decode() for v in surrogates
]
def test_serde_jsonplus_bytes() -> None:
@@ -295,20 +174,6 @@ def test_serde_jsonplus_bytes() -> None:
assert serde.loads_typed(dumped) == some_bytes
def test_deserde_invalid_module() -> None:
serde = JsonPlusSerializer()
load = {
"lc": 2,
"type": "constructor",
"id": ["pprint", "pprint"],
"kwargs": {"object": "HELLO"},
}
with pytest.raises(InvalidModuleError):
serde._revive_lc2(load)
serde = JsonPlusSerializer(allowed_json_modules=[("pprint", "pprint")])
serde.loads_typed(("json", json.dumps(load).encode("utf-8")))
def test_serde_jsonplus_bytearray() -> None:
serde = JsonPlusSerializer()
@@ -319,198 +184,19 @@ def test_serde_jsonplus_bytearray() -> None:
assert serde.loads_typed(dumped) == some_bytearray
@pytest.mark.parametrize(
"arr",
[
np.arange(9, dtype=np.int32).reshape(3, 3),
np.asfortranarray(np.arange(9, dtype=np.float64).reshape(3, 3)),
np.arange(12, dtype=np.int16)[::2].reshape(3, 2),
],
)
def test_serde_jsonplus_numpy_array(arr: np.ndarray) -> None:
def test_loads_cannot_find() -> None:
serde = JsonPlusSerializer()
dumped = serde.dumps_typed(arr)
assert dumped[0] == "msgpack"
result = serde.loads_typed(dumped)
assert isinstance(result, np.ndarray)
assert result.dtype == arr.dtype
assert np.array_equal(result, arr)
dumped = (
"json",
b'{"lc": 2, "type": "constructor", "id": ["tests", "test_jsonplus", "MyPydanticccc"], "method": null, "args": [], "kwargs": {"foo": "foo", "bar": 1}}',
)
assert serde.loads_typed(dumped) is None, "Should return None if cannot find class"
@pytest.mark.parametrize(
"arr",
[
np.arange(6, dtype=np.float32).reshape(2, 3),
np.asfortranarray(np.arange(4, dtype=np.complex128).reshape(2, 2)),
],
)
def test_serde_jsonplus_numpy_array_json_hook(arr: np.ndarray) -> None:
serde = JsonPlusSerializer(__unpack_ext_hook__=_msgpack_ext_hook_to_json)
dumped = serde.dumps_typed(arr)
assert dumped[0] == "msgpack"
result = serde.loads_typed(dumped)
assert isinstance(result, list)
assert result == arr.tolist()
dumped = (
"json",
b'{"lc": 2, "type": "constructor", "id": ["tests", "test_jsonpluss", "MyPydantic"], "method": null, "args": [], "kwargs": {"foo": "foo", "bar": 1}}',
)
@pytest.mark.parametrize(
"df",
[
pd.DataFrame(),
pd.DataFrame({"int_col": [1, 2, 3]}),
pd.DataFrame({"float_col": [1.1, 2.2, 3.3]}),
pd.DataFrame({"str_col": ["a", "b", "c"]}),
pd.DataFrame({"bool_col": [True, False, True]}),
pd.DataFrame(
{
"datetime_col": [
datetime(2024, 1, 1),
datetime(2024, 1, 2),
datetime(2024, 1, 3),
]
}
),
pd.DataFrame(
{
"int_col": [1, 2, 3],
"float_col": [1.1, 2.2, 3.3],
"str_col": ["a", "b", "c"],
}
),
pd.DataFrame(
{
"int_col": [1, 2, None],
"float_col": [1.1, None, 3.3],
"str_col": ["a", None, "c"],
}
),
pytest.param(
pd.DataFrame({"cat_col": pd.Categorical(["a", "b", "a", "c"])}),
marks=pytest.mark.skipif(
sys.version_info >= (3, 14), reason="NotImplementedError on Python 3.14"
),
),
pd.DataFrame(
{
"int8": pd.array([1, 2, 3], dtype="int8"),
"int16": pd.array([10, 20, 30], dtype="int16"),
"int32": pd.array([100, 200, 300], dtype="int32"),
"int64": pd.array([1000, 2000, 3000], dtype="int64"),
"float32": pd.array([1.1, 2.2, 3.3], dtype="float32"),
"float64": pd.array([10.1, 20.2, 30.3], dtype="float64"),
}
),
pd.DataFrame({"value": [1, 2, 3]}, index=["x", "y", "z"]),
pd.DataFrame(
[[1, 2, 3, 4]],
columns=pd.MultiIndex.from_tuples(
[("A", "X"), ("A", "Y"), ("B", "X"), ("B", "Y")]
),
),
pd.DataFrame(
{"value": [1, 2, 3]}, index=pd.date_range("2024-01-01", periods=3, freq="D")
),
pd.DataFrame(
{
"col1": range(1000),
"col2": [f"str_{i}" for i in range(1000)],
"col3": np.random.rand(1000),
}
),
pytest.param(
pd.DataFrame(
{
"tz_datetime": pd.date_range(
"2024-01-01", periods=3, freq="D", tz="UTC"
)
}
),
marks=pytest.mark.skipif(
sys.version_info >= (3, 14), reason="NotImplementedError on Python 3.14"
),
),
pd.DataFrame({"timedelta": pd.to_timedelta([1, 2, 3], unit="D")}),
pytest.param(
pd.DataFrame({"period": pd.period_range("2024-01", periods=3, freq="M")}),
marks=pytest.mark.skipif(
sys.version_info >= (3, 14), reason="NotImplementedError on Python 3.14"
),
),
pd.DataFrame({"interval": pd.interval_range(start=0, end=3, periods=3)}),
pd.DataFrame({"unicode": ["Hello 🌍", "Python 🐍", "Data 📊"]}),
pd.DataFrame({"mixed": [1, "string", [1, 2, 3], {"key": "value"}]}),
pd.DataFrame({"a": [1], "b": ["test"], "c": [3.14]}),
pd.DataFrame({"single": [42]}),
pd.DataFrame(
{
"small": [sys.float_info.min, 0, sys.float_info.max],
"large_int": [-(2**63), 0, 2**63 - 1],
}
),
pd.DataFrame({"special_strings": ["", "null", "None", "NaN", "inf", "-inf"]}),
pd.DataFrame({"bytes_col": [b"hello", b"world", b"\x00\x01\x02"]}),
],
)
def test_serde_jsonplus_pandas_dataframe(df: pd.DataFrame) -> None:
serde = JsonPlusSerializer(pickle_fallback=True)
dumped = serde.dumps_typed(df)
assert dumped[0] == "pickle"
result = serde.loads_typed(dumped)
assert result.equals(df)
@pytest.mark.parametrize(
"series",
[
pd.Series([]),
pd.Series([1, 2, 3]),
pd.Series([1.1, 2.2, 3.3]),
pd.Series(["a", "b", "c"]),
pd.Series([True, False, True]),
pd.Series([datetime(2024, 1, 1), datetime(2024, 1, 2), datetime(2024, 1, 3)]),
pd.Series([1, 2, None]),
pd.Series([1.1, None, 3.3]),
pd.Series(["a", None, "c"]),
pytest.param(
pd.Series(pd.Categorical(["a", "b", "a", "c"])),
marks=pytest.mark.skipif(
sys.version_info >= (3, 14), reason="NotImplementedError on Python 3.14"
),
),
pd.Series([1, 2, 3], dtype="int8"),
pd.Series([10, 20, 30], dtype="int16"),
pd.Series([100, 200, 300], dtype="int32"),
pd.Series([1000, 2000, 3000], dtype="int64"),
pd.Series([1.1, 2.2, 3.3], dtype="float32"),
pd.Series([10.1, 20.2, 30.3], dtype="float64"),
pd.Series([1, 2, 3], index=["x", "y", "z"]),
pd.Series([1, 2, 3], index=pd.date_range("2024-01-01", periods=3, freq="D")),
pd.Series(range(1000)),
pd.Series(pd.date_range("2024-01-01", periods=3, freq="D", tz="UTC")),
pd.Series(pd.to_timedelta([1, 2, 3], unit="D")),
pd.Series(pd.period_range("2024-01", periods=3, freq="M")),
pd.Series(pd.interval_range(start=0, end=3, periods=3)),
pd.Series(["Hello 🌍", "Python 🐍", "Data 📊"]),
pd.Series([1, "string", [1, 2, 3], {"key": "value"}]),
pd.Series([42], name="single"),
pd.Series([sys.float_info.min, 0, sys.float_info.max]),
pd.Series([-(2**63), 0, 2**63 - 1]),
pd.Series(["", "null", "None", "NaN", "inf", "-inf"]),
pd.Series([b"hello", b"world", b"\x00\x01\x02"]),
pd.Series([1, 2, 3], name="named_series"),
pd.Series(
[10, 20],
index=pd.MultiIndex.from_tuples([("a", 1), ("b", 2)], names=["x", "y"]),
),
],
)
def test_serde_jsonplus_pandas_series(series: pd.Series) -> None:
serde = JsonPlusSerializer(pickle_fallback=True)
dumped = serde.dumps_typed(series)
assert dumped[0] == "pickle"
result = serde.loads_typed(dumped)
assert result.equals(series)
assert serde.loads_typed(dumped) is None, "Should return None if cannot find module"
+20 -88
View File
@@ -22,7 +22,8 @@ class TestMemorySaver:
"configurable": {
"thread_id": "thread-1",
"checkpoint_ns": "",
"checkpoint_id": "1",
# for backwards compatibility testing
"thread_ts": "1",
}
}
self.config_2: RunnableConfig = {
@@ -67,37 +68,21 @@ class TestMemorySaver:
},
"metadata": {"run_id": "my_run_id"},
}
self.memory_saver.put(
config, self.chkpnt_2, self.metadata_2, self.chkpnt_2["channel_versions"]
)
self.memory_saver.put(config, self.chkpnt_2, self.metadata_2, {})
checkpoint = self.memory_saver.get_tuple(config)
assert checkpoint is not None
assert checkpoint.metadata == {
**self.metadata_2,
"thread_id": "thread-2",
"run_id": "my_run_id",
}
async def test_search(self) -> None:
def test_search(self) -> None:
# set up test
# save checkpoints
self.memory_saver.put(
self.config_1,
self.chkpnt_1,
self.metadata_1,
self.chkpnt_1["channel_versions"],
)
self.memory_saver.put(
self.config_2,
self.chkpnt_2,
self.metadata_2,
self.chkpnt_2["channel_versions"],
)
self.memory_saver.put(
self.config_3,
self.chkpnt_3,
self.metadata_3,
self.chkpnt_3["channel_versions"],
)
self.memory_saver.put(self.config_1, self.chkpnt_1, self.metadata_1, {})
self.memory_saver.put(self.config_2, self.chkpnt_2, self.metadata_2, {})
self.memory_saver.put(self.config_3, self.chkpnt_3, self.metadata_3, {})
# call method / assertions
query_1 = {"source": "input"} # search by 1 key
@@ -110,11 +95,18 @@ class TestMemorySaver:
search_results_1 = list(self.memory_saver.list(None, filter=query_1))
assert len(search_results_1) == 1
assert search_results_1[0].metadata == self.metadata_1
assert search_results_1[0].metadata == {
"thread_id": "thread-1",
"thread_ts": "1",
**self.metadata_1,
}
search_results_2 = list(self.memory_saver.list(None, filter=query_2))
assert len(search_results_2) == 1
assert search_results_2[0].metadata == self.metadata_2
assert search_results_2[0].metadata == {
"thread_id": "thread-2",
**self.metadata_2,
}
search_results_3 = list(self.memory_saver.list(None, filter=query_3))
assert len(search_results_3) == 3
@@ -134,68 +126,8 @@ class TestMemorySaver:
# TODO: test before and limit params
async def test_asearch(self) -> None:
# set up test
# save checkpoints
self.memory_saver.put(
self.config_1,
self.chkpnt_1,
self.metadata_1,
self.chkpnt_1["channel_versions"],
)
self.memory_saver.put(
self.config_2,
self.chkpnt_2,
self.metadata_2,
self.chkpnt_2["channel_versions"],
)
self.memory_saver.put(
self.config_3,
self.chkpnt_3,
self.metadata_3,
self.chkpnt_3["channel_versions"],
)
# call method / assertions
query_1 = {"source": "input"} # search by 1 key
query_2 = {
"step": 1,
"writes": {"foo": "bar"},
} # search by multiple keys
query_3: dict[str, Any] = {} # search by no keys, return all checkpoints
query_4 = {"source": "update", "step": 1} # no match
def test_memory_saver() -> None:
from langgraph.checkpoint.memory import MemorySaver
search_results_1 = [
c async for c in self.memory_saver.alist(None, filter=query_1)
]
assert len(search_results_1) == 1
assert search_results_1[0].metadata == self.metadata_1
search_results_2 = [
c async for c in self.memory_saver.alist(None, filter=query_2)
]
assert len(search_results_2) == 1
assert search_results_2[0].metadata == self.metadata_2
search_results_3 = [
c async for c in self.memory_saver.alist(None, filter=query_3)
]
assert len(search_results_3) == 3
search_results_4 = [
c async for c in self.memory_saver.alist(None, filter=query_4)
]
assert len(search_results_4) == 0
async def test_memory_saver() -> None:
from langgraph.checkpoint.memory import InMemorySaver
memory_saver = InMemorySaver()
assert isinstance(memory_saver, InMemorySaver)
async with memory_saver as async_memory_saver:
assert async_memory_saver is memory_saver
with memory_saver as sync_memory_saver:
assert sync_memory_saver is memory_saver
assert isinstance(MemorySaver(), InMemorySaver)
-316
View File
@@ -1,316 +0,0 @@
"""Unit tests for Redis cache implementation."""
import time
import pytest
import redis
from langgraph.cache.base import FullKey
from langgraph.cache.redis import RedisCache
class TestRedisCache:
@pytest.fixture(autouse=True)
def setup(self) -> None:
"""Set up test Redis client and cache."""
self.client = redis.Redis(
host="localhost", port=6379, db=0, decode_responses=False
)
try:
self.client.ping()
except redis.ConnectionError:
pytest.skip("Redis server not available")
self.cache: RedisCache = RedisCache(self.client, prefix="test:cache:")
# Clean up before each test
self.client.flushdb()
def teardown_method(self) -> None:
"""Clean up after each test."""
try:
self.client.flushdb()
except Exception:
pass
def test_basic_set_and_get(self) -> None:
"""Test basic set and get operations."""
keys: list[FullKey] = [(("graph", "node"), "key1")]
values = {keys[0]: ({"result": 42}, None)}
# Set value
self.cache.set(values)
# Get value
result = self.cache.get(keys)
assert len(result) == 1
assert result[keys[0]] == {"result": 42}
def test_batch_operations(self) -> None:
"""Test batch set and get operations."""
keys: list[FullKey] = [
(("graph", "node1"), "key1"),
(("graph", "node2"), "key2"),
(("other", "node"), "key3"),
]
values = {
keys[0]: ({"result": 1}, None),
keys[1]: ({"result": 2}, 60), # With TTL
keys[2]: ({"result": 3}, None),
}
# Set values
self.cache.set(values)
# Get all values
result = self.cache.get(keys)
assert len(result) == 3
assert result[keys[0]] == {"result": 1}
assert result[keys[1]] == {"result": 2}
assert result[keys[2]] == {"result": 3}
def test_ttl_behavior(self) -> None:
"""Test TTL (time-to-live) functionality."""
key: FullKey = (("graph", "node"), "ttl_key")
values = {key: ({"data": "expires_soon"}, 1)} # 1 second TTL
# Set with TTL
self.cache.set(values)
# Should be available immediately
result = self.cache.get([key])
assert len(result) == 1
assert result[key] == {"data": "expires_soon"}
# Wait for expiration
time.sleep(1.1)
# Should be expired
result = self.cache.get([key])
assert len(result) == 0
def test_namespace_isolation(self) -> None:
"""Test that different namespaces are isolated."""
key1: FullKey = (("graph1", "node"), "same_key")
key2: FullKey = (("graph2", "node"), "same_key")
values = {key1: ({"graph": 1}, None), key2: ({"graph": 2}, None)}
self.cache.set(values)
result = self.cache.get([key1, key2])
assert result[key1] == {"graph": 1}
assert result[key2] == {"graph": 2}
def test_clear_all(self) -> None:
"""Test clearing all cached values."""
keys: list[FullKey] = [
(("graph", "node1"), "key1"),
(("graph", "node2"), "key2"),
]
values = {keys[0]: ({"result": 1}, None), keys[1]: ({"result": 2}, None)}
self.cache.set(values)
# Verify data exists
result = self.cache.get(keys)
assert len(result) == 2
# Clear all
self.cache.clear()
# Verify data is gone
result = self.cache.get(keys)
assert len(result) == 0
def test_clear_by_namespace(self) -> None:
"""Test clearing cached values by namespace."""
keys: list[FullKey] = [
(("graph1", "node"), "key1"),
(("graph2", "node"), "key2"),
(("graph1", "other"), "key3"),
]
values = {
keys[0]: ({"result": 1}, None),
keys[1]: ({"result": 2}, None),
keys[2]: ({"result": 3}, None),
}
self.cache.set(values)
# Clear only graph1 namespace
self.cache.clear([("graph1", "node"), ("graph1", "other")])
# graph1 should be cleared, graph2 should remain
result = self.cache.get(keys)
assert len(result) == 1
assert result[keys[1]] == {"result": 2}
def test_empty_operations(self) -> None:
"""Test behavior with empty keys/values."""
# Empty get
result = self.cache.get([])
assert result == {}
# Empty set
self.cache.set({}) # Should not raise error
def test_nonexistent_keys(self) -> None:
"""Test getting keys that don't exist."""
keys: list[FullKey] = [(("graph", "node"), "nonexistent")]
result = self.cache.get(keys)
assert len(result) == 0
@pytest.mark.asyncio
async def test_async_operations(self) -> None:
"""Test async set and get operations with sync Redis client."""
# Create sync Redis client and cache (like main integration tests)
client = redis.Redis(host="localhost", port=6379, db=1, decode_responses=False)
try:
client.ping()
except Exception:
pytest.skip("Redis not available")
cache: RedisCache = RedisCache(client, prefix="test:async:")
keys: list[FullKey] = [(("graph", "node"), "async_key")]
values = {keys[0]: ({"async": True}, None)}
# Async set (delegates to sync)
await cache.aset(values)
# Async get (delegates to sync)
result = await cache.aget(keys)
assert len(result) == 1
assert result[keys[0]] == {"async": True}
# Cleanup
client.flushdb()
@pytest.mark.asyncio
async def test_async_clear(self) -> None:
"""Test async clear operations with sync Redis client."""
# Create sync Redis client and cache (like main integration tests)
client = redis.Redis(host="localhost", port=6379, db=1, decode_responses=False)
try:
client.ping()
except Exception:
pytest.skip("Redis not available")
cache: RedisCache = RedisCache(client, prefix="test:async:")
keys: list[FullKey] = [(("graph", "node"), "key")]
values = {keys[0]: ({"data": "test"}, None)}
await cache.aset(values)
# Verify data exists
result = await cache.aget(keys)
assert len(result) == 1
# Clear all (delegates to sync)
await cache.aclear()
# Verify data is gone
result = await cache.aget(keys)
assert len(result) == 0
# Cleanup
client.flushdb()
def test_redis_unavailable_get(self) -> None:
"""Test behavior when Redis is unavailable during get operations."""
# Create cache with non-existent Redis server
bad_client = redis.Redis(
host="nonexistent", port=9999, socket_connect_timeout=0.1
)
cache: RedisCache = RedisCache(bad_client, prefix="test:cache:")
keys: list[FullKey] = [(("graph", "node"), "key")]
result = cache.get(keys)
# Should return empty dict when Redis unavailable
assert result == {}
def test_redis_unavailable_set(self) -> None:
"""Test behavior when Redis is unavailable during set operations."""
# Create cache with non-existent Redis server
bad_client = redis.Redis(
host="nonexistent", port=9999, socket_connect_timeout=0.1
)
cache: RedisCache = RedisCache(bad_client, prefix="test:cache:")
keys: list[FullKey] = [(("graph", "node"), "key")]
values = {keys[0]: ({"data": "test"}, None)}
# Should not raise exception when Redis unavailable
cache.set(values) # Should silently fail
@pytest.mark.asyncio
async def test_redis_unavailable_async(self) -> None:
"""Test async behavior when Redis is unavailable."""
# Create sync cache with non-existent Redis server (like main integration tests)
bad_client = redis.Redis(
host="nonexistent", port=9999, socket_connect_timeout=0.1
)
cache: RedisCache = RedisCache(bad_client, prefix="test:cache:")
keys: list[FullKey] = [(("graph", "node"), "key")]
values = {keys[0]: ({"data": "test"}, None)}
# Should return empty dict for get (delegates to sync)
result = await cache.aget(keys)
assert result == {}
# Should not raise exception for set (delegates to sync)
await cache.aset(values) # Should silently fail
def test_corrupted_data_handling(self) -> None:
"""Test handling of corrupted data in Redis."""
# Set some valid data first
keys: list[FullKey] = [(("graph", "node"), "valid_key")]
values = {keys[0]: ({"data": "valid"}, None)}
self.cache.set(values)
# Manually insert corrupted data
corrupted_key = self.cache._make_key(("graph", "node"), "corrupted_key")
self.client.set(corrupted_key, b"invalid:data:format:too:many:colons")
# Should skip corrupted entry and return only valid ones
all_keys: list[FullKey] = [keys[0], (("graph", "node"), "corrupted_key")]
result = self.cache.get(all_keys)
assert len(result) == 1
assert result[keys[0]] == {"data": "valid"}
def test_key_parsing_edge_cases(self) -> None:
"""Test key parsing with edge cases."""
# Test empty namespace
key1: FullKey = ((), "empty_ns")
values = {key1: ({"data": "empty_ns"}, None)}
self.cache.set(values)
result = self.cache.get([key1])
assert result[key1] == {"data": "empty_ns"}
# Test namespace with special characters
key2: FullKey = (
("graph:with:colons", "node-with-dashes"),
"key_with_underscores",
)
values = {key2: ({"data": "special_chars"}, None)}
self.cache.set(values)
result = self.cache.get([key2])
assert result[key2] == {"data": "special_chars"}
def test_large_data_serialization(self) -> None:
"""Test handling of large data objects."""
# Create a large data structure
large_data = {"large_list": list(range(1000)), "nested": {"data": "x" * 1000}}
key: FullKey = (("graph", "node"), "large_key")
values = {key: (large_data, None)}
self.cache.set(values)
result = self.cache.get([key])
assert len(result) == 1
assert result[key] == large_data
+1 -354
View File
@@ -1,75 +1,17 @@
# mypy: disable-error-code="operator"
import asyncio
import json
from collections.abc import Iterable
from datetime import datetime
from typing import Any
import pytest
from pytest_mock import MockerFixture
from langgraph.store.base import (
GetOp,
InvalidNamespaceError,
Item,
Op,
PutOp,
Result,
get_text_at_path,
)
from langgraph.store.base.batch import AsyncBatchedBaseStore
from langgraph.store.memory import InMemoryStore
from tests.embed_test_utils import CharacterEmbeddings
class MockAsyncBatchedStore(AsyncBatchedBaseStore):
def __init__(self, **kwargs: Any) -> None:
super().__init__()
self._store = InMemoryStore(**kwargs)
def batch(self, ops: Iterable[Op]) -> list[Result]:
return self._store.batch(ops)
async def abatch(self, ops: Iterable[Op]) -> list[Result]:
return self._store.batch(ops)
async def test_async_batch_store_resilience() -> None:
"""Test that AsyncBatchedBaseStore recovers gracefully from task cancellation."""
doc = {"foo": "bar"}
async_store = MockAsyncBatchedStore()
await async_store.aput(("foo", "langgraph", "foo"), "bar", doc)
# Store the original task reference
original_task = async_store._task
assert original_task is not None
assert not original_task.done()
# Cancel the background task
original_task.cancel()
await asyncio.sleep(0.01)
assert original_task.cancelled()
# Perform a new operation - this should trigger _ensure_task() to create a new task
result = await async_store.asearch(("foo", "langgraph", "foo"))
assert len(result) > 0
assert result[0].value == doc
# Verify a new task was created
new_task = async_store._task
assert new_task is not None
assert new_task is not original_task
assert not new_task.done()
# Test that operations continue to work with the new task
doc2 = {"baz": "qux"}
await async_store.aput(("test", "namespace"), "key", doc2)
result2 = await async_store.aget(("test", "namespace"), "key")
assert result2 is not None
assert result2.value == doc2
def test_get_text_at_path() -> None:
nested_data = {
"name": "test",
@@ -138,96 +80,6 @@ def test_get_text_at_path() -> None:
assert get_text_at_path(nested_data, "nested[{invalid}]") == []
async def test_async_batch_store(mocker: MockerFixture) -> None:
abatch = mocker.stub()
class MockStore(AsyncBatchedBaseStore):
def batch(self, ops: Iterable[Op]) -> list[Result]:
raise NotImplementedError
async def abatch(self, ops: Iterable[Op]) -> list[Result]:
assert all(isinstance(op, GetOp) for op in ops)
abatch(ops)
return [
Item(
value={},
key=getattr(op, "key", ""),
namespace=getattr(op, "namespace", ()),
created_at=datetime(2024, 9, 24, 17, 29, 10, 128397),
updated_at=datetime(2024, 9, 24, 17, 29, 10, 128397),
)
for op in ops
]
store = MockStore()
# concurrent calls are batched
results = await asyncio.gather(
store.aget(namespace=("a",), key="b"),
store.aget(namespace=("c",), key="d"),
)
assert results == [
Item(
value={},
key="b",
namespace=("a",),
created_at=datetime(2024, 9, 24, 17, 29, 10, 128397),
updated_at=datetime(2024, 9, 24, 17, 29, 10, 128397),
),
Item(
value={},
key="d",
namespace=("c",),
created_at=datetime(2024, 9, 24, 17, 29, 10, 128397),
updated_at=datetime(2024, 9, 24, 17, 29, 10, 128397),
),
]
assert abatch.call_count == 1
assert [tuple(c.args[0]) for c in abatch.call_args_list] == [
(
GetOp(("a",), "b", refresh_ttl=True),
GetOp(("c",), "d", refresh_ttl=True),
),
]
async def test_async_batch_store_handles_cancellation() -> None:
class MockStore(AsyncBatchedBaseStore):
def batch(self, ops: Iterable[Op]) -> list[Result]:
raise NotImplementedError
async def abatch(self, ops: Iterable[Op]) -> list[Result]:
assert all(isinstance(op, GetOp) for op in ops)
return [
Item(
value={},
key=getattr(op, "key", ""),
namespace=getattr(op, "namespace", ()),
created_at=datetime(2024, 9, 24, 17, 29, 10, 128397),
updated_at=datetime(2024, 9, 24, 17, 29, 10, 128397),
)
for op in ops
]
store = MockStore()
# Simulate cancellation
task = asyncio.create_task(store.aget(namespace=("a",), key="b"))
await asyncio.sleep(0)
task.cancel()
await asyncio.sleep(0)
# Cancelling individual queries against the store should not break the store
result = await store.aget(namespace=("c",), key="d")
assert result == Item(
value={},
key="d",
namespace=("c",),
created_at=datetime(2024, 9, 24, 17, 29, 10, 128397),
updated_at=datetime(2024, 9, 24, 17, 29, 10, 128397),
)
def test_list_namespaces_basic() -> None:
store = InMemoryStore()
@@ -480,105 +332,6 @@ async def test_cannot_put_empty_namespace() -> None:
store.delete(("langgraph", "foo"), "bar")
assert store.get(("langgraph", "foo"), "bar") is None
async_store = MockAsyncBatchedStore()
doc = {"foo": "bar"}
with pytest.raises(InvalidNamespaceError):
await async_store.aput((), "foo", doc)
with pytest.raises(InvalidNamespaceError):
await async_store.aput(("the", "thing.about"), "foo", doc)
with pytest.raises(InvalidNamespaceError):
await async_store.aput(("some", "fun", ""), "foo", doc)
with pytest.raises(InvalidNamespaceError):
await async_store.aput(("langgraph", "foo"), "bar", doc)
await async_store.aput(("foo", "langgraph", "foo"), "bar", doc)
val = await async_store.aget(("foo", "langgraph", "foo"), "bar")
assert val is not None
assert val.value == doc
assert (await async_store.asearch(("foo", "langgraph", "foo")))[0].value == doc
assert (await async_store.asearch(("foo", "langgraph", "foo"), query="bar"))[
0
].value == doc
await async_store.adelete(("foo", "langgraph", "foo"), "bar")
assert (await async_store.aget(("foo", "langgraph", "foo"), "bar")) is None
await async_store.abatch([PutOp(("valid", "namespace"), "key", doc)])
val = await async_store.aget(("valid", "namespace"), "key")
assert val is not None
assert val.value == doc
assert (await async_store.asearch(("valid", "namespace")))[0].value == doc
await async_store.adelete(("valid", "namespace"), "key")
assert (await async_store.aget(("valid", "namespace"), "key")) is None
async def test_async_batch_store_deduplication(mocker: MockerFixture) -> None:
abatch = mocker.spy(InMemoryStore, "batch")
store = MockAsyncBatchedStore()
same_doc = {"value": "same"}
diff_doc = {"value": "different"}
await asyncio.gather(
store.aput(namespace=("test",), key="same", value=same_doc),
store.aput(namespace=("test",), key="different", value=diff_doc),
)
abatch.reset_mock()
results = await asyncio.gather(
store.aget(namespace=("test",), key="same"),
store.aget(namespace=("test",), key="same"),
store.aget(namespace=("test",), key="different"),
)
assert len(results) == 3
assert results[0] == results[1]
assert results[0] != results[2]
assert results[0].value == same_doc # type: ignore
assert results[2].value == diff_doc # type: ignore
assert len(abatch.call_args_list) == 1
ops = list(abatch.call_args_list[0].args[1])
assert len(ops) == 2
assert GetOp(("test",), "same", refresh_ttl=True) in ops
assert GetOp(("test",), "different", refresh_ttl=True) in ops
abatch.reset_mock()
doc1 = {"value": 1}
doc2 = {"value": 2}
results = await asyncio.gather(
store.aput(namespace=("test",), key="key", value=doc1),
store.aput(namespace=("test",), key="key", value=doc2),
)
assert len(abatch.call_args_list) == 1
ops = list(abatch.call_args_list[0].args[1])
assert len(ops) == 1
assert ops[0] == PutOp(("test",), "key", doc2)
assert len(results) == 2
assert all(result is None for result in results)
result = await store.aget(namespace=("test",), key="key")
assert result is not None
assert result.value == doc2
abatch.reset_mock()
results = await asyncio.gather(
store.asearch(("test",), filter={"value": 2}),
store.asearch(("test",), filter={"value": 2}),
)
assert len(abatch.call_args_list) == 1
ops = list(abatch.call_args_list[0].args[1])
assert len(ops) == 1
assert len(results) == 2
assert results[0] == results[1]
assert len(results[0]) == 1
assert results[0][0].value == doc2
abatch.reset_mock()
@pytest.fixture
def fake_embeddings() -> CharacterEmbeddings:
@@ -795,88 +548,6 @@ async def test_async_vector_search_with_filters(
assert results[0].key == "doc3"
async def test_async_batched_vector_search_concurrent(
fake_embeddings: CharacterEmbeddings,
) -> None:
"""Test concurrent vector search operations using async batched store."""
store = MockAsyncBatchedStore(
index={"dims": fake_embeddings.dims, "embed": fake_embeddings}
)
colors = ["red", "blue", "green", "yellow", "purple"]
items = ["apple", "car", "house", "book", "phone"]
scores = [3.0, 3.5, 4.0, 4.5, 5.0]
docs = []
for i in range(50):
color = colors[i % len(colors)]
item = items[i % len(items)]
score = scores[i % len(scores)]
docs.append(
(
f"doc{i}",
{"text": f"{color} {item}", "color": color, "score": score, "index": i},
)
)
coros = [
*[store.aput(("test",), key, value) for key, value in docs],
*[store.adelete(("test",), key) for key, value in docs],
*[store.aput(("test",), key, value) for key, value in docs],
]
await asyncio.gather(*coros)
# Prepare multiple search queries with different filters
search_queries: list[tuple[str, dict[str, Any]]] = [
("apple", {"color": "red"}),
("car", {"color": "blue"}),
("house", {"color": "green"}),
("phone", {"score": {"$gt": 4.99}}),
("book", {"score": {"$lte": 3.5}}),
("apple", {"score": {"$gte": 3.0}, "color": "red"}),
("car", {"score": {"$lt": 5.1}, "color": "blue"}),
("house", {"index": {"$gt": 25}}),
("phone", {"index": {"$lte": 10}}),
]
all_results = await asyncio.gather(
*[
store.asearch(("test",), query=query, filter=filter_)
for query, filter_ in search_queries
]
)
for results, (query, filter_) in zip(all_results, search_queries, strict=False):
assert len(results) > 0, f"No results for query '{query}' with filter {filter_}"
for result in results:
if "color" in filter_:
assert result.value["color"] == filter_["color"]
if "score" in filter_:
score = result.value["score"]
for op, value in filter_["score"].items():
if op == "$gt":
assert score > value
elif op == "$gte":
assert score >= value
elif op == "$lt":
assert score < value
elif op == "$lte":
assert score <= value
if "index" in filter_:
index = result.value["index"]
for op, value in filter_["index"].items():
if op == "$gt":
assert index > value
elif op == "$gte":
assert index >= value
elif op == "$lt":
assert index < value
elif op == "$lte":
assert index <= value
def test_vector_search_pagination(fake_embeddings: CharacterEmbeddings) -> None:
"""Test pagination with vector search."""
store = InMemoryStore(
@@ -950,8 +621,8 @@ async def test_embed_with_path(fake_embeddings: CharacterEmbeddings) -> None:
assert results[0].key != results[1].key
ascore = results[0].score
bscore = results[1].score
assert ascore == bscore
assert ascore is not None and bscore is not None
assert ascore == pytest.approx(bscore, abs=1e-5)
results = await store.asearch(("test",), query="uuu")
assert len(results) == 2
@@ -1021,27 +692,3 @@ async def test_embed_with_path(fake_embeddings: CharacterEmbeddings) -> None:
assert len(results) == 3
doc5_result = next(r for r in results if r.key == "doc5")
assert doc5_result.score is None
def test_non_ascii(fake_embeddings: CharacterEmbeddings) -> None:
"""Test support for non-ascii characters"""
store = InMemoryStore(
index={"dims": fake_embeddings.dims, "embed": fake_embeddings}
)
store.put(("user_123", "memories"), "1", {"text": "这是中文"}) # Chinese
store.put(("user_123", "memories"), "2", {"text": "これは日本語です"}) # Japanese
store.put(("user_123", "memories"), "3", {"text": "이건 한국어야"}) # Korean
store.put(("user_123", "memories"), "4", {"text": "Это русский"}) # Russian
store.put(("user_123", "memories"), "5", {"text": "यह रूसी है"}) # Hindi
result1 = store.search(("user_123", "memories"), query="这是中文")
result2 = store.search(("user_123", "memories"), query="これは日本語です")
result3 = store.search(("user_123", "memories"), query="이건 한국어야")
result4 = store.search(("user_123", "memories"), query="Это русский")
result5 = store.search(("user_123", "memories"), query="यह रूसी है")
assert result1[0].key == "1"
assert result2[0].key == "2"
assert result3[0].key == "3"
assert result4[0].key == "4"
assert result5[0].key == "5"
-1443
View File
File diff suppressed because it is too large. Load diff
-1
View File
@@ -1 +0,0 @@
.langgraph_api/
-21
View File
@@ -1,21 +0,0 @@
MIT License
Copyright (c) 2024 LangChain, Inc.
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.
-40
View File
@@ -1,40 +0,0 @@
.PHONY: test lint format test-integration update-schema bump-version
######################
# TESTING AND COVERAGE
######################
TEST?= "tests/unit_tests"
test:
uv run pytest $(TEST)
test-integration:
uv run pytest tests/integration_tests
######################
# LINTING AND FORMATTING
######################
# Define a variable for Python and notebook files.
PYTHON_FILES=.
MYPY_CACHE=.mypy_cache
lint format: PYTHON_FILES=.
lint_diff format_diff: PYTHON_FILES=$(shell git diff --name-only --relative --diff-filter=d main . | grep -E '\.py$$|\.ipynb$$')
lint_package: PYTHON_FILES=langgraph_cli
lint_tests: PYTHON_FILES=tests
lint_tests: MYPY_CACHE=.mypy_cache_test
lint lint_diff lint_package lint_tests:
uv run ruff check .
[ "$(PYTHON_FILES)" = "" ] || uv run ruff format $(PYTHON_FILES) --diff
[ "$(PYTHON_FILES)" = "" ] || uv run ruff check --select I $(PYTHON_FILES)
[ "$(PYTHON_FILES)" = "" ] || mkdir -p $(MYPY_CACHE) || uv run mypy $(PYTHON_FILES) --cache-dir $(MYPY_CACHE)
format format_diff:
uv run ruff format $(PYTHON_FILES)
uv run ruff check --select I --fix $(PYTHON_FILES)
update-schema:
uv run python generate_schema.py
bump-version:
uv run hatch version patch
-105
View File
@@ -1,105 +0,0 @@
# LangGraph CLI
The official command-line interface for LangGraph, providing tools to create, develop, and deploy LangGraph applications.
## Installation
Install via pip:
```bash
pip install langgraph-cli
```
For development mode with hot reloading:
```bash
pip install "langgraph-cli[inmem]"
```
## Commands
### `langgraph new` 🌱
Create a new LangGraph project from a template
```bash
langgraph new [PATH] --template TEMPLATE_NAME
```
### `langgraph dev` 🏃‍♀️
Run LangGraph API server in development mode with hot reloading
```bash
langgraph dev [OPTIONS]
--host TEXT Host to bind to (default: 127.0.0.1)
--port INTEGER Port to bind to (default: 2024)
--no-reload Disable auto-reload
--debug-port INTEGER Enable remote debugging
--no-browser Skip opening browser window
-c, --config FILE Config file path (default: langgraph.json)
```
### `langgraph up` 🚀
Launch LangGraph API server in Docker
```bash
langgraph up [OPTIONS]
-p, --port INTEGER Port to expose (default: 8123)
--wait Wait for services to start
--watch Restart on file changes
--verbose Show detailed logs
-c, --config FILE Config file path
-d, --docker-compose Additional services file
```
### `langgraph build`
Build a Docker image for your LangGraph application
```bash
langgraph build -t IMAGE_TAG [OPTIONS]
--platform TEXT Target platforms (e.g., linux/amd64,linux/arm64)
--pull / --no-pull Use latest/local base image
-c, --config FILE Config file path
```
### `langgraph dockerfile`
Generate a Dockerfile for custom deployments
```bash
langgraph dockerfile SAVE_PATH [OPTIONS]
-c, --config FILE Config file path
```
## Configuration
The CLI uses a `langgraph.json` configuration file with these key settings:
```json
{
"dependencies": ["langchain_openai", "./your_package"], // Required: Package dependencies
"graphs": {
"my_graph": "./your_package/file.py:graph" // Required: Graph definitions
},
"env": "./.env", // Optional: Environment variables
"python_version": "3.11", // Optional: Python version (3.11/3.12)
"pip_config_file": "./pip.conf", // Optional: pip configuration
"dockerfile_lines": [] // Optional: Additional Dockerfile commands
}
```
See the [full documentation](https://langchain-ai.github.io/langgraph/cloud/reference/cli/) for detailed configuration options.
## Development
To develop the CLI itself:
1. Clone the repository
2. Navigate to the CLI directory: `cd libs/cli`
3. Install development dependencies: `uv pip install`
4. Make your changes to the CLI code
5. Test your changes:
```bash
# Run CLI commands directly
uv run langgraph --help
# Or use the examples
cd examples
uv pip install
uv run langgraph dev # or other commands
```
## License
This project is licensed under the terms specified in the repository's LICENSE file.
Loaded 100 of 407 files, more files were not shown because too many files have changed in this diff. Show more