mirror of
https://github.com/langchain-ai/langgraph.git
synced 2026-11-19 20:31:02 +01:00
Compare commits
10
Commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
1b9b0a686e | ||
|
|
9e02a23682 | ||
|
|
5e70e6f307 | ||
|
|
eb57c06896 | ||
|
|
9284b57ba0 | ||
|
|
25fea591b5 | ||
|
|
b9fe53777f | ||
|
|
35c2e8a679 | ||
|
|
e0fb56c6a3 | ||
|
|
3458a3cecb |
No files matched your search
@@ -1,6 +0,0 @@
|
||||
# Contributing to LangGraph
|
||||
|
||||
Hi there! Thank you for even being interested in contributing to LangGraph.
|
||||
As an open-source project in a rapidly developing field, we are extremely open to contributions, whether they involve new features, improved infrastructure, better documentation, or bug fixes.
|
||||
|
||||
To learn how to contribute to LangGraph, please follow the [contribution guide here](https://docs.langchain.com/oss/python/contributing).
|
||||
@@ -1,28 +1,29 @@
|
||||
name: "\U0001F41B Bug Report"
|
||||
description: Report a bug in LangGraph. To report a security issue, please instead use the security option below. For questions, please use the LangChain Forum at forum.langchain.com.
|
||||
labels: [pending, bug]
|
||||
description: Report a bug in LangGraph. To report a security issue, please instead use the security option below. For questions, please use the GitHub Discussions.
|
||||
labels: ["02 Bug Report"]
|
||||
body:
|
||||
- type: markdown
|
||||
attributes:
|
||||
value: |
|
||||
value: >
|
||||
Thank you for taking the time to file a bug report.
|
||||
|
||||
Use this to report BUGS in LangGraph. For usage questions, feature requests and general design questions, please use the [LangChain Forum](https://forum.langchain.com/).
|
||||
|
||||
|
||||
Use this to report BUGS in LangGraph. For usage questions, feature requests and general design questions, please use [GitHub Discussions](https://github.com/langchain-ai/langgraph/discussions).
|
||||
|
||||
Relevant links to check before filing a bug report to see if your issue has already been reported, fixed or
|
||||
if there's another way to solve your problem:
|
||||
|
||||
* [LangChain Forum](https://forum.langchain.com/),
|
||||
* [LangGraph Github Issues](https://github.com/langchain-ai/langgraph/issues),
|
||||
* [LangChain documentation with the integrated search](https://docs.langchain.com/),
|
||||
* [GitHub search](https://github.com/langchain-ai/langgraph),
|
||||
|
||||
[LangGraph Github Discussions](https://github.com/langchain-ai/langgraph/discussions),
|
||||
[LangGraph Github Issues](https://github.com/langchain-ai/langgraph/issues),
|
||||
[LangGraph how-to guides](https://langchain-ai.github.io/langgraph/how-tos/).
|
||||
[LangChain documentation with the integrated search](https://python.langchain.com/docs/get_started/introduction),
|
||||
[GitHub search](https://github.com/langchain-ai/langgraph),
|
||||
- type: checkboxes
|
||||
id: checks
|
||||
attributes:
|
||||
label: Checked other resources
|
||||
description: Before submitting this issue, please confirm that you have completed all the steps below by checking each option. These steps help ensure your issue is well-defined, relevant, and actionable.
|
||||
options:
|
||||
- label: This is a bug, not a usage question. For questions, please use the LangChain Forum (https://forum.langchain.com/).
|
||||
- label: This is a bug, not a usage question. For questions, please use GitHub Discussions.
|
||||
required: true
|
||||
- label: I added a clear and detailed title that summarizes the issue.
|
||||
required: true
|
||||
@@ -37,7 +38,7 @@ body:
|
||||
attributes:
|
||||
label: Example Code
|
||||
description: |
|
||||
Please add a self-contained, [minimal, reproducible, example](https://stackoverflow.com/help/minimal-reproducible-example) with your use case. Replace this code with your own!
|
||||
Please add a self-contained, [minimal, reproducible, example](https://stackoverflow.com/help/minimal-reproducible-example) with your use case.
|
||||
placeholder: |
|
||||
from langgraph.graph import StateGraph
|
||||
|
||||
@@ -77,7 +78,7 @@ body:
|
||||
attributes:
|
||||
label: System Info
|
||||
description: |
|
||||
Run on your machine: `python -m langchain_core.sys_info`
|
||||
python -m langchain_core.sys_info
|
||||
placeholder: |
|
||||
python -m langchain_core.sys_info
|
||||
validations:
|
||||
|
||||
@@ -1,9 +1,15 @@
|
||||
blank_issues_enabled: false
|
||||
version: 2.1
|
||||
contact_links:
|
||||
- name: Documentation
|
||||
url: https://github.com/langchain-ai/docs/issues/new?template=langgraph.yml
|
||||
about: Report an issue related to the LangGraph documentation
|
||||
- name: LangChain Forum
|
||||
url: https://forum.langchain.com/
|
||||
about: General community discussions and support
|
||||
- name: 🤔 Question or Problem
|
||||
about: Ask a question or ask about a problem in GitHub Discussions.
|
||||
url: https://github.com/langchain-ai/langgraph/discussions/categories/q-a
|
||||
- name: Feature Request
|
||||
url: https://github.com/langchain-ai/langgraph/discussions/categories/ideas
|
||||
about: Suggest a feature or an idea
|
||||
- name: Show and tell
|
||||
about: Show what you built with LangChain
|
||||
url: https://github.com/langchain-ai/langgraph/discussions/categories/show-and-tell
|
||||
- name: Slack
|
||||
url: https://www.langchain.com/join-community
|
||||
about: General community discussions
|
||||
@@ -0,0 +1,19 @@
|
||||
name: Documentation
|
||||
description: Report an issue related to the LangGraph documentation.
|
||||
title: "DOC: <Please write a comprehensive title after the 'DOC: ' prefix>"
|
||||
labels: [03 - Documentation]
|
||||
|
||||
body:
|
||||
- type: textarea
|
||||
attributes:
|
||||
label: "Issue with current documentation:"
|
||||
description: >
|
||||
Please make sure to leave a reference to the document/code you're
|
||||
referring to.
|
||||
|
||||
- type: textarea
|
||||
attributes:
|
||||
label: "Idea or request for content:"
|
||||
description: >
|
||||
Please describe as clearly as possible what topics you think are missing
|
||||
from the current documentation.
|
||||
@@ -1,29 +1,25 @@
|
||||
name: 🔒 Privileged
|
||||
description: You are a LangGraph maintainer, or was asked directly by a maintainer to create an issue here. If not, check the other options.
|
||||
description: You are a LangChain maintainer, or was asked directly by a maintainer to create an issue here. If not, check the other options.
|
||||
body:
|
||||
- type: markdown
|
||||
attributes:
|
||||
value: |
|
||||
Thanks for your interest in LangGraph! 🚀
|
||||
|
||||
If you are not a LangGraph maintainer or were not asked directly by a maintainer to create an issue, then please start the conversation on the [LangChain Forum](https://forum.langchain.com/) instead.
|
||||
|
||||
You are a LangGraph maintainer if you maintain any of the packages inside of the LangGraph repository
|
||||
or are a regular contributor to LangGraph with previous merged merged pull requests.
|
||||
Thanks for your interest in LangChain! 🚀
|
||||
|
||||
If you are not a LangChain maintainer or were not asked directly by a maintainer to create an issue, then please start the conversation in a [Question in GitHub Discussions](https://github.com/langchain-ai/langchain/discussions/categories/q-a) instead.
|
||||
|
||||
You are a LangChain maintainer if you maintain any of the packages inside of the LangChain repository
|
||||
or are a regular contributor to LangChain with previous merged merged pull requests.
|
||||
- type: checkboxes
|
||||
id: privileged
|
||||
attributes:
|
||||
label: Privileged issue
|
||||
description: Confirm that you are allowed to create an issue here.
|
||||
options:
|
||||
- label: I am a LangGraph maintainer, or was asked directly by a LangGraph maintainer to create an issue here.
|
||||
- label: I am a LangChain maintainer, or was asked directly by a LangChain maintainer to create an issue here.
|
||||
required: true
|
||||
- type: textarea
|
||||
id: content
|
||||
attributes:
|
||||
label: Issue Content
|
||||
description: Add the content of the issue here.
|
||||
- type: markdown
|
||||
attributes:
|
||||
value: |
|
||||
Community members should **NOT** work on Privileged issues unless these issues have been explicitly marked with a "help-wanted" tag.
|
||||
@@ -1,31 +0,0 @@
|
||||
Thank you for contributing to LangGraph! Follow these steps to mark your pull request as ready for review. **If any of these steps are not completed, your PR will not be considered for review.**
|
||||
|
||||
- [ ] **PR title**: Follows the format: {TYPE}({SCOPE}): {DESCRIPTION}
|
||||
- Examples:
|
||||
- feat(core): add multi-tenant support
|
||||
- fix(cli): resolve flag parsing error
|
||||
- docs(openai): update API usage examples
|
||||
- Allowed `{TYPE}` values:
|
||||
- feat, fix, docs, style, refactor, perf, test, build, ci, chore, revert, release
|
||||
- Allowed `{SCOPE}` values (optional):
|
||||
- langgraph, docs, cli, checkpoint, checkpoint-postgres, checkpoint-sqlite, prebuilt, scheduler-kafka, sdk-py
|
||||
- Once you've written the title, please delete this checklist item; do not include it in the PR.
|
||||
|
||||
- [ ] **PR message**: ***Delete this entire checklist*** and replace with
|
||||
- **Description:** a description of the change. Include a [closing keyword](https://docs.github.com/en/issues/tracking-your-work-with-issues/using-issues/linking-a-pull-request-to-an-issue#linking-a-pull-request-to-an-issue-using-a-keyword) if applicable.
|
||||
- **Issue:** the issue # it fixes, if applicable
|
||||
- **Dependencies:** any dependencies required for this change
|
||||
- **Twitter handle:** if your PR gets announced, and you'd like a mention, we'll gladly shout you out!
|
||||
|
||||
- [ ] **Add tests and docs**: If you're adding a new integration, you must include:
|
||||
1. A test for the integration, preferably unit tests that do not rely on network access,
|
||||
2. An example notebook showing its use. It lives in `docs/docs/integrations` directory.
|
||||
|
||||
- [ ] **Lint and test**: Run `make format`, `make lint` and `make test` from the root of the package(s) you've modified. We will not consider a PR unless these three are passing in CI. See [contribution guidelines](https://github.com/langchain-ai/langgraph/blob/main/CONTRIBUTING.md) for more.
|
||||
|
||||
Additional guidelines:
|
||||
|
||||
- Make sure optional dependencies are imported within a function.
|
||||
- Please do not add dependencies to `pyproject.toml` files (even optional ones) unless they are **required** for unit tests.
|
||||
- Most PRs should not touch more than one package.
|
||||
- Changes should be backwards compatible.
|
||||
@@ -0,0 +1,88 @@
|
||||
# An action for setting up poetry install with caching.
|
||||
# Using a custom action since the default action does not
|
||||
# take poetry install groups into account.
|
||||
# Action code from:
|
||||
# https://github.com/actions/setup-python/issues/505#issuecomment-1273013236
|
||||
name: poetry-install-with-caching
|
||||
description: Poetry install with support for caching of dependency groups.
|
||||
|
||||
inputs:
|
||||
python-version:
|
||||
description: Python version, supporting MAJOR.MINOR only
|
||||
required: true
|
||||
|
||||
poetry-version:
|
||||
description: Poetry version
|
||||
required: true
|
||||
|
||||
cache-key:
|
||||
description: Cache key to use for manual handling of caching
|
||||
required: true
|
||||
|
||||
runs:
|
||||
using: composite
|
||||
steps:
|
||||
- uses: actions/setup-python@v5
|
||||
name: Setup python ${{ inputs.python-version }}
|
||||
id: setup-python
|
||||
with:
|
||||
python-version: ${{ inputs.python-version }}
|
||||
|
||||
- uses: actions/cache@v3
|
||||
id: cache-bin-poetry
|
||||
name: Cache Poetry binary - Python ${{ inputs.python-version }}
|
||||
env:
|
||||
SEGMENT_DOWNLOAD_TIMEOUT_MIN: "1"
|
||||
with:
|
||||
path: |
|
||||
/opt/pipx/venvs/poetry
|
||||
# This step caches the poetry installation, so make sure it's keyed on the poetry version as well.
|
||||
key: bin-poetry-${{ runner.os }}-${{ runner.arch }}-py-${{ inputs.python-version }}-${{ inputs.poetry-version }}
|
||||
|
||||
- name: Refresh shell hashtable and fixup softlinks
|
||||
if: steps.cache-bin-poetry.outputs.cache-hit == 'true'
|
||||
shell: bash
|
||||
env:
|
||||
POETRY_VERSION: ${{ inputs.poetry-version }}
|
||||
PYTHON_VERSION: ${{ inputs.python-version }}
|
||||
run: |
|
||||
set -eux
|
||||
|
||||
# Refresh the shell hashtable, to ensure correct `which` output.
|
||||
hash -r
|
||||
|
||||
# `actions/cache@v3` doesn't always seem able to correctly unpack softlinks.
|
||||
# Delete and recreate the softlinks pipx expects to have.
|
||||
rm /opt/pipx/venvs/poetry/bin/python
|
||||
cd /opt/pipx/venvs/poetry/bin
|
||||
ln -s "$(which "python$PYTHON_VERSION")" python
|
||||
chmod +x python
|
||||
cd /opt/pipx_bin/
|
||||
ln -s /opt/pipx/venvs/poetry/bin/poetry poetry
|
||||
chmod +x poetry
|
||||
|
||||
# Ensure everything got set up correctly.
|
||||
/opt/pipx/venvs/poetry/bin/python --version
|
||||
/opt/pipx_bin/poetry --version
|
||||
|
||||
- name: Install poetry
|
||||
if: steps.cache-bin-poetry.outputs.cache-hit != 'true'
|
||||
shell: bash
|
||||
env:
|
||||
POETRY_VERSION: ${{ inputs.poetry-version }}
|
||||
PYTHON_VERSION: ${{ inputs.python-version }}
|
||||
# Install poetry using the python version installed by setup-python step.
|
||||
run: pipx install "poetry==$POETRY_VERSION" --python '${{ steps.setup-python.outputs.python-path }}' --verbose
|
||||
|
||||
- name: Restore pip and poetry cached dependencies
|
||||
uses: actions/cache@v3
|
||||
env:
|
||||
SEGMENT_DOWNLOAD_TIMEOUT_MIN: "4"
|
||||
with:
|
||||
path: |
|
||||
~/.cache/pip
|
||||
~/.cache/pypoetry/virtualenvs
|
||||
~/.cache/pypoetry/cache
|
||||
~/.cache/pypoetry/artifacts
|
||||
./.venv
|
||||
key: py-deps-${{ runner.os }}-${{ runner.arch }}-py-${{ inputs.python-version }}-poetry-${{ inputs.poetry-version }}-${{ inputs.cache-key }}-${{ hashFiles('./poetry.lock') }}
|
||||
@@ -1,18 +0,0 @@
|
||||
version: 2
|
||||
updates:
|
||||
- package-ecosystem: "github-actions"
|
||||
directory: "/"
|
||||
schedule:
|
||||
interval: "weekly"
|
||||
|
||||
- package-ecosystem: "pip"
|
||||
directories:
|
||||
- "libs/checkpoint"
|
||||
- "libs/checkpoint-postgres"
|
||||
- "libs/checkpoint-sqlite"
|
||||
- "libs/cli"
|
||||
- "libs/langgraph"
|
||||
- "libs/prebuilt"
|
||||
- "libs/sdk-py"
|
||||
schedule:
|
||||
interval: "weekly"
|
||||
@@ -1,15 +1,10 @@
|
||||
import ast
|
||||
import os
|
||||
from itertools import filterfalse
|
||||
from typing import Dict, List, Tuple
|
||||
from typing import List, Tuple
|
||||
|
||||
ROOT_PATH = os.path.abspath(os.path.join(__file__, "..", "..", ".."))
|
||||
CLIENT_PATH = os.path.join(ROOT_PATH, "libs", "sdk-py", "langgraph_sdk", "client.py")
|
||||
ASYNC_TO_SYNC_METHOD_MAP: Dict[str, str] = {
|
||||
"aclose": "close",
|
||||
"__aenter__": "__enter__",
|
||||
"__aexit__": "__exit__",
|
||||
}
|
||||
|
||||
|
||||
def get_class_methods(node: ast.ClassDef) -> List[str]:
|
||||
@@ -27,7 +22,7 @@ def find_classes(tree: ast.AST) -> List[Tuple[str, List[str]]]:
|
||||
|
||||
def compare_sync_async_methods(sync_methods: List[str], async_methods: List[str]) -> List[str]:
|
||||
sync_set = set(sync_methods)
|
||||
async_set = {ASYNC_TO_SYNC_METHOD_MAP.get(async_method, async_method) for async_method in async_methods}
|
||||
async_set = set(async_methods)
|
||||
missing_in_sync = list(async_set - sync_set)
|
||||
missing_in_async = list(sync_set - async_set)
|
||||
return missing_in_sync + missing_in_async
|
||||
@@ -38,7 +33,7 @@ def main():
|
||||
tree = ast.parse(file.read())
|
||||
|
||||
classes = find_classes(tree)
|
||||
|
||||
|
||||
def is_sync(class_spec: Tuple[str, List[str]]) -> bool:
|
||||
return class_spec[0].startswith("Sync")
|
||||
|
||||
|
||||
@@ -1,164 +1,108 @@
|
||||
import logging
|
||||
import asyncio
|
||||
import json
|
||||
import os
|
||||
import pathlib
|
||||
import sys
|
||||
import time
|
||||
from urllib import error, request
|
||||
|
||||
import langgraph_cli
|
||||
import langgraph_cli.config
|
||||
import langgraph_cli.docker
|
||||
from langgraph_cli.cli import prepare_args_and_stdin
|
||||
from langgraph_cli.constants import DEFAULT_PORT
|
||||
import langgraph_cli.config
|
||||
|
||||
from langgraph_cli.exec import Runner, subp_exec
|
||||
from langgraph_cli.progress import Progress
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
logging.basicConfig(level=logging.INFO)
|
||||
from langgraph_cli.constants import DEFAULT_PORT
|
||||
|
||||
|
||||
def test(config: pathlib.Path, port: int, tag: str, verbose: bool):
|
||||
"""Spin up API with Postgres/Redis via docker compose and wait until ready."""
|
||||
logger.info("Starting test...")
|
||||
def test(
|
||||
config: pathlib.Path,
|
||||
port: int,
|
||||
tag: str,
|
||||
verbose: bool,
|
||||
):
|
||||
with Runner() as runner, Progress(message="Pulling...") as set:
|
||||
# Detect docker/compose capabilities
|
||||
# check docker available
|
||||
capabilities = langgraph_cli.docker.check_capabilities(runner)
|
||||
|
||||
# Validate config and prepare compose stdin/args using built image
|
||||
# open config
|
||||
config_json = langgraph_cli.config.validate_config_file(config)
|
||||
args, stdin = prepare_args_and_stdin(
|
||||
capabilities=capabilities,
|
||||
config_path=config,
|
||||
config=config_json,
|
||||
docker_compose=None,
|
||||
port=port,
|
||||
watch=False,
|
||||
debugger_port=None,
|
||||
debugger_base_url=f"http://127.0.0.1:{port}",
|
||||
postgres_uri=None,
|
||||
api_version=None,
|
||||
image=tag,
|
||||
base_image=None,
|
||||
)
|
||||
|
||||
# Compose up with wait (implies detach), similar to `langgraph up --wait`
|
||||
args_up = [*args, "up", "--remove-orphans", "--wait"]
|
||||
|
||||
compose_cmd = ["docker", "compose"]
|
||||
if capabilities.compose_type == "standalone":
|
||||
compose_cmd = ["docker-compose"]
|
||||
|
||||
set("Starting...")
|
||||
try:
|
||||
runner.run(
|
||||
subp_exec(
|
||||
*compose_cmd,
|
||||
*args_up,
|
||||
input=stdin,
|
||||
verbose=verbose,
|
||||
)
|
||||
set("Running...")
|
||||
args = [
|
||||
"run",
|
||||
"--rm",
|
||||
"-p",
|
||||
f"{port}:8000",
|
||||
]
|
||||
if isinstance(config_json["env"], str):
|
||||
args.extend(
|
||||
[
|
||||
"--env-file",
|
||||
str(config.parent / config_json["env"]),
|
||||
]
|
||||
)
|
||||
except Exception as e: # noqa: BLE001
|
||||
# On failure, show diagnostics then ensure clean teardown
|
||||
sys.stderr.write(f"docker compose up failed: {e}\n")
|
||||
try:
|
||||
sys.stderr.write("\n== docker compose ps ==\n")
|
||||
runner.run(
|
||||
subp_exec(*compose_cmd, *args, "ps", input=stdin, verbose=False)
|
||||
)
|
||||
except Exception:
|
||||
pass
|
||||
try:
|
||||
sys.stderr.write("\n== docker compose logs (api) ==\n")
|
||||
runner.run(
|
||||
subp_exec(
|
||||
*compose_cmd,
|
||||
*args,
|
||||
"logs",
|
||||
"langgraph-api",
|
||||
input=stdin,
|
||||
verbose=False,
|
||||
)
|
||||
)
|
||||
except Exception:
|
||||
pass
|
||||
finally:
|
||||
try:
|
||||
runner.run(
|
||||
subp_exec(
|
||||
*compose_cmd,
|
||||
*args,
|
||||
"down",
|
||||
"-v",
|
||||
"--remove-orphans",
|
||||
input=stdin,
|
||||
verbose=False,
|
||||
)
|
||||
)
|
||||
finally:
|
||||
raise
|
||||
|
||||
set("")
|
||||
base_url = f"http://localhost:{port}"
|
||||
ok_url = f"{base_url}/ok"
|
||||
logger.info(f"Waiting for {ok_url} to respond with 200...")
|
||||
deadline = time.time() + 30
|
||||
last_err: Exception | None = None
|
||||
while time.time() < deadline:
|
||||
try:
|
||||
with request.urlopen(ok_url, timeout=2) as resp:
|
||||
if resp.status == 200:
|
||||
sys.stdout.write(
|
||||
f"""Ready!\n- API: {base_url}\n- /ok: 200 OK\n"""
|
||||
)
|
||||
sys.stdout.flush()
|
||||
break
|
||||
else:
|
||||
last_err = RuntimeError(f"Unexpected status: {resp.status}")
|
||||
logger.error(f"Unexpected status: {resp.status}")
|
||||
except error.URLError as e:
|
||||
logger.error(f"URLError: {e}")
|
||||
last_err = e
|
||||
except Exception as e: # noqa: BLE001
|
||||
logger.error(f"Exception: {e}")
|
||||
last_err = e
|
||||
time.sleep(0.5)
|
||||
else:
|
||||
logger.error("Timeout waiting for /ok to return 200")
|
||||
# Bring stack down before raising
|
||||
args_down = [*args, "down", "-v", "--remove-orphans"]
|
||||
try:
|
||||
runner.run(
|
||||
subp_exec(
|
||||
*compose_cmd,
|
||||
*args_down,
|
||||
input=stdin,
|
||||
verbose=verbose,
|
||||
)
|
||||
)
|
||||
finally:
|
||||
raise SystemExit(
|
||||
f"/ok did not return 202 within timeout. Last error: {last_err}"
|
||||
for k, v in config_json["env"].items():
|
||||
args.extend(
|
||||
[
|
||||
"-e",
|
||||
f"{k}={v}",
|
||||
]
|
||||
)
|
||||
if capabilities.healthcheck_start_interval:
|
||||
args.extend(
|
||||
[
|
||||
"--health-interval",
|
||||
"5s",
|
||||
"--health-retries",
|
||||
"1",
|
||||
"--health-start-period",
|
||||
"10s",
|
||||
"--health-start-interval",
|
||||
"1s",
|
||||
]
|
||||
)
|
||||
else:
|
||||
args.extend(
|
||||
[
|
||||
"--health-interval",
|
||||
"5s",
|
||||
"--health-retries",
|
||||
"2",
|
||||
]
|
||||
)
|
||||
|
||||
_task = None
|
||||
|
||||
def on_stdout(line: str):
|
||||
nonlocal _task
|
||||
if "GET /ok" in line or "Uvicorn running on" in line:
|
||||
set("")
|
||||
sys.stdout.write(
|
||||
f"""Ready!
|
||||
- API: http://localhost:{port}
|
||||
"""
|
||||
)
|
||||
sys.stdout.flush()
|
||||
_task.cancel()
|
||||
return True
|
||||
return False
|
||||
|
||||
async def subp_exec_task(*args, **kwargs):
|
||||
nonlocal _task
|
||||
_task = asyncio.create_task(subp_exec(*args, **kwargs))
|
||||
await _task
|
||||
|
||||
# Clean up: bring compose stack down to free ports for next test
|
||||
logger.info("Test succeeded. Bringing down compose stack...")
|
||||
try:
|
||||
args_down = [*args, "down", "-v", "--remove-orphans"]
|
||||
runner.run(
|
||||
subp_exec(
|
||||
*compose_cmd,
|
||||
*args_down,
|
||||
input=stdin,
|
||||
subp_exec_task(
|
||||
"docker",
|
||||
*args,
|
||||
tag,
|
||||
verbose=verbose,
|
||||
on_stdout=on_stdout,
|
||||
)
|
||||
)
|
||||
logger.info("Compose stack down. Finishing...")
|
||||
except Exception:
|
||||
logger.exception("Failed to bring down compose stack")
|
||||
except asyncio.CancelledError:
|
||||
pass
|
||||
|
||||
logger.info("Test finished")
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
import argparse
|
||||
@@ -166,12 +110,6 @@ if __name__ == "__main__":
|
||||
parser = argparse.ArgumentParser()
|
||||
parser.add_argument("-t", "--tag", type=str)
|
||||
parser.add_argument("-c", "--config", type=str, default="./langgraph.json")
|
||||
parser.add_argument("-p", "--port", type=int, default=DEFAULT_PORT)
|
||||
parser.add_argument("-p", "--port", default=DEFAULT_PORT)
|
||||
args = parser.parse_args()
|
||||
try:
|
||||
test(pathlib.Path(args.config), args.port, args.tag, verbose=True)
|
||||
except BaseException:
|
||||
logger.exception("Test failed")
|
||||
raise
|
||||
|
||||
logger.info("Test execution finished")
|
||||
test(pathlib.Path(args.config), args.port, args.tag, verbose=True)
|
||||
@@ -3,8 +3,8 @@ name: CLI integration test
|
||||
on:
|
||||
workflow_call:
|
||||
|
||||
permissions:
|
||||
contents: read
|
||||
env:
|
||||
POETRY_VERSION: "1.7.1"
|
||||
|
||||
jobs:
|
||||
build:
|
||||
@@ -13,106 +13,62 @@ jobs:
|
||||
matrix:
|
||||
python-version:
|
||||
- "3.10"
|
||||
- "3.14"
|
||||
example:
|
||||
- name: A
|
||||
workdir: libs/cli/examples
|
||||
tag: langgraph-test-a
|
||||
- name: B
|
||||
workdir: libs/cli/examples/graphs
|
||||
tag: langgraph-test-b
|
||||
- name: C
|
||||
workdir: libs/cli/examples/graphs_reqs_a
|
||||
tag: langgraph-test-c
|
||||
- name: D
|
||||
workdir: libs/cli/examples/graphs_reqs_b
|
||||
tag: langgraph-test-d
|
||||
- "3.11"
|
||||
name: "CLI integration test"
|
||||
defaults:
|
||||
run:
|
||||
working-directory: libs/cli
|
||||
steps:
|
||||
- uses: actions/checkout@v6
|
||||
- uses: actions/checkout@v4
|
||||
- name: Get changed files
|
||||
id: changed-files
|
||||
uses: Ana06/get-changed-files@v2.3.0
|
||||
with:
|
||||
filter: "libs/cli/**"
|
||||
- name: Set up Python ${{ matrix.python-version }}
|
||||
- name: Set up Python ${{ matrix.python-version }} + Poetry ${{ env.POETRY_VERSION }}
|
||||
if: steps.changed-files.outputs.all
|
||||
uses: astral-sh/setup-uv@v7
|
||||
uses: "./.github/actions/poetry_setup"
|
||||
with:
|
||||
python-version: ${{ matrix.python-version }}
|
||||
enable-cache: true
|
||||
cache-suffix: "cli-integration-test"
|
||||
ignore-nothing-to-cache: true
|
||||
poetry-version: ${{ env.POETRY_VERSION }}
|
||||
cache-key: integration-test-cli
|
||||
- name: Setup env
|
||||
if: steps.changed-files.outputs.all
|
||||
working-directory: libs/cli/examples
|
||||
run: cat .env.example > .env
|
||||
- name: Install cli globally
|
||||
if: steps.changed-files.outputs.all
|
||||
run: pip install -e .
|
||||
- name: Build and test service ${{ matrix.example.name }}
|
||||
- name: Build and test service A
|
||||
if: steps.changed-files.outputs.all
|
||||
working-directory: ${{ matrix.example.workdir }}
|
||||
env:
|
||||
LANGSMITH_API_KEY: ${{ secrets.LANGSMITH_API_KEY }}
|
||||
working-directory: libs/cli/examples
|
||||
run: |
|
||||
# Build the image for this example
|
||||
langgraph build -t ${{ matrix.example.tag }}
|
||||
# Prepare environment file from local or parent example directory
|
||||
if [ -f .env.example ]; then cp .env.example .env; elif [ -f ../.env.example ]; then cp ../.env.example .env && cp ../.env.example ../.env; fi
|
||||
if [ -n "${{ secrets.LANGSMITH_API_KEY }}" ]; then echo "LANGSMITH_API_KEY=${{ secrets.LANGSMITH_API_KEY }}" >> .env; if [ -f ../.env ]; then echo "LANGSMITH_API_KEY=${{ secrets.LANGSMITH_API_KEY }}" >> ../.env; fi; fi
|
||||
# Run the integration test using the built tag
|
||||
# Compute repo root to reference the shared script robustly
|
||||
REPO_ROOT=$(git rev-parse --show-toplevel)
|
||||
timeout 60 python "$REPO_ROOT/.github/scripts/run_langgraph_cli_test.py" -t ${{ matrix.example.tag }}
|
||||
# The build-arg isn't used; just testing that we accept other args
|
||||
langgraph build -t langgraph-test-a --base-image "langchain/langgraph-trial"
|
||||
cp .env.example .envg
|
||||
timeout 60 python ../../../.github/scripts/run_langgraph_cli_test.py -c langgraph.json -t langgraph-test-a
|
||||
- name: Build and test service B
|
||||
if: steps.changed-files.outputs.all
|
||||
working-directory: libs/cli/examples/graphs
|
||||
run: |
|
||||
langgraph build -t langgraph-test-b --base-image "langchain/langgraph-trial"
|
||||
timeout 60 python ../../../../.github/scripts/run_langgraph_cli_test.py -t langgraph-test-b
|
||||
- name: Build and test service C
|
||||
if: steps.changed-files.outputs.all
|
||||
working-directory: libs/cli/examples/graphs_reqs_a
|
||||
run: |
|
||||
langgraph build -t langgraph-test-c --base-image "langchain/langgraph-trial"
|
||||
timeout 60 python ../../../../.github/scripts/run_langgraph_cli_test.py -t langgraph-test-c
|
||||
- name: Build and test service D
|
||||
if: steps.changed-files.outputs.all
|
||||
working-directory: libs/cli/examples/graphs_reqs_b
|
||||
run: |
|
||||
langgraph build -t langgraph-test-d --base-image "langchain/langgraph-trial"
|
||||
timeout 60 python ../../../../.github/scripts/run_langgraph_cli_test.py -t langgraph-test-d
|
||||
|
||||
- name: Build JS service
|
||||
if: ${{ steps.changed-files.outputs.all && matrix.example.name == 'A' }}
|
||||
if: steps.changed-files.outputs.all
|
||||
working-directory: libs/cli/js-examples
|
||||
run: |
|
||||
langgraph build -t langgraph-test-e
|
||||
|
||||
- name: Build JS monorepo service
|
||||
if: ${{ steps.changed-files.outputs.all && matrix.example.name == 'A' }}
|
||||
working-directory: libs/cli/js-monorepo-example
|
||||
run: |
|
||||
langgraph build -t langgraph-test-f -c apps/agent/langgraph.json --build-command "yarn run turbo build" --install-command "yarn install"
|
||||
|
||||
- name: Build Python monorepo service
|
||||
if: ${{ steps.changed-files.outputs.all && matrix.example.name == 'A' }}
|
||||
working-directory: libs/cli/python-monorepo-example
|
||||
run: |
|
||||
langgraph build -t langgraph-test-g -c apps/agent/langgraph.json
|
||||
cp apps/agent/.env.example apps/agent/.env
|
||||
if [ -n "${{ secrets.LANGSMITH_API_KEY }}" ]; then echo "LANGSMITH_API_KEY=${{ secrets.LANGSMITH_API_KEY }}" >> apps/agent/.env; fi
|
||||
timeout 60 python ../../../.github/scripts/run_langgraph_cli_test.py -t langgraph-test-g -c apps/agent/langgraph.json
|
||||
|
||||
- name: Build and test prerelease reqs service
|
||||
if: ${{ steps.changed-files.outputs.all && matrix.example.name == 'A' }}
|
||||
working-directory: libs/cli/examples/graph_prerelease_reqs
|
||||
run: |
|
||||
langgraph build -t langgraph-test-h
|
||||
cp ../.env.example .env
|
||||
if [ -n "${{ secrets.LANGSMITH_API_KEY }}" ]; then echo "LANGSMITH_API_KEY=${{ secrets.LANGSMITH_API_KEY }}" >> .env; fi
|
||||
timeout 60 python ../../../../.github/scripts/run_langgraph_cli_test.py -t langgraph-test-h
|
||||
echo "Finished starting up langgraph-test-h"
|
||||
LANGGRAPH_VERSION=$(docker run --rm --entrypoint "" langgraph-test-h python -c "import sys; from importlib.metadata import version; v = version('langgraph'); print(v);")
|
||||
if [ "$LANGGRAPH_VERSION" != "1.0.2" ]; then
|
||||
echo "LANGGRAPH_VERSION != 1.0.2; $LANGGRAPH_VERSION"
|
||||
exit 1
|
||||
fi
|
||||
LANGCHAIN_OPENAI_VERSION=$(docker run --rm --entrypoint "" langgraph-test-h python -c "import sys; from importlib.metadata import version; v = version('langchain-openai'); print(v);")
|
||||
if [ "$LANGCHAIN_OPENAI_VERSION" != "1.0.1" ]; then
|
||||
echo "LANGCHAIN_OPENAI_VERSION != 1.0.1; $LANGCHAIN_OPENAI_VERSION"
|
||||
exit 1
|
||||
fi
|
||||
LANGCHAIN_ANTHROPIC_VERSION=$(docker run --rm --entrypoint "" langgraph-test-h python -c "import sys; from importlib.metadata import version; v = version('langchain-anthropic'); print(v);")
|
||||
if [ "$LANGCHAIN_ANTHROPIC_VERSION" != "1.0.0a5" ]; then
|
||||
echo "LANGCHAIN_ANTHROPIC_VERSION != 1.0.0a5; $LANGCHAIN_ANTHROPIC_VERSION"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
- name: Build and test prerelease reqs fail service
|
||||
if: ${{ steps.changed-files.outputs.all && matrix.example.name == 'A' }}
|
||||
working-directory: libs/cli/examples/graph_prerelease_reqs_fail
|
||||
run: |
|
||||
langgraph build -t langgraph-test-i || [ $? -eq 1 ]
|
||||
|
||||
+42
-14
@@ -8,10 +8,9 @@ on:
|
||||
type: string
|
||||
description: "From which folder this pipeline executes"
|
||||
|
||||
permissions:
|
||||
contents: read
|
||||
|
||||
env:
|
||||
POETRY_VERSION: "1.7.1"
|
||||
|
||||
# This env var allows us to get inline annotations when ruff has complaints.
|
||||
RUFF_OUTPUT_FORMAT: github
|
||||
|
||||
@@ -31,34 +30,54 @@ jobs:
|
||||
- "3.12"
|
||||
name: "lint #${{ matrix.python-version }}"
|
||||
steps:
|
||||
- uses: actions/checkout@v6
|
||||
- uses: actions/checkout@v4
|
||||
- name: Get changed files
|
||||
id: changed-files
|
||||
uses: Ana06/get-changed-files@v2.3.0
|
||||
with:
|
||||
filter: "${{ inputs.working-directory }}/**"
|
||||
- name: Set up Python ${{ matrix.python-version }}
|
||||
- name: Set up Python ${{ matrix.python-version }} + Poetry ${{ env.POETRY_VERSION }}
|
||||
if: steps.changed-files.outputs.all
|
||||
uses: astral-sh/setup-uv@v7
|
||||
uses: "./.github/actions/poetry_setup"
|
||||
with:
|
||||
python-version: ${{ matrix.python-version }}
|
||||
enable-cache: true
|
||||
cache-suffix: lint-${{ inputs.working-directory }}
|
||||
poetry-version: ${{ env.POETRY_VERSION }}
|
||||
cache-key: lint-${{ inputs.working-directory }}
|
||||
|
||||
- name: Check Poetry File
|
||||
if: steps.changed-files.outputs.all
|
||||
shell: bash
|
||||
working-directory: ${{ inputs.working-directory }}
|
||||
run: poetry check
|
||||
|
||||
- name: Check lock file
|
||||
if: steps.changed-files.outputs.all
|
||||
shell: bash
|
||||
working-directory: ${{ inputs.working-directory }}
|
||||
run: poetry lock --check
|
||||
|
||||
- name: Install dependencies
|
||||
if: steps.changed-files.outputs.all
|
||||
# Also installs dev/lint/test/typing dependencies, to ensure we have
|
||||
# type hints for as many of our libraries as possible.
|
||||
# This helps catch errors that require dependencies to be spotted, for example:
|
||||
# https://github.com/langchain-ai/langchain/pull/10249/files#diff-935185cd488d015f026dcd9e19616ff62863e8cde8c0bee70318d3ccbca98341
|
||||
#
|
||||
# If you change this configuration, make sure to change the `cache-key`
|
||||
# in the `poetry_setup` action above to stop using the old cache.
|
||||
# It doesn't matter how you change it, any change will cause a cache-bust.
|
||||
working-directory: ${{ inputs.working-directory }}
|
||||
run: uv sync --frozen --group lint
|
||||
run: poetry install --with dev
|
||||
|
||||
- name: Get .mypy_cache to speed up mypy
|
||||
if: steps.changed-files.outputs.all
|
||||
uses: actions/cache@v5
|
||||
uses: actions/cache@v3
|
||||
env:
|
||||
SEGMENT_DOWNLOAD_TIMEOUT_MIN: "2"
|
||||
with:
|
||||
path: |
|
||||
${{ inputs.working-directory }}/.mypy_cache
|
||||
key: mypy-lint-${{ runner.os }}-${{ runner.arch }}-py${{ matrix.python-version }}-${{ inputs.working-directory }}-${{ hashFiles(format('{0}/uv.lock', inputs.working-directory)) }}
|
||||
key: mypy-lint-${{ runner.os }}-${{ runner.arch }}-py${{ matrix.python-version }}-${{ inputs.working-directory }}-${{ hashFiles(format('{0}/poetry.lock', inputs.working-directory)) }}
|
||||
|
||||
- name: Analysing package code with our lint
|
||||
if: steps.changed-files.outputs.all
|
||||
@@ -73,18 +92,27 @@ jobs:
|
||||
|
||||
- name: Install test dependencies
|
||||
if: steps.changed-files.outputs.all
|
||||
# Also installs dev/lint/test/typing dependencies, to ensure we have
|
||||
# type hints for as many of our libraries as possible.
|
||||
# This helps catch errors that require dependencies to be spotted, for example:
|
||||
# https://github.com/langchain-ai/langchain/pull/10249/files#diff-935185cd488d015f026dcd9e19616ff62863e8cde8c0bee70318d3ccbca98341
|
||||
#
|
||||
# If you change this configuration, make sure to change the `cache-key`
|
||||
# in the `poetry_setup` action above to stop using the old cache.
|
||||
# It doesn't matter how you change it, any change will cause a cache-bust.
|
||||
working-directory: ${{ inputs.working-directory }}
|
||||
run: uv sync --group lint
|
||||
run: |
|
||||
poetry install --with dev
|
||||
|
||||
- name: Get .mypy_cache_test to speed up mypy
|
||||
if: steps.changed-files.outputs.all
|
||||
uses: actions/cache@v5
|
||||
uses: actions/cache@v3
|
||||
env:
|
||||
SEGMENT_DOWNLOAD_TIMEOUT_MIN: "2"
|
||||
with:
|
||||
path: |
|
||||
${{ inputs.working-directory }}/.mypy_cache_test
|
||||
key: mypy-test-${{ runner.os }}-${{ runner.arch }}-py${{ matrix.python-version }}-${{ inputs.working-directory }}-${{ hashFiles(format('{0}/uv.lock', inputs.working-directory)) }}
|
||||
key: mypy-test-${{ runner.os }}-${{ runner.arch }}-py${{ matrix.python-version }}-${{ inputs.working-directory }}-${{ hashFiles(format('{0}/poetry.lock', inputs.working-directory)) }}
|
||||
|
||||
- name: Analysing tests with our lint
|
||||
if: steps.changed-files.outputs.all
|
||||
|
||||
+12
-10
@@ -8,8 +8,8 @@ on:
|
||||
type: string
|
||||
description: "From which folder this pipeline executes"
|
||||
|
||||
permissions:
|
||||
contents: read
|
||||
env:
|
||||
POETRY_VERSION: "1.7.1"
|
||||
|
||||
jobs:
|
||||
build:
|
||||
@@ -17,21 +17,21 @@ jobs:
|
||||
strategy:
|
||||
matrix:
|
||||
python-version:
|
||||
- "3.9"
|
||||
- "3.10"
|
||||
- "3.11"
|
||||
- "3.12"
|
||||
- "3.13"
|
||||
- "3.14"
|
||||
|
||||
name: "test #${{ matrix.python-version }}"
|
||||
steps:
|
||||
- uses: actions/checkout@v6
|
||||
- name: Set up Python ${{ matrix.python-version }}
|
||||
uses: astral-sh/setup-uv@v7
|
||||
- uses: actions/checkout@v4
|
||||
- name: Set up Python ${{ matrix.python-version }} + Poetry ${{ env.POETRY_VERSION }}
|
||||
uses: "./.github/actions/poetry_setup"
|
||||
with:
|
||||
python-version: ${{ matrix.python-version }}
|
||||
enable-cache: true
|
||||
cache-suffix: test-${{ inputs.working-directory }}
|
||||
poetry-version: ${{ env.POETRY_VERSION }}
|
||||
cache-key: test-${{ inputs.working-directory }}
|
||||
- name: Login to Docker Hub
|
||||
uses: docker/login-action@v3
|
||||
if: ${{ !github.event.pull_request.head.repo.fork }}
|
||||
@@ -42,12 +42,14 @@ jobs:
|
||||
- name: Install dependencies
|
||||
shell: bash
|
||||
working-directory: ${{ inputs.working-directory }}
|
||||
run: uv sync --frozen --group test --no-dev
|
||||
run: |
|
||||
poetry install --with dev
|
||||
|
||||
- name: Run tests
|
||||
shell: bash
|
||||
working-directory: ${{ inputs.working-directory }}
|
||||
run: make test
|
||||
run: |
|
||||
make test
|
||||
|
||||
- name: Ensure the tests did not create any additional files
|
||||
shell: bash
|
||||
|
||||
@@ -3,8 +3,8 @@ name: test
|
||||
on:
|
||||
workflow_call:
|
||||
|
||||
permissions:
|
||||
contents: read
|
||||
env:
|
||||
POETRY_VERSION: "1.7.1"
|
||||
|
||||
jobs:
|
||||
build:
|
||||
@@ -12,24 +12,24 @@ jobs:
|
||||
strategy:
|
||||
matrix:
|
||||
python-version:
|
||||
- "3.9"
|
||||
- "3.10"
|
||||
- "3.11"
|
||||
- "3.12"
|
||||
- "3.13"
|
||||
- "3.14"
|
||||
|
||||
defaults:
|
||||
run:
|
||||
working-directory: libs/langgraph
|
||||
name: "test #${{ matrix.python-version }}"
|
||||
steps:
|
||||
- uses: actions/checkout@v6
|
||||
- name: Set up Python ${{ matrix.python-version }}
|
||||
uses: astral-sh/setup-uv@v7
|
||||
- uses: actions/checkout@v4
|
||||
- name: Set up Python ${{ matrix.python-version }} + Poetry ${{ env.POETRY_VERSION }}
|
||||
uses: "./.github/actions/poetry_setup"
|
||||
with:
|
||||
python-version: ${{ matrix.python-version }}
|
||||
enable-cache: true
|
||||
cache-suffix: "test-langgraph"
|
||||
poetry-version: ${{ env.POETRY_VERSION }}
|
||||
cache-key: test-langgraph
|
||||
- name: Login to Docker Hub
|
||||
uses: docker/login-action@v3
|
||||
if: ${{ !github.event.pull_request.head.repo.fork }}
|
||||
@@ -39,11 +39,13 @@ jobs:
|
||||
|
||||
- name: Install dependencies
|
||||
shell: bash
|
||||
run: uv sync --frozen --group test --no-dev
|
||||
run: |
|
||||
poetry install --with dev
|
||||
|
||||
- name: Run tests
|
||||
shell: bash
|
||||
run: make test_parallel
|
||||
run: |
|
||||
make test_parallel
|
||||
|
||||
- name: Ensure the tests did not create any additional files
|
||||
shell: bash
|
||||
|
||||
@@ -9,13 +9,12 @@ on:
|
||||
description: "From which folder this pipeline executes"
|
||||
|
||||
env:
|
||||
POETRY_VERSION: "1.7.1"
|
||||
PYTHON_VERSION: "3.10"
|
||||
|
||||
permissions:
|
||||
contents: read
|
||||
|
||||
jobs:
|
||||
build:
|
||||
if: github.ref == 'refs/heads/main'
|
||||
runs-on: ubuntu-latest
|
||||
|
||||
outputs:
|
||||
@@ -23,14 +22,14 @@ jobs:
|
||||
version: ${{ steps.check-version.outputs.version }}
|
||||
|
||||
steps:
|
||||
- uses: actions/checkout@v6
|
||||
- uses: actions/checkout@v4
|
||||
|
||||
- name: Set up Python $${ env.PYTHON_VERSION }}
|
||||
uses: astral-sh/setup-uv@v7
|
||||
- name: Set up Python + Poetry ${{ env.POETRY_VERSION }}
|
||||
uses: "./.github/actions/poetry_setup"
|
||||
with:
|
||||
python-version: ${{ env.PYTHON_VERSION }}
|
||||
enable-cache: true
|
||||
cache-suffix: "release"
|
||||
poetry-version: ${{ env.POETRY_VERSION }}
|
||||
cache-key: release
|
||||
|
||||
# We want to keep this build stage *separate* from the release stage,
|
||||
# so that there's no sharing of permissions between them.
|
||||
@@ -44,11 +43,11 @@ jobs:
|
||||
# > from the publish job.
|
||||
# https://github.com/pypa/gh-action-pypi-publish#non-goals
|
||||
- name: Build project for distribution
|
||||
run: uv build
|
||||
run: poetry build
|
||||
working-directory: ${{ inputs.working-directory }}
|
||||
|
||||
- name: Upload build
|
||||
uses: actions/upload-artifact@v6
|
||||
uses: actions/upload-artifact@v4
|
||||
with:
|
||||
name: test-dist
|
||||
path: ${{ inputs.working-directory }}/dist/
|
||||
@@ -58,8 +57,8 @@ jobs:
|
||||
shell: bash
|
||||
working-directory: ${{ inputs.working-directory }}
|
||||
run: |
|
||||
echo pkg-name=$(grep -m 1 "^name = " pyproject.toml | cut -d '"' -f 2)
|
||||
echo version=$(grep -m 1 "^version = " pyproject.toml | cut -d '"' -f 2)
|
||||
echo pkg-name="$(poetry version | cut -d ' ' -f 1)" >> $GITHUB_OUTPUT
|
||||
echo version="$(poetry version --short)" >> $GITHUB_OUTPUT
|
||||
|
||||
publish:
|
||||
needs:
|
||||
@@ -74,9 +73,9 @@ jobs:
|
||||
id-token: write
|
||||
|
||||
steps:
|
||||
- uses: actions/checkout@v6
|
||||
- uses: actions/checkout@v4
|
||||
|
||||
- uses: actions/download-artifact@v7
|
||||
- uses: actions/download-artifact@v4
|
||||
with:
|
||||
name: test-dist
|
||||
path: ${{ inputs.working-directory }}/dist/
|
||||
|
||||
@@ -0,0 +1,57 @@
|
||||
name: test
|
||||
|
||||
on:
|
||||
workflow_call:
|
||||
|
||||
env:
|
||||
POETRY_VERSION: "1.7.1"
|
||||
|
||||
jobs:
|
||||
build:
|
||||
runs-on: ubuntu-latest
|
||||
strategy:
|
||||
matrix:
|
||||
python-version:
|
||||
- "3.11"
|
||||
- "3.12"
|
||||
|
||||
defaults:
|
||||
run:
|
||||
working-directory: libs/scheduler-kafka
|
||||
name: "test #${{ matrix.python-version }}"
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
- name: Set up Python ${{ matrix.python-version }} + Poetry ${{ env.POETRY_VERSION }}
|
||||
uses: "./.github/actions/poetry_setup"
|
||||
with:
|
||||
python-version: ${{ matrix.python-version }}
|
||||
poetry-version: ${{ env.POETRY_VERSION }}
|
||||
cache-key: test-scheduler-kafka
|
||||
- name: Login to Docker Hub
|
||||
uses: docker/login-action@v3
|
||||
if: ${{ !github.event.pull_request.head.repo.fork }}
|
||||
with:
|
||||
username: ${{ secrets.DOCKERHUB_USERNAME }}
|
||||
password: ${{ secrets.DOCKERHUB_RO_TOKEN }}
|
||||
|
||||
- name: Install dependencies
|
||||
shell: bash
|
||||
run: |
|
||||
poetry install --with dev
|
||||
|
||||
- name: Run tests
|
||||
shell: bash
|
||||
run: |
|
||||
make test
|
||||
|
||||
- name: Ensure the tests did not create any additional files
|
||||
shell: bash
|
||||
run: |
|
||||
set -eu
|
||||
|
||||
STATUS="$(git status)"
|
||||
echo "$STATUS"
|
||||
|
||||
# grep will exit non-zero if the target message isn't found,
|
||||
# and `set -e` above will cause the step to fail.
|
||||
echo "$STATUS" | grep 'nothing to commit, working tree clean'
|
||||
@@ -7,8 +7,8 @@ on:
|
||||
paths:
|
||||
- "libs/**"
|
||||
|
||||
permissions:
|
||||
contents: read
|
||||
env:
|
||||
POETRY_VERSION: "1.7.1"
|
||||
|
||||
jobs:
|
||||
benchmark:
|
||||
@@ -17,20 +17,20 @@ jobs:
|
||||
run:
|
||||
working-directory: libs/langgraph
|
||||
steps:
|
||||
- uses: actions/checkout@v6
|
||||
- uses: actions/checkout@v4
|
||||
- run: SHA=$(git rev-parse HEAD) && echo "SHA=$SHA" >> $GITHUB_ENV
|
||||
- name: Set up Python 3.11
|
||||
uses: astral-sh/setup-uv@v7
|
||||
- name: Set up Python 3.11 + Poetry ${{ env.POETRY_VERSION }}
|
||||
uses: "./.github/actions/poetry_setup"
|
||||
with:
|
||||
python-version: "3.11"
|
||||
enable-cache: true
|
||||
cache-suffix: "bench"
|
||||
poetry-version: ${{ env.POETRY_VERSION }}
|
||||
cache-key: bench
|
||||
- name: Install dependencies
|
||||
run: uv sync --group test
|
||||
run: poetry install --with dev
|
||||
- name: Run benchmarks
|
||||
run: OUTPUT=out/benchmark-baseline.json make -s benchmark
|
||||
- name: Save outputs
|
||||
uses: actions/cache/save@v5
|
||||
uses: actions/cache/save@v4
|
||||
with:
|
||||
key: ${{ runner.os }}-benchmark-baseline-${{ env.SHA }}
|
||||
path: |
|
||||
|
||||
+12
-12
@@ -5,8 +5,8 @@ on:
|
||||
paths:
|
||||
- "libs/**"
|
||||
|
||||
permissions:
|
||||
contents: read
|
||||
env:
|
||||
POETRY_VERSION: "1.7.1"
|
||||
|
||||
jobs:
|
||||
benchmark:
|
||||
@@ -15,22 +15,22 @@ jobs:
|
||||
run:
|
||||
working-directory: libs/langgraph
|
||||
steps:
|
||||
- uses: actions/checkout@v6
|
||||
- uses: actions/checkout@v4
|
||||
- id: files
|
||||
name: Get changed files
|
||||
uses: Ana06/get-changed-files@v2.3.0
|
||||
with:
|
||||
format: json
|
||||
- name: Set up Python 3.11
|
||||
uses: astral-sh/setup-uv@v7
|
||||
- name: Set up Python 3.11 + Poetry ${{ env.POETRY_VERSION }}
|
||||
uses: "./.github/actions/poetry_setup"
|
||||
with:
|
||||
python-version: "3.11"
|
||||
enable-cache: true
|
||||
cache-suffix: "bench"
|
||||
poetry-version: ${{ env.POETRY_VERSION }}
|
||||
cache-key: bench
|
||||
- name: Install dependencies
|
||||
run: uv sync --group test
|
||||
run: poetry install --with dev
|
||||
- name: Download baseline
|
||||
uses: actions/cache/restore@v5
|
||||
uses: actions/cache/restore@v4
|
||||
with:
|
||||
key: ${{ runner.os }}-benchmark-baseline
|
||||
restore-keys: |
|
||||
@@ -43,7 +43,7 @@ jobs:
|
||||
run: |
|
||||
{
|
||||
echo 'OUTPUT<<EOF'
|
||||
make -s benchmark-fast
|
||||
make -s benchmark
|
||||
echo EOF
|
||||
} >> "$GITHUB_OUTPUT"
|
||||
- name: Compare benchmarks
|
||||
@@ -53,11 +53,11 @@ jobs:
|
||||
echo 'OUTPUT<<EOF'
|
||||
mv out/benchmark-baseline.json out/main.json
|
||||
mv out/benchmark.json out/changes.json
|
||||
uv run pyperf compare_to out/main.json out/changes.json --table --group-by-speed
|
||||
poetry run pyperf compare_to out/main.json out/changes.json --table --group-by-speed
|
||||
echo EOF
|
||||
} >> "$GITHUB_OUTPUT"
|
||||
- name: Annotation
|
||||
uses: actions/github-script@v8
|
||||
uses: actions/github-script@v7
|
||||
with:
|
||||
script: |
|
||||
const file = JSON.parse(`${{ steps.files.outputs.added_modified_renamed }}`)[0]
|
||||
|
||||
+77
-56
@@ -3,13 +3,9 @@ name: CI
|
||||
|
||||
on:
|
||||
push:
|
||||
branches:
|
||||
- main
|
||||
branches: [main]
|
||||
pull_request:
|
||||
|
||||
permissions:
|
||||
contents: read
|
||||
|
||||
# If another push to the same PR or branch happens while this workflow is still running,
|
||||
# cancel the earlier run in favor of the next run.
|
||||
#
|
||||
@@ -20,14 +16,17 @@ concurrency:
|
||||
group: ${{ github.workflow }}-${{ github.ref }}
|
||||
cancel-in-progress: true
|
||||
|
||||
env:
|
||||
POETRY_VERSION: "1.7.1"
|
||||
|
||||
jobs:
|
||||
changes:
|
||||
runs-on: ubuntu-latest
|
||||
outputs:
|
||||
python: ${{ steps.filter.outputs.python }}
|
||||
deps: ${{ steps.filter.outputs.deps }}
|
||||
sdk-js: ${{ steps.filter.outputs.sdk-js }}
|
||||
steps:
|
||||
- uses: actions/checkout@v6
|
||||
- uses: actions/checkout@v4
|
||||
- uses: dorny/paths-filter@v3
|
||||
id: filter
|
||||
with:
|
||||
@@ -39,10 +38,10 @@ jobs:
|
||||
- 'libs/checkpoint/**'
|
||||
- 'libs/checkpoint-sqlite/**'
|
||||
- 'libs/checkpoint-postgres/**'
|
||||
- 'libs/scheduler-kafka/**'
|
||||
- 'libs/prebuilt/**'
|
||||
deps:
|
||||
- '**/pyproject.toml'
|
||||
- '**/uv.lock'
|
||||
sdk-js:
|
||||
- 'libs/sdk-js/**'
|
||||
|
||||
lint:
|
||||
needs: changes
|
||||
@@ -57,10 +56,10 @@ jobs:
|
||||
"libs/checkpoint",
|
||||
"libs/checkpoint-sqlite",
|
||||
"libs/checkpoint-postgres",
|
||||
|
||||
"libs/scheduler-kafka",
|
||||
"libs/prebuilt",
|
||||
]
|
||||
if: needs.changes.outputs.python == 'true' || needs.changes.outputs.deps == 'true'
|
||||
if: needs.changes.outputs.python == 'true'
|
||||
uses: ./.github/workflows/_lint.yml
|
||||
with:
|
||||
working-directory: ${{ matrix.working-directory }}
|
||||
@@ -78,9 +77,8 @@ jobs:
|
||||
"libs/checkpoint-sqlite",
|
||||
"libs/checkpoint-postgres",
|
||||
"libs/prebuilt",
|
||||
"libs/sdk-py",
|
||||
]
|
||||
if: needs.changes.outputs.python == 'true' || needs.changes.outputs.deps == 'true'
|
||||
if: needs.changes.outputs.python == 'true'
|
||||
uses: ./.github/workflows/_test.yml
|
||||
with:
|
||||
working-directory: ${{ matrix.working-directory }}
|
||||
@@ -89,78 +87,101 @@ jobs:
|
||||
# NOTE: we're testing langgraph separately because it requires a different matrix
|
||||
test-langgraph:
|
||||
needs: changes
|
||||
if: needs.changes.outputs.python == 'true' || needs.changes.outputs.deps == 'true'
|
||||
if: needs.changes.outputs.python == 'true'
|
||||
name: "cd libs/langgraph"
|
||||
uses: ./.github/workflows/_test_langgraph.yml
|
||||
secrets: inherit
|
||||
|
||||
# NOTE: we're testing scheduler-kafka separately because it requires a different matrix
|
||||
test-scheduler-kafka:
|
||||
needs: changes
|
||||
if: needs.changes.outputs.python == 'true'
|
||||
name: "cd libs/scheduler-kafka"
|
||||
uses: ./.github/workflows/_test_scheduler_kafka.yml
|
||||
secrets: inherit
|
||||
|
||||
check-sdk-methods:
|
||||
needs: changes
|
||||
if: needs.changes.outputs.python == 'true'
|
||||
name: "Check SDK methods matching"
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
- uses: actions/checkout@v6
|
||||
- uses: actions/checkout@v4
|
||||
- name: Set up Python
|
||||
uses: actions/setup-python@v6
|
||||
uses: actions/setup-python@v5
|
||||
with:
|
||||
python-version: "3.11"
|
||||
- name: Run check_sdk_methods script
|
||||
run: python .github/scripts/check_sdk_methods.py
|
||||
|
||||
check-schema:
|
||||
needs: changes
|
||||
if: needs.changes.outputs.python == 'true'
|
||||
name: "Check CLI schema hasn't changed #${{ matrix.python-version }}"
|
||||
runs-on: ubuntu-latest
|
||||
strategy:
|
||||
matrix:
|
||||
python-version:
|
||||
- "3.13"
|
||||
steps:
|
||||
- uses: actions/checkout@v6
|
||||
- name: Set up Python ${{ matrix.python-version }}
|
||||
uses: astral-sh/setup-uv@v7
|
||||
with:
|
||||
python-version: "3.13"
|
||||
enable-cache: true
|
||||
cache-suffix: "schema-check-cli"
|
||||
- name: Install CLI dependencies
|
||||
run: |
|
||||
cd libs/cli
|
||||
uv sync
|
||||
- name: Generate schema and check for changes
|
||||
run: |
|
||||
cd libs/cli
|
||||
# Create a temporary copy of the current schema
|
||||
cp schemas/schema.json schemas/schema.current.json
|
||||
# Generate new schema
|
||||
uv run python generate_schema.py
|
||||
# Compare the new schema with the original
|
||||
if ! diff -q schemas/schema.json schemas/schema.current.json > /dev/null; then
|
||||
echo "Error: Langgraph.json configuration schema has changed. Please run 'uv run python generate_schema.py' in the libs/cli directory and commit the changes."
|
||||
diff schemas/schema.json schemas/schema.current.json
|
||||
exit 1
|
||||
fi
|
||||
echo "Schema check passed - no changes detected"
|
||||
|
||||
integration-test:
|
||||
needs: changes
|
||||
if: needs.changes.outputs.python == 'true' || needs.changes.outputs.deps == 'true'
|
||||
if: needs.changes.outputs.python == 'true'
|
||||
name: CLI integration test
|
||||
uses: ./.github/workflows/_integration_test.yml
|
||||
secrets: inherit
|
||||
|
||||
lint-js:
|
||||
needs: changes
|
||||
if: needs.changes.outputs.sdk-js == 'true'
|
||||
runs-on: ubuntu-latest
|
||||
strategy:
|
||||
matrix:
|
||||
working-directory:
|
||||
- "libs/sdk-js"
|
||||
defaults:
|
||||
run:
|
||||
working-directory: ${{ matrix.working-directory }}
|
||||
steps:
|
||||
- uses: actions/checkout@v3
|
||||
- name: Setup Node.js (LTS)
|
||||
uses: actions/setup-node@v3
|
||||
with:
|
||||
node-version: "20"
|
||||
cache: "yarn"
|
||||
cache-dependency-path: ${{ matrix.working-directory }}/yarn.lock
|
||||
- name: Install dependencies
|
||||
run: yarn install
|
||||
- name: Run lint
|
||||
run: yarn lint
|
||||
- name: Build
|
||||
run: yarn build
|
||||
|
||||
test-js:
|
||||
needs: changes
|
||||
if: needs.changes.outputs.sdk-js == 'true'
|
||||
runs-on: ubuntu-latest
|
||||
strategy:
|
||||
matrix:
|
||||
working-directory:
|
||||
- "libs/sdk-js"
|
||||
defaults:
|
||||
run:
|
||||
working-directory: ${{ matrix.working-directory }}
|
||||
steps:
|
||||
- uses: actions/checkout@v3
|
||||
- name: Setup Node.js (LTS)
|
||||
uses: actions/setup-node@v3
|
||||
with:
|
||||
node-version: "20"
|
||||
cache: "yarn"
|
||||
cache-dependency-path: ${{ matrix.working-directory }}/yarn.lock
|
||||
- name: Install dependencies
|
||||
run: yarn install
|
||||
- name: Run tests
|
||||
run: yarn test
|
||||
|
||||
ci_success:
|
||||
name: "CI Success"
|
||||
needs:
|
||||
[
|
||||
lint,
|
||||
lint-js,
|
||||
test,
|
||||
test-langgraph,
|
||||
check-sdk-methods,
|
||||
check-schema,
|
||||
test-scheduler-kafka,
|
||||
integration-test,
|
||||
test-js,
|
||||
]
|
||||
if: |
|
||||
always()
|
||||
|
||||
@@ -0,0 +1,43 @@
|
||||
---
|
||||
name: CI / cd . / make spell_check
|
||||
|
||||
on:
|
||||
push:
|
||||
branches: [main]
|
||||
pull_request:
|
||||
branches: [main]
|
||||
|
||||
permissions:
|
||||
contents: read
|
||||
|
||||
defaults:
|
||||
run:
|
||||
working-directory: docs
|
||||
|
||||
jobs:
|
||||
codespell:
|
||||
name: (Check for spelling errors)
|
||||
runs-on: ubuntu-latest
|
||||
|
||||
steps:
|
||||
- name: Checkout
|
||||
uses: actions/checkout@v4
|
||||
|
||||
- name: Install Dependencies
|
||||
run: |
|
||||
pip install toml codespell==2.3.0 jupytext
|
||||
|
||||
- name: Extract Ignore Words List
|
||||
run: |
|
||||
# Use a Python script to extract the ignore words list from pyproject.toml
|
||||
python ../.github/workflows/extract_ignored_words_list.py
|
||||
id: extract_ignore_words
|
||||
|
||||
- name: Codespell
|
||||
uses: codespell-project/actions-codespell@v2
|
||||
with:
|
||||
skip: '*.ambr,*.lock,*.ipynb,*.yaml,*.zlib,*.md'
|
||||
ignore_words_list: ${{ steps.extract_ignore_words.outputs.ignore_words_list }}
|
||||
# We do this to avoid spellchecking cell outputs
|
||||
- name: Codespell Notebooks
|
||||
run: make codespell
|
||||
@@ -0,0 +1,170 @@
|
||||
name: Deploy Docs
|
||||
|
||||
on:
|
||||
push:
|
||||
branches:
|
||||
- main
|
||||
pull_request:
|
||||
branches:
|
||||
- main
|
||||
workflow_dispatch:
|
||||
|
||||
env:
|
||||
POETRY_VERSION: "1.7.1"
|
||||
|
||||
permissions:
|
||||
contents: read
|
||||
pages: write
|
||||
id-token: write
|
||||
|
||||
concurrency:
|
||||
group: "pages"
|
||||
cancel-in-progress: false
|
||||
|
||||
defaults:
|
||||
run:
|
||||
working-directory: docs
|
||||
|
||||
jobs:
|
||||
get-changed-files:
|
||||
runs-on: ubuntu-latest
|
||||
outputs:
|
||||
changed-files: ${{ steps.changed-files.outputs.added_modified }}
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
- name: Get changed files
|
||||
id: changed-files
|
||||
uses: Ana06/get-changed-files@v2.3.0
|
||||
with:
|
||||
filter: "docs/docs/**"
|
||||
|
||||
run-changed-notebooks:
|
||||
needs: get-changed-files
|
||||
uses: ./.github/workflows/run_notebooks.yml
|
||||
secrets: inherit
|
||||
with:
|
||||
changed-files: ${{ needs.get-changed-files.outputs.changed-files }}
|
||||
|
||||
deploy:
|
||||
# needs: run-changed-notebooks
|
||||
runs-on: ubuntu-latest
|
||||
timeout-minutes: 10 # Job will be cancelled if it runs for more than 10 minutes
|
||||
env:
|
||||
GITHUB_TOKEN: ${{ secrets.MKDOCS_GITHUB_TOKEN }}
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
with:
|
||||
fetch-depth: 0
|
||||
|
||||
- name: Set up Python + Poetry ${{ env.POETRY_VERSION }}
|
||||
uses: "./.github/actions/poetry_setup"
|
||||
with:
|
||||
python-version: "3.12"
|
||||
poetry-version: ${{ env.POETRY_VERSION }}
|
||||
cache-key: docs
|
||||
|
||||
- name: Use Node.js
|
||||
uses: actions/setup-node@v3
|
||||
with:
|
||||
node-version: "22"
|
||||
cache: "yarn"
|
||||
cache-dependency-path: docs/yarn.lock
|
||||
|
||||
- name: Install dependencies
|
||||
run: |
|
||||
yarn
|
||||
poetry install --with test --with docs --no-root
|
||||
poetry run pip install -U \
|
||||
pytest \
|
||||
pytest-check-links \
|
||||
GitPython \
|
||||
"git+https://github.com/benjamincburns/markdown-exec.git@cc0d39d737e5ffd4b83d23cd8729d7ea16e363c8"
|
||||
|
||||
# we run this installation only for internal PRs
|
||||
# as GITHUB_TOKEN is not available for PRs from outside contributors
|
||||
if [ -n "${GITHUB_TOKEN}" ]; then
|
||||
poetry run pip install "git+https://${GITHUB_TOKEN}@github.com/langchain-ai/mkdocs-material-insiders.git"
|
||||
fi
|
||||
|
||||
poetry run jupyter kernelspec list
|
||||
poetry run python3 -m ipykernel install --user --name=python3
|
||||
npm install -g tslab
|
||||
poetry run tslab install --python=python3
|
||||
poetry run jupyter kernelspec list
|
||||
|
||||
- name: Run unit tests
|
||||
# Run unit tests on the docs build pipeline
|
||||
run: make tests
|
||||
- name: Lint Docs
|
||||
# This step lints the docs using the existing linting set up.
|
||||
# It should be very fast and should not require any external services.
|
||||
run: make lint-docs
|
||||
- name: Build llms-text
|
||||
run: make llms-text
|
||||
- name: Build site
|
||||
run: make build-docs
|
||||
env:
|
||||
MKDOCS_GIT_COMMITTERS_APIKEY: ${{ secrets.MKDOCS_GIT_COMMITTERS_APIKEY }}
|
||||
OPENAI_API_KEY: sf-proj-1234567890 # fake placeholder, shouldn't actually be used
|
||||
ANTHROPIC_API_KEY: sk-ant-api03-1234567890 # fake placeholder, shouldn't actually be used
|
||||
- name: Check links in notebooks
|
||||
env:
|
||||
LANGCHAIN_API_KEY: test
|
||||
run: |
|
||||
if [ "${{ github.event_name }}" == "schedule" ] || [ "${{ github.event_name }}" == "workflow_dispatch" ] || ([ "${{ github.event_name }}" == "push" ] && [ "${{ github.ref }}" == "refs/heads/main" ]); then
|
||||
echo "Running link check on all HTML files matching notebooks in docs directory..."
|
||||
poetry run pytest -v \
|
||||
--check-links-ignore "https://(api|web|docs)\.smith\.langchain\.com/.*" \
|
||||
--check-links-ignore "https://academy\.langchain\.com/.*" \
|
||||
--check-links-ignore "https://x.com/.*" \
|
||||
--check-links-ignore "https://twitter.com/.*" \
|
||||
--check-links-ignore "https://github\.com/.*" \
|
||||
--check-links-ignore "http://localhost:8123/.*" \
|
||||
--check-links-ignore "http://localhost:2024.*" \
|
||||
--check-links-ignore "http://127.0.0.1:.*" \
|
||||
--check-links-ignore "/.*\.(ipynb|html)$" \
|
||||
--check-links-ignore "https://python\.langchain\.com/.*" \
|
||||
--check-links-ignore "https://openai\.com/.*" \
|
||||
--check-links-ignore "https://www\.uber\.com/.*" \
|
||||
--check-links-ignore "https://pepy\.tech/.*" \
|
||||
--check-links $(find site -name "index.html" | grep -v 'storm/index.html')
|
||||
|
||||
else
|
||||
echo "Fetching changes from origin/main..."
|
||||
git fetch origin main
|
||||
echo "Checking for changed notebook files..."
|
||||
CHANGED_FILES=$(git diff --name-only --diff-filter=d origin/main | grep 'docs/docs/.*\.ipynb$' | grep -v 'storm.ipynb' | sed -E 's|^docs/docs/|site/|; s/\.ipynb$/\/index.html/' || true)
|
||||
echo "Changed files: ${CHANGED_FILES}"
|
||||
if [ -n "${CHANGED_FILES}" ]; then
|
||||
echo "Running link check on HTML files matching changed notebook files..."
|
||||
poetry run pytest -v \
|
||||
--check-links-ignore "https://(api|web|docs)\.smith\.langchain\.com/.*" \
|
||||
--check-links-ignore "https://academy\.langchain\.com/.*" \
|
||||
--check-links-ignore "http://localhost:8123/.*" \
|
||||
--check-links-ignore "http://localhost:2024.*" \
|
||||
--check-links-ignore "http://127.0.0.1:.*" \
|
||||
--check-links-ignore "https://x.com/.*" \
|
||||
--check-links-ignore "https://twitter.com/.*" \
|
||||
--check-links-ignore "https://github\.com/.*" \
|
||||
--check-links-ignore "/.*\.(ipynb|html)$" \
|
||||
--check-links ${CHANGED_FILES} \
|
||||
|| ([ $? = 5 ] && exit 0 || exit $?)
|
||||
else
|
||||
echo "No notebook files changed."
|
||||
fi
|
||||
fi
|
||||
|
||||
- name: Configure GitHub Pages
|
||||
if: github.ref == 'refs/heads/main'
|
||||
uses: actions/configure-pages@v4
|
||||
|
||||
- name: Upload Pages Artifact
|
||||
# if: github.ref == 'refs/heads/main'
|
||||
uses: actions/upload-pages-artifact@v3
|
||||
with:
|
||||
path: ./docs/site/
|
||||
|
||||
- name: Deploy to GitHub Pages
|
||||
if: github.ref == 'refs/heads/main'
|
||||
id: deployment
|
||||
uses: actions/deploy-pages@v4
|
||||
@@ -0,0 +1,10 @@
|
||||
import toml
|
||||
|
||||
pyproject_toml = toml.load("pyproject.toml")
|
||||
|
||||
# Extract the ignore words list (adjust the key as per your TOML structure)
|
||||
ignore_words_list = (
|
||||
pyproject_toml.get("tool", {}).get("codespell", {}).get("ignore-words-list")
|
||||
)
|
||||
|
||||
print(f"::set-output name=ignore_words_list::{ignore_words_list}") # noqa: T201
|
||||
@@ -0,0 +1,49 @@
|
||||
name: Check Docs & Links
|
||||
|
||||
on:
|
||||
pull_request:
|
||||
branches:
|
||||
- main
|
||||
push:
|
||||
branches:
|
||||
- main
|
||||
schedule:
|
||||
- cron: "0 5 * * *"
|
||||
workflow_dispatch:
|
||||
|
||||
env:
|
||||
POETRY_VERSION: "1.7.1"
|
||||
|
||||
jobs:
|
||||
markdown-link-check:
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
- name: Checkout code
|
||||
uses: actions/checkout@v4
|
||||
with:
|
||||
fetch-depth: 0
|
||||
|
||||
- name: Check links in Markdown files
|
||||
uses: gaurav-nelson/github-action-markdown-link-check@v1
|
||||
with:
|
||||
folder-path: "docs/"
|
||||
check-modified-files-only: ${{ github.event_name != 'schedule' }}
|
||||
file-path: "./README.md"
|
||||
config-file: "./.markdown-link-check.config.json"
|
||||
|
||||
check-readmes-synced:
|
||||
# This checks that the repo README.md is identical to the libs/langgraph/README.md
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
- name: Checkout code
|
||||
uses: actions/checkout@v4
|
||||
with:
|
||||
fetch-depth: 1
|
||||
|
||||
- name: Check README.md is in sync
|
||||
run: |
|
||||
if ! diff -q README.md libs/langgraph/README.md >/dev/null; then
|
||||
echo "README.md is out of sync with libs/langgraph/README.md"
|
||||
diff -C 3 README.md libs/langgraph/README.md
|
||||
exit 1
|
||||
fi
|
||||
@@ -1,46 +0,0 @@
|
||||
name: PR Title Lint
|
||||
|
||||
permissions:
|
||||
pull-requests: read
|
||||
|
||||
on:
|
||||
pull_request:
|
||||
types: [opened, edited, synchronize]
|
||||
|
||||
jobs:
|
||||
lint-pr-title:
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
- name: Validate PR Title
|
||||
uses: amannn/action-semantic-pull-request@v6
|
||||
env:
|
||||
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
|
||||
with:
|
||||
types: |
|
||||
feat
|
||||
fix
|
||||
docs
|
||||
style
|
||||
refactor
|
||||
perf
|
||||
test
|
||||
build
|
||||
ci
|
||||
chore
|
||||
revert
|
||||
release
|
||||
scopes: |
|
||||
checkpoint
|
||||
checkpoint-postgres
|
||||
checkpoint-sqlite
|
||||
cli
|
||||
langgraph
|
||||
prebuilt
|
||||
scheduler-kafka
|
||||
sdk-py
|
||||
docs
|
||||
ci
|
||||
deps
|
||||
requireScope: false
|
||||
ignoreLabels: |
|
||||
ignore-lint-pr-title
|
||||
@@ -8,14 +8,13 @@ on:
|
||||
type: string
|
||||
default: "libs/langgraph"
|
||||
|
||||
permissions:
|
||||
contents: read
|
||||
|
||||
env:
|
||||
PYTHON_VERSION: "3.11"
|
||||
POETRY_VERSION: "1.7.1"
|
||||
|
||||
jobs:
|
||||
build:
|
||||
if: github.ref == 'refs/heads/main'
|
||||
runs-on: ubuntu-latest
|
||||
|
||||
outputs:
|
||||
@@ -25,14 +24,14 @@ jobs:
|
||||
tag: ${{ steps.check-version.outputs.tag }}
|
||||
|
||||
steps:
|
||||
- uses: actions/checkout@v6
|
||||
- uses: actions/checkout@v4
|
||||
|
||||
- name: Set up Python
|
||||
uses: astral-sh/setup-uv@v7
|
||||
- name: Set up Python + Poetry ${{ env.POETRY_VERSION }}
|
||||
uses: "./.github/actions/poetry_setup"
|
||||
with:
|
||||
python-version: ${{ env.PYTHON_VERSION }}
|
||||
enable-cache: true
|
||||
cache-suffix: "release"
|
||||
poetry-version: ${{ env.POETRY_VERSION }}
|
||||
cache-key: release
|
||||
|
||||
# We want to keep this build stage *separate* from the release stage,
|
||||
# so that there's no sharing of permissions between them.
|
||||
@@ -46,11 +45,11 @@ jobs:
|
||||
# > from the publish job.
|
||||
# https://github.com/pypa/gh-action-pypi-publish#non-goals
|
||||
- name: Build project for distribution
|
||||
run: uv build
|
||||
run: poetry build
|
||||
working-directory: ${{ inputs.working-directory }}
|
||||
|
||||
- name: Upload build
|
||||
uses: actions/upload-artifact@v6
|
||||
uses: actions/upload-artifact@v4
|
||||
with:
|
||||
name: dist
|
||||
path: ${{ inputs.working-directory }}/dist/
|
||||
@@ -60,14 +59,8 @@ jobs:
|
||||
shell: bash
|
||||
working-directory: ${{ inputs.working-directory }}
|
||||
run: |
|
||||
PKG_NAME=$(grep -m 1 "^name = " pyproject.toml | cut -d '"' -f 2)
|
||||
if grep -q 'dynamic.*=.*\[.*"version".*\]' pyproject.toml; then
|
||||
# handle dynamic versioning
|
||||
DIR_NAME=$(echo "$PKG_NAME" | tr '-' '_')
|
||||
VERSION=$(grep -m 1 '^__version__' "${DIR_NAME}/__init__.py" | cut -d '"' -f 2)
|
||||
else
|
||||
VERSION=$(grep -m 1 "^version = " pyproject.toml | cut -d '"' -f 2)
|
||||
fi
|
||||
PKG_NAME="$(poetry version | cut -d ' ' -f 1)"
|
||||
VERSION="$(poetry version --short)"
|
||||
SHORT_PKG_NAME="$(echo "$PKG_NAME" | sed -e 's/langgraph//g' -e 's/-//g')"
|
||||
if [ -z $SHORT_PKG_NAME ]; then
|
||||
TAG="$VERSION"
|
||||
@@ -86,7 +79,7 @@ jobs:
|
||||
outputs:
|
||||
release-body: ${{ steps.generate-release-body.outputs.release-body }}
|
||||
steps:
|
||||
- uses: actions/checkout@v6
|
||||
- uses: actions/checkout@v4
|
||||
with:
|
||||
repository: langchain-ai/langgraph
|
||||
path: langgraph
|
||||
@@ -142,9 +135,7 @@ jobs:
|
||||
needs:
|
||||
- build
|
||||
- release-notes
|
||||
permissions:
|
||||
contents: read
|
||||
id-token: write
|
||||
permissions: write-all
|
||||
uses: ./.github/workflows/_test_release.yml
|
||||
with:
|
||||
working-directory: ${{ inputs.working-directory }}
|
||||
@@ -157,7 +148,7 @@ jobs:
|
||||
- test-pypi-publish
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
- uses: actions/checkout@v6
|
||||
- uses: actions/checkout@v4
|
||||
|
||||
# We explicitly *don't* set up caching here. This ensures our tests are
|
||||
# maximally sensitive to catching breakage.
|
||||
@@ -172,11 +163,11 @@ jobs:
|
||||
# - The package is published, and it breaks on the missing dependency when
|
||||
# used in the real world.
|
||||
|
||||
- name: Set up Python
|
||||
uses: astral-sh/setup-uv@v7
|
||||
- name: Set up Python + Poetry ${{ env.POETRY_VERSION }}
|
||||
uses: "./.github/actions/poetry_setup"
|
||||
with:
|
||||
python-version: ${{ env.PYTHON_VERSION }}
|
||||
enable-cache: true
|
||||
poetry-version: ${{ env.POETRY_VERSION }}
|
||||
|
||||
- name: Import published package
|
||||
shell: bash
|
||||
@@ -194,18 +185,18 @@ jobs:
|
||||
# - attempt install again after 5 seconds if it fails because there is
|
||||
# sometimes a delay in availability on test pypi
|
||||
run: |
|
||||
uv run pip install \
|
||||
poetry run pip install \
|
||||
--extra-index-url https://test.pypi.org/simple/ \
|
||||
"$PKG_NAME==$VERSION" || \
|
||||
( \
|
||||
sleep 5 && \
|
||||
uv run pip install \
|
||||
poetry run pip install \
|
||||
--extra-index-url https://test.pypi.org/simple/ \
|
||||
"$PKG_NAME==$VERSION" \
|
||||
)
|
||||
|
||||
if [[ "$PKG_NAME" == *prebuilt* ]]; then
|
||||
uv run pip install langgraph
|
||||
poetry run pip install langgraph
|
||||
fi
|
||||
|
||||
if [[ "$PKG_NAME" == *checkpoint* || "$PKG_NAME" == *prebuilt* ]]; then
|
||||
@@ -218,10 +209,10 @@ jobs:
|
||||
IMPORT_NAME="$(echo "$PKG_NAME" | sed s/-/_/g)"
|
||||
fi
|
||||
|
||||
uv run python -c "import $IMPORT_NAME; print(dir($IMPORT_NAME))"
|
||||
poetry run python -c "import $IMPORT_NAME; print(dir($IMPORT_NAME))"
|
||||
|
||||
- name: Import test dependencies
|
||||
run: uv sync --group test
|
||||
run: poetry install --with dev
|
||||
working-directory: ${{ inputs.working-directory }}
|
||||
|
||||
# Overwrite the local version of the package with the test PyPI version.
|
||||
@@ -232,7 +223,7 @@ jobs:
|
||||
PKG_NAME: ${{ needs.build.outputs.pkg-name }}
|
||||
VERSION: ${{ needs.build.outputs.version }}
|
||||
run: |
|
||||
uv run pip install \
|
||||
poetry run pip install \
|
||||
--extra-index-url https://test.pypi.org/simple/ \
|
||||
"$PKG_NAME==$VERSION"
|
||||
|
||||
@@ -260,16 +251,16 @@ jobs:
|
||||
working-directory: ${{ inputs.working-directory }}
|
||||
|
||||
steps:
|
||||
- uses: actions/checkout@v6
|
||||
- uses: actions/checkout@v4
|
||||
|
||||
- name: Set up Python
|
||||
uses: astral-sh/setup-uv@v7
|
||||
- name: Set up Python + Poetry ${{ env.POETRY_VERSION }}
|
||||
uses: "./.github/actions/poetry_setup"
|
||||
with:
|
||||
python-version: ${{ env.PYTHON_VERSION }}
|
||||
enable-cache: true
|
||||
cache-suffix: "release"
|
||||
poetry-version: ${{ env.POETRY_VERSION }}
|
||||
cache-key: release
|
||||
|
||||
- uses: actions/download-artifact@v7
|
||||
- uses: actions/download-artifact@v4
|
||||
with:
|
||||
name: dist
|
||||
path: ${{ inputs.working-directory }}/dist/
|
||||
@@ -301,16 +292,16 @@ jobs:
|
||||
working-directory: ${{ inputs.working-directory }}
|
||||
|
||||
steps:
|
||||
- uses: actions/checkout@v6
|
||||
- uses: actions/checkout@v4
|
||||
|
||||
- name: Set up Python
|
||||
uses: astral-sh/setup-uv@v7
|
||||
- name: Set up Python + Poetry ${{ env.POETRY_VERSION }}
|
||||
uses: "./.github/actions/poetry_setup"
|
||||
with:
|
||||
python-version: ${{ env.PYTHON_VERSION }}
|
||||
enable-cache: true
|
||||
cache-suffix: "release"
|
||||
poetry-version: ${{ env.POETRY_VERSION }}
|
||||
cache-key: release
|
||||
|
||||
- uses: actions/download-artifact@v7
|
||||
- uses: actions/download-artifact@v4
|
||||
with:
|
||||
name: dist
|
||||
path: ${{ inputs.working-directory }}/dist/
|
||||
@@ -322,6 +313,5 @@ jobs:
|
||||
token: ${{ secrets.GITHUB_TOKEN }}
|
||||
generateReleaseNotes: false
|
||||
tag: ${{needs.build.outputs.tag}}
|
||||
name: ${{ needs.build.outputs.pkg-name }}==${{ needs.build.outputs.version }}
|
||||
body: ${{ needs.release-notes.outputs.release-body }}
|
||||
commit: ${{ github.sha }}
|
||||
@@ -0,0 +1,38 @@
|
||||
name: JS Release
|
||||
|
||||
on:
|
||||
workflow_dispatch:
|
||||
|
||||
jobs:
|
||||
publish:
|
||||
# Disallow publishing from branches that aren't `main`.
|
||||
if: github.ref == 'refs/heads/main'
|
||||
runs-on: ubuntu-latest
|
||||
|
||||
strategy:
|
||||
matrix:
|
||||
working-directory:
|
||||
- "libs/sdk-js"
|
||||
|
||||
defaults:
|
||||
run:
|
||||
working-directory: ${{ matrix.working-directory }}
|
||||
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
# JS Build
|
||||
- name: Use Node.js
|
||||
uses: actions/setup-node@v3
|
||||
with:
|
||||
node-version: "20"
|
||||
cache: "yarn"
|
||||
cache-dependency-path: ${{ matrix.working-directory }}/yarn.lock
|
||||
|
||||
- name: Install dependencies
|
||||
run: yarn install
|
||||
- name: Build
|
||||
run: yarn build
|
||||
- name: Publish package to NPM
|
||||
run: |
|
||||
echo "//registry.npmjs.org/:_authToken=${{ secrets.NPM_TOKEN }}" > .npmrc
|
||||
npm publish
|
||||
@@ -0,0 +1,82 @@
|
||||
name: Run notebooks
|
||||
|
||||
on:
|
||||
workflow_dispatch:
|
||||
workflow_call:
|
||||
inputs:
|
||||
changed-files:
|
||||
required: false
|
||||
type: string
|
||||
description: "JSON string of changed files"
|
||||
schedule:
|
||||
- cron: '0 13 * * *'
|
||||
|
||||
defaults:
|
||||
run:
|
||||
working-directory: docs
|
||||
|
||||
jobs:
|
||||
build:
|
||||
runs-on: ubuntu-latest
|
||||
strategy:
|
||||
matrix:
|
||||
lib-version:
|
||||
- "development"
|
||||
- "latest"
|
||||
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
- name: Set up Python + Poetry
|
||||
uses: "./.github/actions/poetry_setup"
|
||||
with:
|
||||
python-version: 3.11
|
||||
poetry-version: 1.7.1
|
||||
cache-key: test-langgraph-notebooks
|
||||
|
||||
- name: Install dependencies
|
||||
run: |
|
||||
poetry install --with test
|
||||
poetry run pip install jupyter
|
||||
|
||||
- name: Start services
|
||||
run: make start-services
|
||||
|
||||
- name: Pre-download tiktoken files
|
||||
run: |
|
||||
poetry run python _scripts/download_tiktoken.py
|
||||
|
||||
- name: Prepare notebooks
|
||||
run: |
|
||||
if [ "${{ matrix.lib-version }}" = "development" ]; then
|
||||
poetry run python _scripts/prepare_notebooks_for_ci.py --comment-install-cells
|
||||
else
|
||||
poetry run python _scripts/prepare_notebooks_for_ci.py
|
||||
fi
|
||||
|
||||
- name: Run notebooks
|
||||
env:
|
||||
# these won't actually be used because of the VCR cassettes
|
||||
# but need to set them to avoid triggering getpass()
|
||||
OPENAI_API_KEY: ${{ secrets.OPENAI_API_KEY }}
|
||||
ANTHROPIC_API_KEY: ${{ secrets.ANTHROPIC_API_KEY }}
|
||||
TAVILY_API_KEY: ${{ secrets.TAVILY_API_KEY }}
|
||||
LANGSMITH_API_KEY: ${{ secrets.LANGSMITH_API_KEY }}
|
||||
NOMIC_API_KEY: ${{ secrets.NOMIC_API_KEY }}
|
||||
COHERE_API_KEY: ${{ secrets.COHERE_API_KEY }}
|
||||
FIREWORKS_API_KEY: ${{ secrets.FIREWORKS_API_KEY }}
|
||||
run: |
|
||||
if [ "${{ github.event_name }}" = "workflow_dispatch" ] || [ "${{ github.event_name }}" = "schedule" ]; then
|
||||
echo "Running all notebooks"
|
||||
./_scripts/execute_notebooks.sh
|
||||
else
|
||||
CHANGED_FILES=$(echo '${{ inputs.changed-files }}' | tr ' ' '\n' | sed 's|^docs/docs/|docs/|' | grep '\.ipynb$' || true)
|
||||
if [ -n "$CHANGED_FILES" ]; then
|
||||
echo "Running changed notebooks: $CHANGED_FILES"
|
||||
./_scripts/execute_notebooks.sh $CHANGED_FILES
|
||||
else
|
||||
echo "No notebook files changed, skipping execution"
|
||||
fi
|
||||
fi
|
||||
|
||||
- name: Stop services
|
||||
run: make stop-services
|
||||
@@ -0,0 +1,29 @@
|
||||
name: Check File Size
|
||||
|
||||
on:
|
||||
push:
|
||||
branches:
|
||||
- main
|
||||
pull_request:
|
||||
branches:
|
||||
- main
|
||||
workflow_dispatch:
|
||||
|
||||
jobs:
|
||||
file-size-check:
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
- name: Get changed files
|
||||
id: changed-files
|
||||
uses: tj-actions/changed-files@v44
|
||||
- name: Filter by size
|
||||
# TODO: roll back the web voyager hack
|
||||
run: |
|
||||
large_added_files=$(find ${{ steps.changed-files.outputs.added_files }} -maxdepth 0 -size +1M | grep -v "web_voyager" || true)
|
||||
if [ -n "$large_added_files" ]; then
|
||||
echo "Large files added: $large_added_files"
|
||||
echo "# Large files added:" >> $GITHUB_STEP_SUMMARY
|
||||
echo "$large_added_files" >> $GITHUB_STEP_SUMMARY
|
||||
exit 1
|
||||
fi
|
||||
@@ -1,45 +0,0 @@
|
||||
name: UV Lock Upgrade
|
||||
|
||||
on:
|
||||
schedule:
|
||||
# run at midnight every Sunday
|
||||
- cron: '0 0 * * 0'
|
||||
# allow manual triggering
|
||||
workflow_dispatch:
|
||||
|
||||
permissions:
|
||||
contents: write
|
||||
pull-requests: write
|
||||
|
||||
jobs:
|
||||
upgrade-dependencies:
|
||||
runs-on: ubuntu-latest
|
||||
|
||||
steps:
|
||||
- uses: actions/checkout@v6
|
||||
|
||||
- name: Set up uv
|
||||
uses: astral-sh/setup-uv@v7
|
||||
with:
|
||||
# use minimum supported Python version
|
||||
python-version: "3.10"
|
||||
enable-cache: true
|
||||
cache-suffix: "uv-lock-upgrade"
|
||||
|
||||
- name: Run uv lock --upgrade in all Python packages
|
||||
run: make lock-upgrade
|
||||
|
||||
- name: Create Pull Request
|
||||
uses: peter-evans/create-pull-request@v8
|
||||
with:
|
||||
token: ${{ secrets.GITHUB_TOKEN }}
|
||||
commit-message: "chore(deps): upgrade dependencies with `uv lock --upgrade`"
|
||||
title: "chore(deps): upgrade dependencies with `uv lock --upgrade`"
|
||||
body: |
|
||||
This PR updates the dependencies in all Python packages using `uv lock --upgrade`.
|
||||
|
||||
This is an automated PR created by the UV Lock Upgrade workflow.
|
||||
branch: deps/uv-lock-upgrade
|
||||
delete-branch: true
|
||||
labels: |
|
||||
dependencies
|
||||
+82
-2
@@ -6,6 +6,9 @@ __pycache__/
|
||||
*.py[cod]
|
||||
*$py.class
|
||||
|
||||
# C extensions
|
||||
*.so
|
||||
|
||||
# Distribution / packaging
|
||||
.Python
|
||||
build/
|
||||
@@ -51,12 +54,27 @@ coverage.xml
|
||||
.hypothesis/
|
||||
.pytest_cache/
|
||||
|
||||
# Translations
|
||||
*.mo
|
||||
*.pot
|
||||
|
||||
# Django stuff:
|
||||
*.log
|
||||
local_settings.py
|
||||
db.sqlite3
|
||||
db.sqlite3-journal
|
||||
|
||||
# Flask stuff:
|
||||
instance/
|
||||
.webassets-cache
|
||||
|
||||
# Scrapy stuff:
|
||||
.scrapy
|
||||
|
||||
# Sphinx documentation
|
||||
docs/_build/
|
||||
docs/docs/_build/
|
||||
|
||||
# PyBuilder
|
||||
target/
|
||||
|
||||
@@ -71,9 +89,23 @@ ipython_config.py
|
||||
# pyenv
|
||||
.python-version
|
||||
|
||||
# pipenv
|
||||
# According to pypa/pipenv#598, it is recommended to include Pipfile.lock in version control.
|
||||
# However, in case of collaboration, if having platform-specific dependencies or dependencies
|
||||
# having no cross-platform support, pipenv may install dependencies that don't work, or not
|
||||
# install all needed dependencies.
|
||||
#Pipfile.lock
|
||||
|
||||
# PEP 582; used by e.g. github.com/David-OConnor/pyflow
|
||||
__pypackages__/
|
||||
|
||||
# Celery stuff
|
||||
celerybeat-schedule
|
||||
celerybeat.pid
|
||||
|
||||
# SageMath parsed files
|
||||
*.sage.py
|
||||
|
||||
# Environments
|
||||
.env
|
||||
.envrc
|
||||
@@ -85,6 +117,16 @@ ENV/
|
||||
env.bak/
|
||||
venv.bak/
|
||||
|
||||
# Spyder project settings
|
||||
.spyderproject
|
||||
.spyproject
|
||||
|
||||
# Rope project settings
|
||||
.ropeproject
|
||||
|
||||
# mkdocs documentation
|
||||
/site
|
||||
|
||||
# mypy
|
||||
.mypy_cache/
|
||||
.dmypy.json
|
||||
@@ -96,7 +138,45 @@ dmypy.json
|
||||
# macOS display setting files
|
||||
.DS_Store
|
||||
|
||||
# Wandb directory
|
||||
wandb/
|
||||
|
||||
# asdf tool versions
|
||||
.tool-versions
|
||||
/.ruff_cache/
|
||||
|
||||
*.pkl
|
||||
*.bin
|
||||
|
||||
# integration test artifacts
|
||||
data_map*
|
||||
\[('_type', 'fake'), ('stop', None)]
|
||||
|
||||
# Replit files
|
||||
*replit*
|
||||
|
||||
node_modules
|
||||
docs/.yarn/
|
||||
docs/node_modules/
|
||||
docs/.docusaurus/
|
||||
docs/.cache-loader/
|
||||
docs/_dist
|
||||
docs/api_reference/api_reference.rst
|
||||
docs/api_reference/experimental_api_reference.rst
|
||||
docs/api_reference/_build
|
||||
docs/api_reference/*/
|
||||
!docs/api_reference/_static/
|
||||
!docs/api_reference/templates/
|
||||
!docs/api_reference/themes/
|
||||
docs/docs_skeleton/build
|
||||
docs/docs_skeleton/node_modules
|
||||
docs/docs_skeleton/yarn.lock
|
||||
|
||||
# Any new jupyter notebooks
|
||||
# not intended for the repo
|
||||
Untitled*.ipynb
|
||||
|
||||
Chinook.db
|
||||
|
||||
.vercel
|
||||
.turbo
|
||||
.editorconfig
|
||||
.scratch
|
||||
@@ -0,0 +1,4 @@
|
||||
{
|
||||
"aliveStatusCodes": [200, 206, 402],
|
||||
"ignorePatterns": ["*dcbadge.vercel.app*"]
|
||||
}
|
||||
@@ -1,55 +0,0 @@
|
||||
# AGENTS Instructions
|
||||
|
||||
This repository is a monorepo. Each library lives in a subdirectory under `libs/`.
|
||||
|
||||
When you modify code in any library, run the following commands in that library's directory before creating a pull request:
|
||||
|
||||
- `make format` – run code formatters
|
||||
- `make lint` – run the linter
|
||||
- `make test` – execute the test suite
|
||||
|
||||
To run a particular test file or to pass additional pytest options you can specify the `TEST` variable:
|
||||
|
||||
```txt
|
||||
TEST=path/to/test.py make test
|
||||
```
|
||||
|
||||
Other pytest arguments can also be supplied inside the `TEST` variable.
|
||||
|
||||
## Libraries
|
||||
|
||||
The repository contains several Python and JavaScript/TypeScript libraries.
|
||||
Below is a high-level overview:
|
||||
|
||||
- **checkpoint** – base interfaces for LangGraph checkpointers.
|
||||
- **checkpoint-postgres** – Postgres implementation of the checkpoint saver.
|
||||
- **checkpoint-sqlite** – SQLite implementation of the checkpoint saver.
|
||||
- **cli** – official command-line interface for LangGraph.
|
||||
- **langgraph** – core framework for building stateful, multi-actor agents.
|
||||
- **prebuilt** – high-level APIs for creating and running agents and tools.
|
||||
- **sdk-js** – JS/TS SDK for interacting with the LangGraph REST API.
|
||||
- **sdk-py** – Python SDK for the LangGraph Server API.
|
||||
|
||||
### Dependency map
|
||||
|
||||
The diagram below lists downstream libraries for each production dependency as
|
||||
declared in that library's `pyproject.toml` (or `package.json`).
|
||||
|
||||
```text
|
||||
checkpoint
|
||||
├── checkpoint-postgres
|
||||
├── checkpoint-sqlite
|
||||
├── prebuilt
|
||||
└── langgraph
|
||||
|
||||
prebuilt
|
||||
└── langgraph
|
||||
|
||||
sdk-py
|
||||
├── langgraph
|
||||
└── cli
|
||||
|
||||
sdk-js (standalone)
|
||||
```
|
||||
|
||||
Changes to a library may impact all of its dependents shown above.
|
||||
@@ -1,55 +0,0 @@
|
||||
# AGENTS Instructions
|
||||
|
||||
This repository is a monorepo. Each library lives in a subdirectory under `libs/`.
|
||||
|
||||
When you modify code in any library, run the following commands in that library's directory before creating a pull request:
|
||||
|
||||
- `make format` – run code formatters
|
||||
- `make lint` – run the linter
|
||||
- `make test` – execute the test suite
|
||||
|
||||
To run a particular test file or to pass additional pytest options you can specify the `TEST` variable:
|
||||
|
||||
```
|
||||
TEST=path/to/test.py make test
|
||||
```
|
||||
|
||||
Other pytest arguments can also be supplied inside the `TEST` variable.
|
||||
|
||||
## Libraries
|
||||
|
||||
The repository contains several Python and JavaScript/TypeScript libraries.
|
||||
Below is a high-level overview:
|
||||
|
||||
- **checkpoint** – base interfaces for LangGraph checkpointers.
|
||||
- **checkpoint-postgres** – Postgres implementation of the checkpoint saver.
|
||||
- **checkpoint-sqlite** – SQLite implementation of the checkpoint saver.
|
||||
- **cli** – official command-line interface for LangGraph.
|
||||
- **langgraph** – core framework for building stateful, multi-actor agents.
|
||||
- **prebuilt** – high-level APIs for creating and running agents and tools.
|
||||
- **sdk-js** – JS/TS SDK for interacting with the LangGraph REST API.
|
||||
- **sdk-py** – Python SDK for the LangGraph Server API.
|
||||
|
||||
### Dependency map
|
||||
|
||||
The diagram below lists downstream libraries for each production dependency as
|
||||
declared in that library's `pyproject.toml` (or `package.json`).
|
||||
|
||||
```text
|
||||
checkpoint
|
||||
├── checkpoint-postgres
|
||||
├── checkpoint-sqlite
|
||||
├── prebuilt
|
||||
└── langgraph
|
||||
|
||||
prebuilt
|
||||
└── langgraph
|
||||
|
||||
sdk-py
|
||||
├── langgraph
|
||||
└── cli
|
||||
|
||||
sdk-js (standalone)
|
||||
```
|
||||
|
||||
Changes to a library may impact all of its dependents shown above.
|
||||
+293
@@ -0,0 +1,293 @@
|
||||
# Contributing to LangGraph
|
||||
|
||||
Thank you for being interested in contributing to LangGraph!
|
||||
|
||||
## General guidelines
|
||||
|
||||
Here are some things to keep in mind for all types of contributions:
|
||||
|
||||
- Follow the ["fork and pull request"](https://docs.github.com/en/get-started/exploring-projects-on-github/contributing-to-a-project) workflow.
|
||||
- Fill out the checked-in pull request template when opening pull requests. Note related issues and tag relevant maintainers.
|
||||
- Ensure your PR passes formatting, linting, and testing checks before requesting a review.
|
||||
- If you would like comments or feedback, please open an issue or discussion and tag a maintainer.
|
||||
- Backwards compatibility is key. Your changes must not be breaking, except in case of critical bug and security fixes.
|
||||
- Look for duplicate PRs or issues that have already been opened before opening a new one.
|
||||
- Keep scope as isolated as possible. As a general rule, your changes should not affect more than one package at a time.
|
||||
|
||||
### Bugfixes
|
||||
|
||||
For bug fixes, please open up an issue before proposing a fix to ensure the proposal properly addresses the underlying problem. In general, bug fixes should all have an accompanying unit test that fails before the fix.
|
||||
|
||||
### New features
|
||||
|
||||
For new features, please start a new [discussion](https://github.com/langchain-ai/langgraph/discussions), where the maintainers will help with scoping out the necessary changes.
|
||||
|
||||
## Contribute Documentation
|
||||
|
||||
Documentation is a vital part of LangGraph. We welcome both new documentation for new features and
|
||||
community improvements to our current documentation. Please read the resources below before getting started:
|
||||
|
||||
- [Documentation style guide](#documentation-style-guide)
|
||||
- [Documentation setup](#setup)
|
||||
|
||||
## Documentation Style Guide
|
||||
|
||||
As LangGraph continues to grow, the surface area of documentation required to cover it continues to grow too.
|
||||
This page provides guidelines for anyone writing documentation for LangGraph, as well as some of our philosophies around organization and structure.
|
||||
|
||||
## Philosophy
|
||||
|
||||
LangGraph's documentation follows the [Diataxis framework](https://diataxis.fr).
|
||||
Under this framework, all documentation falls under one of four categories: [Tutorials](#tutorials),
|
||||
[How-to guides](#how-to-guides),
|
||||
[References](#references), and [Explanations (aka conceptual guides)](#conceptual-guide).
|
||||
|
||||
### Tutorials
|
||||
|
||||
Tutorials are lessons that take the reader through a practical activity. Their purpose is to help the user
|
||||
gain understanding of concepts and how they interact by showing one way to achieve some goal in a hands-on way.
|
||||
|
||||
They should **avoid** giving
|
||||
multiple permutations of ways to achieve that goal in-depth. Choice is burdensome. Instead, they should guide a new user through a recommended path to accomplishing a concrete goal. While the end result of a tutorial does not necessarily need to
|
||||
be completely production-ready, it should be useful and practically satisfy the goal that you clearly stated in the tutorial's introduction.
|
||||
|
||||
To quote the Diataxis website:
|
||||
|
||||
> A tutorial serves the user’s *acquisition* of skills and knowledge - their study. Its purpose is not to help the user get something done, but to help them learn.
|
||||
|
||||
In LangGraph, these are often higher level guides that show off end-to-end use cases.
|
||||
|
||||
Some examples include:
|
||||
|
||||
- [Build a Customer Support Bot](https://langchain-ai.github.io/langgraph/tutorials/customer-support/customer-support/)
|
||||
- [Build a SQL Agent](https://langchain-ai.github.io/langgraph/tutorials/sql-agent/)
|
||||
|
||||
Here are some high-level tips on writing a good tutorial:
|
||||
|
||||
- Focus on guiding the user to get something done, but keep in mind the end-goal is more to impart principles than to create a perfect production system.
|
||||
- Be specific, not abstract and follow one path.
|
||||
- No need to go deeply into alternative approaches, but it’s ok to reference them, ideally with a link to an appropriate how-to guide.
|
||||
- Get "a point on the board" as soon as possible - something the user can run that outputs something.
|
||||
- You can iterate and expand afterwards.
|
||||
- Try to frequently checkpoint at given steps where the user can run code and see progress.
|
||||
- Focus on results, not technical explanation.
|
||||
- Crosslink heavily to appropriate conceptual/reference pages
|
||||
- The first time you mention a LangGraph concept, use its full name (e.g. "human-in-the-loop"), and link to its conceptual/other documentation page.
|
||||
- It's also helpful to add a prerequisite callout that links to any pages with necessary background information.
|
||||
- End with a recap/next steps section summarizing what the tutorial covered and future reading, such as related how-to guides.
|
||||
- Use phrases like "Next we can run X & Y. We will expect Z.". Then afterwards, use language like "Notice Z" that recalls our expectations and directs the reader's attention to the topic we are trying to teach.
|
||||
- Do not shy away from repetition.
|
||||
|
||||
### How-to guides
|
||||
|
||||
A how-to guide, as the name implies, demonstrates how to do something discrete and specific.
|
||||
It should assume that the user is already familiar with underlying concepts, and is trying to solve an immediate problem, but
|
||||
should still give some background or list the scenarios where the information contained within can be relevant.
|
||||
They can and should discuss alternatives if one approach may be better than another in certain cases.
|
||||
|
||||
To quote the Diataxis website:
|
||||
|
||||
> A how-to guide serves the work of the already-competent user, whom you can assume to know what they want to do, and to be able to follow your instructions correctly.
|
||||
|
||||
Some examples include:
|
||||
|
||||
- [How to add persistence to your graph](https://langchain-ai.github.io/langgraph/how-tos/persistence/)
|
||||
- [How to view and update past graph state](https://langchain-ai.github.io/langgraph/how-tos/human_in_the_loop/time-travel/)
|
||||
|
||||
Here are some high-level tips on writing a good how-to guide:
|
||||
|
||||
- Clearly explain what you are guiding the user through at the start
|
||||
- Assume higher intent than a tutorial and show what the user needs to do to get that task done
|
||||
- Assume familiarity of concepts, but explain why suggested actions are helpful
|
||||
- Crosslink heavily to conceptual/reference pages
|
||||
- Discuss alternatives and responses to real-world tradeoffs that may arise when solving a problem
|
||||
- Use lots of example code, ideally within complete code blocks that the reader can copy and run.
|
||||
- End with a recap/next steps section summarizing what the tutorial covered and future reading, such as other related how-to guides
|
||||
|
||||
### Conceptual guides
|
||||
|
||||
LangGraph's conceptual guides fall under the **Explanation** quadrant of Diataxis. They should cover LangChain terms and concepts
|
||||
in a more abstract way than how-to guides or tutorials, and should be geared towards curious users interested in
|
||||
gaining a deeper understanding of the framework. Try to avoid excessively large code examples. The goal here is to
|
||||
impart perspective to the user rather than to finish a practical project. These guides should cover **why** things work they way they do.
|
||||
|
||||
|
||||
To quote the Diataxis website:
|
||||
|
||||
> The perspective of explanation is higher and wider than that of the other types. It does not take the user’s eye-level view, as in a how-to guide, or a close-up view of the machinery, like reference material. Its scope in each case is a topic - “an area of knowledge”, that somehow has to be bounded in a reasonable, meaningful way.
|
||||
|
||||
Some examples include:
|
||||
|
||||
- [What does it mean to be agentic?](https://langchain-ai.github.io/langgraph/concepts/high_level/)
|
||||
- [Tool calling](https://langchain-ai.github.io/langgraph/concepts/agentic_concepts/#tool-calling)
|
||||
|
||||
Here are some high-level tips on writing a good conceptual guide:
|
||||
|
||||
- Explain design decisions. Why does concept X exist and why was it designed this way?
|
||||
- Use analogies and reference other concepts and alternatives
|
||||
- Avoid blending in too much reference content
|
||||
- You can and should reference content covered in other guides, but make sure to link to them
|
||||
|
||||
### References
|
||||
|
||||
References contain detailed, low-level information that describes exactly what functionality exists and how to use it.
|
||||
In LangGraph, this is mainly our API reference pages, which are populated from docstrings within code.
|
||||
References pages are generally not read end-to-end, but are consulted as necessary when a user needs to know
|
||||
how to use something specific.
|
||||
|
||||
To quote the Diataxis website:
|
||||
|
||||
> The only purpose of a reference guide is to describe, as succinctly as possible, and in an orderly way. Whereas the content of tutorials and how-to guides are led by needs of the user, reference material is led by the product it describes.
|
||||
|
||||
Many of the reference pages in LangChain are automatically generated from code,
|
||||
but here are some high-level tips on writing a good docstring:
|
||||
|
||||
- Be concise
|
||||
- Discuss special cases and deviations from a user's expectations
|
||||
- Go into detail on required inputs and outputs
|
||||
- Light details on when one might use the feature are fine, but in-depth details belong in other sections.
|
||||
|
||||
Each category serves a distinct purpose and requires a specific approach to writing and structuring the content.
|
||||
|
||||
## General guidelines
|
||||
|
||||
Here are some other guidelines you should think about when writing and organizing documentation.
|
||||
|
||||
We generally do not merge new tutorials from outside contributors without an actue need.
|
||||
We welcome updates as well as new integration docs, how-tos, and references.
|
||||
|
||||
### Avoid duplication
|
||||
|
||||
Multiple pages that cover the same material in depth are difficult to maintain and cause confusion. There should
|
||||
be only one (very rarely two), canonical pages for a given concept or feature. Instead, you should link to other guides.
|
||||
|
||||
### Link to other sections
|
||||
|
||||
Because sections of the docs do not exist in a vacuum, it is important to link to other sections as often as possible
|
||||
to allow a developer to learn more about an unfamiliar topic inline.
|
||||
|
||||
This includes linking to the API references as well as conceptual sections!
|
||||
|
||||
### Be concise
|
||||
|
||||
In general, take a less-is-more approach. If a section with a good explanation of a concept already exists, you should link to it rather than
|
||||
re-explain it, unless the concept you are documenting presents some new wrinkle.
|
||||
|
||||
Be concise, including in code samples.
|
||||
|
||||
### General style
|
||||
|
||||
- Use active voice and present tense whenever possible
|
||||
- Use examples and code snippets to illustrate concepts and usage
|
||||
- Use appropriate header levels (`#`, `##`, `###`, etc.) to organize the content hierarchically
|
||||
- Use fewer cells with more code to make copy/paste easier
|
||||
- Use bullet points and numbered lists to break down information into easily digestible chunks
|
||||
- Use tables (especially for **Reference** sections) and diagrams often to present information visually
|
||||
- Include the table of contents for longer documentation pages to help readers navigate the content, but hide it for shorter pages
|
||||
|
||||
## Setup
|
||||
|
||||
LangChain documentation consists of two components:
|
||||
|
||||
1. Main Documentation: Hosted at [https://langchain-ai.github.io](https://langchain-ai.github.io/langgraph/),
|
||||
this comprehensive resource serves as the primary user-facing documentation.
|
||||
It covers a wide array of topics, including tutorials, use cases, integrations,
|
||||
and more, offering extensive guidance on building with LangGraph.
|
||||
The content for this documentation lives in the `/docs` directory of the monorepo.
|
||||
2. In-code Documentation: This is documentation of the codebase itself, which is also
|
||||
used to generate the externally facing [API Reference](https://langchain-ai.github.io/langgraph/reference/graphs/).
|
||||
The content for the API reference is autogenerated by scanning the docstrings in the codebase. For this reason we ask that developers document their code well.
|
||||
|
||||
We appreciate all contributions to the documentation, whether it be fixing a typo,
|
||||
adding a new tutorial or example and whether it be in the main documentation or the API Reference.
|
||||
|
||||
### 📜 Main Documentation
|
||||
|
||||
The content for the main documentation is located in the `/docs` directory of the monorepo.
|
||||
|
||||
The documentation is written using a combination of ipython notebooks (`.ipynb` files)
|
||||
and markdown (`.md` files). The notebooks are converted to markdown
|
||||
and then built using [MkDocs](https://www.mkdocs.org/).
|
||||
|
||||
Feel free to make contributions to the main documentation! 🥰
|
||||
|
||||
After modifying the documentation:
|
||||
|
||||
1. Run the linting and formatting commands (see below) to ensure that the documentation is well-formatted and free of errors.
|
||||
2. Optionally build the documentation locally to verify that the changes look good.
|
||||
3. Make a pull request with the changes.
|
||||
|
||||
### ⚒️ Linting and Building Documentation Locally
|
||||
|
||||
After writing up the documentation, you may want to lint and build the documentation
|
||||
locally to ensure that it looks good and is free of errors.
|
||||
|
||||
If you're unable to build it locally that's okay as well, as you will be able to
|
||||
see a preview of the documentation on the pull request page.
|
||||
|
||||
From the **monorepo root**, run the following command to install the dependencies:
|
||||
|
||||
```bash
|
||||
poetry install --with docs --no-root
|
||||
```
|
||||
|
||||
#### Building
|
||||
|
||||
The code that builds the documentation is located in the `/docs` directory of the monorepo.
|
||||
|
||||
Before building the documentation, it is always a good idea to clean the build directory:
|
||||
|
||||
```bash
|
||||
make clean-docs
|
||||
```
|
||||
|
||||
You can build and preview the documentation as outlined below:
|
||||
|
||||
```bash
|
||||
make serve-docs
|
||||
```
|
||||
|
||||
#### Linting
|
||||
|
||||
The documentation is linted from the **monorepo root**. To lint it, run the following from there:
|
||||
|
||||
```bash
|
||||
make spellcheck
|
||||
```
|
||||
|
||||
### ️In-code Documentation
|
||||
|
||||
The in-code documentation is autogenerated from docstrings.
|
||||
|
||||
For the API reference to be useful, the codebase must be well-documented. This means that all functions, classes, and methods should have a docstring that explains what they do, what the arguments are, and what the return value is. This is a good practice in general, but it is especially important for LangChain because the API reference is the primary resource for developers to understand how to use the codebase.
|
||||
|
||||
We generally follow the [Google Python Style Guide](https://google.github.io/styleguide/pyguide.html#38-comments-and-docstrings) for docstrings.
|
||||
|
||||
Here is an example of a well-documented function:
|
||||
|
||||
```python
|
||||
|
||||
def my_function(arg1: int, arg2: str) -> float:
|
||||
"""This is a short description of the function. (It should be a single sentence.)
|
||||
|
||||
This is a longer description of the function. It should explain what
|
||||
the function does, what the arguments are, and what the return value is.
|
||||
It should wrap at 88 characters.
|
||||
|
||||
Examples:
|
||||
This is a section for examples of how to use the function.
|
||||
|
||||
.. code-block:: python
|
||||
|
||||
my_function(1, "hello")
|
||||
|
||||
Args:
|
||||
arg1: This is a description of arg1. We do not need to specify the type since
|
||||
it is already specified in the function signature.
|
||||
arg2: This is a description of arg2.
|
||||
|
||||
Returns:
|
||||
This is a description of the return value.
|
||||
"""
|
||||
return 3.14
|
||||
```
|
||||
@@ -1,68 +0,0 @@
|
||||
# Define the directories containing projects
|
||||
LIBS_DIRS := $(wildcard libs/*)
|
||||
|
||||
# Default target
|
||||
.PHONY: all
|
||||
all: lint format lock test
|
||||
|
||||
# Install dependencies for all projects
|
||||
.PHONY: install
|
||||
install:
|
||||
@echo "Creating virtual environment..."
|
||||
@uv venv
|
||||
@for dir in $(LIBS_DIRS); do \
|
||||
if [ -f $$dir/pyproject.toml ]; then \
|
||||
echo "Installing dependencies for $$dir"; \
|
||||
uv pip install -e $$dir; \
|
||||
fi; \
|
||||
done
|
||||
|
||||
# Lint all projects
|
||||
.PHONY: lint
|
||||
lint:
|
||||
@for dir in $(LIBS_DIRS); do \
|
||||
if [ -f $$dir/Makefile ]; then \
|
||||
echo "Running lint in $$dir"; \
|
||||
$(MAKE) -C $$dir lint; \
|
||||
fi; \
|
||||
done
|
||||
|
||||
# Format all projects
|
||||
.PHONY: format
|
||||
format:
|
||||
@for dir in $(LIBS_DIRS); do \
|
||||
if [ -f $$dir/Makefile ]; then \
|
||||
echo "Running format in $$dir"; \
|
||||
$(MAKE) -C $$dir format; \
|
||||
fi; \
|
||||
done
|
||||
|
||||
# Lock all projects
|
||||
.PHONY: lock
|
||||
lock:
|
||||
@for dir in $(LIBS_DIRS); do \
|
||||
if [ -f $$dir/Makefile ]; then \
|
||||
echo "Running lock in $$dir"; \
|
||||
(cd $$dir && uv lock); \
|
||||
fi; \
|
||||
done
|
||||
|
||||
# Lock all projects and upgrade dependencies
|
||||
.PHONY: lock-upgrade
|
||||
lock-upgrade:
|
||||
@for dir in $(LIBS_DIRS); do \
|
||||
if [ -f $$dir/Makefile ]; then \
|
||||
echo "Running lock-upgrade in $$dir"; \
|
||||
(cd $$dir && uv lock --upgrade); \
|
||||
fi; \
|
||||
done
|
||||
|
||||
# Test all projects
|
||||
.PHONY: test
|
||||
test:
|
||||
@for dir in $(LIBS_DIRS); do \
|
||||
if [ -f $$dir/Makefile ]; then \
|
||||
echo "Running test in $$dir"; \
|
||||
$(MAKE) -C $$dir test; \
|
||||
fi; \
|
||||
done
|
||||
@@ -1,91 +1,339 @@
|
||||
<picture class="github-only">
|
||||
<source media="(prefers-color-scheme: light)" srcset="https://langchain-ai.github.io/langgraph/static/wordmark_dark.svg">
|
||||
<source media="(prefers-color-scheme: dark)" srcset="https://langchain-ai.github.io/langgraph/static/wordmark_light.svg">
|
||||
<img alt="LangGraph Logo" src="https://langchain-ai.github.io/langgraph/static/wordmark_dark.svg" width="80%">
|
||||
</picture>
|
||||
# 🦜🕸️LangGraph
|
||||
|
||||
<div>
|
||||
<br>
|
||||
</div>
|
||||
|
||||
[](https://pypi.org/project/langgraph/)
|
||||

|
||||
[](https://pepy.tech/project/langgraph)
|
||||
[](https://github.com/langchain-ai/langgraph/issues)
|
||||
[](https://docs.langchain.com/oss/python/langgraph/overview)
|
||||
[](https://langchain-ai.github.io/langgraph/)
|
||||
|
||||
Trusted by companies shaping the future of agents – including Klarna, Replit, Elastic, and more – LangGraph is a low-level orchestration framework for building, managing, and deploying long-running, stateful agents.
|
||||
⚡ Building language agents as graphs ⚡
|
||||
|
||||
## Get started
|
||||
> [!NOTE]
|
||||
> Looking for the JS version? See the [JS repo](https://github.com/langchain-ai/langgraphjs) and the [JS docs](https://langchain-ai.github.io/langgraphjs/).
|
||||
|
||||
Install LangGraph:
|
||||
## Overview
|
||||
|
||||
```
|
||||
[LangGraph](https://langchain-ai.github.io/langgraph/) is a library for building
|
||||
stateful, multi-actor applications with LLMs, used to create agent and multi-agent
|
||||
workflows. Check out an introductory tutorial [here](https://langchain-ai.github.io/langgraph/tutorials/introduction/).
|
||||
|
||||
|
||||
LangGraph is inspired by [Pregel](https://research.google/pubs/pub37252/) and [Apache Beam](https://beam.apache.org/). The public interface draws inspiration from [NetworkX](https://networkx.org/documentation/latest/). LangGraph is built by LangChain Inc, the creators of LangChain, but can be used without LangChain.
|
||||
|
||||
### Why use LangGraph?
|
||||
|
||||
LangGraph powers [production-grade agents](https://www.langchain.com/built-with-langgraph), trusted by Linkedin, Uber, Klarna, GitLab, and many more. LangGraph provides fine-grained control over both the flow and state of your agent applications. It implements a central [persistence layer](https://langchain-ai.github.io/langgraph/concepts/persistence/), enabling features that are common to most agent architectures:
|
||||
|
||||
- **Memory**: LangGraph persists arbitrary aspects of your application's state,
|
||||
supporting memory of conversations and other updates within and across user
|
||||
interactions;
|
||||
- **Human-in-the-loop**: Because state is checkpointed, execution can be interrupted
|
||||
and resumed, allowing for decisions, validation, and corrections at key stages via
|
||||
human input.
|
||||
|
||||
Standardizing these components allows individuals and teams to focus on the behavior
|
||||
of their agent, instead of its supporting infrastructure.
|
||||
|
||||
Through [LangGraph Platform](#langgraph-platform), LangGraph also provides tooling for
|
||||
the development, deployment, debugging, and monitoring of your applications.
|
||||
|
||||
LangGraph integrates seamlessly with
|
||||
[LangChain](https://python.langchain.com/docs/introduction/) and
|
||||
[LangSmith](https://docs.smith.langchain.com/) (but does not require them).
|
||||
|
||||
To learn more about LangGraph, check out our first LangChain Academy
|
||||
course, *Introduction to LangGraph*, available for free
|
||||
[here](https://academy.langchain.com/courses/intro-to-langgraph).
|
||||
|
||||
### LangGraph Platform
|
||||
|
||||
[LangGraph Platform](https://langchain-ai.github.io/langgraph/concepts/langgraph_platform) is infrastructure for deploying LangGraph agents. It is a commercial solution for deploying agentic applications to production, built on the open-source LangGraph framework. The LangGraph Platform consists of several components that work together to support the development, deployment, debugging, and monitoring of LangGraph applications: [LangGraph Server](https://langchain-ai.github.io/langgraph/concepts/langgraph_server) (APIs), [LangGraph SDKs](https://langchain-ai.github.io/langgraph/concepts/sdk) (clients for the APIs), [LangGraph CLI](https://langchain-ai.github.io/langgraph/concepts/langgraph_cli) (command line tool for building the server), and [LangGraph Studio](https://langchain-ai.github.io/langgraph/concepts/langgraph_studio) (UI/debugger).
|
||||
|
||||
See deployment options [here](https://langchain-ai.github.io/langgraph/concepts/deployment_options/)
|
||||
(includes a free tier).
|
||||
|
||||
Here are some common issues that arise in complex deployments, which LangGraph Platform addresses:
|
||||
|
||||
- **Streaming support**: LangGraph Server provides [multiple streaming modes](https://langchain-ai.github.io/langgraph/concepts/streaming) optimized for various application needs
|
||||
- **Background runs**: Runs agents asynchronously in the background
|
||||
- **Support for long running agents**: Infrastructure that can handle long running processes
|
||||
- **[Double texting](https://langchain-ai.github.io/langgraph/concepts/double_texting)**: Handle the case where you get two messages from the user before the agent can respond
|
||||
- **Handle burstiness**: Task queue for ensuring requests are handled consistently without loss, even under heavy loads
|
||||
|
||||
## Installation
|
||||
|
||||
```shell
|
||||
pip install -U langgraph
|
||||
```
|
||||
|
||||
Create a simple workflow:
|
||||
## Example
|
||||
|
||||
```python
|
||||
from langgraph.graph import START, StateGraph
|
||||
from typing_extensions import TypedDict
|
||||
Let's build a tool-calling [ReAct-style](https://langchain-ai.github.io/langgraph/concepts/agentic_concepts/#react-implementation) agent that uses a search tool!
|
||||
|
||||
|
||||
class State(TypedDict):
|
||||
text: str
|
||||
|
||||
|
||||
def node_a(state: State) -> dict:
|
||||
return {"text": state["text"] + "a"}
|
||||
|
||||
|
||||
def node_b(state: State) -> dict:
|
||||
return {"text": state["text"] + "b"}
|
||||
|
||||
|
||||
graph = StateGraph(State)
|
||||
graph.add_node("node_a", node_a)
|
||||
graph.add_node("node_b", node_b)
|
||||
graph.add_edge(START, "node_a")
|
||||
graph.add_edge("node_a", "node_b")
|
||||
|
||||
print(graph.compile().invoke({"text": ""}))
|
||||
# {'text': 'ab'}
|
||||
```shell
|
||||
pip install langchain-anthropic
|
||||
```
|
||||
|
||||
Get started with the [LangGraph Quickstart](https://docs.langchain.com/oss/python/langgraph/quickstart).
|
||||
```shell
|
||||
export ANTHROPIC_API_KEY=sk-...
|
||||
```
|
||||
|
||||
To quickly build agents with LangChain's `create_agent` (built on LangGraph), see the [LangChain Agents documentation](https://docs.langchain.com/oss/python/langchain/agents).
|
||||
Optionally, we can set up [LangSmith](https://docs.smith.langchain.com/) for best-in-class observability.
|
||||
|
||||
## Core benefits
|
||||
```shell
|
||||
export LANGSMITH_TRACING=true
|
||||
export LANGSMITH_API_KEY=lsv2_sk_...
|
||||
```
|
||||
|
||||
LangGraph provides low-level supporting infrastructure for *any* long-running, stateful workflow or agent. LangGraph does not abstract prompts or architecture, and provides the following central benefits:
|
||||
The simplest way to create a tool-calling agent in LangGraph is to use `create_react_agent`:
|
||||
|
||||
- [Durable execution](https://docs.langchain.com/oss/python/langgraph/durable-execution): Build agents that persist through failures and can run for extended periods, automatically resuming from exactly where they left off.
|
||||
- [Human-in-the-loop](https://docs.langchain.com/oss/python/langgraph/interrupts): Seamlessly incorporate human oversight by inspecting and modifying agent state at any point during execution.
|
||||
- [Comprehensive memory](https://docs.langchain.com/oss/python/langgraph/memory): Create truly stateful agents with both short-term working memory for ongoing reasoning and long-term persistent memory across sessions.
|
||||
- [Debugging with LangSmith](http://www.langchain.com/langsmith): Gain deep visibility into complex agent behavior with visualization tools that trace execution paths, capture state transitions, and provide detailed runtime metrics.
|
||||
- [Production-ready deployment](https://docs.langchain.com/langsmith/app-development): Deploy sophisticated agent systems confidently with scalable infrastructure designed to handle the unique challenges of stateful, long-running workflows.
|
||||
<details open>
|
||||
<summary>High-level implementation</summary>
|
||||
|
||||
## LangGraph’s ecosystem
|
||||
```python
|
||||
from langgraph.prebuilt import create_react_agent
|
||||
from langgraph.checkpoint.memory import MemorySaver
|
||||
from langchain_anthropic import ChatAnthropic
|
||||
from langchain_core.tools import tool
|
||||
|
||||
While LangGraph can be used standalone, it also integrates seamlessly with any LangChain product, giving developers a full suite of tools for building agents. To improve your LLM application development, pair LangGraph with:
|
||||
# Define the tools for the agent to use
|
||||
@tool
|
||||
def search(query: str):
|
||||
"""Call to surf the web."""
|
||||
# This is a placeholder, but don't tell the LLM that...
|
||||
if "sf" in query.lower() or "san francisco" in query.lower():
|
||||
return "It's 60 degrees and foggy."
|
||||
return "It's 90 degrees and sunny."
|
||||
|
||||
- [LangSmith](http://www.langchain.com/langsmith) — Helpful for agent evals and observability. Debug poor-performing LLM app runs, evaluate agent trajectories, gain visibility in production, and improve performance over time.
|
||||
- [LangSmith Deployment](https://docs.langchain.com/langsmith/deployments) — Deploy and scale agents effortlessly with a purpose-built deployment platform for long running, stateful workflows. Discover, reuse, configure, and share agents across teams — and iterate quickly with visual prototyping in [LangGraph Studio](https://docs.langchain.com/oss/python/langgraph/studio).
|
||||
- [LangChain](https://docs.langchain.com/oss/python/langchain/overview) – Provides integrations and composable components to streamline LLM application development.
|
||||
|
||||
> [!NOTE]
|
||||
> Looking for the JS version of LangGraph? See the [JS repo](https://github.com/langchain-ai/langgraphjs) and the [JS docs](https://docs.langchain.com/oss/javascript/langgraph/overview).
|
||||
tools = [search]
|
||||
model = ChatAnthropic(model="claude-3-5-sonnet-latest", temperature=0)
|
||||
|
||||
## Additional resources
|
||||
# Initialize memory to persist state between graph runs
|
||||
checkpointer = MemorySaver()
|
||||
|
||||
- [Guides](https://docs.langchain.com/oss/python/langgraph/guides): Quick, actionable code snippets for topics such as streaming, adding memory & persistence, and design patterns (e.g. branching, subgraphs, etc.).
|
||||
- [Reference](https://reference.langchain.com/python/langgraph/): Detailed reference on core classes, methods, how to use the graph and checkpointing APIs, and higher-level prebuilt components.
|
||||
- [Examples](https://docs.langchain.com/oss/python/langgraph/agentic-rag): Guided examples on getting started with LangGraph.
|
||||
- [LangChain Forum](https://forum.langchain.com/): Connect with the community and share all of your technical questions, ideas, and feedback.
|
||||
- [LangChain Academy](https://academy.langchain.com/courses/intro-to-langgraph): Learn the basics of LangGraph in our free, structured course.
|
||||
- [Case studies](https://www.langchain.com/built-with-langgraph): Hear how industry leaders use LangGraph to ship AI applications at scale.
|
||||
app = create_react_agent(model, tools, checkpointer=checkpointer)
|
||||
|
||||
## Acknowledgements
|
||||
# Use the agent
|
||||
final_state = app.invoke(
|
||||
{"messages": [{"role": "user", "content": "what is the weather in sf"}]},
|
||||
config={"configurable": {"thread_id": 42}}
|
||||
)
|
||||
final_state["messages"][-1].content
|
||||
```
|
||||
```
|
||||
"Based on the search results, I can tell you that the current weather in San Francisco is:\n\nTemperature: 60 degrees Fahrenheit\nConditions: Foggy\n\nSan Francisco is known for its microclimates and frequent fog, especially during the summer months. The temperature of 60°F (about 15.5°C) is quite typical for the city, which tends to have mild temperatures year-round. The fog, often referred to as "Karl the Fog" by locals, is a characteristic feature of San Francisco\'s weather, particularly in the mornings and evenings.\n\nIs there anything else you\'d like to know about the weather in San Francisco or any other location?"
|
||||
```
|
||||
|
||||
LangGraph is inspired by [Pregel](https://research.google/pubs/pub37252/) and [Apache Beam](https://beam.apache.org/). The public interface draws inspiration from [NetworkX](https://networkx.org/documentation/latest/). LangGraph is built by LangChain Inc, the creators of LangChain, but can be used without LangChain.
|
||||
Now when we pass the same <code>"thread_id"</code>, the conversation context is retained via the saved state (i.e. stored list of messages)
|
||||
|
||||
```python
|
||||
final_state = app.invoke(
|
||||
{"messages": [{"role": "user", "content": "what about ny"}]},
|
||||
config={"configurable": {"thread_id": 42}}
|
||||
)
|
||||
final_state["messages"][-1].content
|
||||
```
|
||||
|
||||
```
|
||||
"Based on the search results, I can tell you that the current weather in New York City is:\n\nTemperature: 90 degrees Fahrenheit (approximately 32.2 degrees Celsius)\nConditions: Sunny\n\nThis weather is quite different from what we just saw in San Francisco. New York is experiencing much warmer temperatures right now. Here are a few points to note:\n\n1. The temperature of 90°F is quite hot, typical of summer weather in New York City.\n2. The sunny conditions suggest clear skies, which is great for outdoor activities but also means it might feel even hotter due to direct sunlight.\n3. This kind of weather in New York often comes with high humidity, which can make it feel even warmer than the actual temperature suggests.\n\nIt's interesting to see the stark contrast between San Francisco's mild, foggy weather and New York's hot, sunny conditions. This difference illustrates how varied weather can be across different parts of the United States, even on the same day.\n\nIs there anything else you'd like to know about the weather in New York or any other location?"
|
||||
```
|
||||
</details>
|
||||
|
||||
> [!TIP]
|
||||
> LangGraph is a **low-level** framework that allows you to implement any custom agent
|
||||
architectures. Click on the low-level implementation below to see how to implement a
|
||||
tool-calling agent from scratch.
|
||||
|
||||
<details>
|
||||
<summary>Low-level implementation</summary>
|
||||
|
||||
```python
|
||||
from typing import Literal
|
||||
|
||||
from langchain_anthropic import ChatAnthropic
|
||||
from langchain_core.tools import tool
|
||||
from langgraph.checkpoint.memory import MemorySaver
|
||||
from langgraph.graph import END, START, StateGraph, MessagesState
|
||||
from langgraph.prebuilt import ToolNode
|
||||
|
||||
|
||||
# Define the tools for the agent to use
|
||||
@tool
|
||||
def search(query: str):
|
||||
"""Call to surf the web."""
|
||||
# This is a placeholder, but don't tell the LLM that...
|
||||
if "sf" in query.lower() or "san francisco" in query.lower():
|
||||
return "It's 60 degrees and foggy."
|
||||
return "It's 90 degrees and sunny."
|
||||
|
||||
|
||||
tools = [search]
|
||||
|
||||
tool_node = ToolNode(tools)
|
||||
|
||||
model = ChatAnthropic(model="claude-3-5-sonnet-latest", temperature=0).bind_tools(tools)
|
||||
|
||||
# Define the function that determines whether to continue or not
|
||||
def should_continue(state: MessagesState) -> Literal["tools", END]:
|
||||
messages = state['messages']
|
||||
last_message = messages[-1]
|
||||
# If the LLM makes a tool call, then we route to the "tools" node
|
||||
if last_message.tool_calls:
|
||||
return "tools"
|
||||
# Otherwise, we stop (reply to the user)
|
||||
return END
|
||||
|
||||
|
||||
# Define the function that calls the model
|
||||
def call_model(state: MessagesState):
|
||||
messages = state['messages']
|
||||
response = model.invoke(messages)
|
||||
# We return a list, because this will get added to the existing list
|
||||
return {"messages": [response]}
|
||||
|
||||
|
||||
# Define a new graph
|
||||
workflow = StateGraph(MessagesState)
|
||||
|
||||
# Define the two nodes we will cycle between
|
||||
workflow.add_node("agent", call_model)
|
||||
workflow.add_node("tools", tool_node)
|
||||
|
||||
# Set the entrypoint as `agent`
|
||||
# This means that this node is the first one called
|
||||
workflow.add_edge(START, "agent")
|
||||
|
||||
# We now add a conditional edge
|
||||
workflow.add_conditional_edges(
|
||||
# First, we define the start node. We use `agent`.
|
||||
# This means these are the edges taken after the `agent` node is called.
|
||||
"agent",
|
||||
# Next, we pass in the function that will determine which node is called next.
|
||||
should_continue,
|
||||
)
|
||||
|
||||
# We now add a normal edge from `tools` to `agent`.
|
||||
# This means that after `tools` is called, `agent` node is called next.
|
||||
workflow.add_edge("tools", 'agent')
|
||||
|
||||
# Initialize memory to persist state between graph runs
|
||||
checkpointer = MemorySaver()
|
||||
|
||||
# Finally, we compile it!
|
||||
# This compiles it into a LangChain Runnable,
|
||||
# meaning you can use it as you would any other runnable.
|
||||
# Note that we're (optionally) passing the memory when compiling the graph
|
||||
app = workflow.compile(checkpointer=checkpointer)
|
||||
|
||||
# Use the agent
|
||||
final_state = app.invoke(
|
||||
{"messages": [{"role": "user", "content": "what is the weather in sf"}]},
|
||||
config={"configurable": {"thread_id": 42}}
|
||||
)
|
||||
final_state["messages"][-1].content
|
||||
```
|
||||
|
||||
<b>Step-by-step Breakdown</b>:
|
||||
|
||||
<details>
|
||||
<summary>Initialize the model and tools.</summary>
|
||||
<ul>
|
||||
<li>
|
||||
We use <code>ChatAnthropic</code> as our LLM. <strong>NOTE:</strong> we need to make sure the model knows that it has these tools available to call. We can do this by converting the LangChain tools into the format for OpenAI tool calling using the <code>.bind_tools()</code> method.
|
||||
</li>
|
||||
<li>
|
||||
We define the tools we want to use - a search tool in our case. It is really easy to create your own tools - see documentation here on how to do that <a href="https://python.langchain.com/docs/how_to/custom_tools/">here</a>.
|
||||
</li>
|
||||
</ul>
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary>Initialize graph with state.</summary>
|
||||
|
||||
<ul>
|
||||
<li>We initialize graph (<code>StateGraph</code>) by passing state schema (in our case <code>MessagesState</code>)</li>
|
||||
<li><code>MessagesState</code> is a prebuilt state schema that has one attribute -- a list of LangChain <code>Message</code> objects, as well as logic for merging the updates from each node into the state.</li>
|
||||
</ul>
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary>Define graph nodes.</summary>
|
||||
|
||||
There are two main nodes we need:
|
||||
|
||||
<ul>
|
||||
<li>The <code>agent</code> node: responsible for deciding what (if any) actions to take.</li>
|
||||
<li>The <code>tools</code> node that invokes tools: if the agent decides to take an action, this node will then execute that action.</li>
|
||||
</ul>
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary>Define entry point and graph edges.</summary>
|
||||
|
||||
First, we need to set the entry point for graph execution - <code>agent</code> node.
|
||||
|
||||
Then we define one normal and one conditional edge. Conditional edge means that the destination depends on the contents of the graph's state (<code>MessagesState</code>). In our case, the destination is not known until the agent (LLM) decides.
|
||||
|
||||
<ul>
|
||||
<li>Conditional edge: after the agent is called, we should either:
|
||||
<ul>
|
||||
<li>a. Run tools if the agent said to take an action, OR</li>
|
||||
<li>b. Finish (respond to the user) if the agent did not ask to run tools</li>
|
||||
</ul>
|
||||
</li>
|
||||
<li>Normal edge: after the tools are invoked, the graph should always return to the agent to decide what to do next</li>
|
||||
</ul>
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary>Compile the graph.</summary>
|
||||
|
||||
<ul>
|
||||
<li>
|
||||
When we compile the graph, we turn it into a LangChain
|
||||
<a href="https://python.langchain.com/docs/concepts/runnables/">Runnable</a>,
|
||||
which automatically enables calling <code>.invoke()</code>, <code>.stream()</code> and <code>.batch()</code>
|
||||
with your inputs
|
||||
</li>
|
||||
<li>
|
||||
We can also optionally pass checkpointer object for persisting state between graph runs, and enabling memory,
|
||||
human-in-the-loop workflows, time travel and more. In our case we use <code>MemorySaver</code> -
|
||||
a simple in-memory checkpointer
|
||||
</li>
|
||||
</ul>
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary>Execute the graph.</summary>
|
||||
|
||||
<ol>
|
||||
<li>LangGraph adds the input message to the internal state, then passes the state to the entrypoint node, <code>"agent"</code>.</li>
|
||||
<li>The <code>"agent"</code> node executes, invoking the chat model.</li>
|
||||
<li>The chat model returns an <code>AIMessage</code>. LangGraph adds this to the state.</li>
|
||||
<li>Graph cycles the following steps until there are no more <code>tool_calls</code> on <code>AIMessage</code>:
|
||||
<ul>
|
||||
<li>If <code>AIMessage</code> has <code>tool_calls</code>, <code>"tools"</code> node executes</li>
|
||||
<li>The <code>"agent"</code> node executes again and returns <code>AIMessage</code></li>
|
||||
</ul>
|
||||
</li>
|
||||
<li>Execution progresses to the special <code>END</code> value and outputs the final state. And as a result, we get a list of all our chat messages as output.</li>
|
||||
</ol>
|
||||
</details>
|
||||
|
||||
</details>
|
||||
|
||||
## Documentation
|
||||
|
||||
* [Tutorials](https://langchain-ai.github.io/langgraph/tutorials/): Learn to build with LangGraph through guided examples.
|
||||
* [How-to Guides](https://langchain-ai.github.io/langgraph/how-tos/): Accomplish specific things within LangGraph, from streaming, to adding memory & persistence, to common design patterns (branching, subgraphs, etc.), these are the place to go if you want to copy and run a specific code snippet.
|
||||
* [Conceptual Guides](https://langchain-ai.github.io/langgraph/concepts/high_level/): In-depth explanations of the key concepts and principles behind LangGraph, such as nodes, edges, state and more.
|
||||
* [API Reference](https://langchain-ai.github.io/langgraph/reference/graphs/): Review important classes and methods, simple examples of how to use the graph and checkpointing APIs, higher-level prebuilt components and more.
|
||||
* [LangGraph Platform](https://langchain-ai.github.io/langgraph/concepts/#langgraph-platform): LangGraph Platform is a commercial solution for deploying agentic applications in production, built on the open-source LangGraph framework.
|
||||
|
||||
## Resources
|
||||
|
||||
* [Built with LangGraph](https://www.langchain.com/built-with-langgraph): Hear how industry leaders use LangGraph to ship powerful, production-ready AI applications.
|
||||
|
||||
## Contributing
|
||||
|
||||
For more information on how to contribute, see [here](https://github.com/langchain-ai/langgraph/blob/main/CONTRIBUTING.md).
|
||||
@@ -1,21 +0,0 @@
|
||||
MIT License
|
||||
|
||||
Copyright (c) 2024 LangChain, Inc.
|
||||
|
||||
Permission is hereby granted, free of charge, to any person obtaining a copy
|
||||
of this software and associated documentation files (the "Software"), to deal
|
||||
in the Software without restriction, including without limitation the rights
|
||||
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
||||
copies of the Software, and to permit persons to whom the Software is
|
||||
furnished to do so, subject to the following conditions:
|
||||
|
||||
The above copyright notice and this permission notice shall be included in all
|
||||
copies or substantial portions of the Software.
|
||||
|
||||
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
||||
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
||||
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
||||
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
||||
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
||||
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
||||
SOFTWARE.
|
||||
@@ -18,7 +18,7 @@ POSTGRES_VERSIONS ?= 15 16
|
||||
test_pg_version:
|
||||
@echo "Testing PostgreSQL $(POSTGRES_VERSION)"
|
||||
@POSTGRES_VERSION=$(POSTGRES_VERSION) make start-postgres
|
||||
@uv run pytest $(TEST)
|
||||
@poetry run pytest $(TEST)
|
||||
@EXIT_CODE=$$?; \
|
||||
make stop-postgres; \
|
||||
echo "Finished testing PostgreSQL $(POSTGRES_VERSION); Exit code: $$EXIT_CODE"; \
|
||||
@@ -36,7 +36,7 @@ test:
|
||||
TEST ?= .
|
||||
test_watch:
|
||||
POSTGRES_VERSION=${POSTGRES_VERSION:-16} make start-postgres; \
|
||||
uv run ptw $(TEST); \
|
||||
poetry run ptw $(TEST); \
|
||||
EXIT_CODE=$$?; \
|
||||
make stop-postgres; \
|
||||
exit $$EXIT_CODE
|
||||
@@ -55,12 +55,12 @@ lint_tests: PYTHON_FILES=tests
|
||||
lint_tests: MYPY_CACHE=.mypy_cache_test
|
||||
|
||||
lint lint_diff lint_package lint_tests:
|
||||
uv run ruff check .
|
||||
[ "$(PYTHON_FILES)" = "" ] || uv run ruff format $(PYTHON_FILES) --diff
|
||||
[ "$(PYTHON_FILES)" = "" ] || uv run ruff check --select I $(PYTHON_FILES)
|
||||
poetry run ruff check .
|
||||
[ "$(PYTHON_FILES)" = "" ] || poetry run ruff format $(PYTHON_FILES) --diff
|
||||
[ "$(PYTHON_FILES)" = "" ] || poetry run ruff check --select I $(PYTHON_FILES)
|
||||
[ "$(PYTHON_FILES)" = "" ] || mkdir -p $(MYPY_CACHE)
|
||||
[ "$(PYTHON_FILES)" = "" ] || uv run mypy $(PYTHON_FILES) --cache-dir $(MYPY_CACHE)
|
||||
[ "$(PYTHON_FILES)" = "" ] || poetry run mypy $(PYTHON_FILES) --cache-dir $(MYPY_CACHE)
|
||||
|
||||
format format_diff:
|
||||
uv run ruff format $(PYTHON_FILES)
|
||||
uv run ruff check --select I --fix $(PYTHON_FILES)
|
||||
poetry run ruff format $(PYTHON_FILES)
|
||||
poetry run ruff check --select I --fix $(PYTHON_FILES)
|
||||
@@ -13,20 +13,6 @@ By default `langgraph-checkpoint-postgres` installs `psycopg` (Psycopg 3) withou
|
||||
|
||||
> [!IMPORTANT]
|
||||
> When manually creating Postgres connections and passing them to `PostgresSaver` or `AsyncPostgresSaver`, make sure to include `autocommit=True` and `row_factory=dict_row` (`from psycopg.rows import dict_row`). See a full example in this [how-to guide](https://langchain-ai.github.io/langgraph/how-tos/persistence_postgres/).
|
||||
>
|
||||
> **Why these parameters are required:**
|
||||
> - `autocommit=True`: Required for the `.setup()` method to properly commit the checkpoint tables to the database. Without this, table creation may not be persisted.
|
||||
> - `row_factory=dict_row`: Required because the PostgresSaver implementation accesses database rows using dictionary-style syntax (e.g., `row["column_name"]`). The default `tuple_row` factory returns tuples that only support index-based access (e.g., `row[0]`), which will cause `TypeError` exceptions when the checkpointer tries to access columns by name.
|
||||
>
|
||||
> **Example of incorrect usage:**
|
||||
> ```python
|
||||
> # ❌ This will fail with TypeError during checkpointer operations
|
||||
> with psycopg.connect(DB_URI) as conn: # Missing autocommit=True and row_factory=dict_row
|
||||
> checkpointer = PostgresSaver(conn)
|
||||
> checkpointer.setup() # May not persist tables properly
|
||||
> # Any operation that reads from database will fail with:
|
||||
> # TypeError: tuple indices must be integers or slices, not str
|
||||
> ```
|
||||
|
||||
```python
|
||||
from langgraph.checkpoint.postgres import PostgresSaver
|
||||
@@ -39,7 +25,7 @@ with PostgresSaver.from_conn_string(DB_URI) as checkpointer:
|
||||
# call .setup() the first time you're using the checkpointer
|
||||
checkpointer.setup()
|
||||
checkpoint = {
|
||||
"v": 4,
|
||||
"v": 1,
|
||||
"ts": "2024-07-31T20:14:19.804150+00:00",
|
||||
"id": "1ef4f797-8335-6428-8001-8a1503f9b875",
|
||||
"channel_values": {
|
||||
@@ -61,6 +47,7 @@ with PostgresSaver.from_conn_string(DB_URI) as checkpointer:
|
||||
"start:node": 2
|
||||
}
|
||||
},
|
||||
"pending_sends": [],
|
||||
}
|
||||
|
||||
# store checkpoint
|
||||
@@ -72,44 +59,3 @@ with PostgresSaver.from_conn_string(DB_URI) as checkpointer:
|
||||
# list checkpoints
|
||||
list(checkpointer.list(read_config))
|
||||
```
|
||||
|
||||
### Async
|
||||
|
||||
```python
|
||||
from langgraph.checkpoint.postgres.aio import AsyncPostgresSaver
|
||||
|
||||
async with AsyncPostgresSaver.from_conn_string(DB_URI) as checkpointer:
|
||||
checkpoint = {
|
||||
"v": 4,
|
||||
"ts": "2024-07-31T20:14:19.804150+00:00",
|
||||
"id": "1ef4f797-8335-6428-8001-8a1503f9b875",
|
||||
"channel_values": {
|
||||
"my_key": "meow",
|
||||
"node": "node"
|
||||
},
|
||||
"channel_versions": {
|
||||
"__start__": 2,
|
||||
"my_key": 3,
|
||||
"start:node": 3,
|
||||
"node": 3
|
||||
},
|
||||
"versions_seen": {
|
||||
"__input__": {},
|
||||
"__start__": {
|
||||
"__start__": 1
|
||||
},
|
||||
"node": {
|
||||
"start:node": 2
|
||||
}
|
||||
},
|
||||
}
|
||||
|
||||
# store checkpoint
|
||||
await checkpointer.aput(write_config, checkpoint, {}, {})
|
||||
|
||||
# load checkpoint
|
||||
await checkpointer.aget(read_config)
|
||||
|
||||
# list checkpoints
|
||||
[c async for c in checkpointer.alist(read_config)]
|
||||
```
|
||||
@@ -1,44 +1,38 @@
|
||||
from __future__ import annotations
|
||||
|
||||
import threading
|
||||
from collections import defaultdict
|
||||
from collections.abc import Iterator, Sequence
|
||||
from contextlib import contextmanager
|
||||
from typing import Any
|
||||
from typing import Any, Optional
|
||||
|
||||
from langchain_core.runnables import RunnableConfig
|
||||
from langgraph.checkpoint.base import (
|
||||
WRITES_IDX_MAP,
|
||||
ChannelVersions,
|
||||
Checkpoint,
|
||||
CheckpointMetadata,
|
||||
CheckpointTuple,
|
||||
get_checkpoint_id,
|
||||
get_serializable_checkpoint_metadata,
|
||||
)
|
||||
from langgraph.checkpoint.serde.base import SerializerProtocol
|
||||
from psycopg import Capabilities, Connection, Cursor, Pipeline
|
||||
from psycopg.rows import DictRow, dict_row
|
||||
from psycopg.types.json import Jsonb
|
||||
from psycopg_pool import ConnectionPool
|
||||
|
||||
from langgraph.checkpoint.base import (
|
||||
WRITES_IDX_MAP,
|
||||
ChannelVersions,
|
||||
Checkpoint,
|
||||
CheckpointConfig,
|
||||
CheckpointMetadata,
|
||||
CheckpointTuple,
|
||||
get_checkpoint_id,
|
||||
get_checkpoint_metadata,
|
||||
)
|
||||
from langgraph.checkpoint.postgres import _internal
|
||||
from langgraph.checkpoint.postgres.base import BasePostgresSaver
|
||||
from langgraph.checkpoint.postgres.shallow import ShallowPostgresSaver
|
||||
from langgraph.checkpoint.serde.base import SerializerProtocol
|
||||
|
||||
Conn = _internal.Conn # For backward compatibility
|
||||
|
||||
|
||||
class PostgresSaver(BasePostgresSaver):
|
||||
"""Checkpointer that stores checkpoints in a Postgres database."""
|
||||
|
||||
lock: threading.Lock
|
||||
|
||||
def __init__(
|
||||
self,
|
||||
conn: _internal.Conn,
|
||||
pipe: Pipeline | None = None,
|
||||
serde: SerializerProtocol | None = None,
|
||||
pipe: Optional[Pipeline] = None,
|
||||
serde: Optional[SerializerProtocol] = None,
|
||||
) -> None:
|
||||
super().__init__(serde=serde)
|
||||
if isinstance(conn, ConnectionPool) and pipe is not None:
|
||||
@@ -55,12 +49,12 @@ class PostgresSaver(BasePostgresSaver):
|
||||
@contextmanager
|
||||
def from_conn_string(
|
||||
cls, conn_string: str, *, pipeline: bool = False
|
||||
) -> Iterator[PostgresSaver]:
|
||||
) -> Iterator["PostgresSaver"]:
|
||||
"""Create a new PostgresSaver instance from a connection string.
|
||||
|
||||
Args:
|
||||
conn_string: The Postgres connection info string.
|
||||
pipeline: whether to use Pipeline
|
||||
conn_string (str): The Postgres connection info string.
|
||||
pipeline (bool): whether to use Pipeline
|
||||
|
||||
Returns:
|
||||
PostgresSaver: A new PostgresSaver instance.
|
||||
@@ -94,20 +88,19 @@ class PostgresSaver(BasePostgresSaver):
|
||||
for v, migration in zip(
|
||||
range(version + 1, len(self.MIGRATIONS)),
|
||||
self.MIGRATIONS[version + 1 :],
|
||||
strict=False,
|
||||
):
|
||||
cur.execute(migration)
|
||||
cur.execute("INSERT INTO checkpoint_migrations (v) VALUES (%s)", (v,))
|
||||
cur.execute(f"INSERT INTO checkpoint_migrations (v) VALUES ({v})")
|
||||
if self.pipe:
|
||||
self.pipe.sync()
|
||||
|
||||
def list(
|
||||
self,
|
||||
config: RunnableConfig | None,
|
||||
config: Optional[CheckpointConfig],
|
||||
*,
|
||||
filter: dict[str, Any] | None = None,
|
||||
before: RunnableConfig | None = None,
|
||||
limit: int | None = None,
|
||||
filter: Optional[dict[str, Any]] = None,
|
||||
before: Optional[CheckpointConfig] = None,
|
||||
limit: Optional[int] = None,
|
||||
) -> Iterator[CheckpointTuple]:
|
||||
"""List checkpoints from the database.
|
||||
|
||||
@@ -115,13 +108,13 @@ class PostgresSaver(BasePostgresSaver):
|
||||
on the provided config. The checkpoints are ordered by checkpoint ID in descending order (newest first).
|
||||
|
||||
Args:
|
||||
config: The config to use for listing the checkpoints.
|
||||
filter: Additional filtering criteria for metadata.
|
||||
before: If provided, only checkpoints before the specified checkpoint ID are returned.
|
||||
limit: The maximum number of checkpoints to return.
|
||||
config (CheckpointConfig): The config to use for listing the checkpoints.
|
||||
filter (Optional[Dict[str, Any]]): Additional filtering criteria for metadata. Defaults to None.
|
||||
before (Optional[CheckpointConfig]): If provided, only checkpoints before the specified checkpoint ID are returned. Defaults to None.
|
||||
limit (Optional[int]): The maximum number of checkpoints to return. Defaults to None.
|
||||
|
||||
Yields:
|
||||
An iterator of checkpoint tuples.
|
||||
Iterator[CheckpointTuple]: An iterator of checkpoint tuples.
|
||||
|
||||
Examples:
|
||||
>>> from langgraph.checkpoint.postgres import PostgresSaver
|
||||
@@ -143,57 +136,53 @@ class PostgresSaver(BasePostgresSaver):
|
||||
"""
|
||||
where, args = self._search_where(config, filter, before)
|
||||
query = self.SELECT_SQL + where + " ORDER BY checkpoint_id DESC"
|
||||
params = list(args)
|
||||
if limit is not None:
|
||||
query += " LIMIT %s"
|
||||
params.append(int(limit))
|
||||
if limit:
|
||||
query += f" LIMIT {limit}"
|
||||
# if we change this to use .stream() we need to make sure to close the cursor
|
||||
with self._cursor() as cur:
|
||||
cur.execute(query, params)
|
||||
values = cur.fetchall()
|
||||
if not values:
|
||||
return
|
||||
# migrate pending sends if necessary
|
||||
if to_migrate := [
|
||||
v
|
||||
for v in values
|
||||
if v["checkpoint"]["v"] < 4 and v["parent_checkpoint_id"]
|
||||
]:
|
||||
cur.execute(
|
||||
self.SELECT_PENDING_SENDS_SQL,
|
||||
(
|
||||
values[0]["thread_id"],
|
||||
[v["parent_checkpoint_id"] for v in to_migrate],
|
||||
cur.execute(query, args, binary=True)
|
||||
for value in cur:
|
||||
yield CheckpointTuple(
|
||||
{
|
||||
"configurable": {
|
||||
"thread_id": value["thread_id"],
|
||||
"checkpoint_ns": value["checkpoint_ns"],
|
||||
"checkpoint_id": value["checkpoint_id"],
|
||||
}
|
||||
},
|
||||
self._load_checkpoint(
|
||||
value["checkpoint"],
|
||||
value["channel_values"],
|
||||
value["pending_sends"],
|
||||
),
|
||||
self._load_metadata(value["metadata"]),
|
||||
(
|
||||
{
|
||||
"configurable": {
|
||||
"thread_id": value["thread_id"],
|
||||
"checkpoint_ns": value["checkpoint_ns"],
|
||||
"checkpoint_id": value["parent_checkpoint_id"],
|
||||
}
|
||||
}
|
||||
if value["parent_checkpoint_id"]
|
||||
else None
|
||||
),
|
||||
self._load_writes(value["pending_writes"]),
|
||||
)
|
||||
grouped_by_parent = defaultdict(list)
|
||||
for value in to_migrate:
|
||||
grouped_by_parent[value["parent_checkpoint_id"]].append(value)
|
||||
for sends in cur:
|
||||
for value in grouped_by_parent[sends["checkpoint_id"]]:
|
||||
if value["channel_values"] is None:
|
||||
value["channel_values"] = []
|
||||
self._migrate_pending_sends(
|
||||
sends["sends"],
|
||||
value["checkpoint"],
|
||||
value["channel_values"],
|
||||
)
|
||||
for value in values:
|
||||
yield self._load_checkpoint_tuple(value)
|
||||
|
||||
def get_tuple(self, config: RunnableConfig) -> CheckpointTuple | None:
|
||||
def get_tuple(self, config: CheckpointConfig) -> Optional[CheckpointTuple]:
|
||||
"""Get a checkpoint tuple from the database.
|
||||
|
||||
This method retrieves a checkpoint tuple from the Postgres database based on the
|
||||
provided config. If the config contains a `checkpoint_id` key, the checkpoint with
|
||||
provided config. If the config contains a "checkpoint_id" key, the checkpoint with
|
||||
the matching thread ID and timestamp is retrieved. Otherwise, the latest checkpoint
|
||||
for the given thread ID is retrieved.
|
||||
|
||||
Args:
|
||||
config: The config to use for retrieving the checkpoint.
|
||||
config (CheckpointConfig): The config to use for retrieving the checkpoint.
|
||||
|
||||
Returns:
|
||||
The retrieved checkpoint tuple, or None if no matching checkpoint was found.
|
||||
Optional[CheckpointTuple]: The retrieved checkpoint tuple, or None if no matching checkpoint was found.
|
||||
|
||||
Examples:
|
||||
|
||||
@@ -230,48 +219,58 @@ class PostgresSaver(BasePostgresSaver):
|
||||
cur.execute(
|
||||
self.SELECT_SQL + where,
|
||||
args,
|
||||
binary=True,
|
||||
)
|
||||
value = cur.fetchone()
|
||||
if value is None:
|
||||
return None
|
||||
|
||||
# migrate pending sends if necessary
|
||||
if value["checkpoint"]["v"] < 4 and value["parent_checkpoint_id"]:
|
||||
cur.execute(
|
||||
self.SELECT_PENDING_SENDS_SQL,
|
||||
(thread_id, [value["parent_checkpoint_id"]]),
|
||||
)
|
||||
if sends := cur.fetchone():
|
||||
if value["channel_values"] is None:
|
||||
value["channel_values"] = []
|
||||
self._migrate_pending_sends(
|
||||
sends["sends"],
|
||||
for value in cur:
|
||||
return CheckpointTuple(
|
||||
{
|
||||
"configurable": {
|
||||
"thread_id": thread_id,
|
||||
"checkpoint_ns": checkpoint_ns,
|
||||
"checkpoint_id": value["checkpoint_id"],
|
||||
}
|
||||
},
|
||||
self._load_checkpoint(
|
||||
value["checkpoint"],
|
||||
value["channel_values"],
|
||||
)
|
||||
|
||||
return self._load_checkpoint_tuple(value)
|
||||
value["pending_sends"],
|
||||
),
|
||||
self._load_metadata(value["metadata"]),
|
||||
(
|
||||
{
|
||||
"configurable": {
|
||||
"thread_id": thread_id,
|
||||
"checkpoint_ns": checkpoint_ns,
|
||||
"checkpoint_id": value["parent_checkpoint_id"],
|
||||
}
|
||||
}
|
||||
if value["parent_checkpoint_id"]
|
||||
else None
|
||||
),
|
||||
self._load_writes(value["pending_writes"]),
|
||||
)
|
||||
|
||||
def put(
|
||||
self,
|
||||
config: RunnableConfig,
|
||||
config: CheckpointConfig,
|
||||
checkpoint: Checkpoint,
|
||||
metadata: CheckpointMetadata,
|
||||
new_versions: ChannelVersions,
|
||||
) -> RunnableConfig:
|
||||
) -> CheckpointConfig:
|
||||
"""Save a checkpoint to the database.
|
||||
|
||||
This method saves a checkpoint to the Postgres database. The checkpoint is associated
|
||||
with the provided config and its parent config (if any).
|
||||
|
||||
Args:
|
||||
config: The config to associate with the checkpoint.
|
||||
checkpoint: The checkpoint to save.
|
||||
metadata: Additional metadata to save with the checkpoint.
|
||||
new_versions: New channel versions as of this write.
|
||||
config (CheckpointConfig): The config to associate with the checkpoint.
|
||||
checkpoint (Checkpoint): The checkpoint to save.
|
||||
metadata (CheckpointMetadata): Additional metadata to save with the checkpoint.
|
||||
new_versions (ChannelVersions): New channel versions as of this write.
|
||||
|
||||
Returns:
|
||||
RunnableConfig: Updated configuration after storing the checkpoint.
|
||||
CheckpointConfig: Updated configuration after storing the checkpoint.
|
||||
|
||||
Examples:
|
||||
|
||||
@@ -287,9 +286,11 @@ class PostgresSaver(BasePostgresSaver):
|
||||
configurable = config["configurable"].copy()
|
||||
thread_id = configurable.pop("thread_id")
|
||||
checkpoint_ns = configurable.pop("checkpoint_ns")
|
||||
checkpoint_id = configurable.pop("checkpoint_id", None)
|
||||
checkpoint_id = configurable.pop(
|
||||
"checkpoint_id", configurable.pop("thread_ts", None)
|
||||
)
|
||||
|
||||
copy = checkpoint.copy()
|
||||
copy["channel_values"] = copy["channel_values"].copy()
|
||||
next_config = {
|
||||
"configurable": {
|
||||
"thread_id": thread_id,
|
||||
@@ -298,28 +299,16 @@ class PostgresSaver(BasePostgresSaver):
|
||||
}
|
||||
}
|
||||
|
||||
# inline primitive values in checkpoint table
|
||||
# others are stored in blobs table
|
||||
blob_values = {}
|
||||
for k, v in checkpoint["channel_values"].items():
|
||||
if v is None or isinstance(v, (str, int, float, bool)):
|
||||
pass
|
||||
else:
|
||||
blob_values[k] = copy["channel_values"].pop(k)
|
||||
|
||||
with self._cursor(pipeline=True) as cur:
|
||||
if blob_versions := {
|
||||
k: v for k, v in new_versions.items() if k in blob_values
|
||||
}:
|
||||
cur.executemany(
|
||||
self.UPSERT_CHECKPOINT_BLOBS_SQL,
|
||||
self._dump_blobs(
|
||||
thread_id,
|
||||
checkpoint_ns,
|
||||
blob_values,
|
||||
blob_versions,
|
||||
),
|
||||
)
|
||||
cur.executemany(
|
||||
self.UPSERT_CHECKPOINT_BLOBS_SQL,
|
||||
self._dump_blobs(
|
||||
thread_id,
|
||||
checkpoint_ns,
|
||||
copy.pop("channel_values"), # type: ignore[misc]
|
||||
new_versions,
|
||||
),
|
||||
)
|
||||
cur.execute(
|
||||
self.UPSERT_CHECKPOINTS_SQL,
|
||||
(
|
||||
@@ -327,15 +316,15 @@ class PostgresSaver(BasePostgresSaver):
|
||||
checkpoint_ns,
|
||||
checkpoint["id"],
|
||||
checkpoint_id,
|
||||
Jsonb(copy),
|
||||
Jsonb(get_serializable_checkpoint_metadata(config, metadata)),
|
||||
Jsonb(self._dump_checkpoint(copy)),
|
||||
self._dump_metadata(get_checkpoint_metadata(config, metadata)),
|
||||
),
|
||||
)
|
||||
return next_config
|
||||
|
||||
def put_writes(
|
||||
self,
|
||||
config: RunnableConfig,
|
||||
config: CheckpointConfig,
|
||||
writes: Sequence[tuple[str, Any]],
|
||||
task_id: str,
|
||||
task_path: str = "",
|
||||
@@ -345,9 +334,9 @@ class PostgresSaver(BasePostgresSaver):
|
||||
This method saves intermediate writes associated with a checkpoint to the Postgres database.
|
||||
|
||||
Args:
|
||||
config: Configuration of the related checkpoint.
|
||||
writes: List of writes to store.
|
||||
task_id: Identifier for the task creating the writes.
|
||||
config (CheckpointConfig): Configuration of the related checkpoint.
|
||||
writes (List[Tuple[str, Any]]): List of writes to store.
|
||||
task_id (str): Identifier for the task creating the writes.
|
||||
"""
|
||||
query = (
|
||||
self.UPSERT_CHECKPOINT_WRITES_SQL
|
||||
@@ -367,39 +356,16 @@ class PostgresSaver(BasePostgresSaver):
|
||||
),
|
||||
)
|
||||
|
||||
def delete_thread(self, thread_id: str) -> None:
|
||||
"""Delete all checkpoints and writes associated with a thread ID.
|
||||
|
||||
Args:
|
||||
thread_id: The thread ID to delete.
|
||||
|
||||
Returns:
|
||||
None
|
||||
"""
|
||||
with self._cursor(pipeline=True) as cur:
|
||||
cur.execute(
|
||||
"DELETE FROM checkpoints WHERE thread_id = %s",
|
||||
(str(thread_id),),
|
||||
)
|
||||
cur.execute(
|
||||
"DELETE FROM checkpoint_blobs WHERE thread_id = %s",
|
||||
(str(thread_id),),
|
||||
)
|
||||
cur.execute(
|
||||
"DELETE FROM checkpoint_writes WHERE thread_id = %s",
|
||||
(str(thread_id),),
|
||||
)
|
||||
|
||||
@contextmanager
|
||||
def _cursor(self, *, pipeline: bool = False) -> Iterator[Cursor[DictRow]]:
|
||||
"""Create a database cursor as a context manager.
|
||||
|
||||
Args:
|
||||
pipeline: whether to use pipeline for the DB operations inside the context manager.
|
||||
pipeline (bool): whether to use pipeline for the DB operations inside the context manager.
|
||||
Will be applied regardless of whether the PostgresSaver instance was initialized with a pipeline.
|
||||
If pipeline mode is not supported, will fall back to using transaction context manager.
|
||||
"""
|
||||
with self.lock, _internal.get_connection(self.conn) as conn:
|
||||
with _internal.get_connection(self.conn) as conn:
|
||||
if self.pipe:
|
||||
# a connection in pipeline mode can be used concurrently
|
||||
# in multiple threads/coroutines, but only one cursor can be
|
||||
@@ -415,6 +381,7 @@ class PostgresSaver(BasePostgresSaver):
|
||||
# thread/coroutine at a time, so we acquire a lock
|
||||
if self.supports_pipeline:
|
||||
with (
|
||||
self.lock,
|
||||
conn.pipeline(),
|
||||
conn.cursor(binary=True, row_factory=dict_row) as cur,
|
||||
):
|
||||
@@ -422,55 +389,14 @@ class PostgresSaver(BasePostgresSaver):
|
||||
else:
|
||||
# Use connection's transaction context manager when pipeline mode not supported
|
||||
with (
|
||||
self.lock,
|
||||
conn.transaction(),
|
||||
conn.cursor(binary=True, row_factory=dict_row) as cur,
|
||||
):
|
||||
yield cur
|
||||
else:
|
||||
with conn.cursor(binary=True, row_factory=dict_row) as cur:
|
||||
with self.lock, conn.cursor(binary=True, row_factory=dict_row) as cur:
|
||||
yield cur
|
||||
|
||||
def _load_checkpoint_tuple(self, value: DictRow) -> CheckpointTuple:
|
||||
"""
|
||||
Convert a database row into a CheckpointTuple object.
|
||||
|
||||
Args:
|
||||
value: A row from the database containing checkpoint data.
|
||||
|
||||
Returns:
|
||||
CheckpointTuple: A structured representation of the checkpoint,
|
||||
including its configuration, metadata, parent checkpoint (if any),
|
||||
and pending writes.
|
||||
"""
|
||||
return CheckpointTuple(
|
||||
{
|
||||
"configurable": {
|
||||
"thread_id": value["thread_id"],
|
||||
"checkpoint_ns": value["checkpoint_ns"],
|
||||
"checkpoint_id": value["checkpoint_id"],
|
||||
}
|
||||
},
|
||||
{
|
||||
**value["checkpoint"],
|
||||
"channel_values": {
|
||||
**(value["checkpoint"].get("channel_values") or {}),
|
||||
**self._load_blobs(value["channel_values"]),
|
||||
},
|
||||
},
|
||||
value["metadata"],
|
||||
(
|
||||
{
|
||||
"configurable": {
|
||||
"thread_id": value["thread_id"],
|
||||
"checkpoint_ns": value["checkpoint_ns"],
|
||||
"checkpoint_id": value["parent_checkpoint_id"],
|
||||
}
|
||||
}
|
||||
if value["parent_checkpoint_id"]
|
||||
else None
|
||||
),
|
||||
self._load_writes(value["pending_writes"]),
|
||||
)
|
||||
|
||||
|
||||
__all__ = ["PostgresSaver", "BasePostgresSaver", "ShallowPostgresSaver", "Conn"]
|
||||
__all__ = ["PostgresSaver", "BasePostgresSaver", "Conn"]
|
||||
@@ -1,23 +0,0 @@
|
||||
"""Shared async utility functions for the Postgres checkpoint & storage classes."""
|
||||
|
||||
from collections.abc import AsyncIterator
|
||||
from contextlib import asynccontextmanager
|
||||
|
||||
from psycopg import AsyncConnection
|
||||
from psycopg.rows import DictRow
|
||||
from psycopg_pool import AsyncConnectionPool
|
||||
|
||||
Conn = AsyncConnection[DictRow] | AsyncConnectionPool[AsyncConnection[DictRow]]
|
||||
|
||||
|
||||
@asynccontextmanager
|
||||
async def get_connection(
|
||||
conn: Conn,
|
||||
) -> AsyncIterator[AsyncConnection[DictRow]]:
|
||||
if isinstance(conn, AsyncConnection):
|
||||
yield conn
|
||||
elif isinstance(conn, AsyncConnectionPool):
|
||||
async with conn.connection() as conn:
|
||||
yield conn
|
||||
else:
|
||||
raise TypeError(f"Invalid connection type: {type(conn)}")
|
||||
@@ -2,12 +2,13 @@
|
||||
|
||||
from collections.abc import Iterator
|
||||
from contextlib import contextmanager
|
||||
from typing import Union
|
||||
|
||||
from psycopg import Connection
|
||||
from psycopg.rows import DictRow
|
||||
from psycopg_pool import ConnectionPool
|
||||
|
||||
Conn = Connection[DictRow] | ConnectionPool[Connection[DictRow]]
|
||||
Conn = Union[Connection[DictRow], ConnectionPool[Connection[DictRow]]]
|
||||
|
||||
|
||||
@contextmanager
|
||||
|
||||
@@ -1,582 +0,0 @@
|
||||
from __future__ import annotations
|
||||
|
||||
import asyncio
|
||||
from collections import defaultdict
|
||||
from collections.abc import AsyncIterator, Iterator, Sequence
|
||||
from contextlib import asynccontextmanager
|
||||
from typing import Any
|
||||
|
||||
from langchain_core.runnables import RunnableConfig
|
||||
from langgraph.checkpoint.base import (
|
||||
WRITES_IDX_MAP,
|
||||
ChannelVersions,
|
||||
Checkpoint,
|
||||
CheckpointMetadata,
|
||||
CheckpointTuple,
|
||||
get_checkpoint_id,
|
||||
get_serializable_checkpoint_metadata,
|
||||
)
|
||||
from langgraph.checkpoint.serde.base import SerializerProtocol
|
||||
from psycopg import AsyncConnection, AsyncCursor, AsyncPipeline, Capabilities
|
||||
from psycopg.rows import DictRow, dict_row
|
||||
from psycopg.types.json import Jsonb
|
||||
from psycopg_pool import AsyncConnectionPool
|
||||
|
||||
from langgraph.checkpoint.postgres import _ainternal
|
||||
from langgraph.checkpoint.postgres.base import BasePostgresSaver
|
||||
from langgraph.checkpoint.postgres.shallow import AsyncShallowPostgresSaver
|
||||
|
||||
Conn = _ainternal.Conn # For backward compatibility
|
||||
|
||||
|
||||
class AsyncPostgresSaver(BasePostgresSaver):
|
||||
"""Asynchronous checkpointer that stores checkpoints in a Postgres database."""
|
||||
|
||||
lock: asyncio.Lock
|
||||
|
||||
def __init__(
|
||||
self,
|
||||
conn: _ainternal.Conn,
|
||||
pipe: AsyncPipeline | None = None,
|
||||
serde: SerializerProtocol | None = None,
|
||||
) -> None:
|
||||
super().__init__(serde=serde)
|
||||
if isinstance(conn, AsyncConnectionPool) and pipe is not None:
|
||||
raise ValueError(
|
||||
"Pipeline should be used only with a single AsyncConnection, not AsyncConnectionPool."
|
||||
)
|
||||
|
||||
self.conn = conn
|
||||
self.pipe = pipe
|
||||
self.lock = asyncio.Lock()
|
||||
self.loop = asyncio.get_running_loop()
|
||||
self.supports_pipeline = Capabilities().has_pipeline()
|
||||
|
||||
@classmethod
|
||||
@asynccontextmanager
|
||||
async def from_conn_string(
|
||||
cls,
|
||||
conn_string: str,
|
||||
*,
|
||||
pipeline: bool = False,
|
||||
serde: SerializerProtocol | None = None,
|
||||
) -> AsyncIterator[AsyncPostgresSaver]:
|
||||
"""Create a new AsyncPostgresSaver instance from a connection string.
|
||||
|
||||
Args:
|
||||
conn_string: The Postgres connection info string.
|
||||
pipeline: whether to use AsyncPipeline
|
||||
|
||||
Returns:
|
||||
AsyncPostgresSaver: A new AsyncPostgresSaver instance.
|
||||
"""
|
||||
async with await AsyncConnection.connect(
|
||||
conn_string, autocommit=True, prepare_threshold=0, row_factory=dict_row
|
||||
) as conn:
|
||||
if pipeline:
|
||||
async with conn.pipeline() as pipe:
|
||||
yield cls(conn=conn, pipe=pipe, serde=serde)
|
||||
else:
|
||||
yield cls(conn=conn, serde=serde)
|
||||
|
||||
async def setup(self) -> None:
|
||||
"""Set up the checkpoint database asynchronously.
|
||||
|
||||
This method creates the necessary tables in the Postgres database if they don't
|
||||
already exist and runs database migrations. It MUST be called directly by the user
|
||||
the first time checkpointer is used.
|
||||
"""
|
||||
async with self._cursor() as cur:
|
||||
await cur.execute(self.MIGRATIONS[0])
|
||||
results = await cur.execute(
|
||||
"SELECT v FROM checkpoint_migrations ORDER BY v DESC LIMIT 1"
|
||||
)
|
||||
row = await results.fetchone()
|
||||
if row is None:
|
||||
version = -1
|
||||
else:
|
||||
version = row["v"]
|
||||
for v, migration in zip(
|
||||
range(version + 1, len(self.MIGRATIONS)),
|
||||
self.MIGRATIONS[version + 1 :],
|
||||
strict=False,
|
||||
):
|
||||
await cur.execute(migration)
|
||||
await cur.execute(
|
||||
"INSERT INTO checkpoint_migrations (v) VALUES (%s)", (v,)
|
||||
)
|
||||
if self.pipe:
|
||||
await self.pipe.sync()
|
||||
|
||||
async def alist(
|
||||
self,
|
||||
config: RunnableConfig | None,
|
||||
*,
|
||||
filter: dict[str, Any] | None = None,
|
||||
before: RunnableConfig | None = None,
|
||||
limit: int | None = None,
|
||||
) -> AsyncIterator[CheckpointTuple]:
|
||||
"""List checkpoints from the database asynchronously.
|
||||
|
||||
This method retrieves a list of checkpoint tuples from the Postgres database based
|
||||
on the provided config. The checkpoints are ordered by checkpoint ID in descending order (newest first).
|
||||
|
||||
Args:
|
||||
config: Base configuration for filtering checkpoints.
|
||||
filter: Additional filtering criteria for metadata.
|
||||
before: If provided, only checkpoints before the specified checkpoint ID are returned.
|
||||
limit: Maximum number of checkpoints to return.
|
||||
|
||||
Yields:
|
||||
An asynchronous iterator of matching checkpoint tuples.
|
||||
"""
|
||||
where, args = self._search_where(config, filter, before)
|
||||
query = self.SELECT_SQL + where + " ORDER BY checkpoint_id DESC"
|
||||
params = list(args)
|
||||
if limit is not None:
|
||||
query += " LIMIT %s"
|
||||
params.append(int(limit))
|
||||
# if we change this to use .stream() we need to make sure to close the cursor
|
||||
async with self._cursor() as cur:
|
||||
await cur.execute(query, params, binary=True)
|
||||
values = await cur.fetchall()
|
||||
if not values:
|
||||
return
|
||||
# migrate pending sends if necessary
|
||||
if to_migrate := [
|
||||
v
|
||||
for v in values
|
||||
if v["checkpoint"]["v"] < 4 and v["parent_checkpoint_id"]
|
||||
]:
|
||||
await cur.execute(
|
||||
self.SELECT_PENDING_SENDS_SQL,
|
||||
(
|
||||
values[0]["thread_id"],
|
||||
[v["parent_checkpoint_id"] for v in to_migrate],
|
||||
),
|
||||
)
|
||||
grouped_by_parent = defaultdict(list)
|
||||
for value in to_migrate:
|
||||
grouped_by_parent[value["parent_checkpoint_id"]].append(value)
|
||||
async for sends in cur:
|
||||
for value in grouped_by_parent[sends["checkpoint_id"]]:
|
||||
if value["channel_values"] is None:
|
||||
value["channel_values"] = []
|
||||
self._migrate_pending_sends(
|
||||
sends["sends"],
|
||||
value["checkpoint"],
|
||||
value["channel_values"],
|
||||
)
|
||||
for value in values:
|
||||
yield await self._load_checkpoint_tuple(value)
|
||||
|
||||
async def aget_tuple(self, config: RunnableConfig) -> CheckpointTuple | None:
|
||||
"""Get a checkpoint tuple from the database asynchronously.
|
||||
|
||||
This method retrieves a checkpoint tuple from the Postgres database based on the
|
||||
provided config. If the config contains a `checkpoint_id` key, the checkpoint with
|
||||
the matching thread ID and "checkpoint_id" is retrieved. Otherwise, the latest checkpoint
|
||||
for the given thread ID is retrieved.
|
||||
|
||||
Args:
|
||||
config: The config to use for retrieving the checkpoint.
|
||||
|
||||
Returns:
|
||||
The retrieved checkpoint tuple, or None if no matching checkpoint was found.
|
||||
"""
|
||||
thread_id = config["configurable"]["thread_id"]
|
||||
checkpoint_id = get_checkpoint_id(config)
|
||||
checkpoint_ns = config["configurable"].get("checkpoint_ns", "")
|
||||
if checkpoint_id:
|
||||
args: tuple[Any, ...] = (thread_id, checkpoint_ns, checkpoint_id)
|
||||
where = "WHERE thread_id = %s AND checkpoint_ns = %s AND checkpoint_id = %s"
|
||||
else:
|
||||
args = (thread_id, checkpoint_ns)
|
||||
where = "WHERE thread_id = %s AND checkpoint_ns = %s ORDER BY checkpoint_id DESC LIMIT 1"
|
||||
|
||||
async with self._cursor() as cur:
|
||||
await cur.execute(
|
||||
self.SELECT_SQL + where,
|
||||
args,
|
||||
binary=True,
|
||||
)
|
||||
value = await cur.fetchone()
|
||||
if value is None:
|
||||
return None
|
||||
|
||||
# migrate pending sends if necessary
|
||||
if value["checkpoint"]["v"] < 4 and value["parent_checkpoint_id"]:
|
||||
await cur.execute(
|
||||
self.SELECT_PENDING_SENDS_SQL,
|
||||
(thread_id, [value["parent_checkpoint_id"]]),
|
||||
)
|
||||
if sends := await cur.fetchone():
|
||||
if value["channel_values"] is None:
|
||||
value["channel_values"] = []
|
||||
self._migrate_pending_sends(
|
||||
sends["sends"],
|
||||
value["checkpoint"],
|
||||
value["channel_values"],
|
||||
)
|
||||
|
||||
return await self._load_checkpoint_tuple(value)
|
||||
|
||||
async def aput(
|
||||
self,
|
||||
config: RunnableConfig,
|
||||
checkpoint: Checkpoint,
|
||||
metadata: CheckpointMetadata,
|
||||
new_versions: ChannelVersions,
|
||||
) -> RunnableConfig:
|
||||
"""Save a checkpoint to the database asynchronously.
|
||||
|
||||
This method saves a checkpoint to the Postgres database. The checkpoint is associated
|
||||
with the provided config and its parent config (if any).
|
||||
|
||||
Args:
|
||||
config: The config to associate with the checkpoint.
|
||||
checkpoint: The checkpoint to save.
|
||||
metadata: Additional metadata to save with the checkpoint.
|
||||
new_versions: New channel versions as of this write.
|
||||
|
||||
Returns:
|
||||
RunnableConfig: Updated configuration after storing the checkpoint.
|
||||
"""
|
||||
configurable = config["configurable"].copy()
|
||||
thread_id = configurable.pop("thread_id")
|
||||
checkpoint_ns = configurable.pop("checkpoint_ns")
|
||||
checkpoint_id = configurable.pop("checkpoint_id", None)
|
||||
|
||||
copy = checkpoint.copy()
|
||||
copy["channel_values"] = copy["channel_values"].copy()
|
||||
next_config = {
|
||||
"configurable": {
|
||||
"thread_id": thread_id,
|
||||
"checkpoint_ns": checkpoint_ns,
|
||||
"checkpoint_id": checkpoint["id"],
|
||||
}
|
||||
}
|
||||
|
||||
# inline primitive values in checkpoint table
|
||||
# others are stored in blobs table
|
||||
blob_values = {}
|
||||
for k, v in checkpoint["channel_values"].items():
|
||||
if v is None or isinstance(v, (str, int, float, bool)):
|
||||
pass
|
||||
else:
|
||||
blob_values[k] = copy["channel_values"].pop(k)
|
||||
|
||||
async with self._cursor(pipeline=True) as cur:
|
||||
if blob_versions := {
|
||||
k: v for k, v in new_versions.items() if k in blob_values
|
||||
}:
|
||||
await cur.executemany(
|
||||
self.UPSERT_CHECKPOINT_BLOBS_SQL,
|
||||
await asyncio.to_thread(
|
||||
self._dump_blobs,
|
||||
thread_id,
|
||||
checkpoint_ns,
|
||||
blob_values,
|
||||
blob_versions,
|
||||
),
|
||||
)
|
||||
await cur.execute(
|
||||
self.UPSERT_CHECKPOINTS_SQL,
|
||||
(
|
||||
thread_id,
|
||||
checkpoint_ns,
|
||||
checkpoint["id"],
|
||||
checkpoint_id,
|
||||
Jsonb(copy),
|
||||
Jsonb(get_serializable_checkpoint_metadata(config, metadata)),
|
||||
),
|
||||
)
|
||||
return next_config
|
||||
|
||||
async def aput_writes(
|
||||
self,
|
||||
config: RunnableConfig,
|
||||
writes: Sequence[tuple[str, Any]],
|
||||
task_id: str,
|
||||
task_path: str = "",
|
||||
) -> None:
|
||||
"""Store intermediate writes linked to a checkpoint asynchronously.
|
||||
|
||||
This method saves intermediate writes associated with a checkpoint to the database.
|
||||
|
||||
Args:
|
||||
config: Configuration of the related checkpoint.
|
||||
writes: List of writes to store, each as (channel, value) pair.
|
||||
task_id: Identifier for the task creating the writes.
|
||||
"""
|
||||
query = (
|
||||
self.UPSERT_CHECKPOINT_WRITES_SQL
|
||||
if all(w[0] in WRITES_IDX_MAP for w in writes)
|
||||
else self.INSERT_CHECKPOINT_WRITES_SQL
|
||||
)
|
||||
params = await asyncio.to_thread(
|
||||
self._dump_writes,
|
||||
config["configurable"]["thread_id"],
|
||||
config["configurable"]["checkpoint_ns"],
|
||||
config["configurable"]["checkpoint_id"],
|
||||
task_id,
|
||||
task_path,
|
||||
writes,
|
||||
)
|
||||
async with self._cursor(pipeline=True) as cur:
|
||||
await cur.executemany(query, params)
|
||||
|
||||
async def adelete_thread(self, thread_id: str) -> None:
|
||||
"""Delete all checkpoints and writes associated with a thread ID.
|
||||
|
||||
Args:
|
||||
thread_id: The thread ID to delete.
|
||||
|
||||
Returns:
|
||||
None
|
||||
"""
|
||||
async with self._cursor(pipeline=True) as cur:
|
||||
await cur.execute(
|
||||
"DELETE FROM checkpoints WHERE thread_id = %s",
|
||||
(str(thread_id),),
|
||||
)
|
||||
await cur.execute(
|
||||
"DELETE FROM checkpoint_blobs WHERE thread_id = %s",
|
||||
(str(thread_id),),
|
||||
)
|
||||
await cur.execute(
|
||||
"DELETE FROM checkpoint_writes WHERE thread_id = %s",
|
||||
(str(thread_id),),
|
||||
)
|
||||
|
||||
@asynccontextmanager
|
||||
async def _cursor(
|
||||
self, *, pipeline: bool = False
|
||||
) -> AsyncIterator[AsyncCursor[DictRow]]:
|
||||
"""Create a database cursor as a context manager.
|
||||
|
||||
Args:
|
||||
pipeline: whether to use pipeline for the DB operations inside the context manager.
|
||||
Will be applied regardless of whether the AsyncPostgresSaver instance was initialized with a pipeline.
|
||||
If pipeline mode is not supported, will fall back to using transaction context manager.
|
||||
"""
|
||||
async with self.lock, _ainternal.get_connection(self.conn) as conn:
|
||||
if self.pipe:
|
||||
# a connection in pipeline mode can be used concurrently
|
||||
# in multiple threads/coroutines, but only one cursor can be
|
||||
# used at a time
|
||||
try:
|
||||
async with conn.cursor(binary=True, row_factory=dict_row) as cur:
|
||||
yield cur
|
||||
finally:
|
||||
if pipeline:
|
||||
await self.pipe.sync()
|
||||
elif pipeline:
|
||||
# a connection not in pipeline mode can only be used by one
|
||||
# thread/coroutine at a time, so we acquire a lock
|
||||
if self.supports_pipeline:
|
||||
async with (
|
||||
conn.pipeline(),
|
||||
conn.cursor(binary=True, row_factory=dict_row) as cur,
|
||||
):
|
||||
yield cur
|
||||
else:
|
||||
# Use connection's transaction context manager when pipeline mode not supported
|
||||
async with (
|
||||
conn.transaction(),
|
||||
conn.cursor(binary=True, row_factory=dict_row) as cur,
|
||||
):
|
||||
yield cur
|
||||
else:
|
||||
async with conn.cursor(binary=True, row_factory=dict_row) as cur:
|
||||
yield cur
|
||||
|
||||
async def _load_checkpoint_tuple(self, value: DictRow) -> CheckpointTuple:
|
||||
"""
|
||||
Convert a database row into a CheckpointTuple object.
|
||||
|
||||
Args:
|
||||
value: A row from the database containing checkpoint data.
|
||||
|
||||
Returns:
|
||||
CheckpointTuple: A structured representation of the checkpoint,
|
||||
including its configuration, metadata, parent checkpoint (if any),
|
||||
and pending writes.
|
||||
"""
|
||||
return CheckpointTuple(
|
||||
{
|
||||
"configurable": {
|
||||
"thread_id": value["thread_id"],
|
||||
"checkpoint_ns": value["checkpoint_ns"],
|
||||
"checkpoint_id": value["checkpoint_id"],
|
||||
}
|
||||
},
|
||||
{
|
||||
**value["checkpoint"],
|
||||
"channel_values": {
|
||||
**(value["checkpoint"].get("channel_values") or {}),
|
||||
**self._load_blobs(value["channel_values"]),
|
||||
},
|
||||
},
|
||||
value["metadata"],
|
||||
(
|
||||
{
|
||||
"configurable": {
|
||||
"thread_id": value["thread_id"],
|
||||
"checkpoint_ns": value["checkpoint_ns"],
|
||||
"checkpoint_id": value["parent_checkpoint_id"],
|
||||
}
|
||||
}
|
||||
if value["parent_checkpoint_id"]
|
||||
else None
|
||||
),
|
||||
await asyncio.to_thread(self._load_writes, value["pending_writes"]),
|
||||
)
|
||||
|
||||
def list(
|
||||
self,
|
||||
config: RunnableConfig | None,
|
||||
*,
|
||||
filter: dict[str, Any] | None = None,
|
||||
before: RunnableConfig | None = None,
|
||||
limit: int | None = None,
|
||||
) -> Iterator[CheckpointTuple]:
|
||||
"""List checkpoints from the database.
|
||||
|
||||
This method retrieves a list of checkpoint tuples from the Postgres database based
|
||||
on the provided config. The checkpoints are ordered by checkpoint ID in descending order (newest first).
|
||||
|
||||
Args:
|
||||
config: Base configuration for filtering checkpoints.
|
||||
filter: Additional filtering criteria for metadata.
|
||||
before: If provided, only checkpoints before the specified checkpoint ID are returned.
|
||||
limit: Maximum number of checkpoints to return.
|
||||
|
||||
Yields:
|
||||
An iterator of matching checkpoint tuples.
|
||||
"""
|
||||
try:
|
||||
# check if we are in the main thread, only bg threads can block
|
||||
# we don't check in other methods to avoid the overhead
|
||||
if asyncio.get_running_loop() is self.loop:
|
||||
raise asyncio.InvalidStateError(
|
||||
"Synchronous calls to AsyncPostgresSaver are only allowed from a "
|
||||
"different thread. From the main thread, use the async interface. "
|
||||
"For example, use `checkpointer.alist(...)` or `await "
|
||||
"graph.ainvoke(...)`."
|
||||
)
|
||||
except RuntimeError:
|
||||
pass
|
||||
aiter_ = self.alist(config, filter=filter, before=before, limit=limit)
|
||||
while True:
|
||||
try:
|
||||
yield asyncio.run_coroutine_threadsafe(
|
||||
anext(aiter_), # type: ignore[arg-type] # noqa: F821
|
||||
self.loop,
|
||||
).result()
|
||||
except StopAsyncIteration:
|
||||
break
|
||||
|
||||
def get_tuple(self, config: RunnableConfig) -> CheckpointTuple | None:
|
||||
"""Get a checkpoint tuple from the database.
|
||||
|
||||
This method retrieves a checkpoint tuple from the Postgres database based on the
|
||||
provided config. If the config contains a `checkpoint_id` key, the checkpoint with
|
||||
the matching thread ID and "checkpoint_id" is retrieved. Otherwise, the latest checkpoint
|
||||
for the given thread ID is retrieved.
|
||||
|
||||
Args:
|
||||
config: The config to use for retrieving the checkpoint.
|
||||
|
||||
Returns:
|
||||
The retrieved checkpoint tuple, or None if no matching checkpoint was found.
|
||||
"""
|
||||
try:
|
||||
# check if we are in the main thread, only bg threads can block
|
||||
# we don't check in other methods to avoid the overhead
|
||||
if asyncio.get_running_loop() is self.loop:
|
||||
raise asyncio.InvalidStateError(
|
||||
"Synchronous calls to AsyncPostgresSaver are only allowed from a "
|
||||
"different thread. From the main thread, use the async interface. "
|
||||
"For example, use `await checkpointer.aget_tuple(...)` or `await "
|
||||
"graph.ainvoke(...)`."
|
||||
)
|
||||
except RuntimeError:
|
||||
pass
|
||||
return asyncio.run_coroutine_threadsafe(
|
||||
self.aget_tuple(config), self.loop
|
||||
).result()
|
||||
|
||||
def put(
|
||||
self,
|
||||
config: RunnableConfig,
|
||||
checkpoint: Checkpoint,
|
||||
metadata: CheckpointMetadata,
|
||||
new_versions: ChannelVersions,
|
||||
) -> RunnableConfig:
|
||||
"""Save a checkpoint to the database.
|
||||
|
||||
This method saves a checkpoint to the Postgres database. The checkpoint is associated
|
||||
with the provided config and its parent config (if any).
|
||||
|
||||
Args:
|
||||
config: The config to associate with the checkpoint.
|
||||
checkpoint: The checkpoint to save.
|
||||
metadata: Additional metadata to save with the checkpoint.
|
||||
new_versions: New channel versions as of this write.
|
||||
|
||||
Returns:
|
||||
RunnableConfig: Updated configuration after storing the checkpoint.
|
||||
"""
|
||||
return asyncio.run_coroutine_threadsafe(
|
||||
self.aput(config, checkpoint, metadata, new_versions), self.loop
|
||||
).result()
|
||||
|
||||
def put_writes(
|
||||
self,
|
||||
config: RunnableConfig,
|
||||
writes: Sequence[tuple[str, Any]],
|
||||
task_id: str,
|
||||
task_path: str = "",
|
||||
) -> None:
|
||||
"""Store intermediate writes linked to a checkpoint.
|
||||
|
||||
This method saves intermediate writes associated with a checkpoint to the database.
|
||||
|
||||
Args:
|
||||
config: Configuration of the related checkpoint.
|
||||
writes: List of writes to store, each as (channel, value) pair.
|
||||
task_id: Identifier for the task creating the writes.
|
||||
task_path: Path of the task creating the writes.
|
||||
"""
|
||||
return asyncio.run_coroutine_threadsafe(
|
||||
self.aput_writes(config, writes, task_id, task_path), self.loop
|
||||
).result()
|
||||
|
||||
def delete_thread(self, thread_id: str) -> None:
|
||||
"""Delete all checkpoints and writes associated with a thread ID.
|
||||
|
||||
Args:
|
||||
thread_id: The thread ID to delete.
|
||||
|
||||
Returns:
|
||||
None
|
||||
"""
|
||||
try:
|
||||
# check if we are in the main thread, only bg threads can block
|
||||
# we don't check in other methods to avoid the overhead
|
||||
if asyncio.get_running_loop() is self.loop:
|
||||
raise asyncio.InvalidStateError(
|
||||
"Synchronous calls to AsyncPostgresSaver are only allowed from a "
|
||||
"different thread. From the main thread, use the async interface. "
|
||||
"For example, use `await checkpointer.aget_tuple(...)` or `await "
|
||||
"graph.ainvoke(...)`."
|
||||
)
|
||||
except RuntimeError:
|
||||
pass
|
||||
return asyncio.run_coroutine_threadsafe(
|
||||
self.adelete_thread(thread_id), self.loop
|
||||
).result()
|
||||
|
||||
|
||||
__all__ = ["AsyncPostgresSaver", "AsyncShallowPostgresSaver", "Conn"]
|
||||
@@ -1,34 +1,22 @@
|
||||
from __future__ import annotations
|
||||
|
||||
import random
|
||||
import warnings
|
||||
from collections.abc import Sequence
|
||||
from importlib.metadata import version as get_version
|
||||
from typing import Any, cast
|
||||
from typing import Any, Optional, cast
|
||||
|
||||
from psycopg.types.json import Jsonb
|
||||
|
||||
from langchain_core.runnables import RunnableConfig
|
||||
from langgraph.checkpoint.base import (
|
||||
WRITES_IDX_MAP,
|
||||
BaseCheckpointSaver,
|
||||
ChannelVersions,
|
||||
Checkpoint,
|
||||
CheckpointConfig,
|
||||
CheckpointMetadata,
|
||||
get_checkpoint_id,
|
||||
)
|
||||
from langgraph.checkpoint.serde.types import TASKS
|
||||
from psycopg.types.json import Jsonb
|
||||
from langgraph.checkpoint.serde.jsonplus import JsonPlusSerializer
|
||||
from langgraph.checkpoint.serde.types import TASKS, ChannelProtocol
|
||||
|
||||
MetadataInput = dict[str, Any] | None
|
||||
|
||||
try:
|
||||
major, minor = get_version("langgraph").split(".")[:2]
|
||||
if int(major) == 0 and int(minor) < 5:
|
||||
warnings.warn(
|
||||
"You're using incompatible versions of langgraph and checkpoint-postgres. Please upgrade langgraph to avoid unexpected behavior.",
|
||||
DeprecationWarning,
|
||||
stacklevel=2,
|
||||
)
|
||||
except Exception:
|
||||
# skip version check if running from source
|
||||
pass
|
||||
MetadataInput = Optional[dict[str, Any]]
|
||||
|
||||
"""
|
||||
To add a new migration, add a new string to the MIGRATIONS list.
|
||||
@@ -81,10 +69,10 @@ MIGRATIONS = [
|
||||
"""
|
||||
CREATE INDEX CONCURRENTLY IF NOT EXISTS checkpoint_writes_thread_id_idx ON checkpoint_writes(thread_id);
|
||||
""",
|
||||
"""ALTER TABLE checkpoint_writes ADD COLUMN IF NOT EXISTS task_path TEXT NOT NULL DEFAULT '';""",
|
||||
"""ALTER TABLE checkpoint_writes ADD COLUMN task_path TEXT NOT NULL DEFAULT '';""",
|
||||
]
|
||||
|
||||
SELECT_SQL = """
|
||||
SELECT_SQL = f"""
|
||||
select
|
||||
thread_id,
|
||||
checkpoint,
|
||||
@@ -108,20 +96,17 @@ select
|
||||
where cw.thread_id = checkpoints.thread_id
|
||||
and cw.checkpoint_ns = checkpoints.checkpoint_ns
|
||||
and cw.checkpoint_id = checkpoints.checkpoint_id
|
||||
) as pending_writes
|
||||
) as pending_writes,
|
||||
(
|
||||
select array_agg(array[cw.type::bytea, cw.blob] order by cw.task_path, cw.task_id, cw.idx)
|
||||
from checkpoint_writes cw
|
||||
where cw.thread_id = checkpoints.thread_id
|
||||
and cw.checkpoint_ns = checkpoints.checkpoint_ns
|
||||
and cw.checkpoint_id = checkpoints.parent_checkpoint_id
|
||||
and cw.channel = '{TASKS}'
|
||||
) as pending_sends
|
||||
from checkpoints """
|
||||
|
||||
SELECT_PENDING_SENDS_SQL = f"""
|
||||
select
|
||||
checkpoint_id,
|
||||
array_agg(array[type::bytea, blob] order by task_path, task_id, idx) as sends
|
||||
from checkpoint_writes
|
||||
where thread_id = %s
|
||||
and checkpoint_id = any(%s)
|
||||
and channel = '{TASKS}'
|
||||
group by checkpoint_id
|
||||
"""
|
||||
|
||||
UPSERT_CHECKPOINT_BLOBS_SQL = """
|
||||
INSERT INTO checkpoint_blobs (thread_id, checkpoint_ns, channel, version, type, blob)
|
||||
VALUES (%s, %s, %s, %s, %s, %s)
|
||||
@@ -155,34 +140,31 @@ INSERT_CHECKPOINT_WRITES_SQL = """
|
||||
|
||||
class BasePostgresSaver(BaseCheckpointSaver[str]):
|
||||
SELECT_SQL = SELECT_SQL
|
||||
SELECT_PENDING_SENDS_SQL = SELECT_PENDING_SENDS_SQL
|
||||
MIGRATIONS = MIGRATIONS
|
||||
UPSERT_CHECKPOINT_BLOBS_SQL = UPSERT_CHECKPOINT_BLOBS_SQL
|
||||
UPSERT_CHECKPOINTS_SQL = UPSERT_CHECKPOINTS_SQL
|
||||
UPSERT_CHECKPOINT_WRITES_SQL = UPSERT_CHECKPOINT_WRITES_SQL
|
||||
INSERT_CHECKPOINT_WRITES_SQL = INSERT_CHECKPOINT_WRITES_SQL
|
||||
|
||||
jsonplus_serde = JsonPlusSerializer()
|
||||
supports_pipeline: bool
|
||||
|
||||
def _migrate_pending_sends(
|
||||
def _load_checkpoint(
|
||||
self,
|
||||
pending_sends: list[tuple[bytes, bytes]],
|
||||
checkpoint: dict[str, Any],
|
||||
channel_values: list[tuple[bytes, bytes, bytes]],
|
||||
) -> None:
|
||||
if not pending_sends:
|
||||
return
|
||||
# add to values
|
||||
enc, blob = self.serde.dumps_typed(
|
||||
[self.serde.loads_typed((c.decode(), b)) for c, b in pending_sends],
|
||||
)
|
||||
channel_values.append((TASKS.encode(), enc.encode(), blob))
|
||||
# add to versions
|
||||
checkpoint["channel_versions"][TASKS] = (
|
||||
max(checkpoint["channel_versions"].values())
|
||||
if checkpoint["channel_versions"]
|
||||
else self.get_next_version(None, None)
|
||||
)
|
||||
pending_sends: list[tuple[bytes, bytes]],
|
||||
) -> Checkpoint:
|
||||
return {
|
||||
**checkpoint,
|
||||
"pending_sends": [
|
||||
self.serde.loads_typed((c.decode(), b)) for c, b in pending_sends or []
|
||||
],
|
||||
"channel_values": self._load_blobs(channel_values),
|
||||
}
|
||||
|
||||
def _dump_checkpoint(self, checkpoint: Checkpoint) -> dict[str, Any]:
|
||||
return {**checkpoint, "pending_sends": []}
|
||||
|
||||
def _load_blobs(
|
||||
self, blob_values: list[tuple[bytes, bytes, bytes]]
|
||||
@@ -201,7 +183,7 @@ class BasePostgresSaver(BaseCheckpointSaver[str]):
|
||||
checkpoint_ns: str,
|
||||
values: dict[str, Any],
|
||||
versions: ChannelVersions,
|
||||
) -> list[tuple[str, str, str, str, str, bytes | None]]:
|
||||
) -> list[tuple[str, str, str, str, str, Optional[bytes]]]:
|
||||
if not versions:
|
||||
return []
|
||||
|
||||
@@ -259,7 +241,15 @@ class BasePostgresSaver(BaseCheckpointSaver[str]):
|
||||
for idx, (channel, value) in enumerate(writes)
|
||||
]
|
||||
|
||||
def get_next_version(self, current: str | None, channel: None) -> str:
|
||||
def _load_metadata(self, metadata: dict[str, Any]) -> CheckpointMetadata:
|
||||
return self.jsonplus_serde.loads(self.jsonplus_serde.dumps(metadata))
|
||||
|
||||
def _dump_metadata(self, metadata: CheckpointMetadata) -> str:
|
||||
serialized_metadata = self.jsonplus_serde.dumps(metadata)
|
||||
# NOTE: we're using JSON serializer (not msgpack), so we need to remove null characters before writing
|
||||
return serialized_metadata.decode().replace("\\u0000", "")
|
||||
|
||||
def get_next_version(self, current: Optional[str], channel: ChannelProtocol) -> str:
|
||||
if current is None:
|
||||
current_v = 0
|
||||
elif isinstance(current, int):
|
||||
@@ -272,9 +262,9 @@ class BasePostgresSaver(BaseCheckpointSaver[str]):
|
||||
|
||||
def _search_where(
|
||||
self,
|
||||
config: RunnableConfig | None,
|
||||
config: Optional[CheckpointConfig],
|
||||
filter: MetadataInput,
|
||||
before: RunnableConfig | None = None,
|
||||
before: Optional[CheckpointConfig] = None,
|
||||
) -> tuple[str, list[Any]]:
|
||||
"""Return WHERE clause predicates for alist() given config, filter, before.
|
||||
|
||||
|
||||
@@ -1,967 +0,0 @@
|
||||
import asyncio
|
||||
import threading
|
||||
import warnings
|
||||
from collections.abc import AsyncIterator, Iterator, Sequence
|
||||
from contextlib import asynccontextmanager, contextmanager
|
||||
from typing import Any
|
||||
|
||||
from langchain_core.runnables import RunnableConfig
|
||||
from langgraph.checkpoint.base import (
|
||||
WRITES_IDX_MAP,
|
||||
ChannelVersions,
|
||||
Checkpoint,
|
||||
CheckpointMetadata,
|
||||
CheckpointTuple,
|
||||
get_serializable_checkpoint_metadata,
|
||||
)
|
||||
from langgraph.checkpoint.serde.base import SerializerProtocol
|
||||
from langgraph.checkpoint.serde.types import TASKS
|
||||
from psycopg import (
|
||||
AsyncConnection,
|
||||
AsyncCursor,
|
||||
AsyncPipeline,
|
||||
Capabilities,
|
||||
Connection,
|
||||
Cursor,
|
||||
Pipeline,
|
||||
)
|
||||
from psycopg.rows import DictRow, dict_row
|
||||
from psycopg.types.json import Jsonb
|
||||
from psycopg_pool import AsyncConnectionPool, ConnectionPool
|
||||
|
||||
from langgraph.checkpoint.postgres import _ainternal, _internal
|
||||
from langgraph.checkpoint.postgres.base import BasePostgresSaver
|
||||
|
||||
"""
|
||||
To add a new migration, add a new string to the MIGRATIONS list.
|
||||
The position of the migration in the list is the version number.
|
||||
"""
|
||||
MIGRATIONS = [
|
||||
"""CREATE TABLE IF NOT EXISTS checkpoint_migrations (
|
||||
v INTEGER PRIMARY KEY
|
||||
);""",
|
||||
"""CREATE TABLE IF NOT EXISTS checkpoints (
|
||||
thread_id TEXT NOT NULL,
|
||||
checkpoint_ns TEXT NOT NULL DEFAULT '',
|
||||
type TEXT,
|
||||
checkpoint JSONB NOT NULL,
|
||||
metadata JSONB NOT NULL DEFAULT '{}',
|
||||
PRIMARY KEY (thread_id, checkpoint_ns)
|
||||
);""",
|
||||
"""CREATE TABLE IF NOT EXISTS checkpoint_blobs (
|
||||
thread_id TEXT NOT NULL,
|
||||
checkpoint_ns TEXT NOT NULL DEFAULT '',
|
||||
channel TEXT NOT NULL,
|
||||
type TEXT NOT NULL,
|
||||
blob BYTEA,
|
||||
PRIMARY KEY (thread_id, checkpoint_ns, channel)
|
||||
);""",
|
||||
"""CREATE TABLE IF NOT EXISTS checkpoint_writes (
|
||||
thread_id TEXT NOT NULL,
|
||||
checkpoint_ns TEXT NOT NULL DEFAULT '',
|
||||
checkpoint_id TEXT NOT NULL,
|
||||
task_id TEXT NOT NULL,
|
||||
idx INTEGER NOT NULL,
|
||||
channel TEXT NOT NULL,
|
||||
type TEXT,
|
||||
blob BYTEA NOT NULL,
|
||||
PRIMARY KEY (thread_id, checkpoint_ns, checkpoint_id, task_id, idx)
|
||||
);""",
|
||||
"""
|
||||
CREATE INDEX CONCURRENTLY IF NOT EXISTS checkpoints_thread_id_idx ON checkpoints(thread_id);
|
||||
""",
|
||||
"""
|
||||
CREATE INDEX CONCURRENTLY IF NOT EXISTS checkpoint_blobs_thread_id_idx ON checkpoint_blobs(thread_id);
|
||||
""",
|
||||
"""
|
||||
CREATE INDEX CONCURRENTLY IF NOT EXISTS checkpoint_writes_thread_id_idx ON checkpoint_writes(thread_id);
|
||||
""",
|
||||
"""
|
||||
ALTER TABLE checkpoint_writes ADD COLUMN IF NOT EXISTS task_path TEXT NOT NULL DEFAULT '';
|
||||
""",
|
||||
]
|
||||
|
||||
SELECT_SQL = f"""
|
||||
select
|
||||
thread_id,
|
||||
checkpoint,
|
||||
checkpoint_ns,
|
||||
metadata,
|
||||
(
|
||||
select array_agg(array[bl.channel::bytea, bl.type::bytea, bl.blob])
|
||||
from jsonb_each_text(checkpoint -> 'channel_versions')
|
||||
inner join checkpoint_blobs bl
|
||||
on bl.thread_id = checkpoints.thread_id
|
||||
and bl.checkpoint_ns = checkpoints.checkpoint_ns
|
||||
and bl.channel = jsonb_each_text.key
|
||||
) as channel_values,
|
||||
(
|
||||
select
|
||||
array_agg(array[cw.task_id::text::bytea, cw.channel::bytea, cw.type::bytea, cw.blob] order by cw.task_id, cw.idx)
|
||||
from checkpoint_writes cw
|
||||
where cw.thread_id = checkpoints.thread_id
|
||||
and cw.checkpoint_ns = checkpoints.checkpoint_ns
|
||||
and cw.checkpoint_id = (checkpoint->>'id')
|
||||
) as pending_writes,
|
||||
(
|
||||
select array_agg(array[cw.type::bytea, cw.blob] order by cw.task_path, cw.task_id, cw.idx)
|
||||
from checkpoint_writes cw
|
||||
where cw.thread_id = checkpoints.thread_id
|
||||
and cw.checkpoint_ns = checkpoints.checkpoint_ns
|
||||
and cw.channel = '{TASKS}'
|
||||
) as pending_sends
|
||||
from checkpoints """
|
||||
|
||||
UPSERT_CHECKPOINT_BLOBS_SQL = """
|
||||
INSERT INTO checkpoint_blobs (thread_id, checkpoint_ns, channel, type, blob)
|
||||
VALUES (%s, %s, %s, %s, %s)
|
||||
ON CONFLICT (thread_id, checkpoint_ns, channel) DO UPDATE SET
|
||||
type = EXCLUDED.type,
|
||||
blob = EXCLUDED.blob;
|
||||
"""
|
||||
|
||||
UPSERT_CHECKPOINTS_SQL = """
|
||||
INSERT INTO checkpoints (thread_id, checkpoint_ns, checkpoint, metadata)
|
||||
VALUES (%s, %s, %s, %s)
|
||||
ON CONFLICT (thread_id, checkpoint_ns)
|
||||
DO UPDATE SET
|
||||
checkpoint = EXCLUDED.checkpoint,
|
||||
metadata = EXCLUDED.metadata;
|
||||
"""
|
||||
|
||||
UPSERT_CHECKPOINT_WRITES_SQL = """
|
||||
INSERT INTO checkpoint_writes (thread_id, checkpoint_ns, checkpoint_id, task_id, task_path, idx, channel, type, blob)
|
||||
VALUES (%s, %s, %s, %s, %s, %s, %s, %s, %s)
|
||||
ON CONFLICT (thread_id, checkpoint_ns, checkpoint_id, task_id, idx) DO UPDATE SET
|
||||
channel = EXCLUDED.channel,
|
||||
type = EXCLUDED.type,
|
||||
blob = EXCLUDED.blob;
|
||||
"""
|
||||
|
||||
INSERT_CHECKPOINT_WRITES_SQL = """
|
||||
INSERT INTO checkpoint_writes (thread_id, checkpoint_ns, checkpoint_id, task_id, task_path, idx, channel, type, blob)
|
||||
VALUES (%s, %s, %s, %s, %s, %s, %s, %s, %s)
|
||||
ON CONFLICT (thread_id, checkpoint_ns, checkpoint_id, task_id, idx) DO NOTHING
|
||||
"""
|
||||
|
||||
|
||||
def _dump_blobs(
|
||||
serde: SerializerProtocol,
|
||||
thread_id: str,
|
||||
checkpoint_ns: str,
|
||||
values: dict[str, Any],
|
||||
versions: ChannelVersions,
|
||||
) -> list[tuple[str, str, str, str, bytes | None]]:
|
||||
if not versions:
|
||||
return []
|
||||
|
||||
return [
|
||||
(
|
||||
thread_id,
|
||||
checkpoint_ns,
|
||||
k,
|
||||
*(serde.dumps_typed(values[k]) if k in values else ("empty", None)),
|
||||
)
|
||||
for k in versions
|
||||
]
|
||||
|
||||
|
||||
class ShallowPostgresSaver(BasePostgresSaver):
|
||||
"""A checkpoint saver that uses Postgres to store checkpoints.
|
||||
|
||||
This checkpointer ONLY stores the most recent checkpoint and does NOT retain any history.
|
||||
It is meant to be a light-weight drop-in replacement for the PostgresSaver that
|
||||
supports most of the LangGraph persistence functionality with the exception of time travel.
|
||||
"""
|
||||
|
||||
SELECT_SQL = SELECT_SQL
|
||||
MIGRATIONS = MIGRATIONS
|
||||
UPSERT_CHECKPOINT_BLOBS_SQL = UPSERT_CHECKPOINT_BLOBS_SQL
|
||||
UPSERT_CHECKPOINTS_SQL = UPSERT_CHECKPOINTS_SQL
|
||||
UPSERT_CHECKPOINT_WRITES_SQL = UPSERT_CHECKPOINT_WRITES_SQL
|
||||
INSERT_CHECKPOINT_WRITES_SQL = INSERT_CHECKPOINT_WRITES_SQL
|
||||
|
||||
lock: threading.Lock
|
||||
|
||||
def __init__(
|
||||
self,
|
||||
conn: _internal.Conn,
|
||||
pipe: Pipeline | None = None,
|
||||
serde: SerializerProtocol | None = None,
|
||||
) -> None:
|
||||
warnings.warn(
|
||||
"ShallowPostgresSaver is deprecated as of version 2.0.20 and will be removed in 3.0.0. "
|
||||
"Use PostgresSaver instead, and invoke the graph with `graph.invoke(..., durability='exit')`.",
|
||||
DeprecationWarning,
|
||||
stacklevel=2,
|
||||
)
|
||||
super().__init__(serde=serde)
|
||||
if isinstance(conn, ConnectionPool) and pipe is not None:
|
||||
raise ValueError(
|
||||
"Pipeline should be used only with a single Connection, not ConnectionPool."
|
||||
)
|
||||
|
||||
self.conn = conn
|
||||
self.pipe = pipe
|
||||
self.lock = threading.Lock()
|
||||
self.supports_pipeline = Capabilities().has_pipeline()
|
||||
|
||||
@classmethod
|
||||
@contextmanager
|
||||
def from_conn_string(
|
||||
cls, conn_string: str, *, pipeline: bool = False
|
||||
) -> Iterator["ShallowPostgresSaver"]:
|
||||
"""Create a new ShallowPostgresSaver instance from a connection string.
|
||||
|
||||
Args:
|
||||
conn_string: The Postgres connection info string.
|
||||
pipeline: whether to use Pipeline
|
||||
|
||||
Returns:
|
||||
ShallowPostgresSaver: A new ShallowPostgresSaver instance.
|
||||
"""
|
||||
with Connection.connect(
|
||||
conn_string, autocommit=True, prepare_threshold=0, row_factory=dict_row
|
||||
) as conn:
|
||||
if pipeline:
|
||||
with conn.pipeline() as pipe:
|
||||
yield cls(conn, pipe)
|
||||
else:
|
||||
yield cls(conn)
|
||||
|
||||
def setup(self) -> None:
|
||||
"""Set up the checkpoint database asynchronously.
|
||||
|
||||
This method creates the necessary tables in the Postgres database if they don't
|
||||
already exist and runs database migrations. It MUST be called directly by the user
|
||||
the first time checkpointer is used.
|
||||
"""
|
||||
with self._cursor() as cur:
|
||||
cur.execute(self.MIGRATIONS[0])
|
||||
results = cur.execute(
|
||||
"SELECT v FROM checkpoint_migrations ORDER BY v DESC LIMIT 1"
|
||||
)
|
||||
row = results.fetchone()
|
||||
if row is None:
|
||||
version = -1
|
||||
else:
|
||||
version = row["v"]
|
||||
for v, migration in zip(
|
||||
range(version + 1, len(self.MIGRATIONS)),
|
||||
self.MIGRATIONS[version + 1 :],
|
||||
strict=False,
|
||||
):
|
||||
cur.execute(migration)
|
||||
cur.execute("INSERT INTO checkpoint_migrations (v) VALUES (%s)", (v,))
|
||||
if self.pipe:
|
||||
self.pipe.sync()
|
||||
|
||||
def list(
|
||||
self,
|
||||
config: RunnableConfig | None,
|
||||
*,
|
||||
filter: dict[str, Any] | None = None,
|
||||
before: RunnableConfig | None = None,
|
||||
limit: int | None = None,
|
||||
) -> Iterator[CheckpointTuple]:
|
||||
"""List checkpoints from the database.
|
||||
|
||||
This method retrieves a list of checkpoint tuples from the Postgres database based
|
||||
on the provided config. For ShallowPostgresSaver, this method returns a list with
|
||||
ONLY the most recent checkpoint.
|
||||
"""
|
||||
where, args = self._search_where(config, filter, before)
|
||||
query = self.SELECT_SQL + where
|
||||
params = list(args)
|
||||
if limit is not None:
|
||||
query += " LIMIT %s"
|
||||
params.append(int(limit))
|
||||
with self._cursor() as cur:
|
||||
cur.execute(query, params, binary=True)
|
||||
for value in cur:
|
||||
checkpoint: Checkpoint = {
|
||||
**value["checkpoint"],
|
||||
"channel_values": self._load_blobs(value["channel_values"]),
|
||||
"pending_sends": [
|
||||
self.serde.loads_typed((t.decode(), v))
|
||||
for t, v in value["pending_sends"]
|
||||
]
|
||||
if value["pending_sends"]
|
||||
else [],
|
||||
}
|
||||
yield CheckpointTuple(
|
||||
config={
|
||||
"configurable": {
|
||||
"thread_id": value["thread_id"],
|
||||
"checkpoint_ns": value["checkpoint_ns"],
|
||||
"checkpoint_id": checkpoint["id"],
|
||||
}
|
||||
},
|
||||
checkpoint=checkpoint,
|
||||
metadata=value["metadata"],
|
||||
pending_writes=self._load_writes(value["pending_writes"]),
|
||||
)
|
||||
|
||||
def get_tuple(self, config: RunnableConfig) -> CheckpointTuple | None:
|
||||
"""Get a checkpoint tuple from the database.
|
||||
|
||||
This method retrieves a checkpoint tuple from the Postgres database based on the
|
||||
provided config (matching the thread ID in the config).
|
||||
|
||||
Args:
|
||||
config: The config to use for retrieving the checkpoint.
|
||||
|
||||
Returns:
|
||||
The retrieved checkpoint tuple, or None if no matching checkpoint was found.
|
||||
|
||||
Examples:
|
||||
|
||||
Basic:
|
||||
>>> config = {"configurable": {"thread_id": "1"}}
|
||||
>>> checkpoint_tuple = memory.get_tuple(config)
|
||||
>>> print(checkpoint_tuple)
|
||||
CheckpointTuple(...)
|
||||
|
||||
With timestamp:
|
||||
|
||||
>>> config = {
|
||||
... "configurable": {
|
||||
... "thread_id": "1",
|
||||
... "checkpoint_ns": "",
|
||||
... "checkpoint_id": "1ef4f797-8335-6428-8001-8a1503f9b875",
|
||||
... }
|
||||
... }
|
||||
>>> checkpoint_tuple = memory.get_tuple(config)
|
||||
>>> print(checkpoint_tuple)
|
||||
CheckpointTuple(...)
|
||||
""" # noqa
|
||||
thread_id = config["configurable"]["thread_id"]
|
||||
checkpoint_ns = config["configurable"].get("checkpoint_ns", "")
|
||||
args = (thread_id, checkpoint_ns)
|
||||
where = "WHERE thread_id = %s AND checkpoint_ns = %s"
|
||||
|
||||
with self._cursor() as cur:
|
||||
cur.execute(
|
||||
self.SELECT_SQL + where,
|
||||
args,
|
||||
binary=True,
|
||||
)
|
||||
|
||||
for value in cur:
|
||||
checkpoint: Checkpoint = {
|
||||
**value["checkpoint"],
|
||||
"channel_values": self._load_blobs(value["channel_values"]),
|
||||
"pending_sends": [
|
||||
self.serde.loads_typed((t.decode(), v))
|
||||
for t, v in value["pending_sends"]
|
||||
]
|
||||
if value["pending_sends"]
|
||||
else [],
|
||||
}
|
||||
return CheckpointTuple(
|
||||
config={
|
||||
"configurable": {
|
||||
"thread_id": thread_id,
|
||||
"checkpoint_ns": checkpoint_ns,
|
||||
"checkpoint_id": checkpoint["id"],
|
||||
}
|
||||
},
|
||||
checkpoint=checkpoint,
|
||||
metadata=value["metadata"],
|
||||
pending_writes=self._load_writes(value["pending_writes"]),
|
||||
)
|
||||
|
||||
def put(
|
||||
self,
|
||||
config: RunnableConfig,
|
||||
checkpoint: Checkpoint,
|
||||
metadata: CheckpointMetadata,
|
||||
new_versions: ChannelVersions,
|
||||
) -> RunnableConfig:
|
||||
"""Save a checkpoint to the database.
|
||||
|
||||
This method saves a checkpoint to the Postgres database. The checkpoint is associated
|
||||
with the provided config. For ShallowPostgresSaver, this method saves ONLY the most recent
|
||||
checkpoint and overwrites a previous checkpoint, if it exists.
|
||||
|
||||
Args:
|
||||
config: The config to associate with the checkpoint.
|
||||
checkpoint: The checkpoint to save.
|
||||
metadata: Additional metadata to save with the checkpoint.
|
||||
new_versions: New channel versions as of this write.
|
||||
|
||||
Returns:
|
||||
RunnableConfig: Updated configuration after storing the checkpoint.
|
||||
|
||||
Examples:
|
||||
|
||||
>>> from langgraph.checkpoint.postgres import ShallowPostgresSaver
|
||||
>>> DB_URI = "postgres://postgres:postgres@localhost:5432/postgres?sslmode=disable"
|
||||
>>> with ShallowPostgresSaver.from_conn_string(DB_URI) as memory:
|
||||
>>> config = {"configurable": {"thread_id": "1", "checkpoint_ns": ""}}
|
||||
>>> checkpoint = {"ts": "2024-05-04T06:32:42.235444+00:00", "id": "1ef4f797-8335-6428-8001-8a1503f9b875", "channel_values": {"key": "value"}}
|
||||
>>> saved_config = memory.put(config, checkpoint, {"source": "input", "step": 1, "writes": {"key": "value"}}, {})
|
||||
>>> print(saved_config)
|
||||
{'configurable': {'thread_id': '1', 'checkpoint_ns': '', 'checkpoint_id': '1ef4f797-8335-6428-8001-8a1503f9b875'}}
|
||||
"""
|
||||
configurable = config["configurable"].copy()
|
||||
thread_id = configurable.pop("thread_id")
|
||||
checkpoint_ns = configurable.pop("checkpoint_ns")
|
||||
|
||||
copy = checkpoint.copy()
|
||||
next_config = {
|
||||
"configurable": {
|
||||
"thread_id": thread_id,
|
||||
"checkpoint_ns": checkpoint_ns,
|
||||
"checkpoint_id": checkpoint["id"],
|
||||
}
|
||||
}
|
||||
|
||||
with self._cursor(pipeline=True) as cur:
|
||||
cur.execute(
|
||||
"""DELETE FROM checkpoint_writes
|
||||
WHERE thread_id = %s AND checkpoint_ns = %s AND checkpoint_id NOT IN (%s, %s)""",
|
||||
(
|
||||
thread_id,
|
||||
checkpoint_ns,
|
||||
checkpoint["id"],
|
||||
configurable.get("checkpoint_id", ""),
|
||||
),
|
||||
)
|
||||
cur.executemany(
|
||||
self.UPSERT_CHECKPOINT_BLOBS_SQL,
|
||||
_dump_blobs(
|
||||
self.serde,
|
||||
thread_id,
|
||||
checkpoint_ns,
|
||||
copy.pop("channel_values"), # type: ignore[misc]
|
||||
new_versions,
|
||||
),
|
||||
)
|
||||
cur.execute(
|
||||
self.UPSERT_CHECKPOINTS_SQL,
|
||||
(
|
||||
thread_id,
|
||||
checkpoint_ns,
|
||||
Jsonb(copy),
|
||||
Jsonb(get_serializable_checkpoint_metadata(config, metadata)),
|
||||
),
|
||||
)
|
||||
return next_config
|
||||
|
||||
def put_writes(
|
||||
self,
|
||||
config: RunnableConfig,
|
||||
writes: Sequence[tuple[str, Any]],
|
||||
task_id: str,
|
||||
task_path: str = "",
|
||||
) -> None:
|
||||
"""Store intermediate writes linked to a checkpoint.
|
||||
|
||||
This method saves intermediate writes associated with a checkpoint to the Postgres database.
|
||||
|
||||
Args:
|
||||
config: Configuration of the related checkpoint.
|
||||
writes: List of writes to store.
|
||||
task_id: Identifier for the task creating the writes.
|
||||
"""
|
||||
query = (
|
||||
self.UPSERT_CHECKPOINT_WRITES_SQL
|
||||
if all(w[0] in WRITES_IDX_MAP for w in writes)
|
||||
else self.INSERT_CHECKPOINT_WRITES_SQL
|
||||
)
|
||||
with self._cursor(pipeline=True) as cur:
|
||||
cur.executemany(
|
||||
query,
|
||||
self._dump_writes(
|
||||
config["configurable"]["thread_id"],
|
||||
config["configurable"]["checkpoint_ns"],
|
||||
config["configurable"]["checkpoint_id"],
|
||||
task_id,
|
||||
task_path,
|
||||
writes,
|
||||
),
|
||||
)
|
||||
|
||||
@contextmanager
|
||||
def _cursor(self, *, pipeline: bool = False) -> Iterator[Cursor[DictRow]]:
|
||||
"""Create a database cursor as a context manager.
|
||||
|
||||
Args:
|
||||
pipeline: whether to use pipeline for the DB operations inside the context manager.
|
||||
Will be applied regardless of whether the ShallowPostgresSaver instance was initialized with a pipeline.
|
||||
If pipeline mode is not supported, will fall back to using transaction context manager.
|
||||
"""
|
||||
with _internal.get_connection(self.conn) as conn:
|
||||
if self.pipe:
|
||||
# a connection in pipeline mode can be used concurrently
|
||||
# in multiple threads/coroutines, but only one cursor can be
|
||||
# used at a time
|
||||
try:
|
||||
with conn.cursor(binary=True, row_factory=dict_row) as cur:
|
||||
yield cur
|
||||
finally:
|
||||
if pipeline:
|
||||
self.pipe.sync()
|
||||
elif pipeline:
|
||||
# a connection not in pipeline mode can only be used by one
|
||||
# thread/coroutine at a time, so we acquire a lock
|
||||
if self.supports_pipeline:
|
||||
with (
|
||||
self.lock,
|
||||
conn.pipeline(),
|
||||
conn.cursor(binary=True, row_factory=dict_row) as cur,
|
||||
):
|
||||
yield cur
|
||||
else:
|
||||
# Use connection's transaction context manager when pipeline mode not supported
|
||||
with (
|
||||
self.lock,
|
||||
conn.transaction(),
|
||||
conn.cursor(binary=True, row_factory=dict_row) as cur,
|
||||
):
|
||||
yield cur
|
||||
else:
|
||||
with self.lock, conn.cursor(binary=True, row_factory=dict_row) as cur:
|
||||
yield cur
|
||||
|
||||
|
||||
class AsyncShallowPostgresSaver(BasePostgresSaver):
|
||||
"""A checkpoint saver that uses Postgres to store checkpoints asynchronously.
|
||||
|
||||
This checkpointer ONLY stores the most recent checkpoint and does NOT retain any history.
|
||||
It is meant to be a light-weight drop-in replacement for the AsyncPostgresSaver that
|
||||
supports most of the LangGraph persistence functionality with the exception of time travel.
|
||||
"""
|
||||
|
||||
SELECT_SQL = SELECT_SQL
|
||||
MIGRATIONS = MIGRATIONS
|
||||
UPSERT_CHECKPOINT_BLOBS_SQL = UPSERT_CHECKPOINT_BLOBS_SQL
|
||||
UPSERT_CHECKPOINTS_SQL = UPSERT_CHECKPOINTS_SQL
|
||||
UPSERT_CHECKPOINT_WRITES_SQL = UPSERT_CHECKPOINT_WRITES_SQL
|
||||
INSERT_CHECKPOINT_WRITES_SQL = INSERT_CHECKPOINT_WRITES_SQL
|
||||
lock: asyncio.Lock
|
||||
|
||||
def __init__(
|
||||
self,
|
||||
conn: _ainternal.Conn,
|
||||
pipe: AsyncPipeline | None = None,
|
||||
serde: SerializerProtocol | None = None,
|
||||
) -> None:
|
||||
warnings.warn(
|
||||
"AsyncShallowPostgresSaver is deprecated as of version 2.0.20 and will be removed in 3.0.0. "
|
||||
"Use AsyncPostgresSaver instead, and invoke the graph with `await graph.ainvoke(..., durability='exit')`.",
|
||||
DeprecationWarning,
|
||||
stacklevel=2,
|
||||
)
|
||||
super().__init__(serde=serde)
|
||||
if isinstance(conn, AsyncConnectionPool) and pipe is not None:
|
||||
raise ValueError(
|
||||
"Pipeline should be used only with a single AsyncConnection, not AsyncConnectionPool."
|
||||
)
|
||||
|
||||
self.conn = conn
|
||||
self.pipe = pipe
|
||||
self.lock = asyncio.Lock()
|
||||
self.loop = asyncio.get_running_loop()
|
||||
self.supports_pipeline = Capabilities().has_pipeline()
|
||||
|
||||
@classmethod
|
||||
@asynccontextmanager
|
||||
async def from_conn_string(
|
||||
cls,
|
||||
conn_string: str,
|
||||
*,
|
||||
pipeline: bool = False,
|
||||
serde: SerializerProtocol | None = None,
|
||||
) -> AsyncIterator["AsyncShallowPostgresSaver"]:
|
||||
"""Create a new AsyncShallowPostgresSaver instance from a connection string.
|
||||
|
||||
Args:
|
||||
conn_string: The Postgres connection info string.
|
||||
pipeline: whether to use AsyncPipeline
|
||||
|
||||
Returns:
|
||||
AsyncShallowPostgresSaver: A new AsyncShallowPostgresSaver instance.
|
||||
"""
|
||||
async with await AsyncConnection.connect(
|
||||
conn_string, autocommit=True, prepare_threshold=0, row_factory=dict_row
|
||||
) as conn:
|
||||
if pipeline:
|
||||
async with conn.pipeline() as pipe:
|
||||
yield cls(conn=conn, pipe=pipe, serde=serde)
|
||||
else:
|
||||
yield cls(conn=conn, serde=serde)
|
||||
|
||||
async def setup(self) -> None:
|
||||
"""Set up the checkpoint database asynchronously.
|
||||
|
||||
This method creates the necessary tables in the Postgres database if they don't
|
||||
already exist and runs database migrations. It MUST be called directly by the user
|
||||
the first time checkpointer is used.
|
||||
"""
|
||||
async with self._cursor() as cur:
|
||||
await cur.execute(self.MIGRATIONS[0])
|
||||
results = await cur.execute(
|
||||
"SELECT v FROM checkpoint_migrations ORDER BY v DESC LIMIT 1"
|
||||
)
|
||||
row = await results.fetchone()
|
||||
if row is None:
|
||||
version = -1
|
||||
else:
|
||||
version = row["v"]
|
||||
for v, migration in zip(
|
||||
range(version + 1, len(self.MIGRATIONS)),
|
||||
self.MIGRATIONS[version + 1 :],
|
||||
strict=False,
|
||||
):
|
||||
await cur.execute(migration)
|
||||
await cur.execute(
|
||||
"INSERT INTO checkpoint_migrations (v) VALUES (%s)", (v,)
|
||||
)
|
||||
if self.pipe:
|
||||
await self.pipe.sync()
|
||||
|
||||
async def alist(
|
||||
self,
|
||||
config: RunnableConfig | None,
|
||||
*,
|
||||
filter: dict[str, Any] | None = None,
|
||||
before: RunnableConfig | None = None,
|
||||
limit: int | None = None,
|
||||
) -> AsyncIterator[CheckpointTuple]:
|
||||
"""List checkpoints from the database asynchronously.
|
||||
|
||||
This method retrieves a list of checkpoint tuples from the Postgres database based
|
||||
on the provided config. For ShallowPostgresSaver, this method returns a list with
|
||||
ONLY the most recent checkpoint.
|
||||
"""
|
||||
where, args = self._search_where(config, filter, before)
|
||||
query = self.SELECT_SQL + where
|
||||
params = list(args)
|
||||
if limit is not None:
|
||||
query += " LIMIT %s"
|
||||
params.append(int(limit))
|
||||
async with self._cursor() as cur:
|
||||
await cur.execute(query, params, binary=True)
|
||||
async for value in cur:
|
||||
checkpoint: Checkpoint = {
|
||||
**value["checkpoint"],
|
||||
"channel_values": self._load_blobs(value["channel_values"]),
|
||||
"pending_sends": [
|
||||
self.serde.loads_typed((t.decode(), v))
|
||||
for t, v in value["pending_sends"]
|
||||
]
|
||||
if value["pending_sends"]
|
||||
else [],
|
||||
}
|
||||
yield CheckpointTuple(
|
||||
config={
|
||||
"configurable": {
|
||||
"thread_id": value["thread_id"],
|
||||
"checkpoint_ns": value["checkpoint_ns"],
|
||||
"checkpoint_id": checkpoint["id"],
|
||||
}
|
||||
},
|
||||
checkpoint=checkpoint,
|
||||
metadata=value["metadata"],
|
||||
pending_writes=await asyncio.to_thread(
|
||||
self._load_writes, value["pending_writes"]
|
||||
),
|
||||
)
|
||||
|
||||
async def aget_tuple(self, config: RunnableConfig) -> CheckpointTuple | None:
|
||||
"""Get a checkpoint tuple from the database asynchronously.
|
||||
|
||||
This method retrieves a checkpoint tuple from the Postgres database based on the
|
||||
provided config (matching the thread ID in the config).
|
||||
|
||||
Args:
|
||||
config: The config to use for retrieving the checkpoint.
|
||||
|
||||
Returns:
|
||||
The retrieved checkpoint tuple, or None if no matching checkpoint was found.
|
||||
"""
|
||||
thread_id = config["configurable"]["thread_id"]
|
||||
checkpoint_ns = config["configurable"].get("checkpoint_ns", "")
|
||||
args = (thread_id, checkpoint_ns)
|
||||
where = "WHERE thread_id = %s AND checkpoint_ns = %s"
|
||||
|
||||
async with self._cursor() as cur:
|
||||
await cur.execute(
|
||||
self.SELECT_SQL + where,
|
||||
args,
|
||||
binary=True,
|
||||
)
|
||||
|
||||
async for value in cur:
|
||||
checkpoint: Checkpoint = {
|
||||
**value["checkpoint"],
|
||||
"channel_values": self._load_blobs(value["channel_values"]),
|
||||
"pending_sends": [
|
||||
self.serde.loads_typed((t.decode(), v))
|
||||
for t, v in value["pending_sends"]
|
||||
]
|
||||
if value["pending_sends"]
|
||||
else [],
|
||||
}
|
||||
return CheckpointTuple(
|
||||
config={
|
||||
"configurable": {
|
||||
"thread_id": thread_id,
|
||||
"checkpoint_ns": checkpoint_ns,
|
||||
"checkpoint_id": checkpoint["id"],
|
||||
}
|
||||
},
|
||||
checkpoint=checkpoint,
|
||||
metadata=value["metadata"],
|
||||
pending_writes=await asyncio.to_thread(
|
||||
self._load_writes, value["pending_writes"]
|
||||
),
|
||||
)
|
||||
|
||||
async def aput(
|
||||
self,
|
||||
config: RunnableConfig,
|
||||
checkpoint: Checkpoint,
|
||||
metadata: CheckpointMetadata,
|
||||
new_versions: ChannelVersions,
|
||||
) -> RunnableConfig:
|
||||
"""Save a checkpoint to the database asynchronously.
|
||||
|
||||
This method saves a checkpoint to the Postgres database. The checkpoint is associated
|
||||
with the provided config. For AsyncShallowPostgresSaver, this method saves ONLY the most recent
|
||||
checkpoint and overwrites a previous checkpoint, if it exists.
|
||||
|
||||
Args:
|
||||
config: The config to associate with the checkpoint.
|
||||
checkpoint: The checkpoint to save.
|
||||
metadata: Additional metadata to save with the checkpoint.
|
||||
new_versions: New channel versions as of this write.
|
||||
|
||||
Returns:
|
||||
RunnableConfig: Updated configuration after storing the checkpoint.
|
||||
"""
|
||||
configurable = config["configurable"].copy()
|
||||
thread_id = configurable.pop("thread_id")
|
||||
checkpoint_ns = configurable.pop("checkpoint_ns")
|
||||
|
||||
copy = checkpoint.copy()
|
||||
next_config = {
|
||||
"configurable": {
|
||||
"thread_id": thread_id,
|
||||
"checkpoint_ns": checkpoint_ns,
|
||||
"checkpoint_id": checkpoint["id"],
|
||||
}
|
||||
}
|
||||
|
||||
async with self._cursor(pipeline=True) as cur:
|
||||
await cur.execute(
|
||||
"""DELETE FROM checkpoint_writes
|
||||
WHERE thread_id = %s AND checkpoint_ns = %s AND checkpoint_id NOT IN (%s, %s)""",
|
||||
(
|
||||
thread_id,
|
||||
checkpoint_ns,
|
||||
checkpoint["id"],
|
||||
configurable.get("checkpoint_id", ""),
|
||||
),
|
||||
)
|
||||
await cur.executemany(
|
||||
self.UPSERT_CHECKPOINT_BLOBS_SQL,
|
||||
_dump_blobs(
|
||||
self.serde,
|
||||
thread_id,
|
||||
checkpoint_ns,
|
||||
copy.pop("channel_values"), # type: ignore[misc]
|
||||
new_versions,
|
||||
),
|
||||
)
|
||||
await cur.execute(
|
||||
self.UPSERT_CHECKPOINTS_SQL,
|
||||
(
|
||||
thread_id,
|
||||
checkpoint_ns,
|
||||
Jsonb(copy),
|
||||
Jsonb(get_serializable_checkpoint_metadata(config, metadata)),
|
||||
),
|
||||
)
|
||||
return next_config
|
||||
|
||||
async def aput_writes(
|
||||
self,
|
||||
config: RunnableConfig,
|
||||
writes: Sequence[tuple[str, Any]],
|
||||
task_id: str,
|
||||
task_path: str = "",
|
||||
) -> None:
|
||||
"""Store intermediate writes linked to a checkpoint asynchronously.
|
||||
|
||||
This method saves intermediate writes associated with a checkpoint to the database.
|
||||
|
||||
Args:
|
||||
config: Configuration of the related checkpoint.
|
||||
writes: List of writes to store, each as (channel, value) pair.
|
||||
task_id: Identifier for the task creating the writes.
|
||||
"""
|
||||
query = (
|
||||
self.UPSERT_CHECKPOINT_WRITES_SQL
|
||||
if all(w[0] in WRITES_IDX_MAP for w in writes)
|
||||
else self.INSERT_CHECKPOINT_WRITES_SQL
|
||||
)
|
||||
params = await asyncio.to_thread(
|
||||
self._dump_writes,
|
||||
config["configurable"]["thread_id"],
|
||||
config["configurable"]["checkpoint_ns"],
|
||||
config["configurable"]["checkpoint_id"],
|
||||
task_id,
|
||||
task_path,
|
||||
writes,
|
||||
)
|
||||
async with self._cursor(pipeline=True) as cur:
|
||||
await cur.executemany(query, params)
|
||||
|
||||
@asynccontextmanager
|
||||
async def _cursor(
|
||||
self, *, pipeline: bool = False
|
||||
) -> AsyncIterator[AsyncCursor[DictRow]]:
|
||||
"""Create a database cursor as a context manager.
|
||||
|
||||
Args:
|
||||
pipeline: whether to use pipeline for the DB operations inside the context manager.
|
||||
Will be applied regardless of whether the AsyncShallowPostgresSaver instance was initialized with a pipeline.
|
||||
If pipeline mode is not supported, will fall back to using transaction context manager.
|
||||
"""
|
||||
async with _ainternal.get_connection(self.conn) as conn:
|
||||
if self.pipe:
|
||||
# a connection in pipeline mode can be used concurrently
|
||||
# in multiple threads/coroutines, but only one cursor can be
|
||||
# used at a time
|
||||
try:
|
||||
async with conn.cursor(binary=True, row_factory=dict_row) as cur:
|
||||
yield cur
|
||||
finally:
|
||||
if pipeline:
|
||||
await self.pipe.sync()
|
||||
elif pipeline:
|
||||
# a connection not in pipeline mode can only be used by one
|
||||
# thread/coroutine at a time, so we acquire a lock
|
||||
if self.supports_pipeline:
|
||||
async with (
|
||||
self.lock,
|
||||
conn.pipeline(),
|
||||
conn.cursor(binary=True, row_factory=dict_row) as cur,
|
||||
):
|
||||
yield cur
|
||||
else:
|
||||
# Use connection's transaction context manager when pipeline mode not supported
|
||||
async with (
|
||||
self.lock,
|
||||
conn.transaction(),
|
||||
conn.cursor(binary=True, row_factory=dict_row) as cur,
|
||||
):
|
||||
yield cur
|
||||
else:
|
||||
async with (
|
||||
self.lock,
|
||||
conn.cursor(binary=True, row_factory=dict_row) as cur,
|
||||
):
|
||||
yield cur
|
||||
|
||||
def list(
|
||||
self,
|
||||
config: RunnableConfig | None,
|
||||
*,
|
||||
filter: dict[str, Any] | None = None,
|
||||
before: RunnableConfig | None = None,
|
||||
limit: int | None = None,
|
||||
) -> Iterator[CheckpointTuple]:
|
||||
"""List checkpoints from the database.
|
||||
|
||||
This method retrieves a list of checkpoint tuples from the Postgres database based
|
||||
on the provided config. For ShallowPostgresSaver, this method returns a list with
|
||||
ONLY the most recent checkpoint.
|
||||
"""
|
||||
aiter_ = self.alist(config, filter=filter, before=before, limit=limit)
|
||||
while True:
|
||||
try:
|
||||
yield asyncio.run_coroutine_threadsafe(
|
||||
anext(aiter_), # type: ignore[arg-type] # noqa: F821
|
||||
self.loop,
|
||||
).result()
|
||||
except StopAsyncIteration:
|
||||
break
|
||||
|
||||
def get_tuple(self, config: RunnableConfig) -> CheckpointTuple | None:
|
||||
"""Get a checkpoint tuple from the database.
|
||||
|
||||
This method retrieves a checkpoint tuple from the Postgres database based on the
|
||||
provided config (matching the thread ID in the config).
|
||||
|
||||
Args:
|
||||
config: The config to use for retrieving the checkpoint.
|
||||
|
||||
Returns:
|
||||
The retrieved checkpoint tuple, or None if no matching checkpoint was found.
|
||||
"""
|
||||
try:
|
||||
# check if we are in the main thread, only bg threads can block
|
||||
# we don't check in other methods to avoid the overhead
|
||||
if asyncio.get_running_loop() is self.loop:
|
||||
raise asyncio.InvalidStateError(
|
||||
"Synchronous calls to AsyncShallowPostgresSaver are only allowed from a "
|
||||
"different thread. From the main thread, use the async interface."
|
||||
"For example, use `await checkpointer.aget_tuple(...)` or `await "
|
||||
"graph.ainvoke(...)`."
|
||||
)
|
||||
except RuntimeError:
|
||||
pass
|
||||
return asyncio.run_coroutine_threadsafe(
|
||||
self.aget_tuple(config), self.loop
|
||||
).result()
|
||||
|
||||
def put(
|
||||
self,
|
||||
config: RunnableConfig,
|
||||
checkpoint: Checkpoint,
|
||||
metadata: CheckpointMetadata,
|
||||
new_versions: ChannelVersions,
|
||||
) -> RunnableConfig:
|
||||
"""Save a checkpoint to the database.
|
||||
|
||||
This method saves a checkpoint to the Postgres database. The checkpoint is associated
|
||||
with the provided config. For AsyncShallowPostgresSaver, this method saves ONLY the most recent
|
||||
checkpoint and overwrites a previous checkpoint, if it exists.
|
||||
|
||||
Args:
|
||||
config: The config to associate with the checkpoint.
|
||||
checkpoint: The checkpoint to save.
|
||||
metadata: Additional metadata to save with the checkpoint.
|
||||
new_versions: New channel versions as of this write.
|
||||
|
||||
Returns:
|
||||
RunnableConfig: Updated configuration after storing the checkpoint.
|
||||
"""
|
||||
return asyncio.run_coroutine_threadsafe(
|
||||
self.aput(config, checkpoint, metadata, new_versions), self.loop
|
||||
).result()
|
||||
|
||||
def put_writes(
|
||||
self,
|
||||
config: RunnableConfig,
|
||||
writes: Sequence[tuple[str, Any]],
|
||||
task_id: str,
|
||||
task_path: str = "",
|
||||
) -> None:
|
||||
"""Store intermediate writes linked to a checkpoint.
|
||||
|
||||
This method saves intermediate writes associated with a checkpoint to the database.
|
||||
|
||||
Args:
|
||||
config: Configuration of the related checkpoint.
|
||||
writes: List of writes to store, each as (channel, value) pair.
|
||||
task_id: Identifier for the task creating the writes.
|
||||
task_path: Path of the task creating the writes.
|
||||
"""
|
||||
return asyncio.run_coroutine_threadsafe(
|
||||
self.aput_writes(config, writes, task_id, task_path), self.loop
|
||||
).result()
|
||||
@@ -1,4 +1,3 @@
|
||||
from langgraph.store.postgres.aio import AsyncPostgresStore
|
||||
from langgraph.store.postgres.base import PoolConfig, PostgresStore
|
||||
from langgraph.store.postgres.base import PostgresStore
|
||||
|
||||
__all__ = ["AsyncPostgresStore", "PoolConfig", "PostgresStore"]
|
||||
__all__ = ["PostgresStore"]
|
||||
@@ -1,586 +0,0 @@
|
||||
from __future__ import annotations
|
||||
|
||||
import asyncio
|
||||
import logging
|
||||
from collections.abc import AsyncIterator, Callable, Iterable, Sequence
|
||||
from contextlib import asynccontextmanager
|
||||
from types import TracebackType
|
||||
from typing import Any, cast
|
||||
|
||||
import orjson
|
||||
from langgraph.store.base import (
|
||||
GetOp,
|
||||
ListNamespacesOp,
|
||||
Op,
|
||||
PutOp,
|
||||
Result,
|
||||
SearchOp,
|
||||
)
|
||||
from langgraph.store.base.batch import AsyncBatchedBaseStore
|
||||
from psycopg import AsyncConnection, AsyncCursor, AsyncPipeline, Capabilities
|
||||
from psycopg.rows import DictRow, dict_row
|
||||
from psycopg_pool import AsyncConnectionPool
|
||||
|
||||
from langgraph.checkpoint.postgres import _ainternal
|
||||
from langgraph.store.postgres.base import (
|
||||
PLACEHOLDER,
|
||||
BasePostgresStore,
|
||||
PoolConfig,
|
||||
PostgresIndexConfig,
|
||||
Row,
|
||||
TTLConfig,
|
||||
_decode_ns_bytes,
|
||||
_ensure_index_config,
|
||||
_group_ops,
|
||||
_row_to_item,
|
||||
_row_to_search_item,
|
||||
)
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
class AsyncPostgresStore(AsyncBatchedBaseStore, BasePostgresStore[_ainternal.Conn]):
|
||||
"""Asynchronous Postgres-backed store with optional vector search using pgvector.
|
||||
|
||||
!!! example "Examples"
|
||||
Basic setup and usage:
|
||||
```python
|
||||
from langgraph.store.postgres import AsyncPostgresStore
|
||||
|
||||
conn_string = "postgresql://user:pass@localhost:5432/dbname"
|
||||
|
||||
async with AsyncPostgresStore.from_conn_string(conn_string) as store:
|
||||
await store.setup() # Run migrations. Done once
|
||||
|
||||
# Store and retrieve data
|
||||
await store.aput(("users", "123"), "prefs", {"theme": "dark"})
|
||||
item = await store.aget(("users", "123"), "prefs")
|
||||
```
|
||||
|
||||
Vector search using LangChain embeddings:
|
||||
```python
|
||||
from langchain.embeddings import init_embeddings
|
||||
from langgraph.store.postgres import AsyncPostgresStore
|
||||
|
||||
conn_string = "postgresql://user:pass@localhost:5432/dbname"
|
||||
|
||||
async with AsyncPostgresStore.from_conn_string(
|
||||
conn_string,
|
||||
index={
|
||||
"dims": 1536,
|
||||
"embed": init_embeddings("openai:text-embedding-3-small"),
|
||||
"fields": ["text"] # specify which fields to embed. Default is the whole serialized value
|
||||
}
|
||||
) as store:
|
||||
await store.setup() # Run migrations. Done once
|
||||
|
||||
# Store documents
|
||||
await store.aput(("docs",), "doc1", {"text": "Python tutorial"})
|
||||
await store.aput(("docs",), "doc2", {"text": "TypeScript guide"})
|
||||
await store.aput(("docs",), "doc3", {"text": "Other guide"}, index=False) # don't index
|
||||
|
||||
# Search by similarity
|
||||
results = await store.asearch(("docs",), query="programming guides", limit=2)
|
||||
```
|
||||
|
||||
Using connection pooling for better performance:
|
||||
```python
|
||||
from langgraph.store.postgres import AsyncPostgresStore, PoolConfig
|
||||
|
||||
conn_string = "postgresql://user:pass@localhost:5432/dbname"
|
||||
|
||||
async with AsyncPostgresStore.from_conn_string(
|
||||
conn_string,
|
||||
pool_config=PoolConfig(
|
||||
min_size=5,
|
||||
max_size=20
|
||||
)
|
||||
) as store:
|
||||
await store.setup() # Run migrations. Done once
|
||||
# Use store with connection pooling...
|
||||
```
|
||||
|
||||
Warning:
|
||||
Make sure to:
|
||||
1. Call `setup()` before first use to create necessary tables and indexes
|
||||
2. Have the pgvector extension available to use vector search
|
||||
3. Use Python 3.10+ for async functionality
|
||||
|
||||
Note:
|
||||
Semantic search is disabled by default. You can enable it by providing an `index` configuration
|
||||
when creating the store. Without this configuration, all `index` arguments passed to
|
||||
`put` or `aput` will have no effect.
|
||||
|
||||
Note:
|
||||
If you provide a TTL configuration, you must explicitly call `start_ttl_sweeper()` to begin
|
||||
the background task that removes expired items. Call `stop_ttl_sweeper()` to properly
|
||||
clean up resources when you're done with the store.
|
||||
"""
|
||||
|
||||
__slots__ = (
|
||||
"_deserializer",
|
||||
"pipe",
|
||||
"lock",
|
||||
"supports_pipeline",
|
||||
"index_config",
|
||||
"embeddings",
|
||||
"ttl_config",
|
||||
"_ttl_sweeper_task",
|
||||
"_ttl_stop_event",
|
||||
)
|
||||
supports_ttl: bool = True
|
||||
|
||||
def __init__(
|
||||
self,
|
||||
conn: _ainternal.Conn,
|
||||
*,
|
||||
pipe: AsyncPipeline | None = None,
|
||||
deserializer: Callable[[bytes | orjson.Fragment], dict[str, Any]] | None = None,
|
||||
index: PostgresIndexConfig | None = None,
|
||||
ttl: TTLConfig | None = None,
|
||||
) -> None:
|
||||
if isinstance(conn, AsyncConnectionPool) and pipe is not None:
|
||||
raise ValueError(
|
||||
"Pipeline should be used only with a single AsyncConnection, not AsyncConnectionPool."
|
||||
)
|
||||
super().__init__()
|
||||
self._deserializer = deserializer
|
||||
self.conn = conn
|
||||
self.pipe = pipe
|
||||
self.lock = asyncio.Lock()
|
||||
self.loop = asyncio.get_running_loop()
|
||||
self.supports_pipeline = Capabilities().has_pipeline()
|
||||
self.index_config = index
|
||||
if self.index_config:
|
||||
self.embeddings, self.index_config = _ensure_index_config(self.index_config)
|
||||
else:
|
||||
self.embeddings = None
|
||||
|
||||
self.ttl_config = ttl
|
||||
self._ttl_sweeper_task: asyncio.Task[None] | None = None
|
||||
self._ttl_stop_event = asyncio.Event()
|
||||
|
||||
async def abatch(self, ops: Iterable[Op]) -> list[Result]:
|
||||
grouped_ops, num_ops = _group_ops(ops)
|
||||
results: list[Result] = [None] * num_ops
|
||||
|
||||
async with _ainternal.get_connection(self.conn) as conn:
|
||||
if self.pipe:
|
||||
async with self.pipe:
|
||||
await self._execute_batch(grouped_ops, results, conn)
|
||||
else:
|
||||
await self._execute_batch(grouped_ops, results, conn)
|
||||
|
||||
return results
|
||||
|
||||
@classmethod
|
||||
@asynccontextmanager
|
||||
async def from_conn_string(
|
||||
cls,
|
||||
conn_string: str,
|
||||
*,
|
||||
pipeline: bool = False,
|
||||
pool_config: PoolConfig | None = None,
|
||||
index: PostgresIndexConfig | None = None,
|
||||
ttl: TTLConfig | None = None,
|
||||
) -> AsyncIterator[AsyncPostgresStore]:
|
||||
"""Create a new AsyncPostgresStore instance from a connection string.
|
||||
|
||||
Args:
|
||||
conn_string: The Postgres connection info string.
|
||||
pipeline: Whether to use AsyncPipeline (only for single connections)
|
||||
pool_config: Configuration for the connection pool.
|
||||
If provided, will create a connection pool and use it instead of a single connection.
|
||||
This overrides the `pipeline` argument.
|
||||
index: The embedding config.
|
||||
|
||||
Returns:
|
||||
AsyncPostgresStore: A new AsyncPostgresStore instance.
|
||||
"""
|
||||
if pool_config is not None:
|
||||
pc = pool_config.copy()
|
||||
async with cast(
|
||||
AsyncConnectionPool[AsyncConnection[DictRow]],
|
||||
AsyncConnectionPool(
|
||||
conn_string,
|
||||
min_size=pc.pop("min_size", 1),
|
||||
max_size=pc.pop("max_size", None),
|
||||
kwargs={
|
||||
"autocommit": True,
|
||||
"prepare_threshold": 0,
|
||||
"row_factory": dict_row,
|
||||
**(pc.pop("kwargs", None) or {}),
|
||||
},
|
||||
**cast(dict, pc),
|
||||
),
|
||||
) as pool:
|
||||
yield cls(conn=pool, index=index, ttl=ttl)
|
||||
else:
|
||||
async with await AsyncConnection.connect(
|
||||
conn_string, autocommit=True, prepare_threshold=0, row_factory=dict_row
|
||||
) as conn:
|
||||
if pipeline:
|
||||
async with conn.pipeline() as pipe:
|
||||
yield cls(conn=conn, pipe=pipe, index=index, ttl=ttl)
|
||||
else:
|
||||
yield cls(conn=conn, index=index, ttl=ttl)
|
||||
|
||||
async def setup(self) -> None:
|
||||
"""Set up the store database asynchronously.
|
||||
|
||||
This method creates the necessary tables in the Postgres database if they don't
|
||||
already exist and runs database migrations. It MUST be called directly by the user
|
||||
the first time the store is used.
|
||||
"""
|
||||
|
||||
async def _get_version(cur: AsyncCursor[DictRow], table: str) -> int:
|
||||
await cur.execute(
|
||||
f"""
|
||||
CREATE TABLE IF NOT EXISTS {table} (
|
||||
v INTEGER PRIMARY KEY
|
||||
)
|
||||
"""
|
||||
)
|
||||
await cur.execute(f"SELECT v FROM {table} ORDER BY v DESC LIMIT 1")
|
||||
row = cast(dict, await cur.fetchone())
|
||||
if row is None:
|
||||
version = -1
|
||||
else:
|
||||
version = row["v"]
|
||||
return version
|
||||
|
||||
async with self._cursor() as cur:
|
||||
version = await _get_version(cur, table="store_migrations")
|
||||
for v, sql in enumerate(self.MIGRATIONS[version + 1 :], start=version + 1):
|
||||
await cur.execute(sql)
|
||||
await cur.execute("INSERT INTO store_migrations (v) VALUES (%s)", (v,))
|
||||
|
||||
if self.index_config:
|
||||
version = await _get_version(cur, table="vector_migrations")
|
||||
for v, migration in enumerate(
|
||||
self.VECTOR_MIGRATIONS[version + 1 :], start=version + 1
|
||||
):
|
||||
sql = migration.sql
|
||||
if migration.params:
|
||||
params = {
|
||||
k: v(self) if v is not None and callable(v) else v
|
||||
for k, v in migration.params.items()
|
||||
}
|
||||
if "dims" in params:
|
||||
try:
|
||||
params["dims"] = int(params["dims"])
|
||||
except Exception as e:
|
||||
raise ValueError(
|
||||
f"Invalid dims for vector index: {params['dims']}"
|
||||
) from e
|
||||
if "vector_type" in params:
|
||||
vt = str(params["vector_type"])
|
||||
if vt not in ("vector", "halfvec"):
|
||||
raise ValueError(
|
||||
f"Invalid vector_type for pgvector: {vt}"
|
||||
)
|
||||
params["vector_type"] = vt
|
||||
if "index_type" in params:
|
||||
it = str(params["index_type"])
|
||||
if it not in ("hnsw", "ivfflat"):
|
||||
raise ValueError(
|
||||
f"Invalid index_type for pgvector: {it}"
|
||||
)
|
||||
params["index_type"] = it
|
||||
sql = sql % params
|
||||
await cur.execute(sql)
|
||||
await cur.execute(
|
||||
"INSERT INTO vector_migrations (v) VALUES (%s)", (v,)
|
||||
)
|
||||
|
||||
async def sweep_ttl(self) -> int:
|
||||
"""Delete expired store items based on TTL.
|
||||
|
||||
Returns:
|
||||
int: The number of deleted items.
|
||||
"""
|
||||
async with self._cursor() as cur:
|
||||
await cur.execute(
|
||||
"""
|
||||
DELETE FROM store
|
||||
WHERE expires_at IS NOT NULL AND expires_at < NOW()
|
||||
"""
|
||||
)
|
||||
deleted_count = cur.rowcount
|
||||
return deleted_count
|
||||
|
||||
async def start_ttl_sweeper(
|
||||
self, sweep_interval_minutes: int | None = None
|
||||
) -> asyncio.Task[None]:
|
||||
"""Periodically delete expired store items based on TTL.
|
||||
|
||||
Returns:
|
||||
Task that can be awaited or cancelled.
|
||||
"""
|
||||
if not self.ttl_config:
|
||||
return asyncio.create_task(asyncio.sleep(0))
|
||||
|
||||
if self._ttl_sweeper_task is not None and not self._ttl_sweeper_task.done():
|
||||
return self._ttl_sweeper_task
|
||||
|
||||
self._ttl_stop_event.clear()
|
||||
|
||||
interval = float(
|
||||
sweep_interval_minutes or self.ttl_config.get("sweep_interval_minutes") or 5
|
||||
)
|
||||
logger.info(f"Starting store TTL sweeper with interval {interval} minutes")
|
||||
|
||||
async def _sweep_loop() -> None:
|
||||
while not self._ttl_stop_event.is_set():
|
||||
try:
|
||||
try:
|
||||
await asyncio.wait_for(
|
||||
self._ttl_stop_event.wait(),
|
||||
timeout=interval * 60,
|
||||
)
|
||||
break
|
||||
except asyncio.TimeoutError:
|
||||
pass
|
||||
|
||||
expired_items = await self.sweep_ttl()
|
||||
if expired_items > 0:
|
||||
logger.info(f"Store swept {expired_items} expired items")
|
||||
except asyncio.CancelledError:
|
||||
break
|
||||
except Exception as exc:
|
||||
logger.exception("Store TTL sweep iteration failed", exc_info=exc)
|
||||
|
||||
task = asyncio.create_task(_sweep_loop())
|
||||
task.set_name("ttl_sweeper")
|
||||
self._ttl_sweeper_task = task
|
||||
return task
|
||||
|
||||
async def stop_ttl_sweeper(self, timeout: float | None = None) -> bool:
|
||||
"""Stop the TTL sweeper task if it's running.
|
||||
|
||||
Args:
|
||||
timeout: Maximum time to wait for the task to stop, in seconds.
|
||||
If `None`, wait indefinitely.
|
||||
|
||||
Returns:
|
||||
bool: True if the task was successfully stopped or wasn't running,
|
||||
False if the timeout was reached before the task stopped.
|
||||
"""
|
||||
if self._ttl_sweeper_task is None or self._ttl_sweeper_task.done():
|
||||
return True
|
||||
|
||||
logger.info("Stopping TTL sweeper task")
|
||||
self._ttl_stop_event.set()
|
||||
|
||||
if timeout is not None:
|
||||
try:
|
||||
await asyncio.wait_for(self._ttl_sweeper_task, timeout=timeout)
|
||||
success = True
|
||||
except asyncio.TimeoutError:
|
||||
success = False
|
||||
else:
|
||||
await self._ttl_sweeper_task
|
||||
success = True
|
||||
|
||||
if success:
|
||||
self._ttl_sweeper_task = None
|
||||
logger.info("TTL sweeper task stopped")
|
||||
else:
|
||||
logger.warning("Timed out waiting for TTL sweeper task to stop")
|
||||
|
||||
return success
|
||||
|
||||
async def __aenter__(self) -> AsyncPostgresStore:
|
||||
return self
|
||||
|
||||
async def __aexit__(
|
||||
self,
|
||||
exc_type: type[BaseException] | None,
|
||||
exc_val: BaseException | None,
|
||||
exc_tb: TracebackType | None,
|
||||
) -> None:
|
||||
# Ensure the TTL sweeper task is stopped when exiting the context
|
||||
if hasattr(self, "_ttl_sweeper_task") and self._ttl_sweeper_task is not None:
|
||||
# Set the event to signal the task to stop
|
||||
self._ttl_stop_event.set()
|
||||
# We don't wait for the task to complete here to avoid blocking
|
||||
# The task will clean up itself gracefully
|
||||
|
||||
async def _execute_batch(
|
||||
self,
|
||||
grouped_ops: dict,
|
||||
results: list[Result],
|
||||
conn: AsyncConnection[DictRow],
|
||||
) -> None:
|
||||
async with self._cursor(pipeline=True) as cur:
|
||||
if GetOp in grouped_ops:
|
||||
await self._batch_get_ops(
|
||||
cast(Sequence[tuple[int, GetOp]], grouped_ops[GetOp]),
|
||||
results,
|
||||
cur,
|
||||
)
|
||||
|
||||
if SearchOp in grouped_ops:
|
||||
await self._batch_search_ops(
|
||||
cast(Sequence[tuple[int, SearchOp]], grouped_ops[SearchOp]),
|
||||
results,
|
||||
cur,
|
||||
)
|
||||
|
||||
if ListNamespacesOp in grouped_ops:
|
||||
await self._batch_list_namespaces_ops(
|
||||
cast(
|
||||
Sequence[tuple[int, ListNamespacesOp]],
|
||||
grouped_ops[ListNamespacesOp],
|
||||
),
|
||||
results,
|
||||
cur,
|
||||
)
|
||||
|
||||
if PutOp in grouped_ops:
|
||||
await self._batch_put_ops(
|
||||
cast(Sequence[tuple[int, PutOp]], grouped_ops[PutOp]),
|
||||
cur,
|
||||
)
|
||||
|
||||
async def _batch_get_ops(
|
||||
self,
|
||||
get_ops: Sequence[tuple[int, GetOp]],
|
||||
results: list[Result],
|
||||
cur: AsyncCursor[DictRow],
|
||||
) -> None:
|
||||
for query, params, namespace, items in self._get_batch_GET_ops_queries(get_ops):
|
||||
await cur.execute(query, params)
|
||||
rows = cast(list[Row], await cur.fetchall())
|
||||
key_to_row = {row["key"]: row for row in rows}
|
||||
for idx, key in items:
|
||||
row = key_to_row.get(key)
|
||||
if row:
|
||||
results[idx] = _row_to_item(
|
||||
namespace, row, loader=self._deserializer
|
||||
)
|
||||
else:
|
||||
results[idx] = None
|
||||
|
||||
async def _batch_put_ops(
|
||||
self,
|
||||
put_ops: Sequence[tuple[int, PutOp]],
|
||||
cur: AsyncCursor[DictRow],
|
||||
) -> None:
|
||||
queries, embedding_request = self._prepare_batch_PUT_queries(put_ops)
|
||||
if embedding_request:
|
||||
if self.embeddings is None:
|
||||
# Should not get here since the embedding config is required
|
||||
# to return an embedding_request above
|
||||
raise ValueError(
|
||||
"Embedding configuration is required for vector operations "
|
||||
f"(for semantic search). "
|
||||
f"Please provide an EmbeddingConfig when initializing the {self.__class__.__name__}."
|
||||
)
|
||||
query, txt_params = embedding_request
|
||||
vectors = await self.embeddings.aembed_documents(
|
||||
[param[-1] for param in txt_params]
|
||||
)
|
||||
queries.append(
|
||||
(
|
||||
query,
|
||||
[
|
||||
p
|
||||
for (ns, k, pathname, _), vector in zip(
|
||||
txt_params, vectors, strict=False
|
||||
)
|
||||
for p in (ns, k, pathname, vector)
|
||||
],
|
||||
)
|
||||
)
|
||||
|
||||
for query, params in queries:
|
||||
await cur.execute(query, params)
|
||||
|
||||
async def _batch_search_ops(
|
||||
self,
|
||||
search_ops: Sequence[tuple[int, SearchOp]],
|
||||
results: list[Result],
|
||||
cur: AsyncCursor[DictRow],
|
||||
) -> None:
|
||||
queries, embedding_requests = self._prepare_batch_search_queries(search_ops)
|
||||
|
||||
if embedding_requests and self.embeddings:
|
||||
vectors = await self.embeddings.aembed_documents(
|
||||
[query for _, query in embedding_requests]
|
||||
)
|
||||
for (idx, _), vector in zip(embedding_requests, vectors, strict=False):
|
||||
_paramslist = queries[idx][1]
|
||||
for i in range(len(_paramslist)):
|
||||
if _paramslist[i] is PLACEHOLDER:
|
||||
_paramslist[i] = vector
|
||||
|
||||
for (idx, _), (query, params) in zip(search_ops, queries, strict=False):
|
||||
await cur.execute(query, params)
|
||||
rows = cast(list[Row], await cur.fetchall())
|
||||
items = [
|
||||
_row_to_search_item(
|
||||
_decode_ns_bytes(row["prefix"]), row, loader=self._deserializer
|
||||
)
|
||||
for row in rows
|
||||
]
|
||||
results[idx] = items
|
||||
|
||||
async def _batch_list_namespaces_ops(
|
||||
self,
|
||||
list_ops: Sequence[tuple[int, ListNamespacesOp]],
|
||||
results: list[Result],
|
||||
cur: AsyncCursor[DictRow],
|
||||
) -> None:
|
||||
queries = self._get_batch_list_namespaces_queries(list_ops)
|
||||
for (query, params), (idx, _) in zip(queries, list_ops, strict=False):
|
||||
await cur.execute(query, params)
|
||||
rows = cast(list[dict], await cur.fetchall())
|
||||
namespaces = [_decode_ns_bytes(row["truncated_prefix"]) for row in rows]
|
||||
results[idx] = namespaces
|
||||
|
||||
@asynccontextmanager
|
||||
async def _cursor(
|
||||
self, *, pipeline: bool = False
|
||||
) -> AsyncIterator[AsyncCursor[DictRow]]:
|
||||
"""Create a database cursor as a context manager.
|
||||
|
||||
Args:
|
||||
pipeline: whether to use pipeline for the DB operations inside the context manager.
|
||||
Will be applied regardless of whether the PostgresStore instance was initialized with a pipeline.
|
||||
If pipeline mode is not supported, will fall back to using transaction context manager.
|
||||
"""
|
||||
async with _ainternal.get_connection(self.conn) as conn:
|
||||
if self.pipe:
|
||||
# a connection in pipeline mode can be used concurrently
|
||||
# in multiple threads/coroutines, but only one cursor can be
|
||||
# used at a time
|
||||
try:
|
||||
async with conn.cursor(binary=True, row_factory=dict_row) as cur:
|
||||
yield cur
|
||||
finally:
|
||||
if pipeline:
|
||||
await self.pipe.sync()
|
||||
elif pipeline:
|
||||
# a connection not in pipeline mode can only be used by one
|
||||
# thread/coroutine at a time, so we acquire a lock
|
||||
if self.supports_pipeline:
|
||||
async with (
|
||||
self.lock,
|
||||
conn.pipeline(),
|
||||
conn.cursor(binary=True, row_factory=dict_row) as cur,
|
||||
):
|
||||
yield cur
|
||||
else:
|
||||
async with (
|
||||
self.lock,
|
||||
conn.transaction(),
|
||||
conn.cursor(binary=True, row_factory=dict_row) as cur,
|
||||
):
|
||||
yield cur
|
||||
else:
|
||||
async with (
|
||||
self.lock,
|
||||
conn.cursor(binary=True, row_factory=dict_row) as cur,
|
||||
):
|
||||
yield cur
|
||||
@@ -1,27 +1,34 @@
|
||||
from __future__ import annotations
|
||||
|
||||
import asyncio
|
||||
import concurrent.futures
|
||||
import json
|
||||
import logging
|
||||
import threading
|
||||
from collections import defaultdict
|
||||
from collections.abc import Callable, Iterable, Iterator, Sequence
|
||||
from collections.abc import Iterable, Iterator, Sequence
|
||||
from contextlib import contextmanager
|
||||
from datetime import datetime
|
||||
from typing import (
|
||||
TYPE_CHECKING,
|
||||
Any,
|
||||
Callable,
|
||||
Generic,
|
||||
Literal,
|
||||
NamedTuple,
|
||||
Optional,
|
||||
TypeVar,
|
||||
Union,
|
||||
cast,
|
||||
)
|
||||
|
||||
import orjson
|
||||
from psycopg import Capabilities, Connection, Cursor, Pipeline
|
||||
from psycopg.rows import DictRow, dict_row
|
||||
from psycopg.types.json import Jsonb
|
||||
from psycopg_pool import ConnectionPool
|
||||
from typing_extensions import TypedDict
|
||||
|
||||
from langgraph.checkpoint.postgres import _internal as _pg_internal
|
||||
from langgraph.store.base import (
|
||||
BaseStore,
|
||||
Embeddings,
|
||||
GetOp,
|
||||
IndexConfig,
|
||||
Item,
|
||||
@@ -31,22 +38,10 @@ from langgraph.store.base import (
|
||||
Result,
|
||||
SearchItem,
|
||||
SearchOp,
|
||||
TTLConfig,
|
||||
ensure_embeddings,
|
||||
get_text_at_path,
|
||||
tokenize_path,
|
||||
)
|
||||
from psycopg import Capabilities, Connection, Cursor, Pipeline
|
||||
from psycopg.rows import DictRow, dict_row
|
||||
from psycopg.types.json import Jsonb
|
||||
from psycopg_pool import ConnectionPool
|
||||
from typing_extensions import TypedDict
|
||||
|
||||
from langgraph.checkpoint.postgres import _ainternal as _ainternal
|
||||
from langgraph.checkpoint.postgres import _internal as _pg_internal
|
||||
|
||||
if TYPE_CHECKING:
|
||||
from langchain_core.embeddings import Embeddings
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
@@ -55,8 +50,8 @@ class Migration(NamedTuple):
|
||||
"""A database migration with optional conditions and parameters."""
|
||||
|
||||
sql: str
|
||||
params: dict[str, Any] | None = None
|
||||
condition: Callable[[BasePostgresStore], bool] | None = None
|
||||
params: Optional[dict[str, Any]] = None
|
||||
condition: Optional[Callable[["BasePostgresStore"], bool]] = None
|
||||
|
||||
|
||||
MIGRATIONS: Sequence[str] = [
|
||||
@@ -74,29 +69,13 @@ CREATE TABLE IF NOT EXISTS store (
|
||||
"""
|
||||
-- For faster lookups by prefix
|
||||
CREATE INDEX CONCURRENTLY IF NOT EXISTS store_prefix_idx ON store USING btree (prefix text_pattern_ops);
|
||||
""",
|
||||
"""
|
||||
-- Add expires_at column to store table
|
||||
ALTER TABLE store
|
||||
ADD COLUMN IF NOT EXISTS expires_at TIMESTAMP WITH TIME ZONE,
|
||||
ADD COLUMN IF NOT EXISTS ttl_minutes INT;
|
||||
""",
|
||||
"""
|
||||
-- Add indexes for efficient TTL sweeping
|
||||
CREATE INDEX IF NOT EXISTS idx_store_expires_at ON store (expires_at)
|
||||
WHERE expires_at IS NOT NULL;
|
||||
""",
|
||||
]
|
||||
|
||||
VECTOR_MIGRATIONS: Sequence[Migration] = [
|
||||
Migration(
|
||||
"""
|
||||
DO $$
|
||||
BEGIN
|
||||
IF NOT EXISTS (SELECT 1 FROM pg_extension WHERE extname = 'vector') THEN
|
||||
CREATE EXTENSION vector;
|
||||
END IF;
|
||||
END $$;
|
||||
CREATE EXTENSION IF NOT EXISTS vector;
|
||||
""",
|
||||
),
|
||||
Migration(
|
||||
@@ -144,14 +123,13 @@ CREATE INDEX CONCURRENTLY IF NOT EXISTS store_vectors_embedding_idx ON store_vec
|
||||
]
|
||||
|
||||
|
||||
C = TypeVar("C", bound=_pg_internal.Conn | _ainternal.Conn)
|
||||
C = TypeVar("C", bound=_pg_internal.Conn)
|
||||
|
||||
|
||||
class PoolConfig(TypedDict, total=False):
|
||||
"""Connection pool settings for PostgreSQL connections.
|
||||
|
||||
Controls connection lifecycle and resource utilization:
|
||||
|
||||
- Small pools (1-5) suit low-concurrency workloads
|
||||
- Larger pools handle concurrent requests but consume more resources
|
||||
- Setting max_size prevents resource exhaustion under load
|
||||
@@ -160,14 +138,13 @@ class PoolConfig(TypedDict, total=False):
|
||||
min_size: int
|
||||
"""Minimum number of connections maintained in the pool. Defaults to 1."""
|
||||
|
||||
max_size: int | None
|
||||
max_size: Optional[int]
|
||||
"""Maximum number of connections allowed in the pool. None means unlimited."""
|
||||
|
||||
kwargs: dict
|
||||
"""Additional connection arguments passed to each connection in the pool.
|
||||
|
||||
Default kwargs set automatically:
|
||||
|
||||
- autocommit: True
|
||||
- prepare_threshold: 0
|
||||
- row_factory: dict_row
|
||||
@@ -236,62 +213,27 @@ class BasePostgresStore(Generic[C]):
|
||||
MIGRATIONS = MIGRATIONS
|
||||
VECTOR_MIGRATIONS = VECTOR_MIGRATIONS
|
||||
conn: C
|
||||
_deserializer: Callable[[bytes | orjson.Fragment], dict[str, Any]] | None
|
||||
index_config: PostgresIndexConfig | None
|
||||
_deserializer: Optional[Callable[[Union[bytes, orjson.Fragment]], dict[str, Any]]]
|
||||
index_config: Optional[PostgresIndexConfig]
|
||||
|
||||
def _get_batch_GET_ops_queries(
|
||||
self,
|
||||
get_ops: Sequence[tuple[int, GetOp]],
|
||||
) -> list[tuple[str, tuple, tuple[str, ...], list]]:
|
||||
"""
|
||||
Build queries to fetch (and optionally refresh the TTL of) multiple keys per namespace.
|
||||
|
||||
Each returned element is a tuple of:
|
||||
(sql_query_string, sql_params, namespace, items_for_this_namespace)
|
||||
|
||||
where items_for_this_namespace is the original list of (idx, key, refresh_ttl).
|
||||
"""
|
||||
|
||||
namespace_groups = defaultdict(list)
|
||||
refresh_ttls = defaultdict(list)
|
||||
for idx, op in get_ops:
|
||||
namespace_groups[op.namespace].append((idx, op.key))
|
||||
refresh_ttls[op.namespace].append(op.refresh_ttl)
|
||||
|
||||
results = []
|
||||
for namespace, items in namespace_groups.items():
|
||||
_, keys = zip(*items, strict=False)
|
||||
this_refresh_ttls = refresh_ttls[namespace]
|
||||
|
||||
query = """
|
||||
WITH passed_in AS (
|
||||
SELECT unnest(%s::text[]) AS key,
|
||||
unnest(%s::bool[]) AS do_refresh
|
||||
),
|
||||
updated AS (
|
||||
UPDATE store s
|
||||
SET expires_at = NOW() + (s.ttl_minutes || ' minutes')::interval
|
||||
FROM passed_in p
|
||||
WHERE s.prefix = %s
|
||||
AND s.key = p.key
|
||||
AND p.do_refresh = TRUE
|
||||
AND s.ttl_minutes IS NOT NULL
|
||||
RETURNING s.key
|
||||
)
|
||||
SELECT s.key, s.value, s.created_at, s.updated_at
|
||||
FROM store s
|
||||
JOIN passed_in p ON s.key = p.key
|
||||
WHERE s.prefix = %s
|
||||
_, keys = zip(*items)
|
||||
keys_to_query = ",".join(["%s"] * len(keys))
|
||||
query = f"""
|
||||
SELECT key, value, created_at, updated_at
|
||||
FROM store
|
||||
WHERE prefix = %s AND key IN ({keys_to_query})
|
||||
"""
|
||||
ns_text = _namespace_to_text(namespace)
|
||||
params = (
|
||||
list(keys), # -> unnest(%s::text[])
|
||||
list(this_refresh_ttls), # -> unnest(%s::bool[])
|
||||
ns_text, # -> prefix = %s (for UPDATE)
|
||||
ns_text, # -> prefix = %s (for final SELECT)
|
||||
)
|
||||
params = (_namespace_to_text(namespace), *keys)
|
||||
results.append((query, params, namespace, items))
|
||||
|
||||
return results
|
||||
|
||||
def _prepare_batch_PUT_queries(
|
||||
@@ -299,8 +241,9 @@ class BasePostgresStore(Generic[C]):
|
||||
put_ops: Sequence[tuple[int, PutOp]],
|
||||
) -> tuple[
|
||||
list[tuple[str, Sequence]],
|
||||
tuple[str, Sequence[tuple[str, str, str, str]]] | None,
|
||||
Optional[tuple[str, Sequence[tuple[str, str, str, str]]]],
|
||||
]:
|
||||
# Last-write wins
|
||||
dedupped_ops: dict[tuple[tuple[str, ...], str], PutOp] = {}
|
||||
for _, op in put_ops:
|
||||
dedupped_ops[(op.namespace, op.key)] = op
|
||||
@@ -326,39 +269,25 @@ class BasePostgresStore(Generic[C]):
|
||||
)
|
||||
params = (_namespace_to_text(namespace), *keys)
|
||||
queries.append((query, params))
|
||||
embedding_request: tuple[str, Sequence[tuple[str, str, str, str]]] | None = None
|
||||
embedding_request: Optional[tuple[str, Sequence[tuple[str, str, str, str]]]] = (
|
||||
None
|
||||
)
|
||||
if inserts:
|
||||
values = []
|
||||
insertion_params: list[Any] = []
|
||||
insertion_params = []
|
||||
vector_values = []
|
||||
embedding_request_params = []
|
||||
# Handle TTL expiration
|
||||
|
||||
# First handle main store insertions
|
||||
for op in inserts:
|
||||
values.append("(%s, %s, %s, CURRENT_TIMESTAMP, CURRENT_TIMESTAMP)")
|
||||
insertion_params.extend(
|
||||
(
|
||||
[
|
||||
_namespace_to_text(op.namespace),
|
||||
op.key,
|
||||
Jsonb(cast(dict, op.value)),
|
||||
)
|
||||
]
|
||||
)
|
||||
if op.ttl is not None:
|
||||
values.append(
|
||||
"(%s, %s, %s, CURRENT_TIMESTAMP, CURRENT_TIMESTAMP, NOW() + %s::interval, %s)"
|
||||
)
|
||||
ttl_minutes = float(op.ttl)
|
||||
insertion_params.extend(
|
||||
(
|
||||
f"{ttl_minutes * 60} seconds",
|
||||
ttl_minutes,
|
||||
)
|
||||
)
|
||||
else:
|
||||
values.append(
|
||||
"(%s, %s, %s, CURRENT_TIMESTAMP, CURRENT_TIMESTAMP, NULL, %s)"
|
||||
)
|
||||
insertion_params.append(None)
|
||||
|
||||
# Then handle embeddings if configured
|
||||
if self.index_config:
|
||||
@@ -370,7 +299,7 @@ class BasePostgresStore(Generic[C]):
|
||||
k = op.key
|
||||
|
||||
if op.index is None:
|
||||
paths = cast(dict, self.index_config)["__tokenized_fields"]
|
||||
paths = self.index_config["__tokenized_fields"]
|
||||
else:
|
||||
paths = [(ix, tokenize_path(ix)) for ix in op.index]
|
||||
|
||||
@@ -385,13 +314,11 @@ class BasePostgresStore(Generic[C]):
|
||||
|
||||
values_str = ",".join(values)
|
||||
query = f"""
|
||||
INSERT INTO store (prefix, key, value, created_at, updated_at, expires_at, ttl_minutes)
|
||||
INSERT INTO store (prefix, key, value, created_at, updated_at)
|
||||
VALUES {values_str}
|
||||
ON CONFLICT (prefix, key) DO UPDATE
|
||||
SET value = EXCLUDED.value,
|
||||
updated_at = CURRENT_TIMESTAMP,
|
||||
expires_at = EXCLUDED.expires_at,
|
||||
ttl_minutes = EXCLUDED.ttl_minutes
|
||||
updated_at = CURRENT_TIMESTAMP
|
||||
"""
|
||||
queries.append((query, insertion_params))
|
||||
|
||||
@@ -412,156 +339,120 @@ class BasePostgresStore(Generic[C]):
|
||||
self,
|
||||
search_ops: Sequence[tuple[int, SearchOp]],
|
||||
) -> tuple[
|
||||
list[tuple[str, list[None | str | list[float]]]], # queries, params
|
||||
list[tuple[str, list[Union[None, str, list[float]]]]], # queries, params
|
||||
list[tuple[int, str]], # idx, query_text pairs to embed
|
||||
]:
|
||||
"""
|
||||
Build per-SearchOp SQL queries (with optional TTL refresh) plus embedding requests.
|
||||
Returns:
|
||||
- queries: list of (SQL, param_list)
|
||||
- embedding_requests: list of (original_index_in_search_ops, text_query)
|
||||
"""
|
||||
|
||||
queries = []
|
||||
embedding_requests = []
|
||||
|
||||
for idx, (_, op) in enumerate(search_ops):
|
||||
# Build filter conditions first
|
||||
filter_params = []
|
||||
filter_clauses = []
|
||||
filter_conditions = []
|
||||
if op.filter:
|
||||
for key, value in op.filter.items():
|
||||
if isinstance(value, dict):
|
||||
for op_name, val in value.items():
|
||||
condition, params_ = self._get_filter_condition(
|
||||
condition, filter_params_ = self._get_filter_condition(
|
||||
key, op_name, val
|
||||
)
|
||||
filter_clauses.append(condition)
|
||||
filter_params.extend(params_)
|
||||
filter_conditions.append(condition)
|
||||
filter_params.extend(filter_params_)
|
||||
else:
|
||||
filter_clauses.append("value->%s = %s::jsonb")
|
||||
filter_params.extend([key, orjson.dumps(value).decode("utf-8")])
|
||||
|
||||
ns_condition = "TRUE"
|
||||
ns_param: Sequence[str] | None = None
|
||||
if op.namespace_prefix:
|
||||
ns_condition = "store.prefix LIKE %s"
|
||||
ns_param = (f"{_namespace_to_text(op.namespace_prefix)}%",)
|
||||
else:
|
||||
ns_param = ()
|
||||
|
||||
extra_filters = (
|
||||
" AND " + " AND ".join(filter_clauses) if filter_clauses else ""
|
||||
)
|
||||
filter_conditions.append("value->%s = %s::jsonb")
|
||||
filter_params.extend([key, json.dumps(value)])
|
||||
|
||||
# Vector search branch
|
||||
if op.query and self.index_config:
|
||||
# We'll embed the text later, so record the request.
|
||||
embedding_requests.append((idx, op.query))
|
||||
|
||||
score_operator, post_operator = get_distance_operator(self)
|
||||
post_operator = post_operator.replace("scored", "uniq")
|
||||
vector_type = self.index_config.get("ann_index_config", {}).get(
|
||||
"vector_type", "vector"
|
||||
score_operator, post_operator = _get_distance_operator(self)
|
||||
vector_type = (
|
||||
cast(PostgresIndexConfig, self.index_config)
|
||||
.get("ann_index_config", {})
|
||||
.get("vector_type", "vector")
|
||||
)
|
||||
|
||||
# For hamming bit vectors, or “regular” vectors
|
||||
if (
|
||||
vector_type == "bit"
|
||||
and cast(dict, self.index_config).get("distance_type") == "hamming"
|
||||
and self.index_config.get("distance_type") == "hamming"
|
||||
):
|
||||
score_operator = score_operator % (
|
||||
"%s",
|
||||
cast(dict, self.index_config)["dims"],
|
||||
self.index_config["dims"],
|
||||
)
|
||||
else:
|
||||
if vector_type not in ("vector", "halfvec"):
|
||||
raise ValueError(
|
||||
f"Invalid vector_type for pgvector: {vector_type}"
|
||||
)
|
||||
score_operator = score_operator % ("%s", vector_type)
|
||||
score_operator = score_operator % (
|
||||
"%s",
|
||||
vector_type,
|
||||
)
|
||||
|
||||
vectors_per_doc_estimate = cast(dict, self.index_config)[
|
||||
"__estimated_num_vectors"
|
||||
]
|
||||
vectors_per_doc_estimate = self.index_config["__estimated_num_vectors"]
|
||||
expanded_limit = (op.limit * vectors_per_doc_estimate * 2) + 1
|
||||
|
||||
# “sub_scored” does the main vector search
|
||||
# Then we do DISTINCT ON to drop duplicates if your store can have them
|
||||
# Finally we limit & offset
|
||||
vector_search_cte = f"""
|
||||
SELECT store.prefix, store.key, store.value, store.created_at, store.updated_at,
|
||||
{score_operator} AS neg_score
|
||||
FROM store
|
||||
JOIN store_vectors sv ON store.prefix = sv.prefix AND store.key = sv.key
|
||||
WHERE {ns_condition} {extra_filters}
|
||||
ORDER BY {score_operator} ASC
|
||||
LIMIT %s
|
||||
"""
|
||||
# Vector search with CTE for proper score handling
|
||||
filter_str = (
|
||||
""
|
||||
if not filter_conditions
|
||||
else " AND " + " AND ".join(filter_conditions)
|
||||
)
|
||||
if op.namespace_prefix:
|
||||
prefix_filter_str = f"WHERE s.prefix LIKE %s {filter_str} "
|
||||
ns_args: Sequence = (f"{_namespace_to_text(op.namespace_prefix)}%",)
|
||||
else:
|
||||
ns_args = ()
|
||||
if filter_str:
|
||||
prefix_filter_str = f"WHERE {filter_str} "
|
||||
else:
|
||||
prefix_filter_str = ""
|
||||
|
||||
search_results_sql = f"""
|
||||
WITH scored AS (
|
||||
{vector_search_cte}
|
||||
)
|
||||
SELECT uniq.prefix, uniq.key, uniq.value, uniq.created_at, uniq.updated_at,
|
||||
{post_operator} AS score
|
||||
FROM (
|
||||
SELECT DISTINCT ON (scored.prefix, scored.key)
|
||||
scored.prefix, scored.key, scored.value, scored.created_at, scored.updated_at, scored.neg_score
|
||||
FROM scored
|
||||
ORDER BY scored.prefix, scored.key, scored.neg_score ASC
|
||||
) uniq
|
||||
ORDER BY score DESC
|
||||
base_query = f"""
|
||||
WITH scored AS (
|
||||
SELECT s.prefix, s.key, s.value, s.created_at, s.updated_at, {score_operator} AS neg_score
|
||||
FROM store s
|
||||
JOIN store_vectors sv ON s.prefix = sv.prefix AND s.key = sv.key
|
||||
{prefix_filter_str}
|
||||
ORDER BY {score_operator} ASC
|
||||
LIMIT %s
|
||||
OFFSET %s
|
||||
"""
|
||||
|
||||
search_results_params = [
|
||||
PLACEHOLDER,
|
||||
*ns_param,
|
||||
)
|
||||
SELECT * FROM (
|
||||
SELECT DISTINCT ON (prefix, key)
|
||||
prefix, key, value, created_at, updated_at, {post_operator} as score
|
||||
FROM scored
|
||||
ORDER BY prefix, key, score DESC
|
||||
) AS unique_docs
|
||||
ORDER BY score DESC
|
||||
LIMIT %s
|
||||
OFFSET %s
|
||||
"""
|
||||
params = [
|
||||
_PLACEHOLDER, # Vector placeholder
|
||||
*ns_args,
|
||||
*filter_params,
|
||||
PLACEHOLDER,
|
||||
_PLACEHOLDER,
|
||||
expanded_limit,
|
||||
op.limit,
|
||||
op.offset,
|
||||
]
|
||||
|
||||
# Regular search branch
|
||||
else:
|
||||
base_query = f"""
|
||||
SELECT store.prefix, store.key, store.value, store.created_at, store.updated_at, NULL AS score
|
||||
FROM store
|
||||
WHERE {ns_condition} {extra_filters}
|
||||
ORDER BY store.updated_at DESC
|
||||
LIMIT %s
|
||||
OFFSET %s
|
||||
"""
|
||||
search_results_sql = base_query
|
||||
search_results_params = [
|
||||
*ns_param,
|
||||
*filter_params,
|
||||
op.limit,
|
||||
op.offset,
|
||||
]
|
||||
base_query = """
|
||||
SELECT prefix, key, value, created_at, updated_at
|
||||
FROM store
|
||||
WHERE prefix LIKE %s
|
||||
"""
|
||||
params = [f"{_namespace_to_text(op.namespace_prefix)}%"]
|
||||
|
||||
if op.refresh_ttl:
|
||||
# Wrap entire primary query in a CTE, then perform "update_at"
|
||||
final_sql = f"""
|
||||
WITH search_results AS (
|
||||
{search_results_sql}
|
||||
),
|
||||
updated AS (
|
||||
UPDATE store s
|
||||
SET expires_at = NOW() + (s.ttl_minutes || ' minutes')::interval
|
||||
FROM search_results sr
|
||||
WHERE s.prefix = sr.prefix
|
||||
AND s.key = sr.key
|
||||
AND s.ttl_minutes IS NOT NULL
|
||||
)
|
||||
SELECT sr.prefix, sr.key, sr.value, sr.created_at, sr.updated_at, sr.score
|
||||
FROM search_results sr
|
||||
"""
|
||||
final_params = search_results_params[:] # copy
|
||||
else:
|
||||
final_sql = search_results_sql
|
||||
final_params = search_results_params
|
||||
queries.append((final_sql, final_params))
|
||||
if filter_conditions:
|
||||
params.extend(filter_params)
|
||||
base_query += " AND " + " AND ".join(filter_conditions)
|
||||
|
||||
base_query += " ORDER BY updated_at DESC"
|
||||
base_query += " LIMIT %s OFFSET %s"
|
||||
params.extend([op.limit, op.offset])
|
||||
|
||||
queries.append((base_query, params))
|
||||
|
||||
return queries, embedding_requests
|
||||
|
||||
@@ -571,7 +462,7 @@ class BasePostgresStore(Generic[C]):
|
||||
) -> list[tuple[str, Sequence]]:
|
||||
queries: list[tuple[str, Sequence]] = []
|
||||
for _, op in list_ops:
|
||||
query = r"""
|
||||
query = """
|
||||
SELECT DISTINCT ON (truncated_prefix) truncated_prefix, prefix
|
||||
FROM (
|
||||
SELECT
|
||||
@@ -658,8 +549,7 @@ class PostgresStore(BaseStore, BasePostgresStore[_pg_internal.Conn]):
|
||||
item = store.get(("users", "123"), "prefs")
|
||||
```
|
||||
|
||||
Or using the convenient `from_conn_string` helper:
|
||||
|
||||
Or using the convenient from_conn_string helper:
|
||||
```python
|
||||
from langgraph.store.postgres import PostgresStore
|
||||
|
||||
@@ -696,7 +586,7 @@ class PostgresStore(BaseStore, BasePostgresStore[_pg_internal.Conn]):
|
||||
store.put(("docs",), "doc2", {"text": "Other guide"}, index=False) # don't index
|
||||
|
||||
# Search by similarity
|
||||
results = store.search(("docs",), query="programming guides", limit=2)
|
||||
results = store.search(("docs",), "programming guides", limit=2)
|
||||
```
|
||||
|
||||
Note:
|
||||
@@ -708,11 +598,6 @@ class PostgresStore(BaseStore, BasePostgresStore[_pg_internal.Conn]):
|
||||
Make sure to call `setup()` before first use to create necessary tables and indexes.
|
||||
The pgvector extension must be available to use vector search.
|
||||
|
||||
Note:
|
||||
If you provide a TTL configuration, you must explicitly call `start_ttl_sweeper()` to begin
|
||||
the background thread that removes expired items. Call `stop_ttl_sweeper()` to properly
|
||||
clean up resources when you're done with the store.
|
||||
|
||||
"""
|
||||
|
||||
__slots__ = (
|
||||
@@ -722,19 +607,17 @@ class PostgresStore(BaseStore, BasePostgresStore[_pg_internal.Conn]):
|
||||
"supports_pipeline",
|
||||
"index_config",
|
||||
"embeddings",
|
||||
"_ttl_sweeper_thread",
|
||||
"_ttl_stop_event",
|
||||
)
|
||||
supports_ttl: bool = True
|
||||
|
||||
def __init__(
|
||||
self,
|
||||
conn: _pg_internal.Conn,
|
||||
*,
|
||||
pipe: Pipeline | None = None,
|
||||
deserializer: Callable[[bytes | orjson.Fragment], dict[str, Any]] | None = None,
|
||||
index: PostgresIndexConfig | None = None,
|
||||
ttl: TTLConfig | None = None,
|
||||
pipe: Optional[Pipeline] = None,
|
||||
deserializer: Optional[
|
||||
Callable[[Union[bytes, orjson.Fragment]], dict[str, Any]]
|
||||
] = None,
|
||||
index: Optional[PostgresIndexConfig] = None,
|
||||
) -> None:
|
||||
super().__init__()
|
||||
self._deserializer = deserializer
|
||||
@@ -747,9 +630,6 @@ class PostgresStore(BaseStore, BasePostgresStore[_pg_internal.Conn]):
|
||||
self.embeddings, self.index_config = _ensure_index_config(self.index_config)
|
||||
else:
|
||||
self.embeddings = None
|
||||
self.ttl_config = ttl
|
||||
self._ttl_sweeper_thread: threading.Thread | None = None
|
||||
self._ttl_stop_event = threading.Event()
|
||||
|
||||
@classmethod
|
||||
@contextmanager
|
||||
@@ -758,20 +638,18 @@ class PostgresStore(BaseStore, BasePostgresStore[_pg_internal.Conn]):
|
||||
conn_string: str,
|
||||
*,
|
||||
pipeline: bool = False,
|
||||
pool_config: PoolConfig | None = None,
|
||||
index: PostgresIndexConfig | None = None,
|
||||
ttl: TTLConfig | None = None,
|
||||
) -> Iterator[PostgresStore]:
|
||||
pool_config: Optional[PoolConfig] = None,
|
||||
index: Optional[PostgresIndexConfig] = None,
|
||||
) -> Iterator["PostgresStore"]:
|
||||
"""Create a new PostgresStore instance from a connection string.
|
||||
|
||||
Args:
|
||||
conn_string: The Postgres connection info string.
|
||||
pipeline: whether to use Pipeline
|
||||
pool_config: Configuration for the connection pool.
|
||||
conn_string (str): The Postgres connection info string.
|
||||
pipeline (bool): whether to use Pipeline
|
||||
pool_config (Optional[PoolArgs]): Configuration for the connection pool.
|
||||
If provided, will create a connection pool and use it instead of a single connection.
|
||||
This overrides the `pipeline` argument.
|
||||
index: The index configuration for the store.
|
||||
ttl: The TTL configuration for the store.
|
||||
index (Optional[PostgresIndexConfig]): The index configuration for the store.
|
||||
|
||||
Returns:
|
||||
PostgresStore: A new PostgresStore instance.
|
||||
@@ -793,130 +671,23 @@ class PostgresStore(BaseStore, BasePostgresStore[_pg_internal.Conn]):
|
||||
**cast(dict, pc),
|
||||
),
|
||||
) as pool:
|
||||
yield cls(conn=pool, index=index, ttl=ttl)
|
||||
yield cls(conn=pool, index=index)
|
||||
else:
|
||||
with Connection.connect(
|
||||
conn_string, autocommit=True, prepare_threshold=0, row_factory=dict_row
|
||||
) as conn:
|
||||
if pipeline:
|
||||
with conn.pipeline() as pipe:
|
||||
yield cls(conn, pipe=pipe, index=index, ttl=ttl)
|
||||
yield cls(conn, pipe=pipe, index=index)
|
||||
else:
|
||||
yield cls(conn, index=index, ttl=ttl)
|
||||
|
||||
def sweep_ttl(self) -> int:
|
||||
"""Delete expired store items based on TTL.
|
||||
|
||||
Returns:
|
||||
int: The number of deleted items.
|
||||
"""
|
||||
with self._cursor() as cur:
|
||||
cur.execute(
|
||||
"""
|
||||
DELETE FROM store
|
||||
WHERE expires_at IS NOT NULL AND expires_at < NOW()
|
||||
"""
|
||||
)
|
||||
deleted_count = cur.rowcount
|
||||
return deleted_count
|
||||
|
||||
def start_ttl_sweeper(
|
||||
self, sweep_interval_minutes: int | None = None
|
||||
) -> concurrent.futures.Future[None]:
|
||||
"""Periodically delete expired store items based on TTL.
|
||||
|
||||
Returns:
|
||||
Future that can be waited on or cancelled.
|
||||
"""
|
||||
if not self.ttl_config:
|
||||
future: concurrent.futures.Future[None] = concurrent.futures.Future()
|
||||
future.set_result(None)
|
||||
return future
|
||||
|
||||
if self._ttl_sweeper_thread and self._ttl_sweeper_thread.is_alive():
|
||||
logger.info("TTL sweeper thread is already running")
|
||||
# Return a future that can be used to cancel the existing thread
|
||||
future = concurrent.futures.Future()
|
||||
future.add_done_callback(
|
||||
lambda f: self._ttl_stop_event.set() if f.cancelled() else None
|
||||
)
|
||||
return future
|
||||
|
||||
self._ttl_stop_event.clear()
|
||||
|
||||
interval = float(
|
||||
sweep_interval_minutes or self.ttl_config.get("sweep_interval_minutes") or 5
|
||||
)
|
||||
logger.info(f"Starting store TTL sweeper with interval {interval} minutes")
|
||||
|
||||
future = concurrent.futures.Future()
|
||||
|
||||
def _sweep_loop() -> None:
|
||||
try:
|
||||
while not self._ttl_stop_event.is_set():
|
||||
if self._ttl_stop_event.wait(interval * 60):
|
||||
break
|
||||
|
||||
try:
|
||||
expired_items = self.sweep_ttl()
|
||||
if expired_items > 0:
|
||||
logger.info(f"Store swept {expired_items} expired items")
|
||||
except Exception as exc:
|
||||
logger.exception(
|
||||
"Store TTL sweep iteration failed", exc_info=exc
|
||||
)
|
||||
future.set_result(None)
|
||||
except Exception as exc:
|
||||
future.set_exception(exc)
|
||||
|
||||
thread = threading.Thread(target=_sweep_loop, daemon=True, name="ttl-sweeper")
|
||||
self._ttl_sweeper_thread = thread
|
||||
thread.start()
|
||||
|
||||
future.add_done_callback(
|
||||
lambda f: self._ttl_stop_event.set() if f.cancelled() else None
|
||||
)
|
||||
return future
|
||||
|
||||
def stop_ttl_sweeper(self, timeout: float | None = None) -> bool:
|
||||
"""Stop the TTL sweeper thread if it's running.
|
||||
|
||||
Args:
|
||||
timeout: Maximum time to wait for the thread to stop, in seconds.
|
||||
If `None`, wait indefinitely.
|
||||
|
||||
Returns:
|
||||
bool: True if the thread was successfully stopped or wasn't running,
|
||||
False if the timeout was reached before the thread stopped.
|
||||
"""
|
||||
if not self._ttl_sweeper_thread or not self._ttl_sweeper_thread.is_alive():
|
||||
return True
|
||||
|
||||
logger.info("Stopping TTL sweeper thread")
|
||||
self._ttl_stop_event.set()
|
||||
|
||||
self._ttl_sweeper_thread.join(timeout)
|
||||
success = not self._ttl_sweeper_thread.is_alive()
|
||||
|
||||
if success:
|
||||
self._ttl_sweeper_thread = None
|
||||
logger.info("TTL sweeper thread stopped")
|
||||
else:
|
||||
logger.warning("Timed out waiting for TTL sweeper thread to stop")
|
||||
|
||||
return success
|
||||
|
||||
def __del__(self) -> None:
|
||||
"""Ensure the TTL sweeper thread is stopped when the object is garbage collected."""
|
||||
if hasattr(self, "_ttl_stop_event") and hasattr(self, "_ttl_sweeper_thread"):
|
||||
self.stop_ttl_sweeper(timeout=0.1)
|
||||
yield cls(conn, index=index)
|
||||
|
||||
@contextmanager
|
||||
def _cursor(self, *, pipeline: bool = False) -> Iterator[Cursor[DictRow]]:
|
||||
"""Create a database cursor as a context manager.
|
||||
|
||||
Args:
|
||||
pipeline: whether to use pipeline for the DB operations inside the context manager.
|
||||
pipeline (bool): whether to use pipeline for the DB operations inside the context manager.
|
||||
Will be applied regardless of whether the PostgresStore instance was initialized with a pipeline.
|
||||
If pipeline mode is not supported, will fall back to using transaction context manager.
|
||||
"""
|
||||
@@ -1029,9 +800,7 @@ class PostgresStore(BaseStore, BasePostgresStore[_pg_internal.Conn]):
|
||||
query,
|
||||
[
|
||||
p
|
||||
for (ns, k, pathname, _), vector in zip(
|
||||
txt_params, vectors, strict=False
|
||||
)
|
||||
for (ns, k, pathname, _), vector in zip(txt_params, vectors)
|
||||
for p in (ns, k, pathname, vector)
|
||||
],
|
||||
)
|
||||
@@ -1052,15 +821,13 @@ class PostgresStore(BaseStore, BasePostgresStore[_pg_internal.Conn]):
|
||||
embeddings = self.embeddings.embed_documents(
|
||||
[query for _, query in embedding_requests]
|
||||
)
|
||||
for (idx, _), embedding in zip(
|
||||
embedding_requests, embeddings, strict=False
|
||||
):
|
||||
for (idx, _), embedding in zip(embedding_requests, embeddings):
|
||||
_paramslist = queries[idx][1]
|
||||
for i in range(len(_paramslist)):
|
||||
if _paramslist[i] is PLACEHOLDER:
|
||||
if _paramslist[i] is _PLACEHOLDER:
|
||||
_paramslist[i] = embedding
|
||||
|
||||
for (idx, _), (query, params) in zip(search_ops, queries, strict=False):
|
||||
for (idx, _), (query, params) in zip(search_ops, queries):
|
||||
cur.execute(query, params)
|
||||
rows = cast(list[Row], cur.fetchall())
|
||||
results[idx] = [
|
||||
@@ -1077,14 +844,11 @@ class PostgresStore(BaseStore, BasePostgresStore[_pg_internal.Conn]):
|
||||
cur: Cursor[DictRow],
|
||||
) -> None:
|
||||
for (query, params), (idx, _) in zip(
|
||||
self._get_batch_list_namespaces_queries(list_ops), list_ops, strict=False
|
||||
self._get_batch_list_namespaces_queries(list_ops), list_ops
|
||||
):
|
||||
cur.execute(query, params)
|
||||
results[idx] = [_decode_ns_bytes(row["truncated_prefix"]) for row in cur]
|
||||
|
||||
async def abatch(self, ops: Iterable[Op]) -> list[Result]:
|
||||
return await asyncio.get_running_loop().run_in_executor(None, self.batch, ops)
|
||||
|
||||
def setup(self) -> None:
|
||||
"""Set up the store database.
|
||||
|
||||
@@ -1112,14 +876,8 @@ class PostgresStore(BaseStore, BasePostgresStore[_pg_internal.Conn]):
|
||||
with self._cursor() as cur:
|
||||
version = _get_version(cur, table="store_migrations")
|
||||
for v, sql in enumerate(self.MIGRATIONS[version + 1 :], start=version + 1):
|
||||
try:
|
||||
cur.execute(sql)
|
||||
cur.execute("INSERT INTO store_migrations (v) VALUES (%s)", (v,))
|
||||
except Exception as e:
|
||||
logger.error(
|
||||
f"Failed to apply migration {v}.\nSql={sql}\nError={e}"
|
||||
)
|
||||
raise
|
||||
cur.execute(sql)
|
||||
cur.execute("INSERT INTO store_migrations (v) VALUES (%s)", (v,))
|
||||
|
||||
if self.index_config:
|
||||
version = _get_version(cur, table="vector_migrations")
|
||||
@@ -1134,27 +892,6 @@ class PostgresStore(BaseStore, BasePostgresStore[_pg_internal.Conn]):
|
||||
k: v(self) if v is not None and callable(v) else v
|
||||
for k, v in migration.params.items()
|
||||
}
|
||||
if "dims" in params:
|
||||
try:
|
||||
params["dims"] = int(params["dims"])
|
||||
except Exception as e:
|
||||
raise ValueError(
|
||||
f"Invalid dims for vector index: {params['dims']}"
|
||||
) from e
|
||||
if "vector_type" in params:
|
||||
vt = str(params["vector_type"])
|
||||
if vt not in ("vector", "halfvec"):
|
||||
raise ValueError(
|
||||
f"Invalid vector_type for pgvector: {vt}"
|
||||
)
|
||||
params["vector_type"] = vt
|
||||
if "index_type" in params:
|
||||
it = str(params["index_type"])
|
||||
if it not in ("hnsw", "ivfflat"):
|
||||
raise ValueError(
|
||||
f"Invalid index_type for pgvector: {it}"
|
||||
)
|
||||
params["index_type"] = it
|
||||
sql = sql % params
|
||||
cur.execute(sql)
|
||||
cur.execute("INSERT INTO vector_migrations (v) VALUES (%s)", (v,))
|
||||
@@ -1180,7 +917,7 @@ def _get_vector_type_ops(store: BasePostgresStore) -> str:
|
||||
if not store.index_config:
|
||||
return "vector_cosine_ops"
|
||||
|
||||
config = store.index_config
|
||||
config = cast(PostgresIndexConfig, store.index_config)
|
||||
index_config = config.get("ann_index_config", _DEFAULT_ANN_CONFIG).copy()
|
||||
vector_type = cast(str, index_config.get("vector_type", "vector"))
|
||||
if vector_type not in ("vector", "halfvec"):
|
||||
@@ -1208,44 +945,15 @@ def _get_vector_type_ops(store: BasePostgresStore) -> str:
|
||||
|
||||
|
||||
def _get_index_params(store: Any) -> tuple[str, dict[str, Any]]:
|
||||
"""Get a sanitized index type and configuration based on config.
|
||||
|
||||
Only allow known-safe kinds and integer parameters to avoid SQL injection
|
||||
when constructing DDL strings for index creation.
|
||||
"""
|
||||
"""Get the index type and configuration based on config."""
|
||||
if not store.index_config:
|
||||
return "hnsw", {}
|
||||
|
||||
config = cast(PostgresIndexConfig, store.index_config)
|
||||
raw = config.get("ann_index_config", _DEFAULT_ANN_CONFIG).copy()
|
||||
|
||||
kind = str(raw.pop("kind", "hnsw"))
|
||||
if kind not in ("hnsw", "ivfflat", "flat"):
|
||||
raise ValueError(
|
||||
f"Invalid index kind for pgvector: {kind}. Expected 'hnsw', 'ivfflat', or 'flat'."
|
||||
)
|
||||
|
||||
raw.pop("vector_type", None)
|
||||
|
||||
if kind == "hnsw":
|
||||
allowed_keys = {"m", "ef_construction"}
|
||||
else: # ivfflat/flat
|
||||
allowed_keys = {"lists", "nlist"}
|
||||
|
||||
sanitized: dict[str, int] = {}
|
||||
for k, v in list(raw.items()):
|
||||
if k not in allowed_keys:
|
||||
continue
|
||||
key = "lists" if k == "nlist" else k
|
||||
try:
|
||||
ivalue = int(v) # type: ignore[call-overload]
|
||||
except Exception as e:
|
||||
raise ValueError(f"Invalid index parameter value for {k}: {v}") from e
|
||||
if ivalue <= 0:
|
||||
continue
|
||||
sanitized[key] = ivalue
|
||||
|
||||
return kind, sanitized
|
||||
index_config = config.get("ann_index_config", _DEFAULT_ANN_CONFIG).copy()
|
||||
kind = index_config.pop("kind", "hnsw")
|
||||
index_config.pop("vector_type", None)
|
||||
return kind, index_config
|
||||
|
||||
|
||||
def _namespace_to_text(
|
||||
@@ -1261,7 +969,7 @@ def _row_to_item(
|
||||
namespace: tuple[str, ...],
|
||||
row: Row,
|
||||
*,
|
||||
loader: Callable[[bytes | orjson.Fragment], dict[str, Any]] | None = None,
|
||||
loader: Optional[Callable[[Union[bytes, orjson.Fragment]], dict[str, Any]]] = None,
|
||||
) -> Item:
|
||||
"""Convert a row from the database into an Item.
|
||||
|
||||
@@ -1289,7 +997,7 @@ def _row_to_search_item(
|
||||
namespace: tuple[str, ...],
|
||||
row: Row,
|
||||
*,
|
||||
loader: Callable[[bytes | orjson.Fragment], dict[str, Any]] | None = None,
|
||||
loader: Optional[Callable[[Union[bytes, orjson.Fragment]], dict[str, Any]]] = None,
|
||||
) -> SearchItem:
|
||||
"""Convert a row from the database into an Item."""
|
||||
loader = loader or _json_loads
|
||||
@@ -1320,7 +1028,7 @@ def _group_ops(ops: Iterable[Op]) -> tuple[dict[type, list[tuple[int, Op]]], int
|
||||
return grouped_ops, tot
|
||||
|
||||
|
||||
def _json_loads(content: bytes | orjson.Fragment) -> Any:
|
||||
def _json_loads(content: Union[bytes, orjson.Fragment]) -> Any:
|
||||
if isinstance(content, orjson.Fragment):
|
||||
if hasattr(content, "buf"):
|
||||
content = content.buf
|
||||
@@ -1332,7 +1040,7 @@ def _json_loads(content: bytes | orjson.Fragment) -> Any:
|
||||
return orjson.loads(cast(bytes, content))
|
||||
|
||||
|
||||
def _decode_ns_bytes(namespace: str | bytes | list) -> tuple[str, ...]:
|
||||
def _decode_ns_bytes(namespace: Union[str, bytes, list]) -> tuple[str, ...]:
|
||||
if isinstance(namespace, list):
|
||||
return tuple(namespace)
|
||||
if isinstance(namespace, bytes):
|
||||
@@ -1340,7 +1048,7 @@ def _decode_ns_bytes(namespace: str | bytes | list) -> tuple[str, ...]:
|
||||
return tuple(namespace.split("."))
|
||||
|
||||
|
||||
def get_distance_operator(store: Any) -> tuple[str, str]:
|
||||
def _get_distance_operator(store: Any) -> tuple[str, str]:
|
||||
"""Get the distance operator and score expression based on config."""
|
||||
# Note: Today, we are not using ANN indices due to restrictions
|
||||
# on PGVector's support for mixing vector and non-vector filters
|
||||
@@ -1381,16 +1089,16 @@ def get_distance_operator(store: Any) -> tuple[str, str]:
|
||||
|
||||
def _ensure_index_config(
|
||||
index_config: PostgresIndexConfig,
|
||||
) -> tuple[Embeddings | None, PostgresIndexConfig]:
|
||||
) -> tuple[Optional["Embeddings"], PostgresIndexConfig]:
|
||||
index_config = index_config.copy()
|
||||
tokenized: list[tuple[str, Literal["$"] | list[str]]] = []
|
||||
tokenized: list[tuple[str, Union[Literal["$"], list[str]]]] = []
|
||||
tot = 0
|
||||
fields = index_config.get("fields") or ["$"]
|
||||
if isinstance(fields, str):
|
||||
fields = [fields]
|
||||
if not isinstance(fields, list):
|
||||
raise ValueError(f"Text fields must be a list or a string. Got {fields}")
|
||||
for p in fields:
|
||||
text_fields = index_config.get("text_fields") or ["$"]
|
||||
if isinstance(text_fields, str):
|
||||
text_fields = [text_fields]
|
||||
if not isinstance(text_fields, list):
|
||||
raise ValueError(f"Text fields must be a list or a string. Got {text_fields}")
|
||||
for p in text_fields:
|
||||
if p == "$":
|
||||
tokenized.append((p, "$"))
|
||||
tot += 1
|
||||
@@ -1406,4 +1114,4 @@ def _ensure_index_config(
|
||||
return embeddings, index_config
|
||||
|
||||
|
||||
PLACEHOLDER = object()
|
||||
_PLACEHOLDER = object()
|
||||
Generated
+1258
File diff suppressed because it is too large.
Load diff
@@ -1,62 +1,47 @@
|
||||
[build-system]
|
||||
requires = ["hatchling"]
|
||||
build-backend = "hatchling.build"
|
||||
|
||||
[project]
|
||||
[tool.poetry]
|
||||
name = "langgraph-checkpoint-postgres"
|
||||
version = "3.0.3"
|
||||
version = "2.0.15"
|
||||
description = "Library with a Postgres implementation of LangGraph checkpoint saver."
|
||||
authors = []
|
||||
requires-python = ">=3.10"
|
||||
readme = "README.md"
|
||||
license = "MIT"
|
||||
license-files = ['LICENSE']
|
||||
dependencies = [
|
||||
"langgraph-checkpoint>=2.1.2,<5.0.0",
|
||||
"orjson>=3.10.1",
|
||||
"psycopg>=3.2.0",
|
||||
"psycopg-pool>=3.2.0",
|
||||
]
|
||||
readme = "README.md"
|
||||
repository = "https://www.github.com/langchain-ai/langgraph"
|
||||
packages = [{ include = "langgraph" }]
|
||||
|
||||
[project.urls]
|
||||
Source = "https://github.com/langchain-ai/langgraph/tree/main/libs/checkpoint-postgres"
|
||||
Twitter = "https://x.com/LangChainAI"
|
||||
Slack = "https://www.langchain.com/join-community"
|
||||
Reddit = "https://www.reddit.com/r/LangChain/"
|
||||
[tool.poetry.dependencies]
|
||||
python = "^3.9.0,<4.0"
|
||||
langgraph-checkpoint = "^2.0.15"
|
||||
orjson = ">=3.10.1"
|
||||
psycopg = "^3.2.0"
|
||||
psycopg-pool = "^3.2.0"
|
||||
|
||||
[dependency-groups]
|
||||
test = [
|
||||
"pytest",
|
||||
"anyio",
|
||||
"pytest-asyncio",
|
||||
"pytest-mock",
|
||||
"psycopg[binary]",
|
||||
"langgraph-checkpoint",
|
||||
"pytest-watcher",
|
||||
]
|
||||
lint = [
|
||||
"ruff",
|
||||
"codespell",
|
||||
"mypy",
|
||||
]
|
||||
dev = [
|
||||
{include-group = "test"},
|
||||
{include-group = "lint"},
|
||||
]
|
||||
|
||||
[tool.uv]
|
||||
default-groups = ['dev']
|
||||
|
||||
[tool.uv.sources]
|
||||
langgraph-checkpoint = { path = "../checkpoint", editable = true }
|
||||
|
||||
[tool.hatch.build.targets.wheel]
|
||||
include = ["langgraph"]
|
||||
[tool.poetry.group.dev.dependencies]
|
||||
ruff = "^0.6.2"
|
||||
codespell = "^2.2.0"
|
||||
pytest = "^7.2.1"
|
||||
anyio = "^4.4.0"
|
||||
pytest-asyncio = "^0.21.1"
|
||||
pytest-mock = "^3.11.1"
|
||||
mypy = "^1.10.0"
|
||||
psycopg = {extras = ["binary"], version = ">=3.0.0"}
|
||||
langgraph-checkpoint = {path = "../checkpoint", develop = true}
|
||||
pytest-watcher = "^0.4.3"
|
||||
|
||||
[tool.pytest.ini_options]
|
||||
# --strict-markers will raise errors on unknown marks.
|
||||
# https://docs.pytest.org/en/7.1.x/how-to/mark.html#raising-errors-on-unknown-marks
|
||||
#
|
||||
# https://docs.pytest.org/en/7.1.x/reference/reference.html
|
||||
# --strict-config any warnings encountered while parsing the `pytest`
|
||||
# section of the configuration file raise errors.
|
||||
addopts = "--strict-markers --strict-config --durations=5 -vv"
|
||||
asyncio_mode = "auto"
|
||||
|
||||
|
||||
[build-system]
|
||||
requires = ["poetry-core"]
|
||||
build-backend = "poetry.core.masonry.api"
|
||||
|
||||
[tool.ruff]
|
||||
lint.select = [
|
||||
"E", # pycodestyle
|
||||
@@ -64,10 +49,8 @@ lint.select = [
|
||||
"UP", # pyupgrade
|
||||
"B", # flake8-bugbear
|
||||
"I", # isort
|
||||
"UP", # pyupgrade
|
||||
]
|
||||
lint.ignore = ["E501", "B008"]
|
||||
target-version = "py310"
|
||||
lint.ignore = ["E501", "B008", "UP007", "UP006"]
|
||||
|
||||
[tool.mypy]
|
||||
# https://mypy.readthedocs.io/en/stable/config_file.html
|
||||
|
||||
@@ -1,373 +0,0 @@
|
||||
# type: ignore
|
||||
|
||||
from contextlib import asynccontextmanager
|
||||
from typing import Any
|
||||
from uuid import uuid4
|
||||
|
||||
import pytest
|
||||
from langchain_core.runnables import RunnableConfig
|
||||
from langgraph.checkpoint.base import (
|
||||
EXCLUDED_METADATA_KEYS,
|
||||
Checkpoint,
|
||||
CheckpointMetadata,
|
||||
create_checkpoint,
|
||||
empty_checkpoint,
|
||||
)
|
||||
from langgraph.checkpoint.serde.types import TASKS
|
||||
from psycopg import AsyncConnection
|
||||
from psycopg.rows import dict_row
|
||||
from psycopg_pool import AsyncConnectionPool
|
||||
|
||||
from langgraph.checkpoint.postgres.aio import (
|
||||
AsyncPostgresSaver,
|
||||
AsyncShallowPostgresSaver,
|
||||
)
|
||||
from tests.conftest import DEFAULT_POSTGRES_URI
|
||||
|
||||
|
||||
def _exclude_keys(config: dict[str, Any]) -> dict[str, Any]:
|
||||
return {k: v for k, v in config.items() if k not in EXCLUDED_METADATA_KEYS}
|
||||
|
||||
|
||||
@asynccontextmanager
|
||||
async def _pool_saver():
|
||||
"""Fixture for pool mode testing."""
|
||||
database = f"test_{uuid4().hex[:16]}"
|
||||
# create unique db
|
||||
async with await AsyncConnection.connect(
|
||||
DEFAULT_POSTGRES_URI, autocommit=True
|
||||
) as conn:
|
||||
await conn.execute(f"CREATE DATABASE {database}")
|
||||
try:
|
||||
# yield checkpointer
|
||||
async with AsyncConnectionPool(
|
||||
DEFAULT_POSTGRES_URI + database,
|
||||
max_size=10,
|
||||
kwargs={"autocommit": True, "row_factory": dict_row},
|
||||
) as pool:
|
||||
checkpointer = AsyncPostgresSaver(pool)
|
||||
await checkpointer.setup()
|
||||
yield checkpointer
|
||||
finally:
|
||||
# drop unique db
|
||||
async with await AsyncConnection.connect(
|
||||
DEFAULT_POSTGRES_URI, autocommit=True
|
||||
) as conn:
|
||||
await conn.execute(f"DROP DATABASE {database}")
|
||||
|
||||
|
||||
@asynccontextmanager
|
||||
async def _pipe_saver():
|
||||
"""Fixture for pipeline mode testing."""
|
||||
database = f"test_{uuid4().hex[:16]}"
|
||||
# create unique db
|
||||
async with await AsyncConnection.connect(
|
||||
DEFAULT_POSTGRES_URI, autocommit=True
|
||||
) as conn:
|
||||
await conn.execute(f"CREATE DATABASE {database}")
|
||||
try:
|
||||
async with await AsyncConnection.connect(
|
||||
DEFAULT_POSTGRES_URI + database,
|
||||
autocommit=True,
|
||||
prepare_threshold=0,
|
||||
row_factory=dict_row,
|
||||
) as conn:
|
||||
checkpointer = AsyncPostgresSaver(conn)
|
||||
await checkpointer.setup()
|
||||
async with conn.pipeline() as pipe:
|
||||
checkpointer = AsyncPostgresSaver(conn, pipe=pipe)
|
||||
yield checkpointer
|
||||
finally:
|
||||
# drop unique db
|
||||
async with await AsyncConnection.connect(
|
||||
DEFAULT_POSTGRES_URI, autocommit=True
|
||||
) as conn:
|
||||
await conn.execute(f"DROP DATABASE {database}")
|
||||
|
||||
|
||||
@asynccontextmanager
|
||||
async def _base_saver():
|
||||
"""Fixture for regular connection mode testing."""
|
||||
database = f"test_{uuid4().hex[:16]}"
|
||||
# create unique db
|
||||
async with await AsyncConnection.connect(
|
||||
DEFAULT_POSTGRES_URI, autocommit=True
|
||||
) as conn:
|
||||
await conn.execute(f"CREATE DATABASE {database}")
|
||||
try:
|
||||
async with await AsyncConnection.connect(
|
||||
DEFAULT_POSTGRES_URI + database,
|
||||
autocommit=True,
|
||||
prepare_threshold=0,
|
||||
row_factory=dict_row,
|
||||
) as conn:
|
||||
checkpointer = AsyncPostgresSaver(conn)
|
||||
await checkpointer.setup()
|
||||
yield checkpointer
|
||||
finally:
|
||||
# drop unique db
|
||||
async with await AsyncConnection.connect(
|
||||
DEFAULT_POSTGRES_URI, autocommit=True
|
||||
) as conn:
|
||||
await conn.execute(f"DROP DATABASE {database}")
|
||||
|
||||
|
||||
@asynccontextmanager
|
||||
async def _shallow_saver():
|
||||
"""Fixture for shallow connection mode testing."""
|
||||
database = f"test_{uuid4().hex[:16]}"
|
||||
# create unique db
|
||||
async with await AsyncConnection.connect(
|
||||
DEFAULT_POSTGRES_URI, autocommit=True
|
||||
) as conn:
|
||||
await conn.execute(f"CREATE DATABASE {database}")
|
||||
try:
|
||||
async with await AsyncConnection.connect(
|
||||
DEFAULT_POSTGRES_URI + database,
|
||||
autocommit=True,
|
||||
prepare_threshold=0,
|
||||
row_factory=dict_row,
|
||||
) as conn:
|
||||
checkpointer = AsyncShallowPostgresSaver(conn)
|
||||
await checkpointer.setup()
|
||||
yield checkpointer
|
||||
finally:
|
||||
# drop unique db
|
||||
async with await AsyncConnection.connect(
|
||||
DEFAULT_POSTGRES_URI, autocommit=True
|
||||
) as conn:
|
||||
await conn.execute(f"DROP DATABASE {database}")
|
||||
|
||||
|
||||
@asynccontextmanager
|
||||
async def _saver(name: str):
|
||||
if name == "base":
|
||||
async with _base_saver() as saver:
|
||||
yield saver
|
||||
elif name == "shallow":
|
||||
async with _shallow_saver() as saver:
|
||||
yield saver
|
||||
elif name == "pool":
|
||||
async with _pool_saver() as saver:
|
||||
yield saver
|
||||
elif name == "pipe":
|
||||
async with _pipe_saver() as saver:
|
||||
yield saver
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def test_data():
|
||||
"""Fixture providing test data for checkpoint tests."""
|
||||
config_1: RunnableConfig = {
|
||||
"configurable": {
|
||||
"thread_id": "thread-1",
|
||||
"checkpoint_id": "1",
|
||||
"checkpoint_ns": "",
|
||||
}
|
||||
}
|
||||
config_2: RunnableConfig = {
|
||||
"configurable": {
|
||||
"thread_id": "thread-2",
|
||||
"checkpoint_id": "2",
|
||||
"checkpoint_ns": "",
|
||||
}
|
||||
}
|
||||
config_3: RunnableConfig = {
|
||||
"configurable": {
|
||||
"thread_id": "thread-2",
|
||||
"checkpoint_id": "2-inner",
|
||||
"checkpoint_ns": "inner",
|
||||
}
|
||||
}
|
||||
|
||||
chkpnt_1: Checkpoint = empty_checkpoint()
|
||||
chkpnt_2: Checkpoint = create_checkpoint(chkpnt_1, {}, 1)
|
||||
chkpnt_3: Checkpoint = empty_checkpoint()
|
||||
|
||||
metadata_1: CheckpointMetadata = {
|
||||
"source": "input",
|
||||
"step": 2,
|
||||
"score": 1,
|
||||
}
|
||||
metadata_2: CheckpointMetadata = {
|
||||
"source": "loop",
|
||||
"step": 1,
|
||||
"score": None,
|
||||
}
|
||||
metadata_3: CheckpointMetadata = {}
|
||||
|
||||
return {
|
||||
"configs": [config_1, config_2, config_3],
|
||||
"checkpoints": [chkpnt_1, chkpnt_2, chkpnt_3],
|
||||
"metadata": [metadata_1, metadata_2, metadata_3],
|
||||
}
|
||||
|
||||
|
||||
@pytest.mark.parametrize("saver_name", ["base", "pool", "pipe", "shallow"])
|
||||
async def test_combined_metadata(saver_name: str, test_data) -> None:
|
||||
async with _saver(saver_name) as saver:
|
||||
config = {
|
||||
"configurable": {
|
||||
"thread_id": "thread-2",
|
||||
"checkpoint_ns": "",
|
||||
"__super_private_key": "super_private_value",
|
||||
},
|
||||
"metadata": {"run_id": "my_run_id"},
|
||||
}
|
||||
chkpnt: Checkpoint = create_checkpoint(empty_checkpoint(), {}, 1)
|
||||
metadata: CheckpointMetadata = {
|
||||
"source": "loop",
|
||||
"step": 1,
|
||||
"score": None,
|
||||
}
|
||||
await saver.aput(config, chkpnt, metadata, {})
|
||||
checkpoint = await saver.aget_tuple(config)
|
||||
assert checkpoint.metadata == {
|
||||
**metadata,
|
||||
"run_id": "my_run_id",
|
||||
}
|
||||
|
||||
|
||||
@pytest.mark.parametrize("saver_name", ["base", "pool", "pipe", "shallow"])
|
||||
async def test_asearch(saver_name: str, test_data) -> None:
|
||||
async with _saver(saver_name) as saver:
|
||||
configs = test_data["configs"]
|
||||
checkpoints = test_data["checkpoints"]
|
||||
metadata = test_data["metadata"]
|
||||
|
||||
await saver.aput(configs[0], checkpoints[0], metadata[0], {})
|
||||
await saver.aput(configs[1], checkpoints[1], metadata[1], {})
|
||||
await saver.aput(configs[2], checkpoints[2], metadata[2], {})
|
||||
|
||||
# call method / assertions
|
||||
query_1 = {"source": "input"} # search by 1 key
|
||||
query_2 = {
|
||||
"step": 1,
|
||||
} # search by multiple keys
|
||||
query_3: dict[str, Any] = {} # search by no keys, return all checkpoints
|
||||
query_4 = {"source": "update", "step": 1} # no match
|
||||
|
||||
search_results_1 = [c async for c in saver.alist(None, filter=query_1)]
|
||||
assert len(search_results_1) == 1
|
||||
assert search_results_1[0].metadata == {
|
||||
**_exclude_keys(configs[0]["configurable"]),
|
||||
**metadata[0],
|
||||
}
|
||||
|
||||
search_results_2 = [c async for c in saver.alist(None, filter=query_2)]
|
||||
assert len(search_results_2) == 1
|
||||
assert search_results_2[0].metadata == {
|
||||
**_exclude_keys(configs[1]["configurable"]),
|
||||
**metadata[1],
|
||||
}
|
||||
|
||||
search_results_3 = [c async for c in saver.alist(None, filter=query_3)]
|
||||
assert len(search_results_3) == 3
|
||||
|
||||
search_results_4 = [c async for c in saver.alist(None, filter=query_4)]
|
||||
assert len(search_results_4) == 0
|
||||
|
||||
# search by config (defaults to checkpoints across all namespaces)
|
||||
search_results_5 = [
|
||||
c async for c in saver.alist({"configurable": {"thread_id": "thread-2"}})
|
||||
]
|
||||
assert len(search_results_5) == 2
|
||||
assert {
|
||||
search_results_5[0].config["configurable"]["checkpoint_ns"],
|
||||
search_results_5[1].config["configurable"]["checkpoint_ns"],
|
||||
} == {"", "inner"}
|
||||
|
||||
|
||||
@pytest.mark.parametrize("saver_name", ["base", "pool", "pipe", "shallow"])
|
||||
async def test_null_chars(saver_name: str, test_data) -> None:
|
||||
async with _saver(saver_name) as saver:
|
||||
config = await saver.aput(
|
||||
test_data["configs"][0],
|
||||
test_data["checkpoints"][0],
|
||||
{"my_key": "\x00abc"},
|
||||
{},
|
||||
)
|
||||
assert (await saver.aget_tuple(config)).metadata["my_key"] == "abc" # type: ignore
|
||||
assert [c async for c in saver.alist(None, filter={"my_key": "abc"})][
|
||||
0
|
||||
].metadata["my_key"] == "abc"
|
||||
|
||||
|
||||
@pytest.mark.parametrize("saver_name", ["base", "pool", "pipe"])
|
||||
async def test_pending_sends_migration(saver_name: str) -> None:
|
||||
async with _saver(saver_name) as saver:
|
||||
config = {
|
||||
"configurable": {
|
||||
"thread_id": "thread-1",
|
||||
"checkpoint_ns": "",
|
||||
}
|
||||
}
|
||||
|
||||
# create the first checkpoint
|
||||
# and put some pending sends
|
||||
checkpoint_0 = empty_checkpoint()
|
||||
config = await saver.aput(config, checkpoint_0, {}, {})
|
||||
await saver.aput_writes(
|
||||
config, [(TASKS, "send-1"), (TASKS, "send-2")], task_id="task-1"
|
||||
)
|
||||
await saver.aput_writes(config, [(TASKS, "send-3")], task_id="task-2")
|
||||
|
||||
# check that fetching checkpoint_0 doesn't attach pending sends
|
||||
# (they should be attached to the next checkpoint)
|
||||
tuple_0 = await saver.aget_tuple(config)
|
||||
assert tuple_0.checkpoint["channel_values"] == {}
|
||||
assert tuple_0.checkpoint["channel_versions"] == {}
|
||||
|
||||
# create the second checkpoint
|
||||
checkpoint_1 = create_checkpoint(checkpoint_0, {}, 1)
|
||||
config = await saver.aput(config, checkpoint_1, {}, {})
|
||||
|
||||
# check that pending sends are attached to checkpoint_1
|
||||
tuple_1 = await saver.aget_tuple(config)
|
||||
assert tuple_1.checkpoint["channel_values"] == {
|
||||
TASKS: ["send-1", "send-2", "send-3"]
|
||||
}
|
||||
assert TASKS in tuple_1.checkpoint["channel_versions"]
|
||||
|
||||
# check that list also applies the migration
|
||||
search_results = [
|
||||
c async for c in saver.alist({"configurable": {"thread_id": "thread-1"}})
|
||||
]
|
||||
assert len(search_results) == 2
|
||||
assert search_results[-1].checkpoint["channel_values"] == {}
|
||||
assert search_results[-1].checkpoint["channel_versions"] == {}
|
||||
assert search_results[0].checkpoint["channel_values"] == {
|
||||
TASKS: ["send-1", "send-2", "send-3"]
|
||||
}
|
||||
assert TASKS in search_results[0].checkpoint["channel_versions"]
|
||||
|
||||
|
||||
@pytest.mark.parametrize("saver_name", ["base", "pool", "pipe"])
|
||||
async def test_get_checkpoint_no_channel_values(
|
||||
monkeypatch, saver_name: str, test_data
|
||||
) -> None:
|
||||
"""Backwards compatibility test that verifies a checkpoint with no channel_values key can be retrieved without throwing an error."""
|
||||
async with _saver(saver_name) as saver:
|
||||
config = {
|
||||
"configurable": {
|
||||
"thread_id": "thread-2",
|
||||
"checkpoint_ns": "",
|
||||
"__super_private_key": "super_private_value",
|
||||
},
|
||||
"metadata": {"run_id": "my_run_id"},
|
||||
}
|
||||
chkpnt: Checkpoint = create_checkpoint(empty_checkpoint(), {}, 1)
|
||||
await saver.aput(config, chkpnt, {}, {})
|
||||
|
||||
load_checkpoint_tuple = saver._load_checkpoint_tuple
|
||||
|
||||
def patched_load_checkpoint_tuple(value):
|
||||
value["checkpoint"].pop("channel_values", None)
|
||||
return load_checkpoint_tuple(value)
|
||||
|
||||
monkeypatch.setattr(
|
||||
saver, "_load_checkpoint_tuple", patched_load_checkpoint_tuple
|
||||
)
|
||||
|
||||
checkpoint = await saver.aget_tuple(config)
|
||||
assert checkpoint.checkpoint["channel_values"] == {}
|
||||
@@ -1,687 +0,0 @@
|
||||
# type: ignore
|
||||
from __future__ import annotations
|
||||
|
||||
import asyncio
|
||||
import itertools
|
||||
import uuid
|
||||
from collections.abc import AsyncIterator
|
||||
from concurrent.futures import ThreadPoolExecutor
|
||||
from contextlib import asynccontextmanager
|
||||
from typing import Any
|
||||
|
||||
import pytest
|
||||
from langchain_core.embeddings import Embeddings
|
||||
from langgraph.store.base import (
|
||||
GetOp,
|
||||
Item,
|
||||
ListNamespacesOp,
|
||||
PutOp,
|
||||
SearchOp,
|
||||
)
|
||||
from psycopg import AsyncConnection
|
||||
|
||||
from langgraph.store.postgres import AsyncPostgresStore
|
||||
from tests.conftest import (
|
||||
DEFAULT_URI,
|
||||
VECTOR_TYPES,
|
||||
CharacterEmbeddings,
|
||||
)
|
||||
|
||||
TTL_SECONDS = 6
|
||||
TTL_MINUTES = TTL_SECONDS / 60
|
||||
|
||||
|
||||
@pytest.fixture(scope="function", params=["default", "pipe", "pool"])
|
||||
async def store(request) -> AsyncIterator[AsyncPostgresStore]:
|
||||
database = f"test_{uuid.uuid4().hex[:16]}"
|
||||
uri_parts = DEFAULT_URI.split("/")
|
||||
uri_base = "/".join(uri_parts[:-1])
|
||||
query_params = ""
|
||||
if "?" in uri_parts[-1]:
|
||||
db_name, query_params = uri_parts[-1].split("?", 1)
|
||||
query_params = "?" + query_params
|
||||
|
||||
conn_string = f"{uri_base}/{database}{query_params}"
|
||||
admin_conn_string = DEFAULT_URI
|
||||
ttl_config = {
|
||||
"default_ttl": TTL_MINUTES,
|
||||
"refresh_on_read": True,
|
||||
"sweep_interval_minutes": TTL_MINUTES / 2,
|
||||
}
|
||||
async with await AsyncConnection.connect(
|
||||
admin_conn_string, autocommit=True
|
||||
) as conn:
|
||||
await conn.execute(f"CREATE DATABASE {database}")
|
||||
try:
|
||||
async with AsyncPostgresStore.from_conn_string(
|
||||
conn_string, ttl=ttl_config
|
||||
) as store:
|
||||
store.MIGRATIONS = [
|
||||
(
|
||||
mig.replace("ttl_minutes INT;", "ttl_minutes FLOAT;")
|
||||
if isinstance(mig, str)
|
||||
else mig
|
||||
)
|
||||
for mig in store.MIGRATIONS
|
||||
]
|
||||
await store.setup()
|
||||
async with store._cursor() as cur:
|
||||
# drop the migration index
|
||||
await cur.execute("DROP TABLE IF EXISTS store_migrations")
|
||||
await store.setup() # Will fail if migrations aren't idempotent
|
||||
|
||||
if request.param == "pipe":
|
||||
async with AsyncPostgresStore.from_conn_string(
|
||||
conn_string, pipeline=True, ttl=ttl_config
|
||||
) as store:
|
||||
await store.start_ttl_sweeper()
|
||||
yield store
|
||||
await store.stop_ttl_sweeper()
|
||||
elif request.param == "pool":
|
||||
async with AsyncPostgresStore.from_conn_string(
|
||||
conn_string, pool_config={"min_size": 1, "max_size": 10}, ttl=ttl_config
|
||||
) as store:
|
||||
await store.start_ttl_sweeper()
|
||||
yield store
|
||||
await store.stop_ttl_sweeper()
|
||||
else: # default
|
||||
async with AsyncPostgresStore.from_conn_string(
|
||||
conn_string, ttl=ttl_config
|
||||
) as store:
|
||||
await store.start_ttl_sweeper()
|
||||
yield store
|
||||
await store.stop_ttl_sweeper()
|
||||
finally:
|
||||
async with await AsyncConnection.connect(
|
||||
admin_conn_string, autocommit=True
|
||||
) as conn:
|
||||
await conn.execute(f"DROP DATABASE {database}")
|
||||
|
||||
|
||||
async def test_no_running_loop(store: AsyncPostgresStore) -> None:
|
||||
with pytest.raises(asyncio.InvalidStateError):
|
||||
store.put(("foo", "bar"), "baz", {"val": "baz"})
|
||||
with pytest.raises(asyncio.InvalidStateError):
|
||||
store.get(("foo", "bar"), "baz")
|
||||
with pytest.raises(asyncio.InvalidStateError):
|
||||
store.delete(("foo", "bar"), "baz")
|
||||
with pytest.raises(asyncio.InvalidStateError):
|
||||
store.search(("foo", "bar"))
|
||||
with pytest.raises(asyncio.InvalidStateError):
|
||||
store.list_namespaces(prefix=("foo",))
|
||||
with pytest.raises(asyncio.InvalidStateError):
|
||||
store.batch([PutOp(namespace=("foo", "bar"), key="baz", value={"val": "baz"})])
|
||||
with ThreadPoolExecutor(max_workers=1) as executor:
|
||||
future = executor.submit(store.put, ("foo", "bar"), "baz", {"val": "baz"})
|
||||
result = await asyncio.wrap_future(future)
|
||||
assert result is None
|
||||
future = executor.submit(store.get, ("foo", "bar"), "baz")
|
||||
result = await asyncio.wrap_future(future)
|
||||
assert result.value == {"val": "baz"}
|
||||
result = await asyncio.wrap_future(
|
||||
executor.submit(store.list_namespaces, prefix=("foo",))
|
||||
)
|
||||
|
||||
|
||||
async def test_large_batches(request: Any, store: AsyncPostgresStore) -> None:
|
||||
N = 100 # less important that we are performant here
|
||||
M = 10
|
||||
|
||||
with ThreadPoolExecutor(max_workers=10) as executor:
|
||||
futures = []
|
||||
for m in range(M):
|
||||
for i in range(N):
|
||||
futures += [
|
||||
executor.submit(
|
||||
store.put,
|
||||
("test", "foo", "bar", "baz", str(m % 2)),
|
||||
f"key{i}",
|
||||
value={"foo": "bar" + str(i)},
|
||||
),
|
||||
executor.submit(
|
||||
store.get,
|
||||
("test", "foo", "bar", "baz", str(m % 2)),
|
||||
f"key{i}",
|
||||
),
|
||||
executor.submit(
|
||||
store.list_namespaces,
|
||||
prefix=None,
|
||||
max_depth=m + 1,
|
||||
),
|
||||
executor.submit(
|
||||
store.search,
|
||||
("test",),
|
||||
),
|
||||
executor.submit(
|
||||
store.put,
|
||||
("test", "foo", "bar", "baz", str(m % 2)),
|
||||
f"key{i}",
|
||||
value={"foo": "bar" + str(i)},
|
||||
),
|
||||
executor.submit(
|
||||
store.put,
|
||||
("test", "foo", "bar", "baz", str(m % 2)),
|
||||
f"key{i}",
|
||||
None,
|
||||
),
|
||||
]
|
||||
|
||||
results = await asyncio.gather(
|
||||
*(asyncio.wrap_future(future) for future in futures)
|
||||
)
|
||||
assert len(results) == M * N * 6
|
||||
|
||||
|
||||
async def test_large_batches_async(store: AsyncPostgresStore) -> None:
|
||||
N = 1000
|
||||
M = 10
|
||||
coros = []
|
||||
for m in range(M):
|
||||
for i in range(N):
|
||||
coros.append(
|
||||
store.aput(
|
||||
("test", "foo", "bar", "baz", str(m % 2)),
|
||||
f"key{i}",
|
||||
value={"foo": "bar" + str(i)},
|
||||
)
|
||||
)
|
||||
coros.append(
|
||||
store.aget(
|
||||
("test", "foo", "bar", "baz", str(m % 2)),
|
||||
f"key{i}",
|
||||
)
|
||||
)
|
||||
coros.append(
|
||||
store.alist_namespaces(
|
||||
prefix=None,
|
||||
max_depth=m + 1,
|
||||
)
|
||||
)
|
||||
coros.append(
|
||||
store.asearch(
|
||||
("test",),
|
||||
)
|
||||
)
|
||||
coros.append(
|
||||
store.aput(
|
||||
("test", "foo", "bar", "baz", str(m % 2)),
|
||||
f"key{i}",
|
||||
value={"foo": "bar" + str(i)},
|
||||
)
|
||||
)
|
||||
coros.append(
|
||||
store.adelete(
|
||||
("test", "foo", "bar", "baz", str(m % 2)),
|
||||
f"key{i}",
|
||||
)
|
||||
)
|
||||
|
||||
results = await asyncio.gather(*coros)
|
||||
assert len(results) == M * N * 6
|
||||
|
||||
|
||||
async def test_abatch_order(store: AsyncPostgresStore) -> None:
|
||||
# Setup test data
|
||||
await store.aput(("test", "foo"), "key1", {"data": "value1"})
|
||||
await store.aput(("test", "bar"), "key2", {"data": "value2"})
|
||||
|
||||
ops = [
|
||||
GetOp(namespace=("test", "foo"), key="key1"),
|
||||
PutOp(namespace=("test", "bar"), key="key2", value={"data": "value2"}),
|
||||
SearchOp(
|
||||
namespace_prefix=("test",), filter={"data": "value1"}, limit=10, offset=0
|
||||
),
|
||||
ListNamespacesOp(match_conditions=None, max_depth=None, limit=10, offset=0),
|
||||
GetOp(namespace=("test",), key="key3"),
|
||||
]
|
||||
|
||||
results = await store.abatch(ops)
|
||||
assert len(results) == 5
|
||||
assert isinstance(results[0], Item)
|
||||
assert isinstance(results[0].value, dict)
|
||||
assert results[0].value == {"data": "value1"}
|
||||
assert results[0].key == "key1"
|
||||
assert results[1] is None
|
||||
assert isinstance(results[2], list)
|
||||
assert len(results[2]) == 1
|
||||
assert isinstance(results[3], list)
|
||||
assert ("test", "foo") in results[3] and ("test", "bar") in results[3]
|
||||
assert results[4] is None
|
||||
|
||||
ops_reordered = [
|
||||
SearchOp(namespace_prefix=("test",), filter=None, limit=5, offset=0),
|
||||
GetOp(namespace=("test", "bar"), key="key2"),
|
||||
ListNamespacesOp(match_conditions=None, max_depth=None, limit=5, offset=0),
|
||||
PutOp(namespace=("test",), key="key3", value={"data": "value3"}),
|
||||
GetOp(namespace=("test", "foo"), key="key1"),
|
||||
]
|
||||
|
||||
results_reordered = await store.abatch(ops_reordered)
|
||||
assert len(results_reordered) == 5
|
||||
assert isinstance(results_reordered[0], list)
|
||||
assert len(results_reordered[0]) == 2
|
||||
assert isinstance(results_reordered[1], Item)
|
||||
assert results_reordered[1].value == {"data": "value2"}
|
||||
assert results_reordered[1].key == "key2"
|
||||
assert isinstance(results_reordered[2], list)
|
||||
assert ("test", "foo") in results_reordered[2] and (
|
||||
"test",
|
||||
"bar",
|
||||
) in results_reordered[2]
|
||||
assert results_reordered[3] is None
|
||||
assert isinstance(results_reordered[4], Item)
|
||||
assert results_reordered[4].value == {"data": "value1"}
|
||||
assert results_reordered[4].key == "key1"
|
||||
|
||||
|
||||
async def test_batch_get_ops(store: AsyncPostgresStore) -> None:
|
||||
# Setup test data
|
||||
await store.aput(("test",), "key1", {"data": "value1"})
|
||||
await store.aput(("test",), "key2", {"data": "value2"})
|
||||
|
||||
ops = [
|
||||
GetOp(namespace=("test",), key="key1"),
|
||||
GetOp(namespace=("test",), key="key2"),
|
||||
GetOp(namespace=("test",), key="key3"),
|
||||
]
|
||||
|
||||
results = await store.abatch(ops)
|
||||
|
||||
assert len(results) == 3
|
||||
assert results[0] is not None
|
||||
assert results[1] is not None
|
||||
assert results[2] is None
|
||||
assert results[0].key == "key1"
|
||||
assert results[1].key == "key2"
|
||||
|
||||
|
||||
async def test_batch_put_ops(store: AsyncPostgresStore) -> None:
|
||||
ops = [
|
||||
PutOp(namespace=("test",), key="key1", value={"data": "value1"}),
|
||||
PutOp(namespace=("test",), key="key2", value={"data": "value2"}),
|
||||
PutOp(namespace=("test",), key="key3", value=None),
|
||||
]
|
||||
|
||||
results = await store.abatch(ops)
|
||||
|
||||
assert len(results) == 3
|
||||
assert all(result is None for result in results)
|
||||
|
||||
# Verify the puts worked
|
||||
items = await store.asearch(["test"], limit=10)
|
||||
assert len(items) == 2 # key3 had None value so wasn't stored
|
||||
|
||||
|
||||
async def test_batch_search_ops(store: AsyncPostgresStore) -> None:
|
||||
# Setup test data
|
||||
await store.aput(("test", "foo"), "key1", {"data": "value1"})
|
||||
await store.aput(("test", "bar"), "key2", {"data": "value2"})
|
||||
|
||||
ops = [
|
||||
SearchOp(
|
||||
namespace_prefix=("test",), filter={"data": "value1"}, limit=10, offset=0
|
||||
),
|
||||
SearchOp(namespace_prefix=("test",), filter=None, limit=5, offset=0),
|
||||
]
|
||||
|
||||
results = await store.abatch(ops)
|
||||
|
||||
assert len(results) == 2
|
||||
assert len(results[0]) == 1 # Filtered results
|
||||
assert len(results[1]) == 2 # All results
|
||||
|
||||
|
||||
async def test_batch_list_namespaces_ops(store: AsyncPostgresStore) -> None:
|
||||
# Setup test data
|
||||
await store.aput(("test", "namespace1"), "key1", {"data": "value1"})
|
||||
await store.aput(("test", "namespace2"), "key2", {"data": "value2"})
|
||||
|
||||
ops = [ListNamespacesOp(match_conditions=None, max_depth=None, limit=10, offset=0)]
|
||||
|
||||
results = await store.abatch(ops)
|
||||
|
||||
assert len(results) == 1
|
||||
assert len(results[0]) == 2
|
||||
assert ("test", "namespace1") in results[0]
|
||||
assert ("test", "namespace2") in results[0]
|
||||
|
||||
|
||||
@asynccontextmanager
|
||||
async def _create_vector_store(
|
||||
vector_type: str,
|
||||
distance_type: str,
|
||||
fake_embeddings: CharacterEmbeddings,
|
||||
text_fields: list[str] | None = None,
|
||||
) -> AsyncIterator[AsyncPostgresStore]:
|
||||
"""Create a store with vector search enabled."""
|
||||
|
||||
database = f"test_{uuid.uuid4().hex[:16]}"
|
||||
uri_parts = DEFAULT_URI.split("/")
|
||||
uri_base = "/".join(uri_parts[:-1])
|
||||
query_params = ""
|
||||
if "?" in uri_parts[-1]:
|
||||
db_name, query_params = uri_parts[-1].split("?", 1)
|
||||
query_params = "?" + query_params
|
||||
|
||||
conn_string = f"{uri_base}/{database}{query_params}"
|
||||
admin_conn_string = DEFAULT_URI
|
||||
|
||||
index_config = {
|
||||
"dims": fake_embeddings.dims,
|
||||
"embed": fake_embeddings,
|
||||
"ann_index_config": {
|
||||
"vector_type": vector_type,
|
||||
},
|
||||
"distance_type": distance_type,
|
||||
"fields": text_fields,
|
||||
}
|
||||
|
||||
async with await AsyncConnection.connect(
|
||||
admin_conn_string, autocommit=True
|
||||
) as conn:
|
||||
await conn.execute(f"CREATE DATABASE {database}")
|
||||
try:
|
||||
async with AsyncPostgresStore.from_conn_string(
|
||||
conn_string,
|
||||
index=index_config,
|
||||
) as store:
|
||||
await store.setup()
|
||||
yield store
|
||||
finally:
|
||||
async with await AsyncConnection.connect(
|
||||
admin_conn_string, autocommit=True
|
||||
) as conn:
|
||||
await conn.execute(f"DROP DATABASE {database}")
|
||||
|
||||
|
||||
@pytest.fixture(
|
||||
scope="function",
|
||||
params=[
|
||||
(vector_type, distance_type)
|
||||
for vector_type in VECTOR_TYPES
|
||||
for distance_type in (
|
||||
["hamming"] if vector_type == "bit" else ["l2", "inner_product", "cosine"]
|
||||
)
|
||||
],
|
||||
ids=lambda p: f"{p[0]}_{p[1]}",
|
||||
)
|
||||
async def vector_store(
|
||||
request,
|
||||
fake_embeddings: CharacterEmbeddings,
|
||||
) -> AsyncIterator[AsyncPostgresStore]:
|
||||
"""Create a store with vector search enabled."""
|
||||
vector_type, distance_type = request.param
|
||||
async with _create_vector_store(
|
||||
vector_type, distance_type, fake_embeddings
|
||||
) as store:
|
||||
yield store
|
||||
|
||||
|
||||
async def test_vector_store_initialization(
|
||||
vector_store: AsyncPostgresStore, fake_embeddings: CharacterEmbeddings
|
||||
) -> None:
|
||||
"""Test store initialization with embedding config."""
|
||||
assert vector_store.index_config is not None
|
||||
assert vector_store.index_config["dims"] == fake_embeddings.dims
|
||||
if isinstance(vector_store.index_config["embed"], Embeddings):
|
||||
assert vector_store.index_config["embed"] == fake_embeddings
|
||||
|
||||
|
||||
async def test_vector_insert_with_auto_embedding(
|
||||
vector_store: AsyncPostgresStore,
|
||||
) -> None:
|
||||
"""Test inserting items that get auto-embedded."""
|
||||
docs = [
|
||||
("doc1", {"text": "short text"}),
|
||||
("doc2", {"text": "longer text document"}),
|
||||
("doc3", {"text": "longest text document here"}),
|
||||
("doc4", {"description": "text in description field"}),
|
||||
("doc5", {"content": "text in content field"}),
|
||||
("doc6", {"body": "text in body field"}),
|
||||
]
|
||||
|
||||
for key, value in docs:
|
||||
await vector_store.aput(("test",), key, value)
|
||||
|
||||
results = await vector_store.asearch(("test",), query="long text")
|
||||
assert len(results) > 0
|
||||
|
||||
doc_order = [r.key for r in results]
|
||||
assert "doc2" in doc_order
|
||||
assert "doc3" in doc_order
|
||||
|
||||
|
||||
async def test_vector_update_with_embedding(vector_store: AsyncPostgresStore) -> None:
|
||||
"""Test that updating items properly updates their embeddings."""
|
||||
await vector_store.aput(("test",), "doc1", {"text": "zany zebra Xerxes"})
|
||||
await vector_store.aput(("test",), "doc2", {"text": "something about dogs"})
|
||||
await vector_store.aput(("test",), "doc3", {"text": "text about birds"})
|
||||
|
||||
results_initial = await vector_store.asearch(("test",), query="Zany Xerxes")
|
||||
assert len(results_initial) > 0
|
||||
assert results_initial[0].key == "doc1"
|
||||
initial_score = results_initial[0].score
|
||||
|
||||
await vector_store.aput(("test",), "doc1", {"text": "new text about dogs"})
|
||||
|
||||
results_after = await vector_store.asearch(("test",), query="Zany Xerxes")
|
||||
after_score = next((r.score for r in results_after if r.key == "doc1"), 0.0)
|
||||
assert after_score < initial_score
|
||||
|
||||
results_new = await vector_store.asearch(("test",), query="new text about dogs")
|
||||
for r in results_new:
|
||||
if r.key == "doc1":
|
||||
assert r.score > after_score
|
||||
|
||||
# Don't index this one
|
||||
await vector_store.aput(
|
||||
("test",), "doc4", {"text": "new text about dogs"}, index=False
|
||||
)
|
||||
results_new = await vector_store.asearch(
|
||||
("test",), query="new text about dogs", limit=3
|
||||
)
|
||||
assert not any(r.key == "doc4" for r in results_new)
|
||||
|
||||
|
||||
async def test_vector_search_with_filters(vector_store: AsyncPostgresStore) -> None:
|
||||
"""Test combining vector search with filters."""
|
||||
docs = [
|
||||
("doc1", {"text": "red apple", "color": "red", "score": 4.5}),
|
||||
("doc2", {"text": "red car", "color": "red", "score": 3.0}),
|
||||
("doc3", {"text": "green apple", "color": "green", "score": 4.0}),
|
||||
("doc4", {"text": "blue car", "color": "blue", "score": 3.5}),
|
||||
]
|
||||
|
||||
for key, value in docs:
|
||||
await vector_store.aput(("test",), key, value)
|
||||
|
||||
results = await vector_store.asearch(
|
||||
("test",), query="apple", filter={"color": "red"}
|
||||
)
|
||||
assert len(results) == 2
|
||||
assert results[0].key == "doc1"
|
||||
|
||||
results = await vector_store.asearch(
|
||||
("test",), query="car", filter={"color": "red"}
|
||||
)
|
||||
assert len(results) == 2
|
||||
assert results[0].key == "doc2"
|
||||
|
||||
results = await vector_store.asearch(
|
||||
("test",), query="bbbbluuu", filter={"score": {"$gt": 3.2}}
|
||||
)
|
||||
assert len(results) == 3
|
||||
assert results[0].key == "doc4"
|
||||
|
||||
results = await vector_store.asearch(
|
||||
("test",), query="apple", filter={"score": {"$gte": 4.0}, "color": "green"}
|
||||
)
|
||||
assert len(results) == 1
|
||||
assert results[0].key == "doc3"
|
||||
|
||||
|
||||
async def test_vector_search_pagination(vector_store: AsyncPostgresStore) -> None:
|
||||
"""Test pagination with vector search."""
|
||||
for i in range(5):
|
||||
await vector_store.aput(
|
||||
("test",), f"doc{i}", {"text": f"test document number {i}"}
|
||||
)
|
||||
|
||||
results_page1 = await vector_store.asearch(("test",), query="test", limit=2)
|
||||
results_page2 = await vector_store.asearch(
|
||||
("test",), query="test", limit=2, offset=2
|
||||
)
|
||||
|
||||
assert len(results_page1) == 2
|
||||
assert len(results_page2) == 2
|
||||
assert results_page1[0].key != results_page2[0].key
|
||||
|
||||
all_results = await vector_store.asearch(("test",), query="test", limit=10)
|
||||
assert len(all_results) == 5
|
||||
|
||||
|
||||
async def test_vector_search_edge_cases(vector_store: AsyncPostgresStore) -> None:
|
||||
"""Test edge cases in vector search."""
|
||||
await vector_store.aput(("test",), "doc1", {"text": "test document"})
|
||||
|
||||
perfect_match = await vector_store.asearch(("test",), query="text test document")
|
||||
perfect_score = perfect_match[0].score
|
||||
|
||||
results = await vector_store.asearch(("test",), query="")
|
||||
assert len(results) == 1
|
||||
assert results[0].score is None
|
||||
|
||||
results = await vector_store.asearch(("test",), query=None)
|
||||
assert len(results) == 1
|
||||
assert results[0].score is None
|
||||
|
||||
long_query = "foo " * 100
|
||||
results = await vector_store.asearch(("test",), query=long_query)
|
||||
assert len(results) == 1
|
||||
assert results[0].score < perfect_score
|
||||
|
||||
special_query = "test!@#$%^&*()"
|
||||
results = await vector_store.asearch(("test",), query=special_query)
|
||||
assert len(results) == 1
|
||||
assert results[0].score < perfect_score
|
||||
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
"vector_type,distance_type",
|
||||
[
|
||||
*itertools.product(["vector", "halfvec"], ["cosine", "inner_product", "l2"]),
|
||||
],
|
||||
)
|
||||
async def test_embed_with_path(
|
||||
request: Any,
|
||||
fake_embeddings: CharacterEmbeddings,
|
||||
vector_type: str,
|
||||
distance_type: str,
|
||||
) -> None:
|
||||
"""Test vector search with specific text fields in Postgres store."""
|
||||
async with _create_vector_store(
|
||||
vector_type,
|
||||
distance_type,
|
||||
fake_embeddings,
|
||||
text_fields=["key0", "key1", "key3"],
|
||||
) as store:
|
||||
# This will have 2 vectors representing it
|
||||
doc1 = {
|
||||
# Omit key0 - check it doesn't raise an error
|
||||
"key1": "xxx",
|
||||
"key2": "yyy",
|
||||
"key3": "zzz",
|
||||
}
|
||||
# This will have 3 vectors representing it
|
||||
doc2 = {
|
||||
"key0": "uuu",
|
||||
"key1": "vvv",
|
||||
"key2": "www",
|
||||
"key3": "xxx",
|
||||
}
|
||||
await store.aput(("test",), "doc1", doc1)
|
||||
await store.aput(("test",), "doc2", doc2)
|
||||
|
||||
# doc2.key3 and doc1.key1 both would have the highest score
|
||||
results = await store.asearch(("test",), query="xxx")
|
||||
assert len(results) == 2
|
||||
assert results[0].key != results[1].key
|
||||
ascore = results[0].score
|
||||
bscore = results[1].score
|
||||
assert ascore == pytest.approx(bscore, abs=1e-3)
|
||||
|
||||
results = await store.asearch(("test",), query="uuu")
|
||||
assert len(results) == 2
|
||||
assert results[0].key != results[1].key
|
||||
assert results[0].key == "doc2"
|
||||
assert results[0].score > results[1].score
|
||||
assert ascore == pytest.approx(results[0].score, abs=1e-3)
|
||||
|
||||
# Un-indexed - will have low results for both. Not zero (because we're projecting)
|
||||
# but less than the above.
|
||||
results = await store.asearch(("test",), query="www")
|
||||
assert len(results) == 2
|
||||
assert results[0].score < ascore
|
||||
assert results[1].score < ascore
|
||||
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
"vector_type,distance_type",
|
||||
[
|
||||
*itertools.product(["vector", "halfvec"], ["cosine", "inner_product", "l2"]),
|
||||
],
|
||||
)
|
||||
async def test_search_sorting(
|
||||
request: Any,
|
||||
fake_embeddings: CharacterEmbeddings,
|
||||
vector_type: str,
|
||||
distance_type: str,
|
||||
) -> None:
|
||||
"""Test operation-level field configuration for vector search."""
|
||||
async with _create_vector_store(
|
||||
vector_type,
|
||||
distance_type,
|
||||
fake_embeddings,
|
||||
text_fields=["key1"], # Default fields that won't match our test data
|
||||
) as store:
|
||||
amatch = {
|
||||
"key1": "mmm",
|
||||
}
|
||||
|
||||
await store.aput(("test", "M"), "M", amatch)
|
||||
N = 100
|
||||
for i in range(N):
|
||||
await store.aput(("test", "A"), f"A{i}", {"key1": "no"})
|
||||
for i in range(N):
|
||||
await store.aput(("test", "Z"), f"Z{i}", {"key1": "no"})
|
||||
|
||||
results = await store.asearch(("test",), query="mmm", limit=10)
|
||||
assert len(results) == 10
|
||||
assert len(set(r.key for r in results)) == 10
|
||||
assert results[0].key == "M"
|
||||
assert results[0].score > results[1].score
|
||||
|
||||
|
||||
async def test_store_ttl(store):
|
||||
# Assumes a TTL of 1 minute = 60 seconds
|
||||
ns = ("foo",)
|
||||
await store.start_ttl_sweeper()
|
||||
await store.aput(
|
||||
ns,
|
||||
key="item1",
|
||||
value={"foo": "bar"},
|
||||
ttl=TTL_MINUTES, # type: ignore
|
||||
)
|
||||
await asyncio.sleep(TTL_SECONDS - 2)
|
||||
res = await store.aget(ns, key="item1", refresh_ttl=True)
|
||||
assert res is not None
|
||||
await asyncio.sleep(TTL_SECONDS - 2)
|
||||
results = await store.asearch(ns, query="foo", refresh_ttl=True)
|
||||
assert len(results) == 1
|
||||
await asyncio.sleep(TTL_SECONDS - 2)
|
||||
res = await store.aget(ns, key="item1", refresh_ttl=False)
|
||||
assert res is not None
|
||||
await asyncio.sleep(TTL_SECONDS - 1)
|
||||
# Now has been (TTL_SECONDS-2)*2 > TTL_SECONDS + TTL_SECONDS/2
|
||||
results = await store.asearch(ns, query="bar", refresh_ttl=False)
|
||||
assert len(results) == 0
|
||||
@@ -1,14 +1,14 @@
|
||||
# type: ignore
|
||||
from __future__ import annotations
|
||||
|
||||
import re
|
||||
import time
|
||||
from contextlib import contextmanager
|
||||
from typing import Any
|
||||
from typing import Any, Optional
|
||||
from uuid import uuid4
|
||||
|
||||
import pytest
|
||||
from langchain_core.embeddings import Embeddings
|
||||
from psycopg import Connection
|
||||
|
||||
from langgraph.store.base import (
|
||||
GetOp,
|
||||
Item,
|
||||
@@ -17,8 +17,6 @@ from langgraph.store.base import (
|
||||
PutOp,
|
||||
SearchOp,
|
||||
)
|
||||
from psycopg import Connection
|
||||
|
||||
from langgraph.store.postgres import PostgresStore
|
||||
from tests.conftest import (
|
||||
DEFAULT_URI,
|
||||
@@ -26,9 +24,6 @@ from tests.conftest import (
|
||||
CharacterEmbeddings,
|
||||
)
|
||||
|
||||
TTL_SECONDS = 6
|
||||
TTL_MINUTES = TTL_SECONDS / 60
|
||||
|
||||
|
||||
@pytest.fixture(scope="function", params=["default", "pipe", "pool"])
|
||||
def store(request) -> PostgresStore:
|
||||
@@ -37,56 +32,29 @@ def store(request) -> PostgresStore:
|
||||
uri_base = "/".join(uri_parts[:-1])
|
||||
query_params = ""
|
||||
if "?" in uri_parts[-1]:
|
||||
_, query_params = uri_parts[-1].split("?", 1)
|
||||
db_name, query_params = uri_parts[-1].split("?", 1)
|
||||
query_params = "?" + query_params
|
||||
|
||||
conn_string = f"{uri_base}/{database}{query_params}"
|
||||
admin_conn_string = DEFAULT_URI
|
||||
ttl_config = {
|
||||
"default_ttl": TTL_MINUTES,
|
||||
"refresh_on_read": True,
|
||||
"sweep_interval_minutes": TTL_MINUTES / 2,
|
||||
}
|
||||
|
||||
with Connection.connect(admin_conn_string, autocommit=True) as conn:
|
||||
conn.execute(f"CREATE DATABASE {database}")
|
||||
try:
|
||||
with PostgresStore.from_conn_string(conn_string, ttl=ttl_config) as store:
|
||||
store.MIGRATIONS = [
|
||||
(
|
||||
mig.replace("ttl_minutes INT;", "ttl_minutes FLOAT;")
|
||||
if isinstance(mig, str)
|
||||
else mig
|
||||
)
|
||||
for mig in store.MIGRATIONS
|
||||
]
|
||||
with PostgresStore.from_conn_string(conn_string) as store:
|
||||
store.setup()
|
||||
|
||||
if request.param == "pipe":
|
||||
with PostgresStore.from_conn_string(
|
||||
conn_string,
|
||||
pipeline=True,
|
||||
ttl=ttl_config,
|
||||
) as store:
|
||||
store.start_ttl_sweeper()
|
||||
with PostgresStore.from_conn_string(conn_string, pipeline=True) as store:
|
||||
yield store
|
||||
|
||||
store.stop_ttl_sweeper()
|
||||
elif request.param == "pool":
|
||||
with PostgresStore.from_conn_string(
|
||||
conn_string,
|
||||
pool_config={"min_size": 1, "max_size": 10},
|
||||
ttl=ttl_config,
|
||||
conn_string, pool_config={"min_size": 1, "max_size": 10}
|
||||
) as store:
|
||||
store.start_ttl_sweeper()
|
||||
yield store
|
||||
|
||||
store.stop_ttl_sweeper()
|
||||
else: # default
|
||||
with PostgresStore.from_conn_string(conn_string, ttl=ttl_config) as store:
|
||||
store.start_ttl_sweeper()
|
||||
with PostgresStore.from_conn_string(conn_string) as store:
|
||||
yield store
|
||||
|
||||
store.stop_ttl_sweeper()
|
||||
finally:
|
||||
with Connection.connect(admin_conn_string, autocommit=True) as conn:
|
||||
conn.execute(f"DROP DATABASE {database}")
|
||||
@@ -252,127 +220,134 @@ def test_batch_list_namespaces_ops(store: PostgresStore) -> None:
|
||||
assert all(ns[-1] == "public" for ns in results[2])
|
||||
|
||||
|
||||
def test_basic_store_ops(store) -> None:
|
||||
namespace = ("test", "documents")
|
||||
item_id = "doc1"
|
||||
item_value = {"title": "Test Document", "content": "Hello, World!"}
|
||||
class TestPostgresStore:
|
||||
@pytest.fixture(autouse=True)
|
||||
def setup(self) -> None:
|
||||
with PostgresStore.from_conn_string(DEFAULT_URI) as store:
|
||||
store.setup()
|
||||
|
||||
store.put(namespace, item_id, item_value)
|
||||
item = store.get(namespace, item_id)
|
||||
def test_basic_store_ops(self) -> None:
|
||||
with PostgresStore.from_conn_string(DEFAULT_URI) as store:
|
||||
namespace = ("test", "documents")
|
||||
item_id = "doc1"
|
||||
item_value = {"title": "Test Document", "content": "Hello, World!"}
|
||||
|
||||
assert item
|
||||
assert item.namespace == namespace
|
||||
assert item.key == item_id
|
||||
assert item.value == item_value
|
||||
store.put(namespace, item_id, item_value)
|
||||
item = store.get(namespace, item_id)
|
||||
|
||||
# Test update
|
||||
updated_value = {"title": "Updated Document", "content": "Hello, Updated!"}
|
||||
store.put(namespace, item_id, updated_value)
|
||||
updated_item = store.get(namespace, item_id)
|
||||
assert item
|
||||
assert item.namespace == namespace
|
||||
assert item.key == item_id
|
||||
assert item.value == item_value
|
||||
|
||||
assert updated_item.value == updated_value
|
||||
assert updated_item.updated_at > item.updated_at
|
||||
# Test update
|
||||
updated_value = {"title": "Updated Document", "content": "Hello, Updated!"}
|
||||
store.put(namespace, item_id, updated_value)
|
||||
updated_item = store.get(namespace, item_id)
|
||||
|
||||
# Test get from non-existent namespace
|
||||
different_namespace = ("test", "other_documents")
|
||||
item_in_different_namespace = store.get(different_namespace, item_id)
|
||||
assert item_in_different_namespace is None
|
||||
assert updated_item.value == updated_value
|
||||
assert updated_item.updated_at > item.updated_at
|
||||
|
||||
# Test delete
|
||||
store.delete(namespace, item_id)
|
||||
deleted_item = store.get(namespace, item_id)
|
||||
assert deleted_item is None
|
||||
# Test get from non-existent namespace
|
||||
different_namespace = ("test", "other_documents")
|
||||
item_in_different_namespace = store.get(different_namespace, item_id)
|
||||
assert item_in_different_namespace is None
|
||||
|
||||
# Test delete
|
||||
store.delete(namespace, item_id)
|
||||
deleted_item = store.get(namespace, item_id)
|
||||
assert deleted_item is None
|
||||
|
||||
def test_list_namespaces(store) -> None:
|
||||
# Create test data with various namespaces
|
||||
test_namespaces = [
|
||||
("test", "documents", "public"),
|
||||
("test", "documents", "private"),
|
||||
("test", "images", "public"),
|
||||
("test", "images", "private"),
|
||||
("prod", "documents", "public"),
|
||||
("prod", "documents", "private"),
|
||||
]
|
||||
def test_list_namespaces(self) -> None:
|
||||
with PostgresStore.from_conn_string(DEFAULT_URI) as store:
|
||||
# Create test data with various namespaces
|
||||
test_namespaces = [
|
||||
("test", "documents", "public"),
|
||||
("test", "documents", "private"),
|
||||
("test", "images", "public"),
|
||||
("test", "images", "private"),
|
||||
("prod", "documents", "public"),
|
||||
("prod", "documents", "private"),
|
||||
]
|
||||
|
||||
# Insert test data
|
||||
for namespace in test_namespaces:
|
||||
store.put(namespace, "dummy", {"content": "dummy"})
|
||||
# Insert test data
|
||||
for namespace in test_namespaces:
|
||||
store.put(namespace, "dummy", {"content": "dummy"})
|
||||
|
||||
# Test listing with various filters
|
||||
all_namespaces = store.list_namespaces()
|
||||
assert len(all_namespaces) == len(test_namespaces)
|
||||
# Test listing with various filters
|
||||
all_namespaces = store.list_namespaces()
|
||||
assert len(all_namespaces) == len(test_namespaces)
|
||||
|
||||
# Test prefix filtering
|
||||
test_prefix_namespaces = store.list_namespaces(prefix=["test"])
|
||||
assert len(test_prefix_namespaces) == 4
|
||||
assert all(ns[0] == "test" for ns in test_prefix_namespaces)
|
||||
# Test prefix filtering
|
||||
test_prefix_namespaces = store.list_namespaces(prefix=["test"])
|
||||
assert len(test_prefix_namespaces) == 4
|
||||
assert all(ns[0] == "test" for ns in test_prefix_namespaces)
|
||||
|
||||
# Test suffix filtering
|
||||
public_namespaces = store.list_namespaces(suffix=["public"])
|
||||
assert len(public_namespaces) == 3
|
||||
assert all(ns[-1] == "public" for ns in public_namespaces)
|
||||
# Test suffix filtering
|
||||
public_namespaces = store.list_namespaces(suffix=["public"])
|
||||
assert len(public_namespaces) == 3
|
||||
assert all(ns[-1] == "public" for ns in public_namespaces)
|
||||
|
||||
# Test max depth
|
||||
depth_2_namespaces = store.list_namespaces(max_depth=2)
|
||||
assert all(len(ns) <= 2 for ns in depth_2_namespaces)
|
||||
# Test max depth
|
||||
depth_2_namespaces = store.list_namespaces(max_depth=2)
|
||||
assert all(len(ns) <= 2 for ns in depth_2_namespaces)
|
||||
|
||||
# Test pagination
|
||||
paginated_namespaces = store.list_namespaces(limit=3)
|
||||
assert len(paginated_namespaces) == 3
|
||||
# Test pagination
|
||||
paginated_namespaces = store.list_namespaces(limit=3)
|
||||
assert len(paginated_namespaces) == 3
|
||||
|
||||
# Cleanup
|
||||
for namespace in test_namespaces:
|
||||
store.delete(namespace, "dummy")
|
||||
# Cleanup
|
||||
for namespace in test_namespaces:
|
||||
store.delete(namespace, "dummy")
|
||||
|
||||
def test_search(self) -> None:
|
||||
with PostgresStore.from_conn_string(DEFAULT_URI) as store:
|
||||
# Create test data
|
||||
test_data = [
|
||||
(
|
||||
("test", "docs"),
|
||||
"doc1",
|
||||
{"title": "First Doc", "author": "Alice", "tags": ["important"]},
|
||||
),
|
||||
(
|
||||
("test", "docs"),
|
||||
"doc2",
|
||||
{"title": "Second Doc", "author": "Bob", "tags": ["draft"]},
|
||||
),
|
||||
(
|
||||
("test", "images"),
|
||||
"img1",
|
||||
{"title": "Image 1", "author": "Alice", "tags": ["final"]},
|
||||
),
|
||||
]
|
||||
|
||||
def test_search(store) -> None:
|
||||
# Create test data
|
||||
test_data = [
|
||||
(
|
||||
("test", "docs"),
|
||||
"doc1",
|
||||
{"title": "First Doc", "author": "Alice", "tags": ["important"]},
|
||||
),
|
||||
(
|
||||
("test", "docs"),
|
||||
"doc2",
|
||||
{"title": "Second Doc", "author": "Bob", "tags": ["draft"]},
|
||||
),
|
||||
(
|
||||
("test", "images"),
|
||||
"img1",
|
||||
{"title": "Image 1", "author": "Alice", "tags": ["final"]},
|
||||
),
|
||||
]
|
||||
for namespace, key, value in test_data:
|
||||
store.put(namespace, key, value)
|
||||
|
||||
for namespace, key, value in test_data:
|
||||
store.put(namespace, key, value)
|
||||
# Test basic search
|
||||
all_items = store.search(["test"])
|
||||
assert len(all_items) == 3
|
||||
|
||||
# Test basic search
|
||||
all_items = store.search(["test"])
|
||||
assert len(all_items) == 3
|
||||
# Test namespace filtering
|
||||
docs_items = store.search(["test", "docs"])
|
||||
assert len(docs_items) == 2
|
||||
assert all(item.namespace == ("test", "docs") for item in docs_items)
|
||||
|
||||
# Test namespace filtering
|
||||
docs_items = store.search(["test", "docs"])
|
||||
assert len(docs_items) == 2
|
||||
assert all(item.namespace == ("test", "docs") for item in docs_items)
|
||||
# Test value filtering
|
||||
alice_items = store.search(["test"], filter={"author": "Alice"})
|
||||
assert len(alice_items) == 2
|
||||
assert all(item.value["author"] == "Alice" for item in alice_items)
|
||||
|
||||
# Test value filtering
|
||||
alice_items = store.search(["test"], filter={"author": "Alice"})
|
||||
assert len(alice_items) == 2
|
||||
assert all(item.value["author"] == "Alice" for item in alice_items)
|
||||
# Test pagination
|
||||
paginated_items = store.search(["test"], limit=2)
|
||||
assert len(paginated_items) == 2
|
||||
|
||||
# Test pagination
|
||||
paginated_items = store.search(["test"], limit=2)
|
||||
assert len(paginated_items) == 2
|
||||
offset_items = store.search(["test"], offset=2)
|
||||
assert len(offset_items) == 1
|
||||
|
||||
offset_items = store.search(["test"], offset=2)
|
||||
assert len(offset_items) == 1
|
||||
|
||||
# Cleanup
|
||||
for namespace, key, _ in test_data:
|
||||
store.delete(namespace, key)
|
||||
# Cleanup
|
||||
for namespace, key, _ in test_data:
|
||||
store.delete(namespace, key)
|
||||
|
||||
|
||||
@contextmanager
|
||||
@@ -380,8 +355,7 @@ def _create_vector_store(
|
||||
vector_type: str,
|
||||
distance_type: str,
|
||||
fake_embeddings: Embeddings,
|
||||
text_fields: list[str] | None = None,
|
||||
enable_ttl: bool = True,
|
||||
text_fields: Optional[list[str]] = None,
|
||||
) -> PostgresStore:
|
||||
"""Create a store with vector search enabled."""
|
||||
database = f"test_{uuid4().hex[:16]}"
|
||||
@@ -402,7 +376,7 @@ def _create_vector_store(
|
||||
"vector_type": vector_type,
|
||||
},
|
||||
"distance_type": distance_type,
|
||||
"fields": text_fields,
|
||||
"text_fields": text_fields,
|
||||
}
|
||||
|
||||
with Connection.connect(admin_conn_string, autocommit=True) as conn:
|
||||
@@ -411,32 +385,23 @@ def _create_vector_store(
|
||||
with PostgresStore.from_conn_string(
|
||||
conn_string,
|
||||
index=index_config,
|
||||
ttl={"default_ttl": 2, "refresh_on_read": True} if enable_ttl else None,
|
||||
) as store:
|
||||
store.setup()
|
||||
with store._cursor() as cur:
|
||||
# drop the migration index
|
||||
cur.execute("DROP TABLE IF EXISTS store_migrations")
|
||||
store.setup() # Will fail if migrations aren't idempotent
|
||||
yield store
|
||||
finally:
|
||||
with Connection.connect(admin_conn_string, autocommit=True) as conn:
|
||||
conn.execute(f"DROP DATABASE {database}")
|
||||
|
||||
|
||||
_vector_params = [
|
||||
(vector_type, distance_type, True)
|
||||
for vector_type in VECTOR_TYPES
|
||||
for distance_type in (
|
||||
["hamming"] if vector_type == "bit" else ["l2", "inner_product", "cosine"]
|
||||
)
|
||||
]
|
||||
_vector_params += [(*_vector_params[-1][:2], False)]
|
||||
|
||||
|
||||
@pytest.fixture(
|
||||
scope="function",
|
||||
params=_vector_params,
|
||||
params=[
|
||||
(vector_type, distance_type)
|
||||
for vector_type in VECTOR_TYPES
|
||||
for distance_type in (
|
||||
["hamming"] if vector_type == "bit" else ["l2", "inner_product", "cosine"]
|
||||
)
|
||||
],
|
||||
ids=lambda p: f"{p[0]}_{p[1]}",
|
||||
)
|
||||
def vector_store(
|
||||
@@ -444,10 +409,8 @@ def vector_store(
|
||||
fake_embeddings: Embeddings,
|
||||
) -> PostgresStore:
|
||||
"""Create a store with vector search enabled."""
|
||||
vector_type, distance_type, enable_ttl = request.param
|
||||
with _create_vector_store(
|
||||
vector_type, distance_type, fake_embeddings, enable_ttl=enable_ttl
|
||||
) as store:
|
||||
vector_type, distance_type = request.param
|
||||
with _create_vector_store(vector_type, distance_type, fake_embeddings) as store:
|
||||
yield store
|
||||
|
||||
|
||||
@@ -511,10 +474,7 @@ def test_vector_update_with_embedding(vector_store: PostgresStore) -> None:
|
||||
assert not any(r.key == "doc4" for r in results_new)
|
||||
|
||||
|
||||
@pytest.mark.parametrize("refresh_ttl", [True, False])
|
||||
def test_vector_search_with_filters(
|
||||
vector_store: PostgresStore, refresh_ttl: bool
|
||||
) -> None:
|
||||
def test_vector_search_with_filters(vector_store: PostgresStore) -> None:
|
||||
"""Test combining vector search with filters."""
|
||||
# Insert test documents
|
||||
docs = [
|
||||
@@ -527,23 +487,16 @@ def test_vector_search_with_filters(
|
||||
for key, value in docs:
|
||||
vector_store.put(("test",), key, value)
|
||||
|
||||
results = vector_store.search(
|
||||
("test",), query="apple", filter={"color": "red"}, refresh_ttl=refresh_ttl
|
||||
)
|
||||
results = vector_store.search(("test",), query="apple", filter={"color": "red"})
|
||||
assert len(results) == 2
|
||||
assert results[0].key == "doc1"
|
||||
|
||||
results = vector_store.search(
|
||||
("test",), query="car", filter={"color": "red"}, refresh_ttl=refresh_ttl
|
||||
)
|
||||
results = vector_store.search(("test",), query="car", filter={"color": "red"})
|
||||
assert len(results) == 2
|
||||
assert results[0].key == "doc2"
|
||||
|
||||
results = vector_store.search(
|
||||
("test",),
|
||||
query="bbbbluuu",
|
||||
filter={"score": {"$gt": 3.2}},
|
||||
refresh_ttl=refresh_ttl,
|
||||
("test",), query="bbbbluuu", filter={"score": {"$gt": 3.2}}
|
||||
)
|
||||
assert len(results) == 3
|
||||
assert results[0].key == "doc4"
|
||||
@@ -735,7 +688,7 @@ def test_embed_with_path_operation_config(
|
||||
store.put(("test",), "doc5", doc5, index=False)
|
||||
results = store.search(("test",))
|
||||
assert len(results) == 3
|
||||
assert all(r.score is None for r in results), f"{results}"
|
||||
assert all(r.score is None for r in results)
|
||||
assert any(r.key == "doc5" for r in results)
|
||||
|
||||
results = store.search(("test",), query="hhh")
|
||||
@@ -754,7 +707,7 @@ def _cosine_similarity(X: list[float], Y: list[list[float]]) -> list[float]:
|
||||
|
||||
similarities = []
|
||||
for y in Y:
|
||||
dot_product = sum(a * b for a, b in zip(X, y, strict=False))
|
||||
dot_product = sum(a * b for a, b in zip(X, y))
|
||||
norm1 = sum(a * a for a in X) ** 0.5
|
||||
norm2 = sum(a * a for a in y) ** 0.5
|
||||
similarity = dot_product / (norm1 * norm2) if norm1 > 0 and norm2 > 0 else 0.0
|
||||
@@ -771,7 +724,7 @@ def _inner_product(X: list[float], Y: list[list[float]]) -> list[float]:
|
||||
|
||||
similarities = []
|
||||
for y in Y:
|
||||
similarity = sum(a * b for a, b in zip(X, y, strict=False))
|
||||
similarity = sum(a * b for a, b in zip(X, y))
|
||||
similarities.append(similarity)
|
||||
|
||||
return similarities
|
||||
@@ -785,7 +738,7 @@ def _neg_l2_distance(X: list[float], Y: list[list[float]]) -> list[float]:
|
||||
|
||||
similarities = []
|
||||
for y in Y:
|
||||
similarity = sum((a - b) ** 2 for a, b in zip(X, y, strict=False)) ** 0.5
|
||||
similarity = sum((a - b) ** 2 for a, b in zip(X, y)) ** 0.5
|
||||
similarities.append(-similarity)
|
||||
|
||||
return similarities
|
||||
@@ -837,65 +790,3 @@ def test_nonnull_migrations() -> None:
|
||||
for migration in PostgresStore.MIGRATIONS:
|
||||
statement = _leading_comment_remover.sub("", migration).split()[0]
|
||||
assert statement.strip()
|
||||
|
||||
|
||||
def test_store_ttl(store):
|
||||
# Assumes a TTL of 1 minute = 60 seconds
|
||||
ns = ("foo",)
|
||||
store.put(
|
||||
ns,
|
||||
key="item1",
|
||||
value={"foo": "bar"},
|
||||
ttl=TTL_MINUTES, # type: ignore
|
||||
)
|
||||
time.sleep(TTL_SECONDS - 2)
|
||||
res = store.get(ns, key="item1", refresh_ttl=True)
|
||||
assert res is not None
|
||||
time.sleep(TTL_SECONDS - 2)
|
||||
results = store.search(ns, query="foo", refresh_ttl=True)
|
||||
assert len(results) == 1
|
||||
time.sleep(TTL_SECONDS - 2)
|
||||
res = store.get(ns, key="item1", refresh_ttl=False)
|
||||
assert res is not None
|
||||
time.sleep(TTL_SECONDS - 1)
|
||||
# Now has been (TTL_SECONDS-2)*2 > TTL_SECONDS + TTL_SECONDS/2
|
||||
res = store.search(ns, query="bar", refresh_ttl=False)
|
||||
assert len(res) == 0
|
||||
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
"vector_type,distance_type",
|
||||
[
|
||||
("vector", "cosine"),
|
||||
("vector", "inner_product"),
|
||||
("halfvec", "cosine"),
|
||||
("halfvec", "inner_product"),
|
||||
],
|
||||
)
|
||||
def test_non_ascii(
|
||||
request: Any,
|
||||
fake_embeddings: CharacterEmbeddings,
|
||||
vector_type: str,
|
||||
distance_type: str,
|
||||
) -> None:
|
||||
"""Test support for non-ascii characters"""
|
||||
with _create_vector_store(vector_type, distance_type, fake_embeddings) as store:
|
||||
store.put(("user_123", "memories"), "1", {"text": "这是中文"}) # Chinese
|
||||
store.put(
|
||||
("user_123", "memories"), "2", {"text": "これは日本語です"}
|
||||
) # Japanese
|
||||
store.put(("user_123", "memories"), "3", {"text": "이건 한국어야"}) # Korean
|
||||
store.put(("user_123", "memories"), "4", {"text": "Это русский"}) # Russian
|
||||
store.put(("user_123", "memories"), "5", {"text": "यह रूसी है"}) # Hindi
|
||||
|
||||
result1 = store.search(("user_123", "memories"), query="这是中文")
|
||||
result2 = store.search(("user_123", "memories"), query="これは日本語です")
|
||||
result3 = store.search(("user_123", "memories"), query="이건 한국어야")
|
||||
result4 = store.search(("user_123", "memories"), query="Это русский")
|
||||
result5 = store.search(("user_123", "memories"), query="यह रूसी है")
|
||||
|
||||
assert result1[0].key == "1"
|
||||
assert result2[0].key == "2"
|
||||
assert result3[0].key == "3"
|
||||
assert result4[0].key == "4"
|
||||
assert result5[0].key == "5"
|
||||
@@ -7,6 +7,10 @@ from uuid import uuid4
|
||||
|
||||
import pytest
|
||||
from langchain_core.runnables import RunnableConfig
|
||||
from psycopg import Connection
|
||||
from psycopg.rows import dict_row
|
||||
from psycopg_pool import ConnectionPool
|
||||
|
||||
from langgraph.checkpoint.base import (
|
||||
EXCLUDED_METADATA_KEYS,
|
||||
Checkpoint,
|
||||
@@ -14,11 +18,6 @@ from langgraph.checkpoint.base import (
|
||||
create_checkpoint,
|
||||
empty_checkpoint,
|
||||
)
|
||||
from langgraph.checkpoint.serde.types import TASKS
|
||||
from psycopg import Connection
|
||||
from psycopg.rows import dict_row
|
||||
from psycopg_pool import ConnectionPool
|
||||
|
||||
from langgraph.checkpoint.postgres import PostgresSaver, ShallowPostgresSaver
|
||||
from tests.conftest import DEFAULT_POSTGRES_URI
|
||||
|
||||
@@ -143,7 +142,8 @@ def test_data():
|
||||
config_1: RunnableConfig = {
|
||||
"configurable": {
|
||||
"thread_id": "thread-1",
|
||||
"checkpoint_id": "1",
|
||||
# for backwards compatibility testing
|
||||
"thread_ts": "1",
|
||||
"checkpoint_ns": "",
|
||||
}
|
||||
}
|
||||
@@ -169,11 +169,13 @@ def test_data():
|
||||
metadata_1: CheckpointMetadata = {
|
||||
"source": "input",
|
||||
"step": 2,
|
||||
"writes": {},
|
||||
"score": 1,
|
||||
}
|
||||
metadata_2: CheckpointMetadata = {
|
||||
"source": "loop",
|
||||
"step": 1,
|
||||
"writes": {"foo": "bar"},
|
||||
"score": None,
|
||||
}
|
||||
metadata_3: CheckpointMetadata = {}
|
||||
@@ -200,12 +202,14 @@ def test_combined_metadata(saver_name: str, test_data) -> None:
|
||||
metadata: CheckpointMetadata = {
|
||||
"source": "loop",
|
||||
"step": 1,
|
||||
"writes": {"foo": "bar"},
|
||||
"score": None,
|
||||
}
|
||||
saver.put(config, chkpnt, metadata, {})
|
||||
checkpoint = saver.get_tuple(config)
|
||||
assert checkpoint.metadata == {
|
||||
**metadata,
|
||||
"thread_id": "thread-2",
|
||||
"run_id": "my_run_id",
|
||||
}
|
||||
|
||||
@@ -225,6 +229,7 @@ def test_search(saver_name: str, test_data) -> None:
|
||||
query_1 = {"source": "input"} # search by 1 key
|
||||
query_2 = {
|
||||
"step": 1,
|
||||
"writes": {"foo": "bar"},
|
||||
} # search by multiple keys
|
||||
query_3: dict[str, Any] = {} # search by no keys, return all checkpoints
|
||||
query_4 = {"source": "update", "step": 1} # no match
|
||||
@@ -279,82 +284,3 @@ def test_nonnull_migrations() -> None:
|
||||
for migration in PostgresSaver.MIGRATIONS:
|
||||
statement = _leading_comment_remover.sub("", migration).split()[0]
|
||||
assert statement.strip()
|
||||
|
||||
|
||||
@pytest.mark.parametrize("saver_name", ["base", "pool", "pipe"])
|
||||
def test_pending_sends_migration(saver_name: str) -> None:
|
||||
with _saver(saver_name) as saver:
|
||||
config = {
|
||||
"configurable": {
|
||||
"thread_id": "thread-1",
|
||||
"checkpoint_ns": "",
|
||||
}
|
||||
}
|
||||
|
||||
# create the first checkpoint
|
||||
# and put some pending sends
|
||||
checkpoint_0 = empty_checkpoint()
|
||||
config = saver.put(config, checkpoint_0, {}, {})
|
||||
saver.put_writes(
|
||||
config, [(TASKS, "send-1"), (TASKS, "send-2")], task_id="task-1"
|
||||
)
|
||||
saver.put_writes(config, [(TASKS, "send-3")], task_id="task-2")
|
||||
|
||||
# check that fetching checkpoint_0 doesn't attach pending sends
|
||||
# (they should be attached to the next checkpoint)
|
||||
tuple_0 = saver.get_tuple(config)
|
||||
assert tuple_0.checkpoint["channel_values"] == {}
|
||||
assert tuple_0.checkpoint["channel_versions"] == {}
|
||||
|
||||
# create the second checkpoint
|
||||
checkpoint_1 = create_checkpoint(checkpoint_0, {}, 1)
|
||||
config = saver.put(config, checkpoint_1, {}, {})
|
||||
|
||||
# check that pending sends are attached to checkpoint_1
|
||||
checkpoint_1 = saver.get_tuple(config)
|
||||
assert checkpoint_1.checkpoint["channel_values"] == {
|
||||
TASKS: ["send-1", "send-2", "send-3"]
|
||||
}
|
||||
assert TASKS in checkpoint_1.checkpoint["channel_versions"]
|
||||
|
||||
# check that list also applies the migration
|
||||
search_results = [
|
||||
c for c in saver.list({"configurable": {"thread_id": "thread-1"}})
|
||||
]
|
||||
assert len(search_results) == 2
|
||||
assert search_results[-1].checkpoint["channel_values"] == {}
|
||||
assert search_results[-1].checkpoint["channel_versions"] == {}
|
||||
assert search_results[0].checkpoint["channel_values"] == {
|
||||
TASKS: ["send-1", "send-2", "send-3"]
|
||||
}
|
||||
assert TASKS in search_results[0].checkpoint["channel_versions"]
|
||||
|
||||
|
||||
@pytest.mark.parametrize("saver_name", ["base", "pool", "pipe"])
|
||||
def test_get_checkpoint_no_channel_values(
|
||||
monkeypatch, saver_name: str, test_data
|
||||
) -> None:
|
||||
"""Backwards compatibility test that verifies a checkpoint with no channel_values key can be retrieved without throwing an error."""
|
||||
with _saver(saver_name) as saver:
|
||||
config = {
|
||||
"configurable": {
|
||||
"thread_id": "thread-2",
|
||||
"checkpoint_ns": "",
|
||||
"__super_private_key": "super_private_value",
|
||||
},
|
||||
}
|
||||
chkpnt: Checkpoint = create_checkpoint(empty_checkpoint(), {}, 1)
|
||||
saver.put(config, chkpnt, {}, {})
|
||||
|
||||
load_checkpoint_tuple = saver._load_checkpoint_tuple
|
||||
|
||||
def patched_load_checkpoint_tuple(value):
|
||||
value["checkpoint"].pop("channel_values", None)
|
||||
return load_checkpoint_tuple(value)
|
||||
|
||||
monkeypatch.setattr(
|
||||
saver, "_load_checkpoint_tuple", patched_load_checkpoint_tuple
|
||||
)
|
||||
|
||||
checkpoint = saver.get_tuple(config)
|
||||
assert checkpoint.checkpoint["channel_values"] == {}
|
||||
Generated
-1346
File diff suppressed because it is too large.
Load diff
@@ -1,21 +0,0 @@
|
||||
MIT License
|
||||
|
||||
Copyright (c) 2024 LangChain, Inc.
|
||||
|
||||
Permission is hereby granted, free of charge, to any person obtaining a copy
|
||||
of this software and associated documentation files (the "Software"), to deal
|
||||
in the Software without restriction, including without limitation the rights
|
||||
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
||||
copies of the Software, and to permit persons to whom the Software is
|
||||
furnished to do so, subject to the following conditions:
|
||||
|
||||
The above copyright notice and this permission notice shall be included in all
|
||||
copies or substantial portions of the Software.
|
||||
|
||||
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
||||
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
||||
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
||||
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
||||
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
||||
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
||||
SOFTWARE.
|
||||
@@ -1,37 +0,0 @@
|
||||
.PHONY: test test_watch lint format
|
||||
|
||||
######################
|
||||
# TESTING AND COVERAGE
|
||||
######################
|
||||
|
||||
TEST ?= .
|
||||
|
||||
test:
|
||||
uv run pytest $(TEST)
|
||||
|
||||
test_watch:
|
||||
uv run ptw $(TEST)
|
||||
|
||||
######################
|
||||
# LINTING AND FORMATTING
|
||||
######################
|
||||
|
||||
# Define a variable for Python and notebook files.
|
||||
PYTHON_FILES=.
|
||||
MYPY_CACHE=.mypy_cache
|
||||
lint format: PYTHON_FILES=.
|
||||
lint_diff format_diff: PYTHON_FILES=$(shell git diff --name-only --relative --diff-filter=d main . | grep -E '\.py$$|\.ipynb$$')
|
||||
lint_package: PYTHON_FILES=langgraph
|
||||
lint_tests: PYTHON_FILES=tests
|
||||
lint_tests: MYPY_CACHE=.mypy_cache_test
|
||||
|
||||
lint lint_diff lint_package lint_tests:
|
||||
uv run ruff check .
|
||||
[ "$(PYTHON_FILES)" = "" ] || uv run ruff format $(PYTHON_FILES) --diff
|
||||
[ "$(PYTHON_FILES)" = "" ] || uv run ruff check --select I $(PYTHON_FILES)
|
||||
[ "$(PYTHON_FILES)" = "" ] || mkdir -p $(MYPY_CACHE)
|
||||
[ "$(PYTHON_FILES)" = "" ] || uv run mypy $(PYTHON_FILES) --cache-dir $(MYPY_CACHE)
|
||||
|
||||
format format_diff:
|
||||
uv run ruff format $(PYTHON_FILES)
|
||||
uv run ruff check --select I --fix $(PYTHON_FILES)
|
||||
@@ -1,88 +0,0 @@
|
||||
# LangGraph SQLite Checkpoint
|
||||
|
||||
Implementation of LangGraph CheckpointSaver that uses SQLite DB (both sync and async, via `aiosqlite`)
|
||||
|
||||
## Usage
|
||||
|
||||
```python
|
||||
from langgraph.checkpoint.sqlite import SqliteSaver
|
||||
|
||||
write_config = {"configurable": {"thread_id": "1", "checkpoint_ns": ""}}
|
||||
read_config = {"configurable": {"thread_id": "1"}}
|
||||
|
||||
with SqliteSaver.from_conn_string(":memory:") as checkpointer:
|
||||
checkpoint = {
|
||||
"v": 4,
|
||||
"ts": "2024-07-31T20:14:19.804150+00:00",
|
||||
"id": "1ef4f797-8335-6428-8001-8a1503f9b875",
|
||||
"channel_values": {
|
||||
"my_key": "meow",
|
||||
"node": "node"
|
||||
},
|
||||
"channel_versions": {
|
||||
"__start__": 2,
|
||||
"my_key": 3,
|
||||
"start:node": 3,
|
||||
"node": 3
|
||||
},
|
||||
"versions_seen": {
|
||||
"__input__": {},
|
||||
"__start__": {
|
||||
"__start__": 1
|
||||
},
|
||||
"node": {
|
||||
"start:node": 2
|
||||
}
|
||||
},
|
||||
}
|
||||
|
||||
# store checkpoint
|
||||
checkpointer.put(write_config, checkpoint, {}, {})
|
||||
|
||||
# load checkpoint
|
||||
checkpointer.get(read_config)
|
||||
|
||||
# list checkpoints
|
||||
list(checkpointer.list(read_config))
|
||||
```
|
||||
|
||||
### Async
|
||||
|
||||
```python
|
||||
from langgraph.checkpoint.sqlite.aio import AsyncSqliteSaver
|
||||
|
||||
async with AsyncSqliteSaver.from_conn_string(":memory:") as checkpointer:
|
||||
checkpoint = {
|
||||
"v": 4,
|
||||
"ts": "2024-07-31T20:14:19.804150+00:00",
|
||||
"id": "1ef4f797-8335-6428-8001-8a1503f9b875",
|
||||
"channel_values": {
|
||||
"my_key": "meow",
|
||||
"node": "node"
|
||||
},
|
||||
"channel_versions": {
|
||||
"__start__": 2,
|
||||
"my_key": 3,
|
||||
"start:node": 3,
|
||||
"node": 3
|
||||
},
|
||||
"versions_seen": {
|
||||
"__input__": {},
|
||||
"__start__": {
|
||||
"__start__": 1
|
||||
},
|
||||
"node": {
|
||||
"start:node": 2
|
||||
}
|
||||
},
|
||||
}
|
||||
|
||||
# store checkpoint
|
||||
await checkpointer.aput(write_config, checkpoint, {}, {})
|
||||
|
||||
# load checkpoint
|
||||
await checkpointer.aget(read_config)
|
||||
|
||||
# list checkpoints
|
||||
[c async for c in checkpointer.alist(read_config)]
|
||||
```
|
||||
@@ -1,120 +0,0 @@
|
||||
from __future__ import annotations
|
||||
|
||||
import asyncio
|
||||
import datetime
|
||||
import sqlite3
|
||||
import threading
|
||||
from collections.abc import Mapping, Sequence
|
||||
|
||||
from langgraph.cache.base import BaseCache, FullKey, Namespace, ValueT
|
||||
from langgraph.checkpoint.serde.base import SerializerProtocol
|
||||
|
||||
|
||||
class SqliteCache(BaseCache[ValueT]):
|
||||
"""File-based cache using SQLite."""
|
||||
|
||||
def __init__(
|
||||
self,
|
||||
*,
|
||||
path: str,
|
||||
serde: SerializerProtocol | None = None,
|
||||
) -> None:
|
||||
"""Initialize the cache with a file path."""
|
||||
super().__init__(serde=serde)
|
||||
# SQLite backing store
|
||||
self._conn = sqlite3.connect(
|
||||
path,
|
||||
check_same_thread=False,
|
||||
)
|
||||
# Serialize access to the shared connection across threads
|
||||
self._lock = threading.RLock()
|
||||
# Better concurrency & atomicity
|
||||
self._conn.execute("PRAGMA journal_mode=WAL;")
|
||||
# Schema: key -> (expiry, encoding, value)
|
||||
self._conn.execute(
|
||||
"""CREATE TABLE IF NOT EXISTS cache (
|
||||
ns TEXT,
|
||||
key TEXT,
|
||||
expiry REAL,
|
||||
encoding TEXT NOT NULL,
|
||||
val BLOB NOT NULL,
|
||||
PRIMARY KEY (ns, key)
|
||||
)"""
|
||||
)
|
||||
self._conn.commit()
|
||||
|
||||
def get(self, keys: Sequence[FullKey]) -> dict[FullKey, ValueT]:
|
||||
"""Get the cached values for the given keys."""
|
||||
with self._lock, self._conn:
|
||||
now = datetime.datetime.now(datetime.timezone.utc).timestamp()
|
||||
if not keys:
|
||||
return {}
|
||||
placeholders = ",".join("(?, ?)" for _ in keys)
|
||||
params: list[str] = []
|
||||
for ns_tuple, key in keys:
|
||||
params.extend((",".join(ns_tuple), key))
|
||||
cursor = self._conn.execute(
|
||||
f"SELECT ns, key, expiry, encoding, val FROM cache WHERE (ns, key) IN ({placeholders})",
|
||||
tuple(params),
|
||||
)
|
||||
values: dict[FullKey, ValueT] = {}
|
||||
rows = cursor.fetchall()
|
||||
for ns, key, expiry, encoding, raw in rows:
|
||||
if expiry is not None and now > expiry:
|
||||
# purge expired entry
|
||||
self._conn.execute(
|
||||
"DELETE FROM cache WHERE (ns, key) = (?, ?)", (ns, key)
|
||||
)
|
||||
continue
|
||||
values[(tuple(ns.split(",")), key)] = self.serde.loads_typed(
|
||||
(encoding, raw)
|
||||
)
|
||||
return values
|
||||
|
||||
async def aget(self, keys: Sequence[FullKey]) -> dict[FullKey, ValueT]:
|
||||
"""Asynchronously get the cached values for the given keys."""
|
||||
return await asyncio.to_thread(self.get, keys)
|
||||
|
||||
def set(self, mapping: Mapping[FullKey, tuple[ValueT, int | None]]) -> None:
|
||||
"""Set the cached values for the given keys and TTLs."""
|
||||
with self._lock, self._conn:
|
||||
now = datetime.datetime.now(datetime.timezone.utc)
|
||||
for key, (value, ttl) in mapping.items():
|
||||
if ttl is not None:
|
||||
delta = datetime.timedelta(seconds=ttl)
|
||||
expiry: float | None = (now + delta).timestamp()
|
||||
else:
|
||||
expiry = None
|
||||
encoding, raw = self.serde.dumps_typed(value)
|
||||
self._conn.execute(
|
||||
"INSERT OR REPLACE INTO cache (ns, key, expiry, encoding, val) VALUES (?, ?, ?, ?, ?)",
|
||||
(",".join(key[0]), key[1], expiry, encoding, raw),
|
||||
)
|
||||
|
||||
async def aset(self, mapping: Mapping[FullKey, tuple[ValueT, int | None]]) -> None:
|
||||
"""Asynchronously set the cached values for the given keys and TTLs."""
|
||||
await asyncio.to_thread(self.set, mapping)
|
||||
|
||||
def clear(self, namespaces: Sequence[Namespace] | None = None) -> None:
|
||||
"""Delete the cached values for the given namespaces.
|
||||
If no namespaces are provided, clear all cached values."""
|
||||
with self._lock, self._conn:
|
||||
if namespaces is None:
|
||||
self._conn.execute("DELETE FROM cache")
|
||||
else:
|
||||
placeholders = ",".join("?" for _ in namespaces)
|
||||
self._conn.execute(
|
||||
f"DELETE FROM cache WHERE (ns) IN ({placeholders})",
|
||||
tuple(",".join(key) for key in namespaces),
|
||||
)
|
||||
|
||||
async def aclear(self, namespaces: Sequence[Namespace] | None = None) -> None:
|
||||
"""Asynchronously delete the cached values for the given namespaces.
|
||||
If no namespaces are provided, clear all cached values."""
|
||||
await asyncio.to_thread(self.clear, namespaces)
|
||||
|
||||
def __del__(self) -> None:
|
||||
try:
|
||||
self._conn.close()
|
||||
except Exception:
|
||||
pass
|
||||
@@ -1,556 +0,0 @@
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
import random
|
||||
import sqlite3
|
||||
import threading
|
||||
from collections.abc import AsyncIterator, Iterator, Sequence
|
||||
from contextlib import closing, contextmanager
|
||||
from typing import Any, cast
|
||||
|
||||
from langchain_core.runnables import RunnableConfig
|
||||
from langgraph.checkpoint.base import (
|
||||
WRITES_IDX_MAP,
|
||||
BaseCheckpointSaver,
|
||||
ChannelVersions,
|
||||
Checkpoint,
|
||||
CheckpointMetadata,
|
||||
CheckpointTuple,
|
||||
SerializerProtocol,
|
||||
get_checkpoint_id,
|
||||
get_checkpoint_metadata,
|
||||
)
|
||||
from langgraph.checkpoint.serde.jsonplus import JsonPlusSerializer
|
||||
|
||||
from langgraph.checkpoint.sqlite.utils import search_where
|
||||
|
||||
_AIO_ERROR_MSG = (
|
||||
"The SqliteSaver does not support async methods. "
|
||||
"Consider using AsyncSqliteSaver instead.\n"
|
||||
"from langgraph.checkpoint.sqlite.aio import AsyncSqliteSaver\n"
|
||||
"Note: AsyncSqliteSaver requires the aiosqlite package to use.\n"
|
||||
"Install with:\n`pip install aiosqlite`\n"
|
||||
"See https://langchain-ai.github.io/langgraph/reference/checkpoints/#langgraph.checkpoint.sqlite.aio.AsyncSqliteSaver"
|
||||
"for more information."
|
||||
)
|
||||
|
||||
|
||||
class SqliteSaver(BaseCheckpointSaver[str]):
|
||||
"""A checkpoint saver that stores checkpoints in a SQLite database.
|
||||
|
||||
Note:
|
||||
This class is meant for lightweight, synchronous use cases
|
||||
(demos and small projects) and does not
|
||||
scale to multiple threads.
|
||||
For a similar sqlite saver with `async` support,
|
||||
consider using [AsyncSqliteSaver][langgraph.checkpoint.sqlite.aio.AsyncSqliteSaver].
|
||||
|
||||
Args:
|
||||
conn (sqlite3.Connection): The SQLite database connection.
|
||||
serde (Optional[SerializerProtocol]): The serializer to use for serializing and deserializing checkpoints. Defaults to JsonPlusSerializerCompat.
|
||||
|
||||
Examples:
|
||||
|
||||
>>> import sqlite3
|
||||
>>> from langgraph.checkpoint.sqlite import SqliteSaver
|
||||
>>> from langgraph.graph import StateGraph
|
||||
>>>
|
||||
>>> builder = StateGraph(int)
|
||||
>>> builder.add_node("add_one", lambda x: x + 1)
|
||||
>>> builder.set_entry_point("add_one")
|
||||
>>> builder.set_finish_point("add_one")
|
||||
>>> # Create a new SqliteSaver instance
|
||||
>>> # Note: check_same_thread=False is OK as the implementation uses a lock
|
||||
>>> # to ensure thread safety.
|
||||
>>> conn = sqlite3.connect("checkpoints.sqlite", check_same_thread=False)
|
||||
>>> memory = SqliteSaver(conn)
|
||||
>>> graph = builder.compile(checkpointer=memory)
|
||||
>>> config = {"configurable": {"thread_id": "1"}}
|
||||
>>> graph.get_state(config)
|
||||
>>> result = graph.invoke(3, config)
|
||||
>>> graph.get_state(config)
|
||||
StateSnapshot(values=4, next=(), config={'configurable': {'thread_id': '1', 'checkpoint_ns': '', 'checkpoint_id': '0c62ca34-ac19-445d-bbb0-5b4984975b2a'}}, parent_config=None)
|
||||
""" # noqa
|
||||
|
||||
conn: sqlite3.Connection
|
||||
is_setup: bool
|
||||
|
||||
def __init__(
|
||||
self,
|
||||
conn: sqlite3.Connection,
|
||||
*,
|
||||
serde: SerializerProtocol | None = None,
|
||||
) -> None:
|
||||
super().__init__(serde=serde)
|
||||
self.jsonplus_serde = JsonPlusSerializer()
|
||||
self.conn = conn
|
||||
self.is_setup = False
|
||||
self.lock = threading.Lock()
|
||||
|
||||
@classmethod
|
||||
@contextmanager
|
||||
def from_conn_string(cls, conn_string: str) -> Iterator[SqliteSaver]:
|
||||
"""Create a new SqliteSaver instance from a connection string.
|
||||
|
||||
Args:
|
||||
conn_string: The SQLite connection string.
|
||||
|
||||
Yields:
|
||||
SqliteSaver: A new SqliteSaver instance.
|
||||
|
||||
Examples:
|
||||
|
||||
In memory:
|
||||
|
||||
with SqliteSaver.from_conn_string(":memory:") as memory:
|
||||
...
|
||||
|
||||
To disk:
|
||||
|
||||
with SqliteSaver.from_conn_string("checkpoints.sqlite") as memory:
|
||||
...
|
||||
"""
|
||||
with closing(
|
||||
sqlite3.connect(
|
||||
conn_string,
|
||||
# https://ricardoanderegg.com/posts/python-sqlite-thread-safety/
|
||||
check_same_thread=False,
|
||||
)
|
||||
) as conn:
|
||||
yield cls(conn)
|
||||
|
||||
def setup(self) -> None:
|
||||
"""Set up the checkpoint database.
|
||||
|
||||
This method creates the necessary tables in the SQLite database if they don't
|
||||
already exist. It is called automatically when needed and should not be called
|
||||
directly by the user.
|
||||
"""
|
||||
if self.is_setup:
|
||||
return
|
||||
|
||||
self.conn.executescript(
|
||||
"""
|
||||
PRAGMA journal_mode=WAL;
|
||||
CREATE TABLE IF NOT EXISTS checkpoints (
|
||||
thread_id TEXT NOT NULL,
|
||||
checkpoint_ns TEXT NOT NULL DEFAULT '',
|
||||
checkpoint_id TEXT NOT NULL,
|
||||
parent_checkpoint_id TEXT,
|
||||
type TEXT,
|
||||
checkpoint BLOB,
|
||||
metadata BLOB,
|
||||
PRIMARY KEY (thread_id, checkpoint_ns, checkpoint_id)
|
||||
);
|
||||
CREATE TABLE IF NOT EXISTS writes (
|
||||
thread_id TEXT NOT NULL,
|
||||
checkpoint_ns TEXT NOT NULL DEFAULT '',
|
||||
checkpoint_id TEXT NOT NULL,
|
||||
task_id TEXT NOT NULL,
|
||||
idx INTEGER NOT NULL,
|
||||
channel TEXT NOT NULL,
|
||||
type TEXT,
|
||||
value BLOB,
|
||||
PRIMARY KEY (thread_id, checkpoint_ns, checkpoint_id, task_id, idx)
|
||||
);
|
||||
"""
|
||||
)
|
||||
|
||||
self.is_setup = True
|
||||
|
||||
@contextmanager
|
||||
def cursor(self, transaction: bool = True) -> Iterator[sqlite3.Cursor]:
|
||||
"""Get a cursor for the SQLite database.
|
||||
|
||||
This method returns a cursor for the SQLite database. It is used internally
|
||||
by the SqliteSaver and should not be called directly by the user.
|
||||
|
||||
Args:
|
||||
transaction (bool): Whether to commit the transaction when the cursor is closed. Defaults to True.
|
||||
|
||||
Yields:
|
||||
sqlite3.Cursor: A cursor for the SQLite database.
|
||||
"""
|
||||
with self.lock:
|
||||
self.setup()
|
||||
cur = self.conn.cursor()
|
||||
try:
|
||||
yield cur
|
||||
finally:
|
||||
if transaction:
|
||||
self.conn.commit()
|
||||
cur.close()
|
||||
|
||||
def get_tuple(self, config: RunnableConfig) -> CheckpointTuple | None:
|
||||
"""Get a checkpoint tuple from the database.
|
||||
|
||||
This method retrieves a checkpoint tuple from the SQLite database based on the
|
||||
provided config. If the config contains a `checkpoint_id` key, the checkpoint with
|
||||
the matching thread ID and checkpoint ID is retrieved. Otherwise, the latest checkpoint
|
||||
for the given thread ID is retrieved.
|
||||
|
||||
Args:
|
||||
config: The config to use for retrieving the checkpoint.
|
||||
|
||||
Returns:
|
||||
The retrieved checkpoint tuple, or None if no matching checkpoint was found.
|
||||
|
||||
Examples:
|
||||
|
||||
Basic:
|
||||
>>> config = {"configurable": {"thread_id": "1"}}
|
||||
>>> checkpoint_tuple = memory.get_tuple(config)
|
||||
>>> print(checkpoint_tuple)
|
||||
CheckpointTuple(...)
|
||||
|
||||
With checkpoint ID:
|
||||
|
||||
>>> config = {
|
||||
... "configurable": {
|
||||
... "thread_id": "1",
|
||||
... "checkpoint_ns": "",
|
||||
... "checkpoint_id": "1ef4f797-8335-6428-8001-8a1503f9b875",
|
||||
... }
|
||||
... }
|
||||
>>> checkpoint_tuple = memory.get_tuple(config)
|
||||
>>> print(checkpoint_tuple)
|
||||
CheckpointTuple(...)
|
||||
""" # noqa
|
||||
checkpoint_ns = config["configurable"].get("checkpoint_ns", "")
|
||||
with self.cursor(transaction=False) as cur:
|
||||
# find the latest checkpoint for the thread_id
|
||||
if checkpoint_id := get_checkpoint_id(config):
|
||||
cur.execute(
|
||||
"SELECT thread_id, checkpoint_id, parent_checkpoint_id, type, checkpoint, metadata FROM checkpoints WHERE thread_id = ? AND checkpoint_ns = ? AND checkpoint_id = ?",
|
||||
(
|
||||
str(config["configurable"]["thread_id"]),
|
||||
checkpoint_ns,
|
||||
checkpoint_id,
|
||||
),
|
||||
)
|
||||
else:
|
||||
cur.execute(
|
||||
"SELECT thread_id, checkpoint_id, parent_checkpoint_id, type, checkpoint, metadata FROM checkpoints WHERE thread_id = ? AND checkpoint_ns = ? ORDER BY checkpoint_id DESC LIMIT 1",
|
||||
(str(config["configurable"]["thread_id"]), checkpoint_ns),
|
||||
)
|
||||
# if a checkpoint is found, return it
|
||||
if value := cur.fetchone():
|
||||
(
|
||||
thread_id,
|
||||
checkpoint_id,
|
||||
parent_checkpoint_id,
|
||||
type,
|
||||
checkpoint,
|
||||
metadata,
|
||||
) = value
|
||||
if not get_checkpoint_id(config):
|
||||
config = {
|
||||
"configurable": {
|
||||
"thread_id": thread_id,
|
||||
"checkpoint_ns": checkpoint_ns,
|
||||
"checkpoint_id": checkpoint_id,
|
||||
}
|
||||
}
|
||||
# find any pending writes
|
||||
cur.execute(
|
||||
"SELECT task_id, channel, type, value FROM writes WHERE thread_id = ? AND checkpoint_ns = ? AND checkpoint_id = ? ORDER BY task_id, idx",
|
||||
(
|
||||
str(config["configurable"]["thread_id"]),
|
||||
checkpoint_ns,
|
||||
str(config["configurable"]["checkpoint_id"]),
|
||||
),
|
||||
)
|
||||
# deserialize the checkpoint and metadata
|
||||
return CheckpointTuple(
|
||||
config,
|
||||
self.serde.loads_typed((type, checkpoint)),
|
||||
cast(
|
||||
CheckpointMetadata,
|
||||
json.loads(metadata) if metadata is not None else {},
|
||||
),
|
||||
(
|
||||
{
|
||||
"configurable": {
|
||||
"thread_id": thread_id,
|
||||
"checkpoint_ns": checkpoint_ns,
|
||||
"checkpoint_id": parent_checkpoint_id,
|
||||
}
|
||||
}
|
||||
if parent_checkpoint_id
|
||||
else None
|
||||
),
|
||||
[
|
||||
(task_id, channel, self.serde.loads_typed((type, value)))
|
||||
for task_id, channel, type, value in cur
|
||||
],
|
||||
)
|
||||
|
||||
def list(
|
||||
self,
|
||||
config: RunnableConfig | None,
|
||||
*,
|
||||
filter: dict[str, Any] | None = None,
|
||||
before: RunnableConfig | None = None,
|
||||
limit: int | None = None,
|
||||
) -> Iterator[CheckpointTuple]:
|
||||
"""List checkpoints from the database.
|
||||
|
||||
This method retrieves a list of checkpoint tuples from the SQLite database based
|
||||
on the provided config. The checkpoints are ordered by checkpoint ID in descending order (newest first).
|
||||
|
||||
Args:
|
||||
config: The config to use for listing the checkpoints.
|
||||
filter: Additional filtering criteria for metadata.
|
||||
before: If provided, only checkpoints before the specified checkpoint ID are returned.
|
||||
limit: The maximum number of checkpoints to return.
|
||||
|
||||
Yields:
|
||||
An iterator of checkpoint tuples.
|
||||
|
||||
Examples:
|
||||
>>> from langgraph.checkpoint.sqlite import SqliteSaver
|
||||
>>> with SqliteSaver.from_conn_string(":memory:") as memory:
|
||||
... # Run a graph, then list the checkpoints
|
||||
>>> config = {"configurable": {"thread_id": "1"}}
|
||||
>>> checkpoints = list(memory.list(config, limit=2))
|
||||
>>> print(checkpoints)
|
||||
[CheckpointTuple(...), CheckpointTuple(...)]
|
||||
|
||||
>>> config = {"configurable": {"thread_id": "1"}}
|
||||
>>> before = {"configurable": {"checkpoint_id": "1ef4f797-8335-6428-8001-8a1503f9b875"}}
|
||||
>>> with SqliteSaver.from_conn_string(":memory:") as memory:
|
||||
... # Run a graph, then list the checkpoints
|
||||
>>> checkpoints = list(memory.list(config, before=before))
|
||||
>>> print(checkpoints)
|
||||
[CheckpointTuple(...), ...]
|
||||
"""
|
||||
where, param_values = search_where(config, filter, before)
|
||||
query = f"""SELECT thread_id, checkpoint_ns, checkpoint_id, parent_checkpoint_id, type, checkpoint, metadata
|
||||
FROM checkpoints
|
||||
{where}
|
||||
ORDER BY checkpoint_id DESC"""
|
||||
if limit is not None:
|
||||
query += " LIMIT ?"
|
||||
param_values = (*param_values, limit)
|
||||
with self.cursor(transaction=False) as cur, closing(self.conn.cursor()) as wcur:
|
||||
cur.execute(query, param_values)
|
||||
for (
|
||||
thread_id,
|
||||
checkpoint_ns,
|
||||
checkpoint_id,
|
||||
parent_checkpoint_id,
|
||||
type,
|
||||
checkpoint,
|
||||
metadata,
|
||||
) in cur:
|
||||
wcur.execute(
|
||||
"SELECT task_id, channel, type, value FROM writes WHERE thread_id = ? AND checkpoint_ns = ? AND checkpoint_id = ? ORDER BY task_id, idx",
|
||||
(thread_id, checkpoint_ns, checkpoint_id),
|
||||
)
|
||||
yield CheckpointTuple(
|
||||
{
|
||||
"configurable": {
|
||||
"thread_id": thread_id,
|
||||
"checkpoint_ns": checkpoint_ns,
|
||||
"checkpoint_id": checkpoint_id,
|
||||
}
|
||||
},
|
||||
self.serde.loads_typed((type, checkpoint)),
|
||||
cast(
|
||||
CheckpointMetadata,
|
||||
json.loads(metadata) if metadata is not None else {},
|
||||
),
|
||||
(
|
||||
{
|
||||
"configurable": {
|
||||
"thread_id": thread_id,
|
||||
"checkpoint_ns": checkpoint_ns,
|
||||
"checkpoint_id": parent_checkpoint_id,
|
||||
}
|
||||
}
|
||||
if parent_checkpoint_id
|
||||
else None
|
||||
),
|
||||
[
|
||||
(task_id, channel, self.serde.loads_typed((type, value)))
|
||||
for task_id, channel, type, value in wcur
|
||||
],
|
||||
)
|
||||
|
||||
def put(
|
||||
self,
|
||||
config: RunnableConfig,
|
||||
checkpoint: Checkpoint,
|
||||
metadata: CheckpointMetadata,
|
||||
new_versions: ChannelVersions,
|
||||
) -> RunnableConfig:
|
||||
"""Save a checkpoint to the database.
|
||||
|
||||
This method saves a checkpoint to the SQLite database. The checkpoint is associated
|
||||
with the provided config and its parent config (if any).
|
||||
|
||||
Args:
|
||||
config: The config to associate with the checkpoint.
|
||||
checkpoint: The checkpoint to save.
|
||||
metadata: Additional metadata to save with the checkpoint.
|
||||
new_versions: New channel versions as of this write.
|
||||
|
||||
Returns:
|
||||
RunnableConfig: Updated configuration after storing the checkpoint.
|
||||
|
||||
Examples:
|
||||
|
||||
>>> from langgraph.checkpoint.sqlite import SqliteSaver
|
||||
>>> with SqliteSaver.from_conn_string(":memory:") as memory:
|
||||
>>> config = {"configurable": {"thread_id": "1", "checkpoint_ns": ""}}
|
||||
>>> checkpoint = {"ts": "2024-05-04T06:32:42.235444+00:00", "id": "1ef4f797-8335-6428-8001-8a1503f9b875", "channel_values": {"key": "value"}}
|
||||
>>> saved_config = memory.put(config, checkpoint, {"source": "input", "step": 1, "writes": {"key": "value"}}, {})
|
||||
>>> print(saved_config)
|
||||
{'configurable': {'thread_id': '1', 'checkpoint_ns': '', 'checkpoint_id': '1ef4f797-8335-6428-8001-8a1503f9b875'}}
|
||||
"""
|
||||
thread_id = config["configurable"]["thread_id"]
|
||||
checkpoint_ns = config["configurable"]["checkpoint_ns"]
|
||||
type_, serialized_checkpoint = self.serde.dumps_typed(checkpoint)
|
||||
serialized_metadata = json.dumps(
|
||||
get_checkpoint_metadata(config, metadata), ensure_ascii=False
|
||||
).encode("utf-8", "ignore")
|
||||
with self.cursor() as cur:
|
||||
cur.execute(
|
||||
"INSERT OR REPLACE INTO checkpoints (thread_id, checkpoint_ns, checkpoint_id, parent_checkpoint_id, type, checkpoint, metadata) VALUES (?, ?, ?, ?, ?, ?, ?)",
|
||||
(
|
||||
str(config["configurable"]["thread_id"]),
|
||||
checkpoint_ns,
|
||||
checkpoint["id"],
|
||||
config["configurable"].get("checkpoint_id"),
|
||||
type_,
|
||||
serialized_checkpoint,
|
||||
serialized_metadata,
|
||||
),
|
||||
)
|
||||
return {
|
||||
"configurable": {
|
||||
"thread_id": thread_id,
|
||||
"checkpoint_ns": checkpoint_ns,
|
||||
"checkpoint_id": checkpoint["id"],
|
||||
}
|
||||
}
|
||||
|
||||
def put_writes(
|
||||
self,
|
||||
config: RunnableConfig,
|
||||
writes: Sequence[tuple[str, Any]],
|
||||
task_id: str,
|
||||
task_path: str = "",
|
||||
) -> None:
|
||||
"""Store intermediate writes linked to a checkpoint.
|
||||
|
||||
This method saves intermediate writes associated with a checkpoint to the SQLite database.
|
||||
|
||||
Args:
|
||||
config: Configuration of the related checkpoint.
|
||||
writes: List of writes to store, each as (channel, value) pair.
|
||||
task_id: Identifier for the task creating the writes.
|
||||
task_path: Path of the task creating the writes.
|
||||
"""
|
||||
query = (
|
||||
"INSERT OR REPLACE INTO writes (thread_id, checkpoint_ns, checkpoint_id, task_id, idx, channel, type, value) VALUES (?, ?, ?, ?, ?, ?, ?, ?)"
|
||||
if all(w[0] in WRITES_IDX_MAP for w in writes)
|
||||
else "INSERT OR IGNORE INTO writes (thread_id, checkpoint_ns, checkpoint_id, task_id, idx, channel, type, value) VALUES (?, ?, ?, ?, ?, ?, ?, ?)"
|
||||
)
|
||||
with self.cursor() as cur:
|
||||
cur.executemany(
|
||||
query,
|
||||
[
|
||||
(
|
||||
str(config["configurable"]["thread_id"]),
|
||||
str(config["configurable"]["checkpoint_ns"]),
|
||||
str(config["configurable"]["checkpoint_id"]),
|
||||
task_id,
|
||||
WRITES_IDX_MAP.get(channel, idx),
|
||||
channel,
|
||||
*self.serde.dumps_typed(value),
|
||||
)
|
||||
for idx, (channel, value) in enumerate(writes)
|
||||
],
|
||||
)
|
||||
|
||||
def delete_thread(self, thread_id: str) -> None:
|
||||
"""Delete all checkpoints and writes associated with a thread ID.
|
||||
|
||||
Args:
|
||||
thread_id: The thread ID to delete.
|
||||
|
||||
Returns:
|
||||
None
|
||||
"""
|
||||
with self.cursor() as cur:
|
||||
cur.execute(
|
||||
"DELETE FROM checkpoints WHERE thread_id = ?",
|
||||
(str(thread_id),),
|
||||
)
|
||||
cur.execute(
|
||||
"DELETE FROM writes WHERE thread_id = ?",
|
||||
(str(thread_id),),
|
||||
)
|
||||
|
||||
async def aget_tuple(self, config: RunnableConfig) -> CheckpointTuple | None:
|
||||
"""Get a checkpoint tuple from the database asynchronously.
|
||||
|
||||
Note:
|
||||
This async method is not supported by the SqliteSaver class.
|
||||
Use get_tuple() instead, or consider using [AsyncSqliteSaver][langgraph.checkpoint.sqlite.aio.AsyncSqliteSaver].
|
||||
"""
|
||||
raise NotImplementedError(_AIO_ERROR_MSG)
|
||||
|
||||
async def alist(
|
||||
self,
|
||||
config: RunnableConfig | None,
|
||||
*,
|
||||
filter: dict[str, Any] | None = None,
|
||||
before: RunnableConfig | None = None,
|
||||
limit: int | None = None,
|
||||
) -> AsyncIterator[CheckpointTuple]:
|
||||
"""List checkpoints from the database asynchronously.
|
||||
|
||||
Note:
|
||||
This async method is not supported by the SqliteSaver class.
|
||||
Use list() instead, or consider using [AsyncSqliteSaver][langgraph.checkpoint.sqlite.aio.AsyncSqliteSaver].
|
||||
"""
|
||||
raise NotImplementedError(_AIO_ERROR_MSG)
|
||||
yield
|
||||
|
||||
async def aput(
|
||||
self,
|
||||
config: RunnableConfig,
|
||||
checkpoint: Checkpoint,
|
||||
metadata: CheckpointMetadata,
|
||||
new_versions: ChannelVersions,
|
||||
) -> RunnableConfig:
|
||||
"""Save a checkpoint to the database asynchronously.
|
||||
|
||||
Note:
|
||||
This async method is not supported by the SqliteSaver class.
|
||||
Use put() instead, or consider using [AsyncSqliteSaver][langgraph.checkpoint.sqlite.aio.AsyncSqliteSaver].
|
||||
"""
|
||||
raise NotImplementedError(_AIO_ERROR_MSG)
|
||||
|
||||
def get_next_version(self, current: str | None, channel: None) -> str:
|
||||
"""Generate the next version ID for a channel.
|
||||
|
||||
This method creates a new version identifier for a channel based on its current version.
|
||||
|
||||
Args:
|
||||
current (Optional[str]): The current version identifier of the channel.
|
||||
|
||||
Returns:
|
||||
str: The next version identifier, which is guaranteed to be monotonically increasing.
|
||||
"""
|
||||
if current is None:
|
||||
current_v = 0
|
||||
elif isinstance(current, int):
|
||||
current_v = current
|
||||
else:
|
||||
current_v = int(current.split(".")[0])
|
||||
next_v = current_v + 1
|
||||
next_h = random.random()
|
||||
return f"{next_v:032}.{next_h:016}"
|
||||
@@ -1,611 +0,0 @@
|
||||
from __future__ import annotations
|
||||
|
||||
import asyncio
|
||||
import json
|
||||
import random
|
||||
from collections.abc import AsyncIterator, Callable, Iterator, Sequence
|
||||
from contextlib import asynccontextmanager
|
||||
from typing import Any, TypeVar, cast
|
||||
|
||||
import aiosqlite
|
||||
from langchain_core.runnables import RunnableConfig
|
||||
from langgraph.checkpoint.base import (
|
||||
WRITES_IDX_MAP,
|
||||
BaseCheckpointSaver,
|
||||
ChannelVersions,
|
||||
Checkpoint,
|
||||
CheckpointMetadata,
|
||||
CheckpointTuple,
|
||||
SerializerProtocol,
|
||||
get_checkpoint_id,
|
||||
get_checkpoint_metadata,
|
||||
)
|
||||
from langgraph.checkpoint.serde.jsonplus import JsonPlusSerializer
|
||||
|
||||
from langgraph.checkpoint.sqlite.utils import search_where
|
||||
|
||||
T = TypeVar("T", bound=Callable)
|
||||
|
||||
|
||||
class AsyncSqliteSaver(BaseCheckpointSaver[str]):
|
||||
"""An asynchronous checkpoint saver that stores checkpoints in a SQLite database.
|
||||
|
||||
This class provides an asynchronous interface for saving and retrieving checkpoints
|
||||
using a SQLite database. It's designed for use in asynchronous environments and
|
||||
offers better performance for I/O-bound operations compared to synchronous alternatives.
|
||||
|
||||
Attributes:
|
||||
conn (aiosqlite.Connection): The asynchronous SQLite database connection.
|
||||
serde (SerializerProtocol): The serializer used for encoding/decoding checkpoints.
|
||||
|
||||
Tip:
|
||||
Requires the [aiosqlite](https://pypi.org/project/aiosqlite/) package.
|
||||
Install it with `pip install aiosqlite`.
|
||||
|
||||
Warning:
|
||||
While this class supports asynchronous checkpointing, it is not recommended
|
||||
for production workloads due to limitations in SQLite's write performance.
|
||||
For production use, consider a more robust database like PostgreSQL.
|
||||
|
||||
Tip:
|
||||
Remember to **close the database connection** after executing your code,
|
||||
otherwise, you may see the graph "hang" after execution (since the program
|
||||
will not exit until the connection is closed).
|
||||
|
||||
The easiest way is to use the `async with` statement as shown in the examples.
|
||||
|
||||
```python
|
||||
async with AsyncSqliteSaver.from_conn_string("checkpoints.sqlite") as saver:
|
||||
# Your code here
|
||||
graph = builder.compile(checkpointer=saver)
|
||||
config = {"configurable": {"thread_id": "thread-1"}}
|
||||
async for event in graph.astream_events(..., config, version="v1"):
|
||||
print(event)
|
||||
```
|
||||
|
||||
Examples:
|
||||
Usage within StateGraph:
|
||||
|
||||
```pycon
|
||||
>>> import asyncio
|
||||
>>>
|
||||
>>> from langgraph.checkpoint.sqlite.aio import AsyncSqliteSaver
|
||||
>>> from langgraph.graph import StateGraph
|
||||
>>>
|
||||
>>> async def main():
|
||||
>>> builder = StateGraph(int)
|
||||
>>> builder.add_node("add_one", lambda x: x + 1)
|
||||
>>> builder.set_entry_point("add_one")
|
||||
>>> builder.set_finish_point("add_one")
|
||||
>>> async with AsyncSqliteSaver.from_conn_string("checkpoints.db") as memory:
|
||||
>>> graph = builder.compile(checkpointer=memory)
|
||||
>>> coro = graph.ainvoke(1, {"configurable": {"thread_id": "thread-1"}})
|
||||
>>> print(await asyncio.gather(coro))
|
||||
>>>
|
||||
>>> asyncio.run(main())
|
||||
Output: [2]
|
||||
```
|
||||
Raw usage:
|
||||
|
||||
```pycon
|
||||
>>> import asyncio
|
||||
>>> import aiosqlite
|
||||
>>> from langgraph.checkpoint.sqlite.aio import AsyncSqliteSaver
|
||||
>>>
|
||||
>>> async def main():
|
||||
>>> async with aiosqlite.connect("checkpoints.db") as conn:
|
||||
... saver = AsyncSqliteSaver(conn)
|
||||
... config = {"configurable": {"thread_id": "1", "checkpoint_ns": ""}}
|
||||
... checkpoint = {"ts": "2023-05-03T10:00:00Z", "data": {"key": "value"}, "id": "0c62ca34-ac19-445d-bbb0-5b4984975b2a"}
|
||||
... saved_config = await saver.aput(config, checkpoint, {}, {})
|
||||
... print(saved_config)
|
||||
>>> asyncio.run(main())
|
||||
{'configurable': {'thread_id': '1', 'checkpoint_ns': '', 'checkpoint_id': '0c62ca34-ac19-445d-bbb0-5b4984975b2a'}}
|
||||
```
|
||||
"""
|
||||
|
||||
lock: asyncio.Lock
|
||||
is_setup: bool
|
||||
|
||||
def __init__(
|
||||
self,
|
||||
conn: aiosqlite.Connection,
|
||||
*,
|
||||
serde: SerializerProtocol | None = None,
|
||||
):
|
||||
super().__init__(serde=serde)
|
||||
self.jsonplus_serde = JsonPlusSerializer()
|
||||
self.conn = conn
|
||||
self.lock = asyncio.Lock()
|
||||
self.loop = asyncio.get_running_loop()
|
||||
self.is_setup = False
|
||||
|
||||
@classmethod
|
||||
@asynccontextmanager
|
||||
async def from_conn_string(
|
||||
cls, conn_string: str
|
||||
) -> AsyncIterator[AsyncSqliteSaver]:
|
||||
"""Create a new AsyncSqliteSaver instance from a connection string.
|
||||
|
||||
Args:
|
||||
conn_string: The SQLite connection string.
|
||||
|
||||
Yields:
|
||||
AsyncSqliteSaver: A new AsyncSqliteSaver instance.
|
||||
"""
|
||||
async with aiosqlite.connect(conn_string) as conn:
|
||||
yield cls(conn)
|
||||
|
||||
def get_tuple(self, config: RunnableConfig) -> CheckpointTuple | None:
|
||||
"""Get a checkpoint tuple from the database.
|
||||
|
||||
This method retrieves a checkpoint tuple from the SQLite database based on the
|
||||
provided config. If the config contains a `checkpoint_id` key, the checkpoint with
|
||||
the matching thread ID and checkpoint ID is retrieved. Otherwise, the latest checkpoint
|
||||
for the given thread ID is retrieved.
|
||||
|
||||
Args:
|
||||
config: The config to use for retrieving the checkpoint.
|
||||
|
||||
Returns:
|
||||
The retrieved checkpoint tuple, or None if no matching checkpoint was found.
|
||||
"""
|
||||
try:
|
||||
# check if we are in the main thread, only bg threads can block
|
||||
# we don't check in other methods to avoid the overhead
|
||||
if asyncio.get_running_loop() is self.loop:
|
||||
raise asyncio.InvalidStateError(
|
||||
"Synchronous calls to AsyncSqliteSaver are only allowed from a "
|
||||
"different thread. From the main thread, use the async interface. "
|
||||
"For example, use `await checkpointer.aget_tuple(...)` or `await "
|
||||
"graph.ainvoke(...)`."
|
||||
)
|
||||
except RuntimeError:
|
||||
pass
|
||||
return asyncio.run_coroutine_threadsafe(
|
||||
self.aget_tuple(config), self.loop
|
||||
).result()
|
||||
|
||||
def list(
|
||||
self,
|
||||
config: RunnableConfig | None,
|
||||
*,
|
||||
filter: dict[str, Any] | None = None,
|
||||
before: RunnableConfig | None = None,
|
||||
limit: int | None = None,
|
||||
) -> Iterator[CheckpointTuple]:
|
||||
"""List checkpoints from the database asynchronously.
|
||||
|
||||
This method retrieves a list of checkpoint tuples from the SQLite database based
|
||||
on the provided config. The checkpoints are ordered by checkpoint ID in descending order (newest first).
|
||||
|
||||
Args:
|
||||
config: Base configuration for filtering checkpoints.
|
||||
filter: Additional filtering criteria for metadata.
|
||||
before: If provided, only checkpoints before the specified checkpoint ID are returned.
|
||||
limit: Maximum number of checkpoints to return.
|
||||
|
||||
Yields:
|
||||
An iterator of matching checkpoint tuples.
|
||||
"""
|
||||
try:
|
||||
# check if we are in the main thread, only bg threads can block
|
||||
# we don't check in other methods to avoid the overhead
|
||||
if asyncio.get_running_loop() is self.loop:
|
||||
raise asyncio.InvalidStateError(
|
||||
"Synchronous calls to AsyncSqliteSaver are only allowed from a "
|
||||
"different thread. From the main thread, use the async interface. "
|
||||
"For example, use `checkpointer.alist(...)` or `await "
|
||||
"graph.ainvoke(...)`."
|
||||
)
|
||||
except RuntimeError:
|
||||
pass
|
||||
aiter_ = self.alist(config, filter=filter, before=before, limit=limit)
|
||||
while True:
|
||||
try:
|
||||
yield asyncio.run_coroutine_threadsafe(
|
||||
anext(aiter_), # type: ignore[arg-type] # noqa: F821
|
||||
self.loop,
|
||||
).result()
|
||||
except StopAsyncIteration:
|
||||
break
|
||||
|
||||
def put(
|
||||
self,
|
||||
config: RunnableConfig,
|
||||
checkpoint: Checkpoint,
|
||||
metadata: CheckpointMetadata,
|
||||
new_versions: ChannelVersions,
|
||||
) -> RunnableConfig:
|
||||
"""Save a checkpoint to the database.
|
||||
|
||||
This method saves a checkpoint to the SQLite database. The checkpoint is associated
|
||||
with the provided config and its parent config (if any).
|
||||
|
||||
Args:
|
||||
config: The config to associate with the checkpoint.
|
||||
checkpoint: The checkpoint to save.
|
||||
metadata: Additional metadata to save with the checkpoint.
|
||||
new_versions: New channel versions as of this write.
|
||||
|
||||
Returns:
|
||||
RunnableConfig: Updated configuration after storing the checkpoint.
|
||||
"""
|
||||
return asyncio.run_coroutine_threadsafe(
|
||||
self.aput(config, checkpoint, metadata, new_versions), self.loop
|
||||
).result()
|
||||
|
||||
def put_writes(
|
||||
self,
|
||||
config: RunnableConfig,
|
||||
writes: Sequence[tuple[str, Any]],
|
||||
task_id: str,
|
||||
task_path: str = "",
|
||||
) -> None:
|
||||
return asyncio.run_coroutine_threadsafe(
|
||||
self.aput_writes(config, writes, task_id, task_path), self.loop
|
||||
).result()
|
||||
|
||||
def delete_thread(self, thread_id: str) -> None:
|
||||
"""Delete all checkpoints and writes associated with a thread ID.
|
||||
|
||||
Args:
|
||||
thread_id: The thread ID to delete.
|
||||
|
||||
Returns:
|
||||
None
|
||||
"""
|
||||
try:
|
||||
# check if we are in the main thread, only bg threads can block
|
||||
# we don't check in other methods to avoid the overhead
|
||||
if asyncio.get_running_loop() is self.loop:
|
||||
raise asyncio.InvalidStateError(
|
||||
"Synchronous calls to AsyncSqliteSaver are only allowed from a "
|
||||
"different thread. From the main thread, use the async interface. "
|
||||
"For example, use `checkpointer.alist(...)` or `await "
|
||||
"graph.ainvoke(...)`."
|
||||
)
|
||||
except RuntimeError:
|
||||
pass
|
||||
return asyncio.run_coroutine_threadsafe(
|
||||
self.adelete_thread(thread_id), self.loop
|
||||
).result()
|
||||
|
||||
async def setup(self) -> None:
|
||||
"""Set up the checkpoint database asynchronously.
|
||||
|
||||
This method creates the necessary tables in the SQLite database if they don't
|
||||
already exist. It is called automatically when needed and should not be called
|
||||
directly by the user.
|
||||
"""
|
||||
async with self.lock:
|
||||
if self.is_setup:
|
||||
return
|
||||
if not self.conn.is_alive():
|
||||
await self.conn
|
||||
async with self.conn.executescript(
|
||||
"""
|
||||
PRAGMA journal_mode=WAL;
|
||||
CREATE TABLE IF NOT EXISTS checkpoints (
|
||||
thread_id TEXT NOT NULL,
|
||||
checkpoint_ns TEXT NOT NULL DEFAULT '',
|
||||
checkpoint_id TEXT NOT NULL,
|
||||
parent_checkpoint_id TEXT,
|
||||
type TEXT,
|
||||
checkpoint BLOB,
|
||||
metadata BLOB,
|
||||
PRIMARY KEY (thread_id, checkpoint_ns, checkpoint_id)
|
||||
);
|
||||
CREATE TABLE IF NOT EXISTS writes (
|
||||
thread_id TEXT NOT NULL,
|
||||
checkpoint_ns TEXT NOT NULL DEFAULT '',
|
||||
checkpoint_id TEXT NOT NULL,
|
||||
task_id TEXT NOT NULL,
|
||||
idx INTEGER NOT NULL,
|
||||
channel TEXT NOT NULL,
|
||||
type TEXT,
|
||||
value BLOB,
|
||||
PRIMARY KEY (thread_id, checkpoint_ns, checkpoint_id, task_id, idx)
|
||||
);
|
||||
"""
|
||||
):
|
||||
await self.conn.commit()
|
||||
|
||||
self.is_setup = True
|
||||
|
||||
async def aget_tuple(self, config: RunnableConfig) -> CheckpointTuple | None:
|
||||
"""Get a checkpoint tuple from the database asynchronously.
|
||||
|
||||
This method retrieves a checkpoint tuple from the SQLite database based on the
|
||||
provided config. If the config contains a `checkpoint_id` key, the checkpoint with
|
||||
the matching thread ID and checkpoint ID is retrieved. Otherwise, the latest checkpoint
|
||||
for the given thread ID is retrieved.
|
||||
|
||||
Args:
|
||||
config: The config to use for retrieving the checkpoint.
|
||||
|
||||
Returns:
|
||||
The retrieved checkpoint tuple, or None if no matching checkpoint was found.
|
||||
"""
|
||||
await self.setup()
|
||||
checkpoint_ns = config["configurable"].get("checkpoint_ns", "")
|
||||
async with self.lock, self.conn.cursor() as cur:
|
||||
# find the latest checkpoint for the thread_id
|
||||
if checkpoint_id := get_checkpoint_id(config):
|
||||
await cur.execute(
|
||||
"SELECT thread_id, checkpoint_id, parent_checkpoint_id, type, checkpoint, metadata FROM checkpoints WHERE thread_id = ? AND checkpoint_ns = ? AND checkpoint_id = ?",
|
||||
(
|
||||
str(config["configurable"]["thread_id"]),
|
||||
checkpoint_ns,
|
||||
checkpoint_id,
|
||||
),
|
||||
)
|
||||
else:
|
||||
await cur.execute(
|
||||
"SELECT thread_id, checkpoint_id, parent_checkpoint_id, type, checkpoint, metadata FROM checkpoints WHERE thread_id = ? AND checkpoint_ns = ? ORDER BY checkpoint_id DESC LIMIT 1",
|
||||
(str(config["configurable"]["thread_id"]), checkpoint_ns),
|
||||
)
|
||||
# if a checkpoint is found, return it
|
||||
if value := await cur.fetchone():
|
||||
(
|
||||
thread_id,
|
||||
checkpoint_id,
|
||||
parent_checkpoint_id,
|
||||
type,
|
||||
checkpoint,
|
||||
metadata,
|
||||
) = value
|
||||
if not get_checkpoint_id(config):
|
||||
config = {
|
||||
"configurable": {
|
||||
"thread_id": thread_id,
|
||||
"checkpoint_ns": checkpoint_ns,
|
||||
"checkpoint_id": checkpoint_id,
|
||||
}
|
||||
}
|
||||
# find any pending writes
|
||||
await cur.execute(
|
||||
"SELECT task_id, channel, type, value FROM writes WHERE thread_id = ? AND checkpoint_ns = ? AND checkpoint_id = ? ORDER BY task_id, idx",
|
||||
(
|
||||
str(config["configurable"]["thread_id"]),
|
||||
checkpoint_ns,
|
||||
str(config["configurable"]["checkpoint_id"]),
|
||||
),
|
||||
)
|
||||
# deserialize the checkpoint and metadata
|
||||
return CheckpointTuple(
|
||||
config,
|
||||
self.serde.loads_typed((type, checkpoint)),
|
||||
cast(
|
||||
CheckpointMetadata,
|
||||
(json.loads(metadata) if metadata is not None else {}),
|
||||
),
|
||||
(
|
||||
{
|
||||
"configurable": {
|
||||
"thread_id": thread_id,
|
||||
"checkpoint_ns": checkpoint_ns,
|
||||
"checkpoint_id": parent_checkpoint_id,
|
||||
}
|
||||
}
|
||||
if parent_checkpoint_id
|
||||
else None
|
||||
),
|
||||
[
|
||||
(task_id, channel, self.serde.loads_typed((type, value)))
|
||||
async for task_id, channel, type, value in cur
|
||||
],
|
||||
)
|
||||
|
||||
async def alist(
|
||||
self,
|
||||
config: RunnableConfig | None,
|
||||
*,
|
||||
filter: dict[str, Any] | None = None,
|
||||
before: RunnableConfig | None = None,
|
||||
limit: int | None = None,
|
||||
) -> AsyncIterator[CheckpointTuple]:
|
||||
"""List checkpoints from the database asynchronously.
|
||||
|
||||
This method retrieves a list of checkpoint tuples from the SQLite database based
|
||||
on the provided config. The checkpoints are ordered by checkpoint ID in descending order (newest first).
|
||||
|
||||
Args:
|
||||
config: Base configuration for filtering checkpoints.
|
||||
filter: Additional filtering criteria for metadata.
|
||||
before: If provided, only checkpoints before the specified checkpoint ID are returned.
|
||||
limit: Maximum number of checkpoints to return.
|
||||
|
||||
Yields:
|
||||
An asynchronous iterator of matching checkpoint tuples.
|
||||
"""
|
||||
await self.setup()
|
||||
where, params = search_where(config, filter, before)
|
||||
query = f"""SELECT thread_id, checkpoint_ns, checkpoint_id, parent_checkpoint_id, type, checkpoint, metadata
|
||||
FROM checkpoints
|
||||
{where}
|
||||
ORDER BY checkpoint_id DESC"""
|
||||
if limit is not None:
|
||||
query += " LIMIT ?"
|
||||
params = (*params, limit)
|
||||
async with (
|
||||
self.lock,
|
||||
self.conn.execute(query, params) as cur,
|
||||
self.conn.cursor() as wcur,
|
||||
):
|
||||
async for (
|
||||
thread_id,
|
||||
checkpoint_ns,
|
||||
checkpoint_id,
|
||||
parent_checkpoint_id,
|
||||
type,
|
||||
checkpoint,
|
||||
metadata,
|
||||
) in cur:
|
||||
await wcur.execute(
|
||||
"SELECT task_id, channel, type, value FROM writes WHERE thread_id = ? AND checkpoint_ns = ? AND checkpoint_id = ? ORDER BY task_id, idx",
|
||||
(thread_id, checkpoint_ns, checkpoint_id),
|
||||
)
|
||||
yield CheckpointTuple(
|
||||
{
|
||||
"configurable": {
|
||||
"thread_id": thread_id,
|
||||
"checkpoint_ns": checkpoint_ns,
|
||||
"checkpoint_id": checkpoint_id,
|
||||
}
|
||||
},
|
||||
self.serde.loads_typed((type, checkpoint)),
|
||||
cast(
|
||||
CheckpointMetadata,
|
||||
(json.loads(metadata) if metadata is not None else {}),
|
||||
),
|
||||
(
|
||||
{
|
||||
"configurable": {
|
||||
"thread_id": thread_id,
|
||||
"checkpoint_ns": checkpoint_ns,
|
||||
"checkpoint_id": parent_checkpoint_id,
|
||||
}
|
||||
}
|
||||
if parent_checkpoint_id
|
||||
else None
|
||||
),
|
||||
[
|
||||
(task_id, channel, self.serde.loads_typed((type, value)))
|
||||
async for task_id, channel, type, value in wcur
|
||||
],
|
||||
)
|
||||
|
||||
async def aput(
|
||||
self,
|
||||
config: RunnableConfig,
|
||||
checkpoint: Checkpoint,
|
||||
metadata: CheckpointMetadata,
|
||||
new_versions: ChannelVersions,
|
||||
) -> RunnableConfig:
|
||||
"""Save a checkpoint to the database asynchronously.
|
||||
|
||||
This method saves a checkpoint to the SQLite database. The checkpoint is associated
|
||||
with the provided config and its parent config (if any).
|
||||
|
||||
Args:
|
||||
config: The config to associate with the checkpoint.
|
||||
checkpoint: The checkpoint to save.
|
||||
metadata: Additional metadata to save with the checkpoint.
|
||||
new_versions: New channel versions as of this write.
|
||||
|
||||
Returns:
|
||||
RunnableConfig: Updated configuration after storing the checkpoint.
|
||||
"""
|
||||
await self.setup()
|
||||
thread_id = config["configurable"]["thread_id"]
|
||||
checkpoint_ns = config["configurable"]["checkpoint_ns"]
|
||||
type_, serialized_checkpoint = self.serde.dumps_typed(checkpoint)
|
||||
serialized_metadata = json.dumps(
|
||||
get_checkpoint_metadata(config, metadata), ensure_ascii=False
|
||||
).encode("utf-8", "ignore")
|
||||
async with (
|
||||
self.lock,
|
||||
self.conn.execute(
|
||||
"INSERT OR REPLACE INTO checkpoints (thread_id, checkpoint_ns, checkpoint_id, parent_checkpoint_id, type, checkpoint, metadata) VALUES (?, ?, ?, ?, ?, ?, ?)",
|
||||
(
|
||||
str(config["configurable"]["thread_id"]),
|
||||
checkpoint_ns,
|
||||
checkpoint["id"],
|
||||
config["configurable"].get("checkpoint_id"),
|
||||
type_,
|
||||
serialized_checkpoint,
|
||||
serialized_metadata,
|
||||
),
|
||||
),
|
||||
):
|
||||
await self.conn.commit()
|
||||
return {
|
||||
"configurable": {
|
||||
"thread_id": thread_id,
|
||||
"checkpoint_ns": checkpoint_ns,
|
||||
"checkpoint_id": checkpoint["id"],
|
||||
}
|
||||
}
|
||||
|
||||
async def aput_writes(
|
||||
self,
|
||||
config: RunnableConfig,
|
||||
writes: Sequence[tuple[str, Any]],
|
||||
task_id: str,
|
||||
task_path: str = "",
|
||||
) -> None:
|
||||
"""Store intermediate writes linked to a checkpoint asynchronously.
|
||||
|
||||
This method saves intermediate writes associated with a checkpoint to the database.
|
||||
|
||||
Args:
|
||||
config: Configuration of the related checkpoint.
|
||||
writes: List of writes to store, each as (channel, value) pair.
|
||||
task_id: Identifier for the task creating the writes.
|
||||
task_path: Path of the task creating the writes.
|
||||
"""
|
||||
query = (
|
||||
"INSERT OR REPLACE INTO writes (thread_id, checkpoint_ns, checkpoint_id, task_id, idx, channel, type, value) VALUES (?, ?, ?, ?, ?, ?, ?, ?)"
|
||||
if all(w[0] in WRITES_IDX_MAP for w in writes)
|
||||
else "INSERT OR IGNORE INTO writes (thread_id, checkpoint_ns, checkpoint_id, task_id, idx, channel, type, value) VALUES (?, ?, ?, ?, ?, ?, ?, ?)"
|
||||
)
|
||||
await self.setup()
|
||||
async with self.lock, self.conn.cursor() as cur:
|
||||
await cur.executemany(
|
||||
query,
|
||||
[
|
||||
(
|
||||
str(config["configurable"]["thread_id"]),
|
||||
str(config["configurable"]["checkpoint_ns"]),
|
||||
str(config["configurable"]["checkpoint_id"]),
|
||||
task_id,
|
||||
WRITES_IDX_MAP.get(channel, idx),
|
||||
channel,
|
||||
*self.serde.dumps_typed(value),
|
||||
)
|
||||
for idx, (channel, value) in enumerate(writes)
|
||||
],
|
||||
)
|
||||
await self.conn.commit()
|
||||
|
||||
async def adelete_thread(self, thread_id: str) -> None:
|
||||
"""Delete all checkpoints and writes associated with a thread ID.
|
||||
|
||||
Args:
|
||||
thread_id: The thread ID to delete.
|
||||
|
||||
Returns:
|
||||
None
|
||||
"""
|
||||
async with self.lock, self.conn.cursor() as cur:
|
||||
await cur.execute(
|
||||
"DELETE FROM checkpoints WHERE thread_id = ?",
|
||||
(str(thread_id),),
|
||||
)
|
||||
await cur.execute(
|
||||
"DELETE FROM writes WHERE thread_id = ?",
|
||||
(str(thread_id),),
|
||||
)
|
||||
await self.conn.commit()
|
||||
|
||||
def get_next_version(self, current: str | None, channel: None) -> str:
|
||||
"""Generate the next version ID for a channel.
|
||||
|
||||
This method creates a new version identifier for a channel based on its current version.
|
||||
|
||||
Args:
|
||||
current (Optional[str]): The current version identifier of the channel.
|
||||
|
||||
Returns:
|
||||
str: The next version identifier, which is guaranteed to be monotonically increasing.
|
||||
"""
|
||||
if current is None:
|
||||
current_v = 0
|
||||
elif isinstance(current, int):
|
||||
current_v = current
|
||||
else:
|
||||
current_v = int(current.split(".")[0])
|
||||
next_v = current_v + 1
|
||||
next_h = random.random()
|
||||
return f"{next_v:032}.{next_h:016}"
|
||||
@@ -1,116 +0,0 @@
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
import re
|
||||
from collections.abc import Sequence
|
||||
from typing import Any
|
||||
|
||||
from langchain_core.runnables import RunnableConfig
|
||||
from langgraph.checkpoint.base import get_checkpoint_id
|
||||
|
||||
_FILTER_PATTERN = re.compile(r"^[a-zA-Z0-9_.-]+$")
|
||||
|
||||
|
||||
def _validate_filter_key(key: str) -> None:
|
||||
"""Validate that a filter key is safe for use in SQL queries.
|
||||
|
||||
Args:
|
||||
key: The filter key to validate
|
||||
|
||||
Raises:
|
||||
ValueError: If the key contains invalid characters that could enable SQL injection
|
||||
"""
|
||||
# Allow alphanumeric characters, underscores, dots, and hyphens
|
||||
# This covers typical JSON property names while preventing SQL injection
|
||||
if not _FILTER_PATTERN.match(key):
|
||||
raise ValueError(
|
||||
f"Invalid filter key: '{key}'. Filter keys must contain only alphanumeric characters, underscores, dots, and hyphens."
|
||||
)
|
||||
|
||||
|
||||
def _metadata_predicate(
|
||||
metadata_filter: dict[str, Any],
|
||||
) -> tuple[Sequence[str], Sequence[Any]]:
|
||||
"""Return WHERE clause predicates for (a)search() given metadata filter.
|
||||
|
||||
This method returns a tuple of a string and a tuple of values. The string
|
||||
is the parametered WHERE clause predicate (excluding the WHERE keyword):
|
||||
"column1 = ? AND column2 IS ?". The tuple of values contains the values
|
||||
for each of the corresponding parameters.
|
||||
"""
|
||||
|
||||
def _where_value(query_value: Any) -> tuple[str, Any]:
|
||||
"""Return tuple of operator and value for WHERE clause predicate."""
|
||||
if query_value is None:
|
||||
return ("IS ?", None)
|
||||
elif (
|
||||
isinstance(query_value, str)
|
||||
or isinstance(query_value, int)
|
||||
or isinstance(query_value, float)
|
||||
):
|
||||
return ("= ?", query_value)
|
||||
elif isinstance(query_value, bool):
|
||||
return ("= ?", 1 if query_value else 0)
|
||||
elif isinstance(query_value, dict) or isinstance(query_value, list):
|
||||
# query value for JSON object cannot have trailing space after separators (, :)
|
||||
# SQLite json_extract() returns JSON string without whitespace
|
||||
return ("= ?", json.dumps(query_value, separators=(",", ":")))
|
||||
else:
|
||||
return ("= ?", str(query_value))
|
||||
|
||||
predicates = []
|
||||
param_values = []
|
||||
|
||||
# process metadata query
|
||||
for query_key, query_value in metadata_filter.items():
|
||||
_validate_filter_key(query_key)
|
||||
operator, param_value = _where_value(query_value)
|
||||
predicates.append(
|
||||
f"json_extract(CAST(metadata AS TEXT), '$.{query_key}') {operator}"
|
||||
)
|
||||
param_values.append(param_value)
|
||||
|
||||
return (predicates, param_values)
|
||||
|
||||
|
||||
def search_where(
|
||||
config: RunnableConfig | None,
|
||||
filter: dict[str, Any] | None,
|
||||
before: RunnableConfig | None = None,
|
||||
) -> tuple[str, Sequence[Any]]:
|
||||
"""Return WHERE clause predicates for (a)search() given metadata filter
|
||||
and `before` config.
|
||||
|
||||
This method returns a tuple of a string and a tuple of values. The string
|
||||
is the parametered WHERE clause predicate (including the WHERE keyword):
|
||||
"WHERE column1 = ? AND column2 IS ?". The tuple of values contains the
|
||||
values for each of the corresponding parameters.
|
||||
"""
|
||||
wheres = []
|
||||
param_values = []
|
||||
|
||||
# construct predicate for config filter
|
||||
if config is not None:
|
||||
wheres.append("thread_id = ?")
|
||||
param_values.append(config["configurable"]["thread_id"])
|
||||
checkpoint_ns = config["configurable"].get("checkpoint_ns")
|
||||
if checkpoint_ns is not None:
|
||||
wheres.append("checkpoint_ns = ?")
|
||||
param_values.append(checkpoint_ns)
|
||||
|
||||
if checkpoint_id := get_checkpoint_id(config):
|
||||
wheres.append("checkpoint_id = ?")
|
||||
param_values.append(checkpoint_id)
|
||||
|
||||
# construct predicate for metadata filter
|
||||
if filter:
|
||||
metadata_predicates, metadata_values = _metadata_predicate(filter)
|
||||
wheres.extend(metadata_predicates)
|
||||
param_values.extend(metadata_values)
|
||||
|
||||
# construct predicate for `before`
|
||||
if before is not None:
|
||||
wheres.append("checkpoint_id < ?")
|
||||
param_values.append(get_checkpoint_id(before))
|
||||
|
||||
return ("WHERE " + " AND ".join(wheres) if wheres else "", param_values)
|
||||
@@ -1,4 +0,0 @@
|
||||
from langgraph.store.sqlite.aio import AsyncSqliteStore
|
||||
from langgraph.store.sqlite.base import SqliteStore
|
||||
|
||||
__all__ = ["AsyncSqliteStore", "SqliteStore"]
|
||||
@@ -1,623 +0,0 @@
|
||||
from __future__ import annotations
|
||||
|
||||
import asyncio
|
||||
import logging
|
||||
from collections import defaultdict
|
||||
from collections.abc import AsyncIterator, Callable, Iterable, Sequence
|
||||
from contextlib import asynccontextmanager
|
||||
from types import TracebackType
|
||||
from typing import Any, cast
|
||||
|
||||
import aiosqlite
|
||||
import orjson
|
||||
import sqlite_vec # type: ignore[import-untyped]
|
||||
from langgraph.store.base import (
|
||||
GetOp,
|
||||
ListNamespacesOp,
|
||||
Op,
|
||||
PutOp,
|
||||
Result,
|
||||
SearchOp,
|
||||
TTLConfig,
|
||||
)
|
||||
from langgraph.store.base.batch import AsyncBatchedBaseStore
|
||||
|
||||
from langgraph.store.sqlite.base import (
|
||||
_PLACEHOLDER,
|
||||
BaseSqliteStore,
|
||||
SqliteIndexConfig,
|
||||
_decode_ns_text,
|
||||
_ensure_index_config,
|
||||
_group_ops,
|
||||
_row_to_item,
|
||||
_row_to_search_item,
|
||||
)
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
class AsyncSqliteStore(AsyncBatchedBaseStore, BaseSqliteStore):
|
||||
"""Asynchronous SQLite-backed store with optional vector search.
|
||||
|
||||
This class provides an asynchronous interface for storing and retrieving data
|
||||
using a SQLite database with support for vector search capabilities.
|
||||
|
||||
Examples:
|
||||
Basic setup and usage:
|
||||
```python
|
||||
from langgraph.store.sqlite import AsyncSqliteStore
|
||||
|
||||
async with AsyncSqliteStore.from_conn_string(":memory:") as store:
|
||||
await store.setup() # Run migrations
|
||||
|
||||
# Store and retrieve data
|
||||
await store.aput(("users", "123"), "prefs", {"theme": "dark"})
|
||||
item = await store.aget(("users", "123"), "prefs")
|
||||
```
|
||||
|
||||
Vector search using LangChain embeddings:
|
||||
```python
|
||||
from langchain_openai import OpenAIEmbeddings
|
||||
from langgraph.store.sqlite import AsyncSqliteStore
|
||||
|
||||
async with AsyncSqliteStore.from_conn_string(
|
||||
":memory:",
|
||||
index={
|
||||
"dims": 1536,
|
||||
"embed": OpenAIEmbeddings(),
|
||||
"fields": ["text"] # specify which fields to embed
|
||||
}
|
||||
) as store:
|
||||
await store.setup() # Run migrations once
|
||||
|
||||
# Store documents
|
||||
await store.aput(("docs",), "doc1", {"text": "Python tutorial"})
|
||||
await store.aput(("docs",), "doc2", {"text": "TypeScript guide"})
|
||||
await store.aput(("docs",), "doc3", {"text": "Other guide"}, index=False) # don't index
|
||||
|
||||
# Search by similarity
|
||||
results = await store.asearch(("docs",), query="programming guides", limit=2)
|
||||
```
|
||||
|
||||
Warning:
|
||||
Make sure to call `setup()` before first use to create necessary tables and indexes.
|
||||
|
||||
Note:
|
||||
This class requires the aiosqlite package. Install with `pip install aiosqlite`.
|
||||
"""
|
||||
|
||||
def __init__(
|
||||
self,
|
||||
conn: aiosqlite.Connection,
|
||||
*,
|
||||
deserializer: Callable[[bytes | str | orjson.Fragment], dict[str, Any]]
|
||||
| None = None,
|
||||
index: SqliteIndexConfig | None = None,
|
||||
ttl: TTLConfig | None = None,
|
||||
):
|
||||
"""Initialize the async SQLite store.
|
||||
|
||||
Args:
|
||||
conn: The SQLite database connection.
|
||||
deserializer: Optional custom deserializer function for values.
|
||||
index: Optional vector search configuration.
|
||||
ttl: Optional time-to-live configuration.
|
||||
"""
|
||||
super().__init__()
|
||||
self._deserializer = deserializer
|
||||
self.conn = conn
|
||||
self.lock = asyncio.Lock()
|
||||
self.loop = asyncio.get_running_loop()
|
||||
self.is_setup = False
|
||||
self.index_config = index
|
||||
if self.index_config:
|
||||
self.embeddings, self.index_config = _ensure_index_config(self.index_config)
|
||||
else:
|
||||
self.embeddings = None
|
||||
self.ttl_config = ttl
|
||||
self._ttl_sweeper_task: asyncio.Task[None] | None = None
|
||||
self._ttl_stop_event = asyncio.Event()
|
||||
|
||||
@classmethod
|
||||
@asynccontextmanager
|
||||
async def from_conn_string(
|
||||
cls,
|
||||
conn_string: str,
|
||||
*,
|
||||
index: SqliteIndexConfig | None = None,
|
||||
ttl: TTLConfig | None = None,
|
||||
) -> AsyncIterator[AsyncSqliteStore]:
|
||||
"""Create a new AsyncSqliteStore instance from a connection string.
|
||||
|
||||
Args:
|
||||
conn_string: The SQLite connection string.
|
||||
index: Optional vector search configuration.
|
||||
ttl: Optional time-to-live configuration.
|
||||
|
||||
Returns:
|
||||
An AsyncSqliteStore instance wrapped in an async context manager.
|
||||
"""
|
||||
async with aiosqlite.connect(conn_string, isolation_level=None) as conn:
|
||||
yield cls(conn, index=index, ttl=ttl)
|
||||
|
||||
async def setup(self) -> None:
|
||||
"""Set up the store database.
|
||||
|
||||
This method creates the necessary tables in the SQLite database if they don't
|
||||
already exist and runs database migrations. It should be called before first use.
|
||||
"""
|
||||
async with self.lock:
|
||||
if self.is_setup:
|
||||
return
|
||||
|
||||
# Create migrations table if it doesn't exist
|
||||
await self.conn.execute(
|
||||
"""
|
||||
CREATE TABLE IF NOT EXISTS store_migrations (
|
||||
v INTEGER PRIMARY KEY
|
||||
)
|
||||
"""
|
||||
)
|
||||
|
||||
# Check current migration version
|
||||
async with self.conn.execute(
|
||||
"SELECT v FROM store_migrations ORDER BY v DESC LIMIT 1"
|
||||
) as cur:
|
||||
row = await cur.fetchone()
|
||||
if row is None:
|
||||
version = -1
|
||||
else:
|
||||
version = row[0]
|
||||
|
||||
# Apply migrations
|
||||
for v, sql in enumerate(self.MIGRATIONS[version + 1 :], start=version + 1):
|
||||
await self.conn.executescript(sql)
|
||||
await self.conn.execute(
|
||||
"INSERT INTO store_migrations (v) VALUES (?)", (v,)
|
||||
)
|
||||
|
||||
# Apply vector migrations if index config is provided
|
||||
if self.index_config:
|
||||
# Create vector migrations table if it doesn't exist
|
||||
await self.conn.enable_load_extension(True)
|
||||
await self.conn.load_extension(sqlite_vec.loadable_path())
|
||||
await self.conn.enable_load_extension(False)
|
||||
await self.conn.execute(
|
||||
"""
|
||||
CREATE TABLE IF NOT EXISTS vector_migrations (
|
||||
v INTEGER PRIMARY KEY
|
||||
)
|
||||
"""
|
||||
)
|
||||
|
||||
# Check current vector migration version
|
||||
async with self.conn.execute(
|
||||
"SELECT v FROM vector_migrations ORDER BY v DESC LIMIT 1"
|
||||
) as cur:
|
||||
row = await cur.fetchone()
|
||||
if row is None:
|
||||
version = -1
|
||||
else:
|
||||
version = row[0]
|
||||
|
||||
# Apply vector migrations
|
||||
for v, sql in enumerate(
|
||||
self.VECTOR_MIGRATIONS[version + 1 :], start=version + 1
|
||||
):
|
||||
await self.conn.executescript(sql)
|
||||
await self.conn.execute(
|
||||
"INSERT INTO vector_migrations (v) VALUES (?)", (v,)
|
||||
)
|
||||
|
||||
self.is_setup = True
|
||||
|
||||
@asynccontextmanager
|
||||
async def _cursor(
|
||||
self, *, transaction: bool = True
|
||||
) -> AsyncIterator[aiosqlite.Cursor]:
|
||||
"""Get a cursor for the SQLite database.
|
||||
|
||||
Args:
|
||||
transaction: Whether to use a transaction for database operations.
|
||||
|
||||
Yields:
|
||||
An SQLite cursor object.
|
||||
"""
|
||||
if not self.is_setup:
|
||||
await self.setup()
|
||||
async with self.lock:
|
||||
if transaction:
|
||||
await self.conn.execute("BEGIN")
|
||||
|
||||
async with self.conn.cursor() as cur:
|
||||
try:
|
||||
yield cur
|
||||
finally:
|
||||
if transaction:
|
||||
await self.conn.execute("COMMIT")
|
||||
|
||||
async def sweep_ttl(self) -> int:
|
||||
"""Delete expired store items based on TTL.
|
||||
|
||||
Returns:
|
||||
int: The number of deleted items.
|
||||
"""
|
||||
async with self._cursor() as cur:
|
||||
await cur.execute(
|
||||
"""
|
||||
DELETE FROM store
|
||||
WHERE expires_at IS NOT NULL AND expires_at < CURRENT_TIMESTAMP
|
||||
"""
|
||||
)
|
||||
deleted_count = cur.rowcount
|
||||
return deleted_count
|
||||
|
||||
async def start_ttl_sweeper(
|
||||
self, sweep_interval_minutes: int | None = None
|
||||
) -> asyncio.Task[None]:
|
||||
"""Periodically delete expired store items based on TTL.
|
||||
|
||||
Returns:
|
||||
Task that can be awaited or cancelled.
|
||||
"""
|
||||
if not self.ttl_config:
|
||||
return asyncio.create_task(asyncio.sleep(0))
|
||||
|
||||
if self._ttl_sweeper_task is not None and not self._ttl_sweeper_task.done():
|
||||
return self._ttl_sweeper_task
|
||||
|
||||
self._ttl_stop_event.clear()
|
||||
|
||||
interval = float(
|
||||
sweep_interval_minutes or self.ttl_config.get("sweep_interval_minutes") or 5
|
||||
)
|
||||
logger.info(f"Starting store TTL sweeper with interval {interval} minutes")
|
||||
|
||||
async def _sweep_loop() -> None:
|
||||
while not self._ttl_stop_event.is_set():
|
||||
try:
|
||||
try:
|
||||
await asyncio.wait_for(
|
||||
self._ttl_stop_event.wait(),
|
||||
timeout=interval * 60,
|
||||
)
|
||||
break
|
||||
except asyncio.TimeoutError:
|
||||
pass
|
||||
|
||||
expired_items = await self.sweep_ttl()
|
||||
if expired_items > 0:
|
||||
logger.info(f"Store swept {expired_items} expired items")
|
||||
except asyncio.CancelledError:
|
||||
break
|
||||
except Exception as exc:
|
||||
logger.exception("Store TTL sweep iteration failed", exc_info=exc)
|
||||
|
||||
task = asyncio.create_task(_sweep_loop())
|
||||
task.set_name("ttl_sweeper")
|
||||
self._ttl_sweeper_task = task
|
||||
return task
|
||||
|
||||
async def stop_ttl_sweeper(self, timeout: float | None = None) -> bool:
|
||||
"""Stop the TTL sweeper task if it's running.
|
||||
|
||||
Args:
|
||||
timeout: Maximum time to wait for the task to stop, in seconds.
|
||||
If `None`, wait indefinitely.
|
||||
|
||||
Returns:
|
||||
bool: True if the task was successfully stopped or wasn't running,
|
||||
False if the timeout was reached before the task stopped.
|
||||
"""
|
||||
if self._ttl_sweeper_task is None or self._ttl_sweeper_task.done():
|
||||
return True
|
||||
|
||||
logger.info("Stopping TTL sweeper task")
|
||||
self._ttl_stop_event.set()
|
||||
|
||||
if timeout is not None:
|
||||
try:
|
||||
await asyncio.wait_for(self._ttl_sweeper_task, timeout=timeout)
|
||||
success = True
|
||||
except asyncio.TimeoutError:
|
||||
success = False
|
||||
else:
|
||||
await self._ttl_sweeper_task
|
||||
success = True
|
||||
|
||||
if success:
|
||||
self._ttl_sweeper_task = None
|
||||
logger.info("TTL sweeper task stopped")
|
||||
else:
|
||||
logger.warning("Timed out waiting for TTL sweeper task to stop")
|
||||
|
||||
return success
|
||||
|
||||
async def __aenter__(self) -> AsyncSqliteStore:
|
||||
return self
|
||||
|
||||
async def __aexit__(
|
||||
self,
|
||||
exc_type: type[BaseException] | None,
|
||||
exc_val: BaseException | None,
|
||||
exc_tb: TracebackType | None,
|
||||
) -> None:
|
||||
# Ensure the TTL sweeper task is stopped when exiting the context
|
||||
if hasattr(self, "_ttl_sweeper_task") and self._ttl_sweeper_task is not None:
|
||||
# Set the event to signal the task to stop
|
||||
self._ttl_stop_event.set()
|
||||
# We don't wait for the task to complete here to avoid blocking
|
||||
# The task will clean up itself gracefully
|
||||
|
||||
async def abatch(self, ops: Iterable[Op]) -> list[Result]:
|
||||
"""Execute a batch of operations asynchronously.
|
||||
|
||||
Args:
|
||||
ops: Iterable of operations to execute.
|
||||
|
||||
Returns:
|
||||
List of operation results.
|
||||
"""
|
||||
grouped_ops, num_ops = _group_ops(ops)
|
||||
results: list[Result] = [None] * num_ops
|
||||
|
||||
async with self._cursor(transaction=True) as cur:
|
||||
if GetOp in grouped_ops:
|
||||
await self._batch_get_ops(
|
||||
cast(Sequence[tuple[int, GetOp]], grouped_ops[GetOp]), results, cur
|
||||
)
|
||||
|
||||
if SearchOp in grouped_ops:
|
||||
await self._batch_search_ops(
|
||||
cast(Sequence[tuple[int, SearchOp]], grouped_ops[SearchOp]),
|
||||
results,
|
||||
cur,
|
||||
)
|
||||
|
||||
if ListNamespacesOp in grouped_ops:
|
||||
await self._batch_list_namespaces_ops(
|
||||
cast(
|
||||
Sequence[tuple[int, ListNamespacesOp]],
|
||||
grouped_ops[ListNamespacesOp],
|
||||
),
|
||||
results,
|
||||
cur,
|
||||
)
|
||||
|
||||
if PutOp in grouped_ops:
|
||||
await self._batch_put_ops(
|
||||
cast(Sequence[tuple[int, PutOp]], grouped_ops[PutOp]), cur
|
||||
)
|
||||
|
||||
return results
|
||||
|
||||
async def _batch_get_ops(
|
||||
self,
|
||||
get_ops: Sequence[tuple[int, GetOp]],
|
||||
results: list[Result],
|
||||
cur: aiosqlite.Cursor,
|
||||
) -> None:
|
||||
"""Process batch GET operations.
|
||||
|
||||
Args:
|
||||
get_ops: Sequence of GET operations.
|
||||
results: List to store results in.
|
||||
cur: Database cursor.
|
||||
"""
|
||||
# Group all queries by namespace to execute all operations for each namespace together
|
||||
namespace_queries = defaultdict(list)
|
||||
for prepared_query in self._get_batch_GET_ops_queries(get_ops):
|
||||
namespace_queries[prepared_query.namespace].append(prepared_query)
|
||||
|
||||
# Process each namespace's operations
|
||||
for namespace, queries in namespace_queries.items():
|
||||
# Execute TTL refresh queries first
|
||||
for query in queries:
|
||||
if query.kind == "refresh":
|
||||
try:
|
||||
await cur.execute(query.query, query.params)
|
||||
except Exception as e:
|
||||
raise ValueError(
|
||||
f"Error executing TTL refresh: \n{query.query}\n{query.params}\n{e}"
|
||||
) from e
|
||||
|
||||
# Then execute GET queries and process results
|
||||
for query in queries:
|
||||
if query.kind == "get":
|
||||
try:
|
||||
await cur.execute(query.query, query.params)
|
||||
except Exception as e:
|
||||
raise ValueError(
|
||||
f"Error executing GET query: \n{query.query}\n{query.params}\n{e}"
|
||||
) from e
|
||||
|
||||
rows = await cur.fetchall()
|
||||
key_to_row = {
|
||||
row[0]: {
|
||||
"key": row[0],
|
||||
"value": row[1],
|
||||
"created_at": row[2],
|
||||
"updated_at": row[3],
|
||||
"expires_at": row[4] if len(row) > 4 else None,
|
||||
"ttl_minutes": row[5] if len(row) > 5 else None,
|
||||
}
|
||||
for row in rows
|
||||
}
|
||||
|
||||
# Process results for this query
|
||||
for idx, key in query.items:
|
||||
row = key_to_row.get(key)
|
||||
if row:
|
||||
results[idx] = _row_to_item(
|
||||
namespace, row, loader=self._deserializer
|
||||
)
|
||||
else:
|
||||
results[idx] = None
|
||||
|
||||
async def _batch_put_ops(
|
||||
self,
|
||||
put_ops: Sequence[tuple[int, PutOp]],
|
||||
cur: aiosqlite.Cursor,
|
||||
) -> None:
|
||||
"""Process batch PUT operations.
|
||||
|
||||
Args:
|
||||
put_ops: Sequence of PUT operations.
|
||||
cur: Database cursor.
|
||||
"""
|
||||
queries, embedding_request = self._prepare_batch_PUT_queries(put_ops)
|
||||
if embedding_request:
|
||||
if self.embeddings is None:
|
||||
# Should not get here since the embedding config is required
|
||||
# to return an embedding_request above
|
||||
raise ValueError(
|
||||
"Embedding configuration is required for vector operations "
|
||||
f"(for semantic search). "
|
||||
f"Please provide an Embeddings when initializing the {self.__class__.__name__}."
|
||||
)
|
||||
|
||||
query, txt_params = embedding_request
|
||||
# Update the params to replace the raw text with the vectors
|
||||
vectors = await self.embeddings.aembed_documents(
|
||||
[param[-1] for param in txt_params]
|
||||
)
|
||||
|
||||
# Convert vectors to SQLite-friendly format
|
||||
vector_params = []
|
||||
for (ns, k, pathname, _), vector in zip(txt_params, vectors, strict=False):
|
||||
vector_params.extend(
|
||||
[ns, k, pathname, sqlite_vec.serialize_float32(vector)]
|
||||
)
|
||||
|
||||
queries.append((query, vector_params))
|
||||
|
||||
for query, params in queries:
|
||||
await cur.execute(query, params)
|
||||
|
||||
async def _batch_search_ops(
|
||||
self,
|
||||
search_ops: Sequence[tuple[int, SearchOp]],
|
||||
results: list[Result],
|
||||
cur: aiosqlite.Cursor,
|
||||
) -> None:
|
||||
"""Process batch SEARCH operations.
|
||||
|
||||
Args:
|
||||
search_ops: Sequence of SEARCH operations.
|
||||
results: List to store results in.
|
||||
cur: Database cursor.
|
||||
"""
|
||||
prepared_queries, embedding_requests = self._prepare_batch_search_queries(
|
||||
search_ops
|
||||
)
|
||||
|
||||
# Setup dot_product function if it doesn't exist
|
||||
if embedding_requests and self.embeddings:
|
||||
vectors = await self.embeddings.aembed_documents(
|
||||
[query for _, query in embedding_requests]
|
||||
)
|
||||
|
||||
for (embed_req_idx, _), embedding in zip(
|
||||
embedding_requests, vectors, strict=False
|
||||
):
|
||||
# Find the corresponding query in prepared_queries
|
||||
# The embed_req_idx is the original index in search_ops, which should map to prepared_queries
|
||||
if embed_req_idx < len(prepared_queries):
|
||||
_params_list: list = prepared_queries[embed_req_idx][1]
|
||||
for i, param in enumerate(_params_list):
|
||||
if param is _PLACEHOLDER:
|
||||
_params_list[i] = sqlite_vec.serialize_float32(embedding)
|
||||
else:
|
||||
logger.warning(
|
||||
f"Embedding request index {embed_req_idx} out of bounds for prepared_queries."
|
||||
)
|
||||
|
||||
for (original_op_idx, _), (query, params, needs_refresh) in zip(
|
||||
search_ops, prepared_queries, strict=False
|
||||
):
|
||||
await cur.execute(query, params)
|
||||
rows = await cur.fetchall()
|
||||
|
||||
if needs_refresh and rows and self.ttl_config:
|
||||
keys_to_refresh = []
|
||||
for row_data in rows:
|
||||
# Assuming row_data[0] is prefix (text), row_data[1] is key (text)
|
||||
# These are raw text values directly from the DB.
|
||||
keys_to_refresh.append((row_data[0], row_data[1]))
|
||||
|
||||
if keys_to_refresh:
|
||||
updates_by_prefix = defaultdict(list)
|
||||
for prefix_text, key_text in keys_to_refresh:
|
||||
updates_by_prefix[prefix_text].append(key_text)
|
||||
|
||||
for prefix_text, key_list in updates_by_prefix.items():
|
||||
placeholders = ",".join(["?"] * len(key_list))
|
||||
update_query = f"""
|
||||
UPDATE store
|
||||
SET expires_at = DATETIME(CURRENT_TIMESTAMP, '+' || ttl_minutes || ' minutes')
|
||||
WHERE prefix = ? AND key IN ({placeholders}) AND ttl_minutes IS NOT NULL
|
||||
"""
|
||||
update_params = (prefix_text, *key_list)
|
||||
try:
|
||||
await cur.execute(update_query, update_params)
|
||||
except Exception as e:
|
||||
logger.error(
|
||||
f"Error during TTL refresh update for search: {e}"
|
||||
)
|
||||
|
||||
# Process rows into items
|
||||
if "score" in query: # Vector search query
|
||||
items = [
|
||||
_row_to_search_item(
|
||||
_decode_ns_text(row[0]), # prefix
|
||||
{
|
||||
"key": row[1], # key
|
||||
"value": row[2], # value
|
||||
"created_at": row[3],
|
||||
"updated_at": row[4],
|
||||
"expires_at": row[5] if len(row) > 5 else None,
|
||||
"ttl_minutes": row[6] if len(row) > 6 else None,
|
||||
"score": row[7] if len(row) > 7 else None,
|
||||
},
|
||||
loader=self._deserializer,
|
||||
)
|
||||
for row in rows
|
||||
]
|
||||
else: # Regular search query
|
||||
items = [
|
||||
_row_to_search_item(
|
||||
_decode_ns_text(row[0]), # prefix
|
||||
{
|
||||
"key": row[1], # key
|
||||
"value": row[2], # value
|
||||
"created_at": row[3],
|
||||
"updated_at": row[4],
|
||||
"expires_at": row[5] if len(row) > 5 else None,
|
||||
"ttl_minutes": row[6] if len(row) > 6 else None,
|
||||
},
|
||||
loader=self._deserializer,
|
||||
)
|
||||
for row in rows
|
||||
]
|
||||
|
||||
results[original_op_idx] = items
|
||||
|
||||
async def _batch_list_namespaces_ops(
|
||||
self,
|
||||
list_ops: Sequence[tuple[int, ListNamespacesOp]],
|
||||
results: list[Result],
|
||||
cur: aiosqlite.Cursor,
|
||||
) -> None:
|
||||
"""Process batch LIST NAMESPACES operations.
|
||||
|
||||
Args:
|
||||
list_ops: Sequence of LIST NAMESPACES operations.
|
||||
results: List to store results in.
|
||||
cur: Database cursor.
|
||||
"""
|
||||
queries = self._get_batch_list_namespaces_queries(list_ops)
|
||||
for (query, params), (idx, _) in zip(queries, list_ops, strict=False):
|
||||
await cur.execute(query, params)
|
||||
|
||||
rows = await cur.fetchall()
|
||||
results[idx] = [_decode_ns_text(row[0]) for row in rows]
|
||||
File diff suppressed because it is too large.
Load diff
@@ -1,84 +0,0 @@
|
||||
[build-system]
|
||||
requires = ["hatchling"]
|
||||
build-backend = "hatchling.build"
|
||||
|
||||
[project]
|
||||
name = "langgraph-checkpoint-sqlite"
|
||||
version = "3.0.2"
|
||||
description = "Library with a SQLite implementation of LangGraph checkpoint saver."
|
||||
authors = []
|
||||
requires-python = ">=3.10"
|
||||
readme = "README.md"
|
||||
license = "MIT"
|
||||
license-files = ['LICENSE']
|
||||
dependencies = [
|
||||
"langgraph-checkpoint>=3,<5.0.0",
|
||||
"aiosqlite>=0.20",
|
||||
"sqlite-vec>=0.1.6",
|
||||
]
|
||||
|
||||
[project.urls]
|
||||
Source = "https://github.com/langchain-ai/langgraph/tree/main/libs/checkpoint-sqlite"
|
||||
Twitter = "https://x.com/LangChainAI"
|
||||
Slack = "https://www.langchain.com/join-community"
|
||||
Reddit = "https://www.reddit.com/r/LangChain/"
|
||||
|
||||
[dependency-groups]
|
||||
test = [
|
||||
"pytest",
|
||||
"pytest-asyncio",
|
||||
"pytest-mock",
|
||||
"pytest-watcher",
|
||||
"langgraph-checkpoint",
|
||||
"pytest-retry>=1.7.0",
|
||||
]
|
||||
lint = [
|
||||
"ruff",
|
||||
"codespell",
|
||||
"mypy",
|
||||
]
|
||||
dev = [
|
||||
{include-group = "test"},
|
||||
{include-group = "lint"},
|
||||
]
|
||||
|
||||
[tool.uv]
|
||||
default-groups = ['dev']
|
||||
|
||||
[tool.uv.sources]
|
||||
langgraph-checkpoint = { path = "../checkpoint", editable = true }
|
||||
|
||||
[tool.hatch.build.targets.wheel]
|
||||
include = ["langgraph"]
|
||||
|
||||
[tool.pytest.ini_options]
|
||||
addopts = "--strict-markers --strict-config --durations=5 -vv"
|
||||
asyncio_mode = "auto"
|
||||
|
||||
[tool.ruff]
|
||||
lint.select = [
|
||||
"E", # pycodestyle
|
||||
"F", # Pyflakes
|
||||
"UP", # pyupgrade
|
||||
"B", # flake8-bugbear
|
||||
"I", # isort
|
||||
"UP", # pyupgrade
|
||||
]
|
||||
lint.ignore = ["E501", "B008"]
|
||||
target-version = "py310"
|
||||
|
||||
[tool.pytest-watcher]
|
||||
now = true
|
||||
delay = 0.1
|
||||
runner_args = ["--ff", "-v", "--tb", "short"]
|
||||
patterns = ["*.py"]
|
||||
|
||||
[tool.mypy]
|
||||
# https://mypy.readthedocs.io/en/stable/config_file.html
|
||||
disallow_untyped_defs = "True"
|
||||
explicit_package_bases = "True"
|
||||
warn_no_return = "False"
|
||||
warn_unused_ignores = "True"
|
||||
warn_redundant_casts = "True"
|
||||
allow_redefinition = "True"
|
||||
disable_error_code = "typeddict-item, return-value"
|
||||
@@ -1,190 +0,0 @@
|
||||
from typing import Any
|
||||
|
||||
import pytest
|
||||
from langchain_core.runnables import RunnableConfig
|
||||
from langgraph.checkpoint.base import (
|
||||
Checkpoint,
|
||||
CheckpointMetadata,
|
||||
create_checkpoint,
|
||||
empty_checkpoint,
|
||||
)
|
||||
|
||||
from langgraph.checkpoint.sqlite.aio import AsyncSqliteSaver
|
||||
|
||||
|
||||
class TestAsyncSqliteSaver:
|
||||
@pytest.fixture(autouse=True)
|
||||
def setup(self) -> None:
|
||||
# objects for test setup
|
||||
self.config_1: RunnableConfig = {
|
||||
"configurable": {
|
||||
"thread_id": "thread-1",
|
||||
"checkpoint_id": "1",
|
||||
"checkpoint_ns": "",
|
||||
}
|
||||
}
|
||||
self.config_2: RunnableConfig = {
|
||||
"configurable": {
|
||||
"thread_id": "thread-2",
|
||||
"checkpoint_id": "2",
|
||||
"checkpoint_ns": "",
|
||||
}
|
||||
}
|
||||
self.config_3: RunnableConfig = {
|
||||
"configurable": {
|
||||
"thread_id": "thread-2",
|
||||
"checkpoint_id": "2-inner",
|
||||
"checkpoint_ns": "inner",
|
||||
}
|
||||
}
|
||||
|
||||
self.chkpnt_1: Checkpoint = empty_checkpoint()
|
||||
self.chkpnt_2: Checkpoint = create_checkpoint(self.chkpnt_1, {}, 1)
|
||||
self.chkpnt_3: Checkpoint = empty_checkpoint()
|
||||
|
||||
self.metadata_1: CheckpointMetadata = {
|
||||
"source": "input",
|
||||
"step": 2,
|
||||
"writes": {},
|
||||
"score": 1,
|
||||
}
|
||||
self.metadata_2: CheckpointMetadata = {
|
||||
"source": "loop",
|
||||
"step": 1,
|
||||
"writes": {"foo": "bar"},
|
||||
"score": None,
|
||||
}
|
||||
self.metadata_3: CheckpointMetadata = {}
|
||||
|
||||
async def test_combined_metadata(self) -> None:
|
||||
async with AsyncSqliteSaver.from_conn_string(":memory:") as saver:
|
||||
config: RunnableConfig = {
|
||||
"configurable": {
|
||||
"thread_id": "thread-2",
|
||||
"checkpoint_ns": "",
|
||||
"__super_private_key": "super_private_value",
|
||||
},
|
||||
"metadata": {"run_id": "my_run_id"},
|
||||
}
|
||||
await saver.aput(config, self.chkpnt_2, self.metadata_2, {})
|
||||
checkpoint = await saver.aget_tuple(config)
|
||||
assert checkpoint is not None and checkpoint.metadata == {
|
||||
**self.metadata_2,
|
||||
"run_id": "my_run_id",
|
||||
}
|
||||
|
||||
async def test_asearch(self) -> None:
|
||||
async with AsyncSqliteSaver.from_conn_string(":memory:") as saver:
|
||||
await saver.aput(self.config_1, self.chkpnt_1, self.metadata_1, {})
|
||||
await saver.aput(self.config_2, self.chkpnt_2, self.metadata_2, {})
|
||||
await saver.aput(self.config_3, self.chkpnt_3, self.metadata_3, {})
|
||||
|
||||
# call method / assertions
|
||||
query_1 = {"source": "input"} # search by 1 key
|
||||
query_2 = {
|
||||
"step": 1,
|
||||
"writes": {"foo": "bar"},
|
||||
} # search by multiple keys
|
||||
query_3: dict[str, Any] = {} # search by no keys, return all checkpoints
|
||||
query_4 = {"source": "update", "step": 1} # no match
|
||||
|
||||
search_results_1 = [c async for c in saver.alist(None, filter=query_1)]
|
||||
assert len(search_results_1) == 1
|
||||
assert search_results_1[0].metadata == self.metadata_1
|
||||
|
||||
search_results_2 = [c async for c in saver.alist(None, filter=query_2)]
|
||||
assert len(search_results_2) == 1
|
||||
assert search_results_2[0].metadata == self.metadata_2
|
||||
|
||||
search_results_3 = [c async for c in saver.alist(None, filter=query_3)]
|
||||
assert len(search_results_3) == 3
|
||||
|
||||
search_results_4 = [c async for c in saver.alist(None, filter=query_4)]
|
||||
assert len(search_results_4) == 0
|
||||
|
||||
# search by config (defaults to checkpoints across all namespaces)
|
||||
search_results_5 = [
|
||||
c
|
||||
async for c in saver.alist({"configurable": {"thread_id": "thread-2"}})
|
||||
]
|
||||
assert len(search_results_5) == 2
|
||||
assert {
|
||||
search_results_5[0].config["configurable"]["checkpoint_ns"],
|
||||
search_results_5[1].config["configurable"]["checkpoint_ns"],
|
||||
} == {"", "inner"}
|
||||
|
||||
# Test limit param
|
||||
search_results_6 = [
|
||||
c
|
||||
async for c in saver.alist(
|
||||
{"configurable": {"thread_id": "thread-2"}}, limit=1
|
||||
)
|
||||
]
|
||||
assert len(search_results_6) == 1
|
||||
assert search_results_6[0].config["configurable"]["thread_id"] == "thread-2"
|
||||
|
||||
# Test before param
|
||||
search_results_7 = [
|
||||
c async for c in saver.alist(None, before=search_results_5[1].config)
|
||||
]
|
||||
assert len(search_results_7) == 1
|
||||
assert search_results_7[0].config["configurable"]["thread_id"] == "thread-1"
|
||||
|
||||
async def test_limit_parameter_sql_injection_prevention(self) -> None:
|
||||
"""Test that the limit parameter properly uses parameterized queries to prevent SQL injection."""
|
||||
async with AsyncSqliteSaver.from_conn_string(":memory:") as saver:
|
||||
# Setup: Create multiple checkpoints
|
||||
for i in range(5):
|
||||
config: RunnableConfig = {
|
||||
"configurable": {
|
||||
"thread_id": f"thread-{i}",
|
||||
"checkpoint_ns": "",
|
||||
}
|
||||
}
|
||||
checkpoint = empty_checkpoint()
|
||||
metadata: CheckpointMetadata = {"index": i}
|
||||
await saver.aput(config, checkpoint, metadata, {})
|
||||
|
||||
# Test that limit works correctly with valid integer
|
||||
results = [c async for c in saver.alist(None, limit=2)]
|
||||
assert len(results) == 2
|
||||
|
||||
# Test that limit=0 returns no results
|
||||
results = [c async for c in saver.alist(None, limit=0)]
|
||||
assert len(results) == 0
|
||||
|
||||
# Test that limit=None returns all results
|
||||
results = [c async for c in saver.alist(None, limit=None)]
|
||||
assert len(results) == 5
|
||||
|
||||
# Test explicit SQL injection attempt via limit parameter
|
||||
# Even if type checking is bypassed and a malicious string is passed,
|
||||
# the parameterized query will treat it as a value, not SQL code
|
||||
# This would cause an error (can't convert string to int for LIMIT),
|
||||
# which is the correct secure behavior
|
||||
malicious_limits = [
|
||||
"1; DROP TABLE checkpoints; --",
|
||||
"1 OR 1=1",
|
||||
"999999 UNION SELECT * FROM checkpoints",
|
||||
]
|
||||
|
||||
for malicious_limit in malicious_limits:
|
||||
# The parameterized query should safely reject non-integer limits
|
||||
# or convert them in a way that prevents SQL injection
|
||||
try:
|
||||
# Bypass type checking by casting
|
||||
results = [
|
||||
c
|
||||
async for c in saver.alist(None, limit=malicious_limit) # type: ignore
|
||||
]
|
||||
# If it doesn't raise an error, it should at least not execute the injection
|
||||
# SQLite's parameter binding will try to convert the string to an integer
|
||||
# which will either fail or treat it as 0
|
||||
except Exception:
|
||||
# Expected: SQLite should reject invalid limit values
|
||||
pass
|
||||
|
||||
# Verify the checkpoints table still exists and has all data
|
||||
# (would have been dropped if injection succeeded)
|
||||
results = [c async for c in saver.alist(None, limit=None)]
|
||||
assert len(results) == 5
|
||||
@@ -1,719 +0,0 @@
|
||||
# mypy: disable-error-code="union-attr,arg-type,index,operator"
|
||||
import asyncio
|
||||
import os
|
||||
import tempfile
|
||||
import uuid
|
||||
from collections.abc import AsyncIterator, Generator, Iterable
|
||||
from contextlib import asynccontextmanager
|
||||
from typing import cast
|
||||
|
||||
import pytest
|
||||
from langgraph.store.base import (
|
||||
GetOp,
|
||||
Item,
|
||||
ListNamespacesOp,
|
||||
PutOp,
|
||||
SearchOp,
|
||||
)
|
||||
|
||||
from langgraph.store.sqlite import AsyncSqliteStore
|
||||
from langgraph.store.sqlite.base import SqliteIndexConfig
|
||||
from tests.test_store import CharacterEmbeddings
|
||||
|
||||
|
||||
@pytest.fixture(scope="function", params=["memory", "file"])
|
||||
async def store(request: pytest.FixtureRequest) -> AsyncIterator[AsyncSqliteStore]:
|
||||
"""Create an AsyncSqliteStore for testing."""
|
||||
if request.param == "memory":
|
||||
# In-memory store
|
||||
async with AsyncSqliteStore.from_conn_string(":memory:") as store:
|
||||
await store.setup()
|
||||
yield store
|
||||
else:
|
||||
# Temporary file store
|
||||
temp_file = tempfile.NamedTemporaryFile(delete=False)
|
||||
temp_file.close()
|
||||
try:
|
||||
async with AsyncSqliteStore.from_conn_string(temp_file.name) as store:
|
||||
await store.setup()
|
||||
yield store
|
||||
finally:
|
||||
os.unlink(temp_file.name)
|
||||
|
||||
|
||||
@pytest.fixture(scope="function")
|
||||
def fake_embeddings() -> CharacterEmbeddings:
|
||||
"""Create fake embeddings for testing."""
|
||||
return CharacterEmbeddings(dims=500)
|
||||
|
||||
|
||||
@asynccontextmanager
|
||||
async def create_vector_store(
|
||||
fake_embeddings: CharacterEmbeddings,
|
||||
conn_string: str = ":memory:",
|
||||
text_fields: list[str] | None = None,
|
||||
) -> AsyncIterator[AsyncSqliteStore]:
|
||||
"""Create an AsyncSqliteStore with vector search capabilities."""
|
||||
index_config: SqliteIndexConfig = {
|
||||
"dims": fake_embeddings.dims,
|
||||
"embed": fake_embeddings,
|
||||
"text_fields": text_fields,
|
||||
}
|
||||
|
||||
async with AsyncSqliteStore.from_conn_string(
|
||||
conn_string, index=index_config
|
||||
) as store:
|
||||
await store.setup()
|
||||
yield store
|
||||
|
||||
|
||||
@pytest.fixture(scope="function", params=["memory", "file"])
|
||||
def conn_string(request: pytest.FixtureRequest) -> Generator[str, None, None]:
|
||||
if request.param == "memory":
|
||||
yield ":memory:"
|
||||
else:
|
||||
temp_file = tempfile.NamedTemporaryFile(delete=False)
|
||||
temp_file.close()
|
||||
try:
|
||||
yield temp_file.name
|
||||
finally:
|
||||
os.unlink(temp_file.name)
|
||||
|
||||
|
||||
async def test_no_running_loop(store: AsyncSqliteStore) -> None:
|
||||
"""Test that sync methods raise proper errors in the main thread."""
|
||||
with pytest.raises(asyncio.InvalidStateError):
|
||||
store.put(("foo", "bar"), "baz", {"val": "baz"})
|
||||
with pytest.raises(asyncio.InvalidStateError):
|
||||
store.get(("foo", "bar"), "baz")
|
||||
with pytest.raises(asyncio.InvalidStateError):
|
||||
store.delete(("foo", "bar"), "baz")
|
||||
with pytest.raises(asyncio.InvalidStateError):
|
||||
store.search(("foo", "bar"))
|
||||
with pytest.raises(asyncio.InvalidStateError):
|
||||
store.list_namespaces(prefix=("foo",))
|
||||
with pytest.raises(asyncio.InvalidStateError):
|
||||
store.batch([PutOp(namespace=("foo", "bar"), key="baz", value={"val": "baz"})])
|
||||
|
||||
|
||||
async def test_large_batches_async(store: AsyncSqliteStore) -> None:
|
||||
"""Test processing large batch operations asynchronously."""
|
||||
N = 100
|
||||
M = 10
|
||||
coros = []
|
||||
for m in range(M):
|
||||
for i in range(N):
|
||||
coros.append(
|
||||
store.aput(
|
||||
("test", "foo", "bar", "baz", str(m % 2)),
|
||||
f"key{i}",
|
||||
value={"foo": "bar" + str(i)},
|
||||
)
|
||||
)
|
||||
coros.append(
|
||||
asyncio.create_task(
|
||||
store.aget(
|
||||
("test", "foo", "bar", "baz", str(m % 2)),
|
||||
f"key{i}",
|
||||
)
|
||||
)
|
||||
)
|
||||
coros.append(
|
||||
asyncio.create_task(
|
||||
store.alist_namespaces(
|
||||
prefix=None,
|
||||
max_depth=m + 1,
|
||||
)
|
||||
)
|
||||
)
|
||||
coros.append(
|
||||
asyncio.create_task(
|
||||
store.asearch(
|
||||
("test",),
|
||||
)
|
||||
)
|
||||
)
|
||||
coros.append(
|
||||
store.aput(
|
||||
("test", "foo", "bar", "baz", str(m % 2)),
|
||||
f"key{i}",
|
||||
value={"foo": "bar" + str(i)},
|
||||
)
|
||||
)
|
||||
coros.append(
|
||||
store.adelete(
|
||||
("test", "foo", "bar", "baz", str(m % 2)),
|
||||
f"key{i}",
|
||||
)
|
||||
)
|
||||
|
||||
results = await asyncio.gather(*coros)
|
||||
assert len(results) == M * N * 6
|
||||
|
||||
|
||||
async def test_abatch_order(store: AsyncSqliteStore) -> None:
|
||||
"""Test ordering of batch operations in async context."""
|
||||
# Setup test data
|
||||
await store.aput(("test", "foo"), "key1", {"data": "value1"})
|
||||
await store.aput(("test", "bar"), "key2", {"data": "value2"})
|
||||
|
||||
ops = [
|
||||
GetOp(namespace=("test", "foo"), key="key1"),
|
||||
PutOp(namespace=("test", "bar"), key="key2", value={"data": "value2"}),
|
||||
SearchOp(
|
||||
namespace_prefix=("test",), filter={"data": "value1"}, limit=10, offset=0
|
||||
),
|
||||
ListNamespacesOp(match_conditions=None, max_depth=None, limit=10, offset=0),
|
||||
GetOp(namespace=("test",), key="key3"),
|
||||
]
|
||||
|
||||
results = await store.abatch(
|
||||
cast(Iterable[GetOp | PutOp | SearchOp | ListNamespacesOp], ops)
|
||||
)
|
||||
assert len(results) == 5
|
||||
assert isinstance(results[0], Item)
|
||||
assert isinstance(results[0].value, dict)
|
||||
assert results[0].value == {"data": "value1"}
|
||||
assert results[0].key == "key1"
|
||||
assert results[1] is None # Put operation returns None
|
||||
assert isinstance(results[2], list)
|
||||
# SQLite query implementation might return different results
|
||||
# Just check that we get a list back and don't check the exact content
|
||||
assert isinstance(results[3], list)
|
||||
assert len(results[3]) > 0
|
||||
assert results[4] is None # Non-existent key returns None
|
||||
|
||||
# Test reordered operations
|
||||
ops_reordered = [
|
||||
SearchOp(namespace_prefix=("test",), filter=None, limit=5, offset=0),
|
||||
GetOp(namespace=("test", "bar"), key="key2"),
|
||||
ListNamespacesOp(match_conditions=None, max_depth=None, limit=5, offset=0),
|
||||
PutOp(namespace=("test",), key="key3", value={"data": "value3"}),
|
||||
GetOp(namespace=("test", "foo"), key="key1"),
|
||||
]
|
||||
|
||||
results_reordered = await store.abatch(
|
||||
cast(Iterable[GetOp | PutOp | SearchOp | ListNamespacesOp], ops_reordered)
|
||||
)
|
||||
assert len(results_reordered) == 5
|
||||
assert isinstance(results_reordered[0], list)
|
||||
assert len(results_reordered[0]) >= 2 # Should find at least our two test items
|
||||
assert isinstance(results_reordered[1], Item)
|
||||
assert results_reordered[1].value == {"data": "value2"}
|
||||
assert results_reordered[1].key == "key2"
|
||||
assert isinstance(results_reordered[2], list)
|
||||
assert len(results_reordered[2]) > 0
|
||||
assert results_reordered[3] is None # Put operation returns None
|
||||
assert isinstance(results_reordered[4], Item)
|
||||
assert results_reordered[4].value == {"data": "value1"}
|
||||
assert results_reordered[4].key == "key1"
|
||||
|
||||
|
||||
async def test_batch_get_ops(store: AsyncSqliteStore) -> None:
|
||||
"""Test GET operations in batch context."""
|
||||
# Setup test data
|
||||
await store.aput(("test",), "key1", {"data": "value1"})
|
||||
await store.aput(("test",), "key2", {"data": "value2"})
|
||||
|
||||
ops = [
|
||||
GetOp(namespace=("test",), key="key1"),
|
||||
GetOp(namespace=("test",), key="key2"),
|
||||
GetOp(namespace=("test",), key="key3"), # Non-existent key
|
||||
]
|
||||
|
||||
results = await store.abatch(ops)
|
||||
|
||||
assert len(results) == 3
|
||||
assert results[0] is not None
|
||||
assert results[1] is not None
|
||||
assert results[2] is None
|
||||
if results[0] is not None:
|
||||
assert results[0].key == "key1"
|
||||
if results[1] is not None:
|
||||
assert results[1].key == "key2"
|
||||
|
||||
|
||||
async def test_batch_put_ops(store: AsyncSqliteStore) -> None:
|
||||
"""Test PUT operations in batch context."""
|
||||
ops = [
|
||||
PutOp(namespace=("test",), key="key1", value={"data": "value1"}),
|
||||
PutOp(namespace=("test",), key="key2", value={"data": "value2"}),
|
||||
PutOp(namespace=("test",), key="key3", value=None), # Delete operation
|
||||
]
|
||||
|
||||
results = await store.abatch(ops)
|
||||
assert len(results) == 3
|
||||
assert all(result is None for result in results)
|
||||
|
||||
# Verify the puts worked
|
||||
items = await store.asearch(("test",), limit=10)
|
||||
assert len(items) == 2 # key3 had None value so wasn't stored
|
||||
|
||||
|
||||
async def test_batch_search_ops(store: AsyncSqliteStore) -> None:
|
||||
"""Test SEARCH operations in batch context."""
|
||||
# Setup test data
|
||||
await store.aput(("test", "foo"), "key1", {"data": "value1"})
|
||||
await store.aput(("test", "bar"), "key2", {"data": "value2"})
|
||||
|
||||
ops = [
|
||||
SearchOp(
|
||||
namespace_prefix=("test",), filter={"data": "value1"}, limit=10, offset=0
|
||||
),
|
||||
SearchOp(namespace_prefix=("test",), filter=None, limit=5, offset=0),
|
||||
]
|
||||
|
||||
results = await store.abatch(ops)
|
||||
|
||||
assert len(results) == 2
|
||||
# SQLite query implementation might return different results
|
||||
# Just check that we get lists back and don't check the exact content
|
||||
assert isinstance(results[0], list)
|
||||
assert isinstance(results[1], list)
|
||||
assert len(results[1]) >= 1 # We should at least find some results
|
||||
|
||||
|
||||
async def test_batch_list_namespaces_ops(store: AsyncSqliteStore) -> None:
|
||||
"""Test LIST NAMESPACES operations in batch context."""
|
||||
# Setup test data
|
||||
await store.aput(("test", "namespace1"), "key1", {"data": "value1"})
|
||||
await store.aput(("test", "namespace2"), "key2", {"data": "value2"})
|
||||
|
||||
ops = [ListNamespacesOp(match_conditions=None, max_depth=None, limit=10, offset=0)]
|
||||
|
||||
results = await store.abatch(ops)
|
||||
|
||||
assert len(results) == 1
|
||||
if isinstance(results[0], list):
|
||||
assert len(results[0]) == 2
|
||||
assert ("test", "namespace1") in results[0]
|
||||
assert ("test", "namespace2") in results[0]
|
||||
|
||||
|
||||
async def test_vector_store_initialization(
|
||||
fake_embeddings: CharacterEmbeddings,
|
||||
) -> None:
|
||||
"""Test store initialization with embedding config."""
|
||||
async with create_vector_store(fake_embeddings) as store:
|
||||
assert store.index_config is not None
|
||||
assert store.index_config["dims"] == fake_embeddings.dims
|
||||
if hasattr(store.index_config.get("embed"), "embed_documents"):
|
||||
assert store.index_config["embed"] == fake_embeddings
|
||||
|
||||
|
||||
async def test_vector_insert_with_auto_embedding(
|
||||
fake_embeddings: CharacterEmbeddings,
|
||||
conn_string: str,
|
||||
) -> None:
|
||||
"""Test inserting items that get auto-embedded."""
|
||||
async with create_vector_store(fake_embeddings, conn_string=conn_string) as store:
|
||||
docs = [
|
||||
("doc1", {"text": "short text"}),
|
||||
("doc2", {"text": "longer text document"}),
|
||||
("doc3", {"text": "longest text document here"}),
|
||||
("doc4", {"description": "text in description field"}),
|
||||
("doc5", {"content": "text in content field"}),
|
||||
("doc6", {"body": "text in body field"}),
|
||||
]
|
||||
|
||||
for key, value in docs:
|
||||
await store.aput(("test",), key, value)
|
||||
|
||||
results = await store.asearch(("test",), query="long text")
|
||||
assert len(results) > 0
|
||||
|
||||
doc_order = [r.key for r in results]
|
||||
assert "doc2" in doc_order
|
||||
assert "doc3" in doc_order
|
||||
|
||||
|
||||
async def test_vector_update_with_embedding(
|
||||
fake_embeddings: CharacterEmbeddings,
|
||||
conn_string: str,
|
||||
) -> None:
|
||||
"""Test that updating items properly updates their embeddings."""
|
||||
async with create_vector_store(fake_embeddings, conn_string=conn_string) as store:
|
||||
await store.aput(("test",), "doc1", {"text": "zany zebra Xerxes"})
|
||||
await store.aput(("test",), "doc2", {"text": "something about dogs"})
|
||||
await store.aput(("test",), "doc3", {"text": "text about birds"})
|
||||
|
||||
results_initial = await store.asearch(("test",), query="Zany Xerxes")
|
||||
assert len(results_initial) > 0
|
||||
assert results_initial[0].score is not None
|
||||
assert results_initial[0].key == "doc1"
|
||||
initial_score = results_initial[0].score
|
||||
|
||||
await store.aput(("test",), "doc1", {"text": "new text about dogs"})
|
||||
|
||||
results_after = await store.asearch(("test",), query="Zany Xerxes")
|
||||
after_score = next((r.score for r in results_after if r.key == "doc1"), 0.0)
|
||||
assert (
|
||||
after_score is not None
|
||||
and initial_score is not None
|
||||
and after_score < initial_score
|
||||
)
|
||||
|
||||
results_new = await store.asearch(("test",), query="new text about dogs")
|
||||
for r in results_new:
|
||||
if r.key == "doc1":
|
||||
assert (
|
||||
r.score is not None
|
||||
and after_score is not None
|
||||
and r.score > after_score
|
||||
)
|
||||
|
||||
# Don't index this one
|
||||
await store.aput(
|
||||
("test",), "doc4", {"text": "new text about dogs"}, index=False
|
||||
)
|
||||
results_new = await store.asearch(
|
||||
("test",), query="new text about dogs", limit=3
|
||||
)
|
||||
assert not any(r.key == "doc4" for r in results_new)
|
||||
|
||||
|
||||
async def test_vector_search_with_filters(
|
||||
fake_embeddings: CharacterEmbeddings,
|
||||
conn_string: str,
|
||||
) -> None:
|
||||
"""Test combining vector search with filters."""
|
||||
async with create_vector_store(fake_embeddings, conn_string=conn_string) as store:
|
||||
docs = [
|
||||
("doc1", {"text": "red apple", "color": "red", "score": 4.5}),
|
||||
("doc2", {"text": "red car", "color": "red", "score": 3.0}),
|
||||
("doc3", {"text": "green apple", "color": "green", "score": 4.0}),
|
||||
("doc4", {"text": "blue car", "color": "blue", "score": 3.5}),
|
||||
]
|
||||
|
||||
for key, value in docs:
|
||||
await store.aput(("test",), key, value)
|
||||
|
||||
# Vector search with filters can be inconsistent in test environments
|
||||
# Skip asserting exact results as we've already validated the functionality
|
||||
# in the synchronous tests
|
||||
_ = await store.asearch(("test",), query="apple", filter={"color": "red"})
|
||||
|
||||
# Skip asserting exact results as we've already validated the functionality
|
||||
# in the synchronous tests
|
||||
_ = await store.asearch(("test",), query="car", filter={"color": "red"})
|
||||
|
||||
# Skip asserting exact results as we've already validated the functionality
|
||||
# in the synchronous tests
|
||||
_ = await store.asearch(
|
||||
("test",), query="bbbbluuu", filter={"score": {"$gt": 3.2}}
|
||||
)
|
||||
|
||||
# Skip asserting exact results as we've already validated the functionality
|
||||
# in the synchronous tests
|
||||
_ = await store.asearch(
|
||||
("test",), query="apple", filter={"score": {"$gte": 4.0}, "color": "green"}
|
||||
)
|
||||
|
||||
|
||||
async def test_vector_search_pagination(fake_embeddings: CharacterEmbeddings) -> None:
|
||||
"""Test pagination with vector search."""
|
||||
async with create_vector_store(fake_embeddings) as store:
|
||||
for i in range(5):
|
||||
await store.aput(
|
||||
("test",), f"doc{i}", {"text": f"test document number {i}"}
|
||||
)
|
||||
|
||||
results_page1 = await store.asearch(("test",), query="test", limit=2)
|
||||
results_page2 = await store.asearch(("test",), query="test", limit=2, offset=2)
|
||||
|
||||
assert len(results_page1) == 2
|
||||
assert len(results_page2) == 2
|
||||
assert results_page1[0].key != results_page2[0].key
|
||||
|
||||
all_results = await store.asearch(("test",), query="test", limit=10)
|
||||
assert len(all_results) == 5
|
||||
|
||||
|
||||
async def test_vector_search_edge_cases(fake_embeddings: CharacterEmbeddings) -> None:
|
||||
"""Test edge cases in vector search."""
|
||||
async with create_vector_store(fake_embeddings) as store:
|
||||
await store.aput(("test",), "doc1", {"text": "test document"})
|
||||
|
||||
results = await store.asearch(("test",), query="")
|
||||
assert len(results) == 1
|
||||
|
||||
results = await store.asearch(("test",), query=None)
|
||||
assert len(results) == 1
|
||||
|
||||
long_query = "test " * 100
|
||||
results = await store.asearch(("test",), query=long_query)
|
||||
assert len(results) == 1
|
||||
|
||||
special_query = "test!@#$%^&*()"
|
||||
results = await store.asearch(("test",), query=special_query)
|
||||
assert len(results) == 1
|
||||
|
||||
|
||||
async def test_embed_with_path(
|
||||
fake_embeddings: CharacterEmbeddings,
|
||||
) -> None:
|
||||
"""Test vector search with specific text fields in SQLite store."""
|
||||
async with create_vector_store(
|
||||
fake_embeddings, text_fields=["key0", "key1", "key3"]
|
||||
) as store:
|
||||
# This will have 2 vectors representing it
|
||||
doc1 = {
|
||||
# Omit key0 - check it doesn't raise an error
|
||||
"key1": "xxx",
|
||||
"key2": "yyy",
|
||||
"key3": "zzz",
|
||||
}
|
||||
# This will have 3 vectors representing it
|
||||
doc2 = {
|
||||
"key0": "uuu",
|
||||
"key1": "vvv",
|
||||
"key2": "www",
|
||||
"key3": "xxx",
|
||||
}
|
||||
await store.aput(("test",), "doc1", doc1)
|
||||
await store.aput(("test",), "doc2", doc2)
|
||||
|
||||
# doc2.key3 and doc1.key1 both would have the highest score
|
||||
results = await store.asearch(("test",), query="xxx")
|
||||
assert len(results) == 2
|
||||
assert results[0].key != results[1].key
|
||||
assert results[0].score > 0.9
|
||||
assert results[1].score > 0.9
|
||||
|
||||
# ~Only match doc2
|
||||
results = await store.asearch(("test",), query="uuu")
|
||||
assert len(results) == 2
|
||||
assert results[0].key != results[1].key
|
||||
assert results[0].key == "doc2"
|
||||
assert results[0].score > results[1].score
|
||||
|
||||
# Un-indexed - will have low results for both. Not zero (because we're projecting)
|
||||
# but less than the above.
|
||||
results = await store.asearch(("test",), query="www")
|
||||
assert len(results) == 2
|
||||
assert results[0].score < 0.9
|
||||
assert results[1].score < 0.9
|
||||
|
||||
|
||||
async def test_basic_store_ops(
|
||||
fake_embeddings: CharacterEmbeddings,
|
||||
) -> None:
|
||||
"""Test vector search with specific text fields in SQLite store."""
|
||||
async with create_vector_store(
|
||||
fake_embeddings, text_fields=["key0", "key1", "key3"]
|
||||
) as store:
|
||||
uid = uuid.uuid4().hex
|
||||
namespace = (uid, "test", "documents")
|
||||
item_id = "doc1"
|
||||
item_value = {"title": "Test Document", "content": "Hello, World!"}
|
||||
results = await store.asearch((uid,))
|
||||
assert len(results) == 0
|
||||
|
||||
await store.aput(namespace, item_id, item_value)
|
||||
item = await store.aget(namespace, item_id)
|
||||
|
||||
assert item is not None
|
||||
assert item.namespace == namespace
|
||||
assert item.key == item_id
|
||||
assert item.value == item_value
|
||||
assert item.created_at is not None
|
||||
assert item.updated_at is not None
|
||||
|
||||
updated_value = {
|
||||
"title": "Updated Test Document",
|
||||
"content": "Hello, LangGraph!",
|
||||
}
|
||||
await asyncio.sleep(1.01)
|
||||
await store.aput(namespace, item_id, updated_value)
|
||||
updated_item = await store.aget(namespace, item_id)
|
||||
assert updated_item is not None
|
||||
|
||||
assert updated_item.value == updated_value
|
||||
assert updated_item.updated_at > item.updated_at
|
||||
different_namespace = (uid, "test", "other_documents")
|
||||
item_in_different_namespace = await store.aget(different_namespace, item_id)
|
||||
assert item_in_different_namespace is None
|
||||
|
||||
new_item_id = "doc2"
|
||||
new_item_value = {"title": "Another Document", "content": "Greetings!"}
|
||||
await store.aput(namespace, new_item_id, new_item_value)
|
||||
|
||||
items = await store.asearch((uid, "test"), limit=10)
|
||||
assert len(items) == 2
|
||||
assert any(item.key == item_id for item in items)
|
||||
assert any(item.key == new_item_id for item in items)
|
||||
|
||||
namespaces = await store.alist_namespaces(prefix=(uid, "test"))
|
||||
assert (uid, "test", "documents") in namespaces
|
||||
|
||||
await store.adelete(namespace, item_id)
|
||||
await store.adelete(namespace, new_item_id)
|
||||
deleted_item = await store.aget(namespace, item_id)
|
||||
assert deleted_item is None
|
||||
|
||||
deleted_item = await store.aget(namespace, new_item_id)
|
||||
assert deleted_item is None
|
||||
|
||||
empty_search_results = await store.asearch((uid, "test"), limit=10)
|
||||
assert len(empty_search_results) == 0
|
||||
|
||||
|
||||
async def test_list_namespaces(
|
||||
fake_embeddings: CharacterEmbeddings,
|
||||
) -> None:
|
||||
"""Test list namespaces functionality with various filters."""
|
||||
async with create_vector_store(
|
||||
fake_embeddings, text_fields=["key0", "key1", "key3"]
|
||||
) as store:
|
||||
test_pref = str(uuid.uuid4())
|
||||
test_namespaces = [
|
||||
(test_pref, "test", "documents", "public", test_pref),
|
||||
(test_pref, "test", "documents", "private", test_pref),
|
||||
(test_pref, "test", "images", "public", test_pref),
|
||||
(test_pref, "test", "images", "private", test_pref),
|
||||
(test_pref, "prod", "documents", "public", test_pref),
|
||||
(test_pref, "prod", "documents", "some", "nesting", "public", test_pref),
|
||||
(test_pref, "prod", "documents", "private", test_pref),
|
||||
]
|
||||
|
||||
# Add test data
|
||||
for namespace in test_namespaces:
|
||||
await store.aput(namespace, "dummy", {"content": "dummy"})
|
||||
|
||||
# Test prefix filtering
|
||||
prefix_result = await store.alist_namespaces(prefix=(test_pref, "test"))
|
||||
assert len(prefix_result) == 4
|
||||
assert all(ns[1] == "test" for ns in prefix_result)
|
||||
|
||||
# Test specific prefix
|
||||
specific_prefix_result = await store.alist_namespaces(
|
||||
prefix=(test_pref, "test", "documents")
|
||||
)
|
||||
assert len(specific_prefix_result) == 2
|
||||
assert all(ns[1:3] == ("test", "documents") for ns in specific_prefix_result)
|
||||
|
||||
# Test suffix filtering
|
||||
suffix_result = await store.alist_namespaces(suffix=("public", test_pref))
|
||||
assert len(suffix_result) == 4
|
||||
assert all(ns[-2] == "public" for ns in suffix_result)
|
||||
|
||||
# Test combined prefix and suffix
|
||||
prefix_suffix_result = await store.alist_namespaces(
|
||||
prefix=(test_pref, "test"), suffix=("public", test_pref)
|
||||
)
|
||||
assert len(prefix_suffix_result) == 2
|
||||
assert all(
|
||||
ns[1] == "test" and ns[-2] == "public" for ns in prefix_suffix_result
|
||||
)
|
||||
|
||||
# Test wildcard in prefix
|
||||
wildcard_prefix_result = await store.alist_namespaces(
|
||||
prefix=(test_pref, "*", "documents")
|
||||
)
|
||||
assert len(wildcard_prefix_result) == 5
|
||||
assert all(ns[2] == "documents" for ns in wildcard_prefix_result)
|
||||
|
||||
# Test wildcard in suffix
|
||||
wildcard_suffix_result = await store.alist_namespaces(
|
||||
suffix=("*", "public", test_pref)
|
||||
)
|
||||
assert len(wildcard_suffix_result) == 4
|
||||
assert all(ns[-2] == "public" for ns in wildcard_suffix_result)
|
||||
|
||||
wildcard_single = await store.alist_namespaces(
|
||||
suffix=("some", "*", "public", test_pref)
|
||||
)
|
||||
assert len(wildcard_single) == 1
|
||||
assert wildcard_single[0] == (
|
||||
test_pref,
|
||||
"prod",
|
||||
"documents",
|
||||
"some",
|
||||
"nesting",
|
||||
"public",
|
||||
test_pref,
|
||||
)
|
||||
|
||||
# Test max depth
|
||||
max_depth_result = await store.alist_namespaces(max_depth=3)
|
||||
assert all(len(ns) <= 3 for ns in max_depth_result)
|
||||
|
||||
max_depth_result = await store.alist_namespaces(
|
||||
max_depth=4, prefix=(test_pref, "*", "documents")
|
||||
)
|
||||
assert len(set(res for res in max_depth_result)) == len(max_depth_result) == 5
|
||||
|
||||
# Test pagination
|
||||
limit_result = await store.alist_namespaces(prefix=(test_pref,), limit=3)
|
||||
assert len(limit_result) == 3
|
||||
|
||||
offset_result = await store.alist_namespaces(prefix=(test_pref,), offset=3)
|
||||
assert len(offset_result) == len(test_namespaces) - 3
|
||||
|
||||
empty_prefix_result = await store.alist_namespaces(prefix=(test_pref,))
|
||||
assert len(empty_prefix_result) == len(test_namespaces)
|
||||
assert set(empty_prefix_result) == set(test_namespaces)
|
||||
|
||||
# Clean up
|
||||
for namespace in test_namespaces:
|
||||
await store.adelete(namespace, "dummy")
|
||||
|
||||
|
||||
async def test_search_items(
|
||||
fake_embeddings: CharacterEmbeddings,
|
||||
) -> None:
|
||||
"""Test search_items functionality by calling store methods directly."""
|
||||
base = "test_search_items"
|
||||
test_namespaces = [
|
||||
(base, "documents", "user1"),
|
||||
(base, "documents", "user2"),
|
||||
(base, "reports", "department1"),
|
||||
(base, "reports", "department2"),
|
||||
]
|
||||
test_items = [
|
||||
{"title": "Doc 1", "author": "John Doe", "tags": ["important"]},
|
||||
{"title": "Doc 2", "author": "Jane Smith", "tags": ["draft"]},
|
||||
{"title": "Report A", "author": "John Doe", "tags": ["final"]},
|
||||
{"title": "Report B", "author": "Alice Johnson", "tags": ["draft"]},
|
||||
]
|
||||
|
||||
async with create_vector_store(
|
||||
fake_embeddings, text_fields=["key0", "key1", "key3"]
|
||||
) as store:
|
||||
# Insert test data
|
||||
for ns, item in zip(test_namespaces, test_items, strict=False):
|
||||
key = f"item_{ns[-1]}"
|
||||
await store.aput(ns, key, item)
|
||||
|
||||
# 1. Search documents
|
||||
docs = await store.asearch((base, "documents"))
|
||||
assert len(docs) == 2
|
||||
assert all(item.namespace[1] == "documents" for item in docs)
|
||||
|
||||
# 2. Search reports
|
||||
reports = await store.asearch((base, "reports"))
|
||||
assert len(reports) == 2
|
||||
assert all(item.namespace[1] == "reports" for item in reports)
|
||||
|
||||
# 3. Pagination
|
||||
first_page = await store.asearch((base,), limit=2, offset=0)
|
||||
second_page = await store.asearch((base,), limit=2, offset=2)
|
||||
assert len(first_page) == 2
|
||||
assert len(second_page) == 2
|
||||
keys_page1 = {item.key for item in first_page}
|
||||
keys_page2 = {item.key for item in second_page}
|
||||
assert keys_page1.isdisjoint(keys_page2)
|
||||
all_items = await store.asearch((base,))
|
||||
assert len(all_items) == 4
|
||||
|
||||
john_items = await store.asearch((base,), filter={"author": "John Doe"})
|
||||
assert len(john_items) == 2
|
||||
assert all(item.value["author"] == "John Doe" for item in john_items)
|
||||
|
||||
draft_items = await store.asearch((base,), filter={"tags": ["draft"]})
|
||||
assert len(draft_items) == 2
|
||||
assert all("draft" in item.value["tags"] for item in draft_items)
|
||||
|
||||
for ns in test_namespaces:
|
||||
key = f"item_{ns[-1]}"
|
||||
await store.adelete(ns, key)
|
||||
@@ -1,309 +0,0 @@
|
||||
from typing import Any, cast
|
||||
|
||||
import pytest
|
||||
from langchain_core.runnables import RunnableConfig
|
||||
from langgraph.checkpoint.base import (
|
||||
Checkpoint,
|
||||
CheckpointMetadata,
|
||||
create_checkpoint,
|
||||
empty_checkpoint,
|
||||
)
|
||||
|
||||
from langgraph.checkpoint.sqlite import SqliteSaver
|
||||
from langgraph.checkpoint.sqlite.utils import _metadata_predicate, search_where
|
||||
|
||||
|
||||
class TestSqliteSaver:
|
||||
@pytest.fixture(autouse=True)
|
||||
def setup(self) -> None:
|
||||
# objects for test setup
|
||||
self.config_1: RunnableConfig = {
|
||||
"configurable": {
|
||||
"thread_id": "thread-1",
|
||||
# for backwards compatibility testing
|
||||
"checkpoint_id": "1",
|
||||
"checkpoint_ns": "",
|
||||
}
|
||||
}
|
||||
self.config_2: RunnableConfig = {
|
||||
"configurable": {
|
||||
"thread_id": "thread-2",
|
||||
"checkpoint_id": "2",
|
||||
"checkpoint_ns": "",
|
||||
}
|
||||
}
|
||||
self.config_3: RunnableConfig = {
|
||||
"configurable": {
|
||||
"thread_id": "thread-2",
|
||||
"checkpoint_id": "2-inner",
|
||||
"checkpoint_ns": "inner",
|
||||
}
|
||||
}
|
||||
|
||||
self.chkpnt_1: Checkpoint = empty_checkpoint()
|
||||
self.chkpnt_2: Checkpoint = create_checkpoint(self.chkpnt_1, {}, 1)
|
||||
self.chkpnt_3: Checkpoint = empty_checkpoint()
|
||||
|
||||
self.metadata_1: CheckpointMetadata = {
|
||||
"source": "input",
|
||||
"step": 2,
|
||||
"writes": {},
|
||||
"score": 1,
|
||||
}
|
||||
self.metadata_2: CheckpointMetadata = {
|
||||
"source": "loop",
|
||||
"step": 1,
|
||||
"writes": {"foo": "bar"},
|
||||
"score": None,
|
||||
}
|
||||
self.metadata_3: CheckpointMetadata = {}
|
||||
|
||||
def test_combined_metadata(self) -> None:
|
||||
with SqliteSaver.from_conn_string(":memory:") as saver:
|
||||
config: RunnableConfig = {
|
||||
"configurable": {
|
||||
"thread_id": "thread-2",
|
||||
"checkpoint_ns": "",
|
||||
"__super_private_key": "super_private_value",
|
||||
},
|
||||
"metadata": {"run_id": "my_run_id"},
|
||||
}
|
||||
saver.put(config, self.chkpnt_2, self.metadata_2, {})
|
||||
checkpoint = saver.get_tuple(config)
|
||||
assert checkpoint is not None and checkpoint.metadata == {
|
||||
**self.metadata_2,
|
||||
"run_id": "my_run_id",
|
||||
}
|
||||
|
||||
def test_search(self) -> None:
|
||||
with SqliteSaver.from_conn_string(":memory:") as saver:
|
||||
# set up test
|
||||
# save checkpoints
|
||||
saver.put(self.config_1, self.chkpnt_1, self.metadata_1, {})
|
||||
saver.put(self.config_2, self.chkpnt_2, self.metadata_2, {})
|
||||
saver.put(self.config_3, self.chkpnt_3, self.metadata_3, {})
|
||||
|
||||
# call method / assertions
|
||||
query_1 = {"source": "input"} # search by 1 key
|
||||
query_2 = {
|
||||
"step": 1,
|
||||
"writes": {"foo": "bar"},
|
||||
} # search by multiple keys
|
||||
query_3: dict[str, Any] = {} # search by no keys, return all checkpoints
|
||||
query_4 = {"source": "update", "step": 1} # no match
|
||||
|
||||
search_results_1 = list(saver.list(None, filter=query_1))
|
||||
assert len(search_results_1) == 1
|
||||
assert search_results_1[0].metadata == self.metadata_1
|
||||
|
||||
search_results_2 = list(saver.list(None, filter=query_2))
|
||||
assert len(search_results_2) == 1
|
||||
assert search_results_2[0].metadata == self.metadata_2
|
||||
|
||||
search_results_3 = list(saver.list(None, filter=query_3))
|
||||
assert len(search_results_3) == 3
|
||||
|
||||
search_results_4 = list(saver.list(None, filter=query_4))
|
||||
assert len(search_results_4) == 0
|
||||
|
||||
# search by config (defaults to checkpoints across all namespaces)
|
||||
search_results_5 = list(
|
||||
saver.list({"configurable": {"thread_id": "thread-2"}})
|
||||
)
|
||||
assert len(search_results_5) == 2
|
||||
assert {
|
||||
search_results_5[0].config["configurable"]["checkpoint_ns"],
|
||||
search_results_5[1].config["configurable"]["checkpoint_ns"],
|
||||
} == {"", "inner"}
|
||||
|
||||
# search with before param
|
||||
search_results_6 = list(saver.list(None, before=search_results_5[1].config))
|
||||
assert len(search_results_6) == 1
|
||||
assert search_results_6[0].config["configurable"]["thread_id"] == "thread-1"
|
||||
|
||||
# search with limit param
|
||||
search_results_7 = list(
|
||||
saver.list({"configurable": {"thread_id": "thread-2"}}, limit=1)
|
||||
)
|
||||
assert len(search_results_7) == 1
|
||||
assert search_results_7[0].config["configurable"]["thread_id"] == "thread-2"
|
||||
|
||||
def test_search_where(self) -> None:
|
||||
# call method / assertions
|
||||
expected_predicate_1 = "WHERE json_extract(CAST(metadata AS TEXT), '$.source') = ? AND json_extract(CAST(metadata AS TEXT), '$.step') = ? AND json_extract(CAST(metadata AS TEXT), '$.writes') = ? AND json_extract(CAST(metadata AS TEXT), '$.score') = ? AND checkpoint_id < ?"
|
||||
expected_param_values_1 = ["input", 2, "{}", 1, "1"]
|
||||
assert search_where(
|
||||
None, cast(dict[str, Any], self.metadata_1), self.config_1
|
||||
) == (
|
||||
expected_predicate_1,
|
||||
expected_param_values_1,
|
||||
)
|
||||
|
||||
def test_metadata_predicate(self) -> None:
|
||||
# call method / assertions
|
||||
expected_predicate_1 = [
|
||||
"json_extract(CAST(metadata AS TEXT), '$.source') = ?",
|
||||
"json_extract(CAST(metadata AS TEXT), '$.step') = ?",
|
||||
"json_extract(CAST(metadata AS TEXT), '$.writes') = ?",
|
||||
"json_extract(CAST(metadata AS TEXT), '$.score') = ?",
|
||||
]
|
||||
expected_predicate_2 = [
|
||||
"json_extract(CAST(metadata AS TEXT), '$.source') = ?",
|
||||
"json_extract(CAST(metadata AS TEXT), '$.step') = ?",
|
||||
"json_extract(CAST(metadata AS TEXT), '$.writes') = ?",
|
||||
"json_extract(CAST(metadata AS TEXT), '$.score') IS ?",
|
||||
]
|
||||
expected_predicate_3: list[str] = []
|
||||
|
||||
expected_param_values_1 = ["input", 2, "{}", 1]
|
||||
expected_param_values_2 = ["loop", 1, '{"foo":"bar"}', None]
|
||||
expected_param_values_3: list[Any] = []
|
||||
|
||||
assert _metadata_predicate(cast(dict[str, Any], self.metadata_1)) == (
|
||||
expected_predicate_1,
|
||||
expected_param_values_1,
|
||||
)
|
||||
assert _metadata_predicate(cast(dict[str, Any], self.metadata_2)) == (
|
||||
expected_predicate_2,
|
||||
expected_param_values_2,
|
||||
)
|
||||
assert _metadata_predicate(cast(dict[str, Any], self.metadata_3)) == (
|
||||
expected_predicate_3,
|
||||
expected_param_values_3,
|
||||
)
|
||||
|
||||
async def test_informative_async_errors(self) -> None:
|
||||
with SqliteSaver.from_conn_string(":memory:") as saver:
|
||||
# call method / assertions
|
||||
with pytest.raises(NotImplementedError, match="AsyncSqliteSaver"):
|
||||
await saver.aget(self.config_1)
|
||||
with pytest.raises(NotImplementedError, match="AsyncSqliteSaver"):
|
||||
await saver.aget_tuple(self.config_1)
|
||||
with pytest.raises(NotImplementedError, match="AsyncSqliteSaver"):
|
||||
async for _ in saver.alist(self.config_1):
|
||||
pass
|
||||
|
||||
def test_metadata_predicate_sql_injection_prevention(self) -> None:
|
||||
"""Test that _metadata_predicate rejects malicious filter keys."""
|
||||
# Test various SQL injection payloads
|
||||
malicious_keys = [
|
||||
"x') OR '1'='1", # Boolean-based injection
|
||||
"x') OR 1=1 --", # Comment-based injection
|
||||
"x') UNION SELECT 1,2,3,4,5,6,7 --", # UNION-based injection
|
||||
"access') = 'public' OR '1'='1' OR json_extract(value, '$.", # Complex injection
|
||||
"'; DROP TABLE checkpoints; --", # Destructive injection
|
||||
]
|
||||
|
||||
for malicious_key in malicious_keys:
|
||||
with pytest.raises(ValueError, match="Invalid filter key"):
|
||||
_metadata_predicate({malicious_key: "dummy"})
|
||||
|
||||
def test_checkpoint_search_sql_injection_prevention(self) -> None:
|
||||
"""Test that SQL injection via malicious filter keys is prevented in checkpoint search."""
|
||||
with SqliteSaver.from_conn_string(":memory:") as saver:
|
||||
# Setup: Create checkpoints with different metadata
|
||||
config_public: RunnableConfig = {
|
||||
"configurable": {
|
||||
"thread_id": "thread-public",
|
||||
"checkpoint_ns": "",
|
||||
}
|
||||
}
|
||||
config_private: RunnableConfig = {
|
||||
"configurable": {
|
||||
"thread_id": "thread-private",
|
||||
"checkpoint_ns": "",
|
||||
}
|
||||
}
|
||||
|
||||
checkpoint_public = empty_checkpoint()
|
||||
checkpoint_private = empty_checkpoint()
|
||||
|
||||
metadata_public: CheckpointMetadata = {
|
||||
"access": "public",
|
||||
"data": "public information",
|
||||
}
|
||||
metadata_private: CheckpointMetadata = {
|
||||
"access": "private",
|
||||
"data": "secret information",
|
||||
"password": "secret123",
|
||||
}
|
||||
|
||||
saver.put(config_public, checkpoint_public, metadata_public, {})
|
||||
saver.put(config_private, checkpoint_private, metadata_private, {})
|
||||
|
||||
# Normal query - should return only public checkpoint
|
||||
normal_results = list(saver.list(None, filter={"access": "public"}))
|
||||
assert len(normal_results) == 1
|
||||
assert normal_results[0].metadata["access"] == "public"
|
||||
|
||||
# SQL injection attempt should raise ValueError
|
||||
malicious_key = (
|
||||
"access') = 'public' OR '1'='1' OR json_extract(metadata, '$."
|
||||
)
|
||||
|
||||
with pytest.raises(ValueError, match="Invalid filter key"):
|
||||
list(saver.list(None, filter={malicious_key: "dummy"}))
|
||||
|
||||
def test_limit_parameter_sql_injection_prevention(self) -> None:
|
||||
"""Test that the limit parameter properly uses parameterized queries to prevent SQL injection."""
|
||||
with SqliteSaver.from_conn_string(":memory:") as saver:
|
||||
# Setup: Create multiple checkpoints
|
||||
for i in range(5):
|
||||
config: RunnableConfig = {
|
||||
"configurable": {
|
||||
"thread_id": f"thread-{i}",
|
||||
"checkpoint_ns": "",
|
||||
}
|
||||
}
|
||||
checkpoint = empty_checkpoint()
|
||||
metadata: CheckpointMetadata = {"index": i}
|
||||
saver.put(config, checkpoint, metadata, {})
|
||||
|
||||
# Test that limit works correctly with valid integer
|
||||
results = list(saver.list(None, limit=2))
|
||||
assert len(results) == 2
|
||||
|
||||
# Test that limit=0 returns no results
|
||||
results = list(saver.list(None, limit=0))
|
||||
assert len(results) == 0
|
||||
|
||||
# Test that limit=None returns all results
|
||||
results = list(saver.list(None, limit=None))
|
||||
assert len(results) == 5
|
||||
|
||||
def test_metadata_filter_keys_with_hyphens_and_digits(self) -> None:
|
||||
"""Metadata keys with hyphens and digit-start should be filterable.
|
||||
|
||||
This exposes incorrect JSON path handling (unquoted segments) by asserting
|
||||
that such filters successfully match saved checkpoints.
|
||||
"""
|
||||
with SqliteSaver.from_conn_string(":memory:") as saver:
|
||||
config: RunnableConfig = {
|
||||
"configurable": {
|
||||
"thread_id": "thread-hyphen-digit",
|
||||
"checkpoint_ns": "",
|
||||
}
|
||||
}
|
||||
checkpoint = empty_checkpoint()
|
||||
metadata: CheckpointMetadata = {
|
||||
"access-level": "public",
|
||||
"user": {"access-level": "nested", "123abc": "ok2"},
|
||||
"123abc": "ok",
|
||||
}
|
||||
saver.put(config, checkpoint, metadata, {})
|
||||
|
||||
# Top-level hyphenated key
|
||||
results = list(saver.list(None, filter={"access-level": "public"}))
|
||||
assert len(results) == 1
|
||||
|
||||
# Nested hyphenated key via dotted path
|
||||
results = list(saver.list(None, filter={"user.access-level": "nested"}))
|
||||
assert len(results) == 1
|
||||
|
||||
# Top-level digit-starting key
|
||||
results = list(saver.list(None, filter={"123abc": "ok"}))
|
||||
assert len(results) == 1
|
||||
|
||||
# Nested digit-starting key via dotted path
|
||||
results = list(saver.list(None, filter={"user.123abc": "ok2"}))
|
||||
assert len(results) == 1
|
||||
File diff suppressed because it is too large.
Load diff
@@ -1,429 +0,0 @@
|
||||
"""Test SQLite store Time-To-Live (TTL) functionality."""
|
||||
|
||||
import asyncio
|
||||
import os
|
||||
import tempfile
|
||||
import time
|
||||
from collections.abc import Generator
|
||||
|
||||
import pytest
|
||||
from langgraph.store.base import TTLConfig
|
||||
|
||||
from langgraph.store.sqlite import SqliteStore
|
||||
from langgraph.store.sqlite.aio import AsyncSqliteStore
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def temp_db_file() -> Generator[str, None, None]:
|
||||
"""Create a temporary database file for testing."""
|
||||
fd, path = tempfile.mkstemp()
|
||||
os.close(fd)
|
||||
yield path
|
||||
os.unlink(path)
|
||||
|
||||
|
||||
def test_ttl_basic(temp_db_file: str) -> None:
|
||||
"""Test basic TTL functionality with synchronous API."""
|
||||
ttl_seconds = 1
|
||||
ttl_minutes = ttl_seconds / 60
|
||||
|
||||
with SqliteStore.from_conn_string(
|
||||
temp_db_file, ttl={"default_ttl": ttl_minutes}
|
||||
) as store:
|
||||
store.setup()
|
||||
|
||||
store.put(("test",), "item1", {"value": "test"})
|
||||
|
||||
item = store.get(("test",), "item1")
|
||||
assert item is not None
|
||||
assert item.value["value"] == "test"
|
||||
|
||||
time.sleep(ttl_seconds + 1.0)
|
||||
|
||||
store.sweep_ttl()
|
||||
|
||||
item = store.get(("test",), "item1")
|
||||
assert item is None
|
||||
|
||||
|
||||
@pytest.mark.flaky(retries=3)
|
||||
def test_ttl_refresh(temp_db_file: str) -> None:
|
||||
"""Test TTL refresh on read."""
|
||||
ttl_seconds = 1
|
||||
ttl_minutes = ttl_seconds / 60
|
||||
|
||||
with SqliteStore.from_conn_string(
|
||||
temp_db_file, ttl={"default_ttl": ttl_minutes, "refresh_on_read": True}
|
||||
) as store:
|
||||
store.setup()
|
||||
|
||||
# Store an item with TTL
|
||||
store.put(("test",), "item1", {"value": "test"})
|
||||
|
||||
# Sleep almost to expiration
|
||||
time.sleep(ttl_seconds - 0.5)
|
||||
swept = store.sweep_ttl()
|
||||
assert swept == 0
|
||||
|
||||
# Get the item and refresh TTL
|
||||
item = store.get(("test",), "item1", refresh_ttl=True)
|
||||
assert item is not None
|
||||
|
||||
time.sleep(ttl_seconds - 0.5)
|
||||
swept = store.sweep_ttl()
|
||||
assert swept == 0
|
||||
|
||||
# Get the item, should still be there
|
||||
item = store.get(("test",), "item1")
|
||||
assert item is not None
|
||||
assert item.value["value"] == "test"
|
||||
|
||||
# Sleep again but don't refresh this time
|
||||
time.sleep(ttl_seconds + 0.75)
|
||||
|
||||
swept = store.sweep_ttl()
|
||||
assert swept == 1
|
||||
|
||||
# Item should be gone now
|
||||
item = store.get(("test",), "item1")
|
||||
assert item is None
|
||||
|
||||
|
||||
def test_ttl_sweeper(temp_db_file: str) -> None:
|
||||
"""Test TTL sweeper thread."""
|
||||
ttl_seconds = 2
|
||||
ttl_minutes = ttl_seconds / 60
|
||||
|
||||
ttl_config: TTLConfig = {
|
||||
"default_ttl": ttl_minutes,
|
||||
"sweep_interval_minutes": ttl_minutes / 2,
|
||||
}
|
||||
with SqliteStore.from_conn_string(
|
||||
temp_db_file,
|
||||
ttl=ttl_config,
|
||||
) as store:
|
||||
store.setup()
|
||||
|
||||
# Start the TTL sweeper
|
||||
store.start_ttl_sweeper()
|
||||
|
||||
# Store an item with TTL
|
||||
store.put(("test",), "item1", {"value": "test"})
|
||||
|
||||
# Item should be there initially
|
||||
item = store.get(("test",), "item1")
|
||||
assert item is not None
|
||||
|
||||
# Wait for TTL to expire and the sweeper to run
|
||||
time.sleep(ttl_seconds + (ttl_seconds / 2) + 0.5)
|
||||
|
||||
# Item should be gone now (swept automatically)
|
||||
item = store.get(("test",), "item1")
|
||||
assert item is None
|
||||
|
||||
# Stop the sweeper
|
||||
store.stop_ttl_sweeper()
|
||||
|
||||
|
||||
@pytest.mark.flaky(retries=3)
|
||||
def test_ttl_custom_value(temp_db_file: str) -> None:
|
||||
"""Test TTL with custom value per item."""
|
||||
with SqliteStore.from_conn_string(temp_db_file) as store:
|
||||
store.setup()
|
||||
|
||||
# Store items with different TTLs
|
||||
store.put(("test",), "item1", {"value": "short"}, ttl=1 / 60) # 1 second
|
||||
store.put(("test",), "item2", {"value": "long"}, ttl=3 / 60) # 3 seconds
|
||||
|
||||
# Item with short TTL
|
||||
time.sleep(2) # Wait for short TTL
|
||||
store.sweep_ttl()
|
||||
|
||||
# Short TTL item should be gone, long TTL item should remain
|
||||
item1 = store.get(("test",), "item1")
|
||||
item2 = store.get(("test",), "item2")
|
||||
assert item1 is None
|
||||
assert item2 is not None
|
||||
|
||||
# Wait for the second item's TTL
|
||||
time.sleep(4)
|
||||
store.sweep_ttl()
|
||||
|
||||
# Now both should be gone
|
||||
item2 = store.get(("test",), "item2")
|
||||
assert item2 is None
|
||||
|
||||
|
||||
@pytest.mark.flaky(retries=3)
|
||||
def test_ttl_override_default(temp_db_file: str) -> None:
|
||||
"""Test overriding default TTL at the item level."""
|
||||
with SqliteStore.from_conn_string(
|
||||
temp_db_file,
|
||||
ttl={"default_ttl": 5 / 60}, # 5 seconds default
|
||||
) as store:
|
||||
store.setup()
|
||||
|
||||
# Store an item with shorter than default TTL
|
||||
store.put(("test",), "item1", {"value": "override"}, ttl=1 / 60) # 1 second
|
||||
|
||||
# Store an item with default TTL
|
||||
store.put(("test",), "item2", {"value": "default"}) # Uses default 5 seconds
|
||||
|
||||
# Store an item with no TTL
|
||||
store.put(("test",), "item3", {"value": "permanent"}, ttl=None)
|
||||
|
||||
# Wait for the override TTL to expire
|
||||
time.sleep(2)
|
||||
store.sweep_ttl()
|
||||
|
||||
# Check results
|
||||
item1 = store.get(("test",), "item1")
|
||||
item2 = store.get(("test",), "item2")
|
||||
item3 = store.get(("test",), "item3")
|
||||
|
||||
assert item1 is None # Should be expired
|
||||
assert item2 is not None # Default TTL, should still be there
|
||||
assert item3 is not None # No TTL, should still be there
|
||||
|
||||
# Wait for default TTL to expire
|
||||
time.sleep(4)
|
||||
store.sweep_ttl()
|
||||
|
||||
# Check results again
|
||||
item2 = store.get(("test",), "item2")
|
||||
item3 = store.get(("test",), "item3")
|
||||
|
||||
assert item2 is None # Default TTL item should be gone
|
||||
assert item3 is not None # No TTL item should still be there
|
||||
|
||||
|
||||
@pytest.mark.flaky(retries=3)
|
||||
def test_search_with_ttl(temp_db_file: str) -> None:
|
||||
"""Test TTL with search operations."""
|
||||
ttl_seconds = 1
|
||||
ttl_minutes = ttl_seconds / 60
|
||||
|
||||
with SqliteStore.from_conn_string(
|
||||
temp_db_file, ttl={"default_ttl": ttl_minutes}
|
||||
) as store:
|
||||
store.setup()
|
||||
|
||||
# Store items
|
||||
store.put(("test",), "item1", {"value": "apple"})
|
||||
store.put(("test",), "item2", {"value": "banana"})
|
||||
|
||||
# Search before expiration
|
||||
results = store.search(("test",), filter={"value": "apple"})
|
||||
assert len(results) == 1
|
||||
assert results[0].key == "item1"
|
||||
|
||||
# Wait for TTL to expire
|
||||
time.sleep(ttl_seconds + 1)
|
||||
store.sweep_ttl()
|
||||
|
||||
# Search after expiration
|
||||
results = store.search(("test",), filter={"value": "apple"})
|
||||
assert len(results) == 0
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_async_ttl_basic(temp_db_file: str) -> None:
|
||||
"""Test basic TTL functionality with asynchronous API."""
|
||||
ttl_seconds = 1
|
||||
ttl_minutes = ttl_seconds / 60
|
||||
|
||||
async with AsyncSqliteStore.from_conn_string(
|
||||
temp_db_file, ttl={"default_ttl": ttl_minutes}
|
||||
) as store:
|
||||
await store.setup()
|
||||
|
||||
# Store an item with TTL
|
||||
await store.aput(("test",), "item1", {"value": "test"})
|
||||
|
||||
# Get the item before expiration
|
||||
item = await store.aget(("test",), "item1")
|
||||
assert item is not None
|
||||
assert item.value["value"] == "test"
|
||||
|
||||
# Wait for TTL to expire
|
||||
await asyncio.sleep(ttl_seconds + 1.0)
|
||||
|
||||
# Manual sweep needed without the sweeper thread
|
||||
await store.sweep_ttl()
|
||||
|
||||
# Item should be gone now
|
||||
item = await store.aget(("test",), "item1")
|
||||
assert item is None
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
@pytest.mark.flaky(retries=3)
|
||||
async def test_async_ttl_refresh(temp_db_file: str) -> None:
|
||||
"""Test TTL refresh on read with async API."""
|
||||
ttl_seconds = 1
|
||||
ttl_minutes = ttl_seconds / 60
|
||||
|
||||
async with AsyncSqliteStore.from_conn_string(
|
||||
temp_db_file, ttl={"default_ttl": ttl_minutes, "refresh_on_read": True}
|
||||
) as store:
|
||||
await store.setup()
|
||||
|
||||
# Store an item with TTL
|
||||
await store.aput(("test",), "item1", {"value": "test"})
|
||||
|
||||
# Sleep almost to expiration
|
||||
await asyncio.sleep(ttl_seconds - 0.5)
|
||||
|
||||
# Get the item and refresh TTL
|
||||
item = await store.aget(("test",), "item1", refresh_ttl=True)
|
||||
assert item is not None
|
||||
|
||||
# Sleep again - without refresh, would have expired by now
|
||||
await asyncio.sleep(ttl_seconds - 0.5)
|
||||
|
||||
# Get the item, should still be there
|
||||
item = await store.aget(("test",), "item1")
|
||||
assert item is not None
|
||||
assert item.value["value"] == "test"
|
||||
|
||||
# Sleep again but don't refresh this time
|
||||
await asyncio.sleep(ttl_seconds + 1.0)
|
||||
|
||||
# Manual sweep
|
||||
await store.sweep_ttl()
|
||||
|
||||
# Item should be gone now
|
||||
item = await store.aget(("test",), "item1")
|
||||
assert item is None
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_async_ttl_sweeper(temp_db_file: str) -> None:
|
||||
"""Test TTL sweeper thread with async API."""
|
||||
ttl_seconds = 2
|
||||
ttl_minutes = ttl_seconds / 60
|
||||
|
||||
ttl_config: TTLConfig = {
|
||||
"default_ttl": ttl_minutes,
|
||||
"sweep_interval_minutes": ttl_minutes / 2,
|
||||
}
|
||||
|
||||
async with AsyncSqliteStore.from_conn_string(
|
||||
temp_db_file,
|
||||
ttl=ttl_config,
|
||||
) as store:
|
||||
await store.setup()
|
||||
|
||||
# Start the TTL sweeper
|
||||
await store.start_ttl_sweeper()
|
||||
|
||||
# Store an item with TTL
|
||||
await store.aput(("test",), "item1", {"value": "test"})
|
||||
|
||||
# Item should be there initially
|
||||
item = await store.aget(("test",), "item1")
|
||||
assert item is not None
|
||||
|
||||
# Wait for TTL to expire and the sweeper to run
|
||||
await asyncio.sleep(ttl_seconds + (ttl_seconds / 2) + 0.5)
|
||||
|
||||
# Item should be gone now (swept automatically)
|
||||
item = await store.aget(("test",), "item1")
|
||||
assert item is None
|
||||
|
||||
# Stop the sweeper
|
||||
await store.stop_ttl_sweeper()
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
@pytest.mark.flaky(retries=3)
|
||||
async def test_async_search_with_ttl(temp_db_file: str) -> None:
|
||||
"""Test TTL with search operations using async API."""
|
||||
ttl_seconds = 1
|
||||
ttl_minutes = ttl_seconds / 60
|
||||
|
||||
async with AsyncSqliteStore.from_conn_string(
|
||||
temp_db_file, ttl={"default_ttl": ttl_minutes}
|
||||
) as store:
|
||||
await store.setup()
|
||||
|
||||
# Store items
|
||||
await store.aput(("test",), "item1", {"value": "apple"})
|
||||
await store.aput(("test",), "item2", {"value": "banana"})
|
||||
|
||||
# Search before expiration
|
||||
results = await store.asearch(("test",), filter={"value": "apple"})
|
||||
assert len(results) == 1
|
||||
assert results[0].key == "item1"
|
||||
|
||||
# Wait for TTL to expire
|
||||
await asyncio.sleep(ttl_seconds + 1)
|
||||
await store.sweep_ttl()
|
||||
|
||||
# Search after expiration
|
||||
results = await store.asearch(("test",), filter={"value": "apple"})
|
||||
assert len(results) == 0
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
@pytest.mark.flaky(retries=3)
|
||||
async def test_async_asearch_refresh_ttl(temp_db_file: str) -> None:
|
||||
"""Test TTL refresh on asearch with async API."""
|
||||
ttl_seconds = 4.0 # Increased TTL for less sensitivity to timing
|
||||
ttl_minutes = ttl_seconds / 60.0
|
||||
|
||||
async with AsyncSqliteStore.from_conn_string(
|
||||
temp_db_file, ttl={"default_ttl": ttl_minutes, "refresh_on_read": True}
|
||||
) as store:
|
||||
await store.setup()
|
||||
|
||||
namespace = ("docs", "user1")
|
||||
# t=0: items put, expire at t=4.0s
|
||||
await store.aput(namespace, "item1", {"text": "content1", "id": 1})
|
||||
await store.aput(namespace, "item2", {"text": "content2", "id": 2})
|
||||
|
||||
# t=3.0s: (after sleep ttl_seconds * 0.75 = 3s)
|
||||
await asyncio.sleep(ttl_seconds * 0.75)
|
||||
|
||||
# Perform asearch with refresh_ttl=True for item1.
|
||||
# item1's TTL should be refreshed. New expiry: t=3.0s + 4.0s = t=7.0s.
|
||||
# item2's TTL is not affected. Expires at t=4.0s.
|
||||
searched_items = await store.asearch(
|
||||
namespace, filter={"id": 1}, refresh_ttl=True
|
||||
)
|
||||
assert len(searched_items) == 1
|
||||
assert searched_items[0].key == "item1"
|
||||
|
||||
# t=5.0s: (after sleep ttl_seconds * 0.5 = 2s more. Total elapsed: 3s + 2s = 5s)
|
||||
await asyncio.sleep(ttl_seconds * 0.5)
|
||||
# At this point:
|
||||
# - item1 (refreshed by asearch) should expire at t=7.0s. Should be ALIVE.
|
||||
# - item2 (original TTL) should have expired at t=4.0s. Should be GONE after sweep.
|
||||
|
||||
await store.sweep_ttl()
|
||||
|
||||
# Check item1 (should exist due to asearch refresh)
|
||||
item1_check1 = await store.aget(namespace, "item1", refresh_ttl=False)
|
||||
assert item1_check1 is not None, (
|
||||
"Item1 should exist after asearch refresh and first sweep"
|
||||
)
|
||||
assert item1_check1.value["text"] == "content1"
|
||||
|
||||
# Check item2 (should be gone)
|
||||
item2_check1 = await store.aget(namespace, "item2", refresh_ttl=False)
|
||||
assert item2_check1 is None, (
|
||||
"Item2 should be gone after its original TTL expired"
|
||||
)
|
||||
|
||||
# t=7.5s: (after sleep ttl_seconds * 0.625 = 2.5s more. Total elapsed: 5s + 2.5s = 7.5s)
|
||||
await asyncio.sleep(ttl_seconds * 0.625)
|
||||
# At this point:
|
||||
# - item1 (refreshed by asearch, expired at t=7.0s) should be GONE after sweep.
|
||||
|
||||
await store.sweep_ttl()
|
||||
|
||||
# Check item1 again (should be gone now)
|
||||
item1_final_check = await store.aget(namespace, "item1", refresh_ttl=False)
|
||||
assert item1_final_check is None, (
|
||||
"Item1 should be gone after its refreshed TTL expired"
|
||||
)
|
||||
Generated
-1137
File diff suppressed because it is too large.
Load diff
@@ -7,10 +7,10 @@
|
||||
TEST ?= .
|
||||
|
||||
test:
|
||||
uv run pytest $(TEST)
|
||||
poetry run pytest $(TEST)
|
||||
|
||||
test_watch:
|
||||
uv run ptw $(TEST)
|
||||
poetry run ptw $(TEST)
|
||||
|
||||
######################
|
||||
# LINTING AND FORMATTING
|
||||
@@ -26,12 +26,12 @@ lint_tests: PYTHON_FILES=tests
|
||||
lint_tests: MYPY_CACHE=.mypy_cache_test
|
||||
|
||||
lint lint_diff lint_package lint_tests:
|
||||
uv run ruff check .
|
||||
[ "$(PYTHON_FILES)" = "" ] || uv run ruff format $(PYTHON_FILES) --diff
|
||||
[ "$(PYTHON_FILES)" = "" ] || uv run ruff check --select I $(PYTHON_FILES)
|
||||
poetry run ruff check .
|
||||
[ "$(PYTHON_FILES)" = "" ] || poetry run ruff format $(PYTHON_FILES) --diff
|
||||
[ "$(PYTHON_FILES)" = "" ] || poetry run ruff check --select I $(PYTHON_FILES)
|
||||
[ "$(PYTHON_FILES)" = "" ] || mkdir -p $(MYPY_CACHE)
|
||||
[ "$(PYTHON_FILES)" = "" ] || uv run mypy $(PYTHON_FILES) --cache-dir $(MYPY_CACHE)
|
||||
[ "$(PYTHON_FILES)" = "" ] || poetry run mypy $(PYTHON_FILES) --cache-dir $(MYPY_CACHE)
|
||||
|
||||
format format_diff:
|
||||
uv run ruff format $(PYTHON_FILES)
|
||||
uv run ruff check --select I --fix $(PYTHON_FILES)
|
||||
poetry run ruff format $(PYTHON_FILES)
|
||||
poetry run ruff check --select I --fix $(PYTHON_FILES)
|
||||
@@ -36,24 +36,22 @@ Each checkpointer should conform to `langgraph.checkpoint.base.BaseCheckpointSav
|
||||
|
||||
- `.put` - Store a checkpoint with its configuration and metadata.
|
||||
- `.put_writes` - Store intermediate writes linked to a checkpoint (i.e. pending writes).
|
||||
- `.get_tuple` - Fetch a checkpoint tuple using for a given configuration (`thread_id` and `checkpoint_id`).
|
||||
- `.get_tuple` - Fetch a checkpoint tuple using for a given configuration (`thread_id` and `thread_ts`).
|
||||
- `.list` - List checkpoints that match a given configuration and filter criteria.
|
||||
- `.delete_thread()` - Delete all checkpoints and writes associated with a thread.
|
||||
- `.get_next_version()` - Generate the next version ID for a channel.
|
||||
|
||||
If the checkpointer will be used with asynchronous graph execution (i.e. executing the graph via `.ainvoke`, `.astream`, `.abatch`), checkpointer must implement asynchronous versions of the above methods (`.aput`, `.aput_writes`, `.aget_tuple`, `.alist`). Similarly, the checkpointer must implement `.adelete_thread()` if asynchronous thread cleanup is desired. The base class provides a default implementation of `.get_next_version()` that generates an integer sequence starting from 1, but this method should be overridden for custom versioning schemes.
|
||||
If the checkpointer will be used with asynchronous graph execution (i.e. executing the graph via `.ainvoke`, `.astream`, `.abatch`), checkpointer must implement asynchronous versions of the above methods (`.aput`, `.aput_writes`, `.aget_tuple`, `.alist`).
|
||||
|
||||
## Usage
|
||||
|
||||
```python
|
||||
from langgraph.checkpoint.memory import InMemorySaver
|
||||
from langgraph.checkpoint.memory import MemorySaver
|
||||
|
||||
write_config = {"configurable": {"thread_id": "1", "checkpoint_ns": ""}}
|
||||
read_config = {"configurable": {"thread_id": "1"}}
|
||||
|
||||
checkpointer = InMemorySaver()
|
||||
checkpointer = MemorySaver()
|
||||
checkpoint = {
|
||||
"v": 4,
|
||||
"v": 1,
|
||||
"ts": "2024-07-31T20:14:19.804150+00:00",
|
||||
"id": "1ef4f797-8335-6428-8001-8a1503f9b875",
|
||||
"channel_values": {
|
||||
@@ -75,6 +73,7 @@ checkpoint = {
|
||||
"start:node": 2
|
||||
}
|
||||
},
|
||||
"pending_sends": [],
|
||||
}
|
||||
|
||||
# store checkpoint
|
||||
|
||||
@@ -1,48 +0,0 @@
|
||||
from __future__ import annotations
|
||||
|
||||
from abc import ABC, abstractmethod
|
||||
from collections.abc import Mapping, Sequence
|
||||
from typing import Generic, TypeVar
|
||||
|
||||
from langgraph.checkpoint.serde.base import SerializerProtocol
|
||||
from langgraph.checkpoint.serde.jsonplus import JsonPlusSerializer
|
||||
|
||||
ValueT = TypeVar("ValueT")
|
||||
Namespace = tuple[str, ...]
|
||||
FullKey = tuple[Namespace, str]
|
||||
|
||||
|
||||
class BaseCache(ABC, Generic[ValueT]):
|
||||
"""Base class for a cache."""
|
||||
|
||||
serde: SerializerProtocol = JsonPlusSerializer(pickle_fallback=False)
|
||||
|
||||
def __init__(self, *, serde: SerializerProtocol | None = None) -> None:
|
||||
"""Initialize the cache with a serializer."""
|
||||
self.serde = serde or self.serde
|
||||
|
||||
@abstractmethod
|
||||
def get(self, keys: Sequence[FullKey]) -> dict[FullKey, ValueT]:
|
||||
"""Get the cached values for the given keys."""
|
||||
|
||||
@abstractmethod
|
||||
async def aget(self, keys: Sequence[FullKey]) -> dict[FullKey, ValueT]:
|
||||
"""Asynchronously get the cached values for the given keys."""
|
||||
|
||||
@abstractmethod
|
||||
def set(self, pairs: Mapping[FullKey, tuple[ValueT, int | None]]) -> None:
|
||||
"""Set the cached values for the given keys and TTLs."""
|
||||
|
||||
@abstractmethod
|
||||
async def aset(self, pairs: Mapping[FullKey, tuple[ValueT, int | None]]) -> None:
|
||||
"""Asynchronously set the cached values for the given keys and TTLs."""
|
||||
|
||||
@abstractmethod
|
||||
def clear(self, namespaces: Sequence[Namespace] | None = None) -> None:
|
||||
"""Delete the cached values for the given namespaces.
|
||||
If no namespaces are provided, clear all cached values."""
|
||||
|
||||
@abstractmethod
|
||||
async def aclear(self, namespaces: Sequence[Namespace] | None = None) -> None:
|
||||
"""Asynchronously delete the cached values for the given namespaces.
|
||||
If no namespaces are provided, clear all cached values."""
|
||||
@@ -1,73 +0,0 @@
|
||||
from __future__ import annotations
|
||||
|
||||
import datetime
|
||||
import threading
|
||||
from collections.abc import Mapping, Sequence
|
||||
|
||||
from langgraph.cache.base import BaseCache, FullKey, Namespace, ValueT
|
||||
from langgraph.checkpoint.serde.base import SerializerProtocol
|
||||
|
||||
|
||||
class InMemoryCache(BaseCache[ValueT]):
|
||||
def __init__(self, *, serde: SerializerProtocol | None = None):
|
||||
super().__init__(serde=serde)
|
||||
self._cache: dict[Namespace, dict[str, tuple[str, bytes, float | None]]] = {}
|
||||
self._lock = threading.RLock()
|
||||
|
||||
def get(self, keys: Sequence[FullKey]) -> dict[FullKey, ValueT]:
|
||||
"""Get the cached values for the given keys."""
|
||||
with self._lock:
|
||||
if not keys:
|
||||
return {}
|
||||
now = datetime.datetime.now(datetime.timezone.utc).timestamp()
|
||||
values: dict[FullKey, ValueT] = {}
|
||||
for ns_tuple, key in keys:
|
||||
ns = Namespace(ns_tuple)
|
||||
if ns in self._cache and key in self._cache[ns]:
|
||||
enc, val, expiry = self._cache[ns][key]
|
||||
if expiry is None or now < expiry:
|
||||
values[(ns, key)] = self.serde.loads_typed((enc, val))
|
||||
else:
|
||||
del self._cache[ns][key]
|
||||
return values
|
||||
|
||||
async def aget(self, keys: Sequence[FullKey]) -> dict[FullKey, ValueT]:
|
||||
"""Asynchronously get the cached values for the given keys."""
|
||||
return self.get(keys)
|
||||
|
||||
def set(self, keys: Mapping[FullKey, tuple[ValueT, int | None]]) -> None:
|
||||
"""Set the cached values for the given keys."""
|
||||
with self._lock:
|
||||
now = datetime.datetime.now(datetime.timezone.utc)
|
||||
for (ns, key), (value, ttl) in keys.items():
|
||||
if ttl is not None:
|
||||
delta = datetime.timedelta(seconds=ttl)
|
||||
expiry: float | None = (now + delta).timestamp()
|
||||
else:
|
||||
expiry = None
|
||||
if ns not in self._cache:
|
||||
self._cache[ns] = {}
|
||||
self._cache[ns][key] = (
|
||||
*self.serde.dumps_typed(value),
|
||||
expiry,
|
||||
)
|
||||
|
||||
async def aset(self, keys: Mapping[FullKey, tuple[ValueT, int | None]]) -> None:
|
||||
"""Asynchronously set the cached values for the given keys."""
|
||||
self.set(keys)
|
||||
|
||||
def clear(self, namespaces: Sequence[Namespace] | None = None) -> None:
|
||||
"""Delete the cached values for the given namespaces.
|
||||
If no namespaces are provided, clear all cached values."""
|
||||
with self._lock:
|
||||
if namespaces is None:
|
||||
self._cache.clear()
|
||||
else:
|
||||
for ns in namespaces:
|
||||
if ns in self._cache:
|
||||
del self._cache[ns]
|
||||
|
||||
async def aclear(self, namespaces: Sequence[Namespace] | None = None) -> None:
|
||||
"""Asynchronously delete the cached values for the given namespaces.
|
||||
If no namespaces are provided, clear all cached values."""
|
||||
self.clear(namespaces)
|
||||
-144
@@ -1,144 +0,0 @@
|
||||
from __future__ import annotations
|
||||
|
||||
from collections.abc import Mapping, Sequence
|
||||
from typing import Any
|
||||
|
||||
from langgraph.cache.base import BaseCache, FullKey, Namespace, ValueT
|
||||
from langgraph.checkpoint.serde.base import SerializerProtocol
|
||||
|
||||
|
||||
class RedisCache(BaseCache[ValueT]):
|
||||
"""Redis-based cache implementation with TTL support."""
|
||||
|
||||
def __init__(
|
||||
self,
|
||||
redis: Any,
|
||||
*,
|
||||
serde: SerializerProtocol | None = None,
|
||||
prefix: str = "langgraph:cache:",
|
||||
) -> None:
|
||||
"""Initialize the cache with a Redis client.
|
||||
|
||||
Args:
|
||||
redis: Redis client instance (sync or async)
|
||||
serde: Serializer to use for values
|
||||
prefix: Key prefix for all cached values
|
||||
"""
|
||||
super().__init__(serde=serde)
|
||||
self.redis = redis
|
||||
self.prefix = prefix
|
||||
|
||||
def _make_key(self, ns: Namespace, key: str) -> str:
|
||||
"""Create a Redis key from namespace and key."""
|
||||
ns_str = ":".join(ns) if ns else ""
|
||||
return f"{self.prefix}{ns_str}:{key}" if ns_str else f"{self.prefix}{key}"
|
||||
|
||||
def _parse_key(self, redis_key: str) -> tuple[Namespace, str]:
|
||||
"""Parse a Redis key back to namespace and key."""
|
||||
if not redis_key.startswith(self.prefix):
|
||||
raise ValueError(
|
||||
f"Key {redis_key} does not start with prefix {self.prefix}"
|
||||
)
|
||||
|
||||
remaining = redis_key[len(self.prefix) :]
|
||||
if ":" in remaining:
|
||||
parts = remaining.split(":")
|
||||
key = parts[-1]
|
||||
ns_parts = parts[:-1]
|
||||
return (tuple(ns_parts), key)
|
||||
else:
|
||||
return (tuple(), remaining)
|
||||
|
||||
def get(self, keys: Sequence[FullKey]) -> dict[FullKey, ValueT]:
|
||||
"""Get the cached values for the given keys."""
|
||||
if not keys:
|
||||
return {}
|
||||
|
||||
# Build Redis keys
|
||||
redis_keys = [self._make_key(ns, key) for ns, key in keys]
|
||||
|
||||
# Get values from Redis using MGET
|
||||
try:
|
||||
raw_values = self.redis.mget(redis_keys)
|
||||
except Exception:
|
||||
# If Redis is unavailable, return empty dict
|
||||
return {}
|
||||
|
||||
values: dict[FullKey, ValueT] = {}
|
||||
for i, raw_value in enumerate(raw_values):
|
||||
if raw_value is not None:
|
||||
try:
|
||||
# Deserialize the value
|
||||
encoding, data = raw_value.split(b":", 1)
|
||||
values[keys[i]] = self.serde.loads_typed((encoding.decode(), data))
|
||||
except Exception:
|
||||
# Skip corrupted entries
|
||||
continue
|
||||
|
||||
return values
|
||||
|
||||
async def aget(self, keys: Sequence[FullKey]) -> dict[FullKey, ValueT]:
|
||||
"""Asynchronously get the cached values for the given keys."""
|
||||
return self.get(keys)
|
||||
|
||||
def set(self, mapping: Mapping[FullKey, tuple[ValueT, int | None]]) -> None:
|
||||
"""Set the cached values for the given keys and TTLs."""
|
||||
if not mapping:
|
||||
return
|
||||
|
||||
# Use pipeline for efficient batch operations
|
||||
pipe = self.redis.pipeline()
|
||||
|
||||
for (ns, key), (value, ttl) in mapping.items():
|
||||
redis_key = self._make_key(ns, key)
|
||||
encoding, data = self.serde.dumps_typed(value)
|
||||
|
||||
# Store as "encoding:data" format
|
||||
serialized_value = f"{encoding}:".encode() + data
|
||||
|
||||
if ttl is not None:
|
||||
pipe.setex(redis_key, ttl, serialized_value)
|
||||
else:
|
||||
pipe.set(redis_key, serialized_value)
|
||||
|
||||
try:
|
||||
pipe.execute()
|
||||
except Exception:
|
||||
# Silently fail if Redis is unavailable
|
||||
pass
|
||||
|
||||
async def aset(self, mapping: Mapping[FullKey, tuple[ValueT, int | None]]) -> None:
|
||||
"""Asynchronously set the cached values for the given keys and TTLs."""
|
||||
self.set(mapping)
|
||||
|
||||
def clear(self, namespaces: Sequence[Namespace] | None = None) -> None:
|
||||
"""Delete the cached values for the given namespaces.
|
||||
If no namespaces are provided, clear all cached values."""
|
||||
try:
|
||||
if namespaces is None:
|
||||
# Clear all keys with our prefix
|
||||
pattern = f"{self.prefix}*"
|
||||
keys = self.redis.keys(pattern)
|
||||
if keys:
|
||||
self.redis.delete(*keys)
|
||||
else:
|
||||
# Clear specific namespaces
|
||||
keys_to_delete = []
|
||||
for ns in namespaces:
|
||||
ns_str = ":".join(ns) if ns else ""
|
||||
pattern = (
|
||||
f"{self.prefix}{ns_str}:*" if ns_str else f"{self.prefix}*"
|
||||
)
|
||||
keys = self.redis.keys(pattern)
|
||||
keys_to_delete.extend(keys)
|
||||
|
||||
if keys_to_delete:
|
||||
self.redis.delete(*keys_to_delete)
|
||||
except Exception:
|
||||
# Silently fail if Redis is unavailable
|
||||
pass
|
||||
|
||||
async def aclear(self, namespaces: Sequence[Namespace] | None = None) -> None:
|
||||
"""Asynchronously delete the cached values for the given namespaces.
|
||||
If no namespaces are provided, clear all cached values."""
|
||||
self.clear(namespaces)
|
||||
@@ -1,17 +1,19 @@
|
||||
from __future__ import annotations
|
||||
|
||||
from collections.abc import AsyncIterator, Iterator, Mapping, Sequence
|
||||
from collections.abc import Iterator, Mapping, Sequence
|
||||
from datetime import datetime, timezone
|
||||
from typing import ( # noqa: UP035
|
||||
Any,
|
||||
Dict,
|
||||
Generic,
|
||||
List,
|
||||
Literal,
|
||||
NamedTuple,
|
||||
Optional,
|
||||
Tuple,
|
||||
TypedDict,
|
||||
TypeVar,
|
||||
Union,
|
||||
)
|
||||
|
||||
from langchain_core.runnables import RunnableConfig
|
||||
|
||||
from langgraph.checkpoint.base.id import uuid6
|
||||
from langgraph.checkpoint.serde.base import SerializerProtocol, maybe_add_typed_methods
|
||||
from langgraph.checkpoint.serde.jsonplus import JsonPlusSerializer
|
||||
@@ -21,10 +23,29 @@ from langgraph.checkpoint.serde.types import (
|
||||
RESUME,
|
||||
SCHEDULED,
|
||||
ChannelProtocol,
|
||||
SendProtocol,
|
||||
)
|
||||
|
||||
V = TypeVar("V", int, float, str)
|
||||
PendingWrite = tuple[str, str, Any]
|
||||
PendingWrite = Tuple[str, str, Any]
|
||||
|
||||
|
||||
class CheckpointConfig(TypedDict, total=False):
|
||||
"""Configuration for a Runnable."""
|
||||
|
||||
metadata: dict[str, Any]
|
||||
"""
|
||||
Metadata for this call and any sub-calls (eg. a Chain calling an LLM).
|
||||
Keys should be strings, values should be JSON-serializable.
|
||||
"""
|
||||
|
||||
configurable: dict[str, Any]
|
||||
"""
|
||||
Runtime values for attributes previously made configurable on this Runnable,
|
||||
or sub-Runnables, through .configurable_fields() or .configurable_alternatives().
|
||||
Check .output_schema() for a description of the attributes that have been made
|
||||
configurable.
|
||||
"""
|
||||
|
||||
|
||||
# Marked as total=False to allow for future expansion.
|
||||
@@ -34,17 +55,22 @@ class CheckpointMetadata(TypedDict, total=False):
|
||||
source: Literal["input", "loop", "update", "fork"]
|
||||
"""The source of the checkpoint.
|
||||
|
||||
- `"input"`: The checkpoint was created from an input to invoke/stream/batch.
|
||||
- `"loop"`: The checkpoint was created from inside the pregel loop.
|
||||
- `"update"`: The checkpoint was created from a manual state update.
|
||||
- `"fork"`: The checkpoint was created as a copy of another checkpoint.
|
||||
- "input": The checkpoint was created from an input to invoke/stream/batch.
|
||||
- "loop": The checkpoint was created from inside the pregel loop.
|
||||
- "update": The checkpoint was created from a manual state update.
|
||||
- "fork": The checkpoint was created as a copy of another checkpoint.
|
||||
"""
|
||||
step: int
|
||||
"""The step number of the checkpoint.
|
||||
|
||||
`-1` for the first `"input"` checkpoint.
|
||||
`0` for the first `"loop"` checkpoint.
|
||||
`...` for the `nth` checkpoint afterwards.
|
||||
-1 for the first "input" checkpoint.
|
||||
0 for the first "loop" checkpoint.
|
||||
... for the nth checkpoint afterwards.
|
||||
"""
|
||||
writes: dict[str, Any]
|
||||
"""The writes that were made between the previous checkpoint and this one.
|
||||
|
||||
Mapping from node name to writes emitted by that node.
|
||||
"""
|
||||
parents: dict[str, str]
|
||||
"""The IDs of the parent checkpoints.
|
||||
@@ -53,41 +79,52 @@ class CheckpointMetadata(TypedDict, total=False):
|
||||
"""
|
||||
|
||||
|
||||
ChannelVersions = dict[str, str | int | float]
|
||||
class TaskInfo(TypedDict):
|
||||
status: Literal["scheduled", "success", "error"]
|
||||
|
||||
|
||||
ChannelVersions = dict[str, Union[str, int, float]]
|
||||
|
||||
|
||||
class Checkpoint(TypedDict):
|
||||
"""State snapshot at a given point in time."""
|
||||
|
||||
v: int
|
||||
"""The version of the checkpoint format. Currently `1`."""
|
||||
"""The version of the checkpoint format. Currently 1."""
|
||||
id: str
|
||||
"""The ID of the checkpoint.
|
||||
|
||||
This is both unique and monotonically increasing, so can be used for sorting
|
||||
checkpoints from first to last."""
|
||||
"""The ID of the checkpoint. This is both unique and monotonically
|
||||
increasing, so can be used for sorting checkpoints from first to last."""
|
||||
ts: str
|
||||
"""The timestamp of the checkpoint in ISO 8601 format."""
|
||||
channel_values: dict[str, Any]
|
||||
"""The values of the channels at the time of the checkpoint.
|
||||
|
||||
Mapping from channel name to deserialized channel snapshot value.
|
||||
"""
|
||||
channel_versions: ChannelVersions
|
||||
"""The versions of the channels at the time of the checkpoint.
|
||||
|
||||
The keys are channel names and the values are monotonically increasing
|
||||
version strings for each channel.
|
||||
"""
|
||||
versions_seen: dict[str, ChannelVersions]
|
||||
"""Map from node ID to map from channel name to version seen.
|
||||
|
||||
This keeps track of the versions of the channels that each node has seen.
|
||||
Used to determine which nodes to execute next.
|
||||
"""
|
||||
updated_channels: list[str] | None
|
||||
"""The channels that were updated in this checkpoint.
|
||||
"""
|
||||
pending_sends: List[SendProtocol]
|
||||
"""List of inputs pushed to nodes but not yet processed.
|
||||
Cleared by the next checkpoint."""
|
||||
|
||||
|
||||
def empty_checkpoint() -> Checkpoint:
|
||||
return Checkpoint(
|
||||
v=1,
|
||||
id=str(uuid6(clock_seq=-2)),
|
||||
ts=datetime.now(timezone.utc).isoformat(),
|
||||
channel_values={},
|
||||
channel_versions={},
|
||||
versions_seen={},
|
||||
pending_sends=[],
|
||||
)
|
||||
|
||||
|
||||
def copy_checkpoint(checkpoint: Checkpoint) -> Checkpoint:
|
||||
@@ -99,18 +136,48 @@ def copy_checkpoint(checkpoint: Checkpoint) -> Checkpoint:
|
||||
channel_versions=checkpoint["channel_versions"].copy(),
|
||||
versions_seen={k: v.copy() for k, v in checkpoint["versions_seen"].items()},
|
||||
pending_sends=checkpoint.get("pending_sends", []).copy(),
|
||||
updated_channels=checkpoint.get("updated_channels", None),
|
||||
)
|
||||
|
||||
|
||||
def create_checkpoint(
|
||||
checkpoint: Checkpoint,
|
||||
channels: Optional[Mapping[str, ChannelProtocol]],
|
||||
step: int,
|
||||
*,
|
||||
id: Optional[str] = None,
|
||||
) -> Checkpoint:
|
||||
"""Create a checkpoint for the given channels."""
|
||||
ts = datetime.now(timezone.utc).isoformat()
|
||||
if channels is None:
|
||||
values = checkpoint["channel_values"]
|
||||
else:
|
||||
values = {}
|
||||
for k, v in channels.items():
|
||||
if k not in checkpoint["channel_versions"]:
|
||||
continue
|
||||
try:
|
||||
values[k] = v.checkpoint()
|
||||
except EmptyChannelError:
|
||||
pass
|
||||
return Checkpoint(
|
||||
v=1,
|
||||
ts=ts,
|
||||
id=id or str(uuid6(clock_seq=step)),
|
||||
channel_values=values,
|
||||
channel_versions=checkpoint["channel_versions"],
|
||||
versions_seen=checkpoint["versions_seen"],
|
||||
pending_sends=checkpoint.get("pending_sends", []),
|
||||
)
|
||||
|
||||
|
||||
class CheckpointTuple(NamedTuple):
|
||||
"""A tuple containing a checkpoint and its associated data."""
|
||||
|
||||
config: RunnableConfig
|
||||
config: CheckpointConfig
|
||||
checkpoint: Checkpoint
|
||||
metadata: CheckpointMetadata
|
||||
parent_config: RunnableConfig | None = None
|
||||
pending_writes: list[PendingWrite] | None = None
|
||||
parent_config: Optional[CheckpointConfig] = None
|
||||
pending_writes: Optional[List[PendingWrite]] = None
|
||||
|
||||
|
||||
class BaseCheckpointSaver(Generic[V]):
|
||||
@@ -132,39 +199,30 @@ class BaseCheckpointSaver(Generic[V]):
|
||||
def __init__(
|
||||
self,
|
||||
*,
|
||||
serde: SerializerProtocol | None = None,
|
||||
serde: Optional[SerializerProtocol] = None,
|
||||
) -> None:
|
||||
self.serde = maybe_add_typed_methods(serde or self.serde)
|
||||
|
||||
@property
|
||||
def config_specs(self) -> list:
|
||||
"""Define the configuration options for the checkpoint saver.
|
||||
|
||||
Returns:
|
||||
list: List of configuration field specs.
|
||||
"""
|
||||
return []
|
||||
|
||||
def get(self, config: RunnableConfig) -> Checkpoint | None:
|
||||
def get(self, config: CheckpointConfig) -> Optional[Checkpoint]:
|
||||
"""Fetch a checkpoint using the given configuration.
|
||||
|
||||
Args:
|
||||
config: Configuration specifying which checkpoint to retrieve.
|
||||
config (CheckpointConfig): Configuration specifying which checkpoint to retrieve.
|
||||
|
||||
Returns:
|
||||
The requested checkpoint, or `None` if not found.
|
||||
Optional[Checkpoint]: The requested checkpoint, or None if not found.
|
||||
"""
|
||||
if value := self.get_tuple(config):
|
||||
return value.checkpoint
|
||||
|
||||
def get_tuple(self, config: RunnableConfig) -> CheckpointTuple | None:
|
||||
def get_tuple(self, config: CheckpointConfig) -> Optional[CheckpointTuple]:
|
||||
"""Fetch a checkpoint tuple using the given configuration.
|
||||
|
||||
Args:
|
||||
config: Configuration specifying which checkpoint to retrieve.
|
||||
config (CheckpointConfig): Configuration specifying which checkpoint to retrieve.
|
||||
|
||||
Returns:
|
||||
The requested checkpoint tuple, or `None` if not found.
|
||||
Optional[CheckpointTuple]: The requested checkpoint tuple, or None if not found.
|
||||
|
||||
Raises:
|
||||
NotImplementedError: Implement this method in your custom checkpoint saver.
|
||||
@@ -173,22 +231,22 @@ class BaseCheckpointSaver(Generic[V]):
|
||||
|
||||
def list(
|
||||
self,
|
||||
config: RunnableConfig | None,
|
||||
config: Optional[CheckpointConfig],
|
||||
*,
|
||||
filter: dict[str, Any] | None = None,
|
||||
before: RunnableConfig | None = None,
|
||||
limit: int | None = None,
|
||||
filter: Optional[Dict[str, Any]] = None,
|
||||
before: Optional[CheckpointConfig] = None,
|
||||
limit: Optional[int] = None,
|
||||
) -> Iterator[CheckpointTuple]:
|
||||
"""List checkpoints that match the given criteria.
|
||||
|
||||
Args:
|
||||
config: Base configuration for filtering checkpoints.
|
||||
filter: Additional filtering criteria.
|
||||
before: List checkpoints created before this configuration.
|
||||
limit: Maximum number of checkpoints to return.
|
||||
config (Optional[CheckpointConfig]): Base configuration for filtering checkpoints.
|
||||
filter (Optional[Dict[str, Any]]): Additional filtering criteria.
|
||||
before (Optional[CheckpointConfig]): List checkpoints created before this configuration.
|
||||
limit (Optional[int]): Maximum number of checkpoints to return.
|
||||
|
||||
Returns:
|
||||
Iterator of matching checkpoint tuples.
|
||||
Iterator[CheckpointTuple]: Iterator of matching checkpoint tuples.
|
||||
|
||||
Raises:
|
||||
NotImplementedError: Implement this method in your custom checkpoint saver.
|
||||
@@ -197,21 +255,21 @@ class BaseCheckpointSaver(Generic[V]):
|
||||
|
||||
def put(
|
||||
self,
|
||||
config: RunnableConfig,
|
||||
config: CheckpointConfig,
|
||||
checkpoint: Checkpoint,
|
||||
metadata: CheckpointMetadata,
|
||||
new_versions: ChannelVersions,
|
||||
) -> RunnableConfig:
|
||||
) -> CheckpointConfig:
|
||||
"""Store a checkpoint with its configuration and metadata.
|
||||
|
||||
Args:
|
||||
config: Configuration for the checkpoint.
|
||||
checkpoint: The checkpoint to store.
|
||||
metadata: Additional metadata for the checkpoint.
|
||||
new_versions: New channel versions as of this write.
|
||||
config (CheckpointConfig): Configuration for the checkpoint.
|
||||
checkpoint (Checkpoint): The checkpoint to store.
|
||||
metadata (CheckpointMetadata): Additional metadata for the checkpoint.
|
||||
new_versions (ChannelVersions): New channel versions as of this write.
|
||||
|
||||
Returns:
|
||||
RunnableConfig: Updated configuration after storing the checkpoint.
|
||||
CheckpointConfig: Updated configuration after storing the checkpoint.
|
||||
|
||||
Raises:
|
||||
NotImplementedError: Implement this method in your custom checkpoint saver.
|
||||
@@ -220,150 +278,33 @@ class BaseCheckpointSaver(Generic[V]):
|
||||
|
||||
def put_writes(
|
||||
self,
|
||||
config: RunnableConfig,
|
||||
writes: Sequence[tuple[str, Any]],
|
||||
config: CheckpointConfig,
|
||||
writes: Sequence[Tuple[str, Any]],
|
||||
task_id: str,
|
||||
task_path: str = "",
|
||||
) -> None:
|
||||
"""Store intermediate writes linked to a checkpoint.
|
||||
|
||||
Args:
|
||||
config: Configuration of the related checkpoint.
|
||||
writes: List of writes to store.
|
||||
task_id: Identifier for the task creating the writes.
|
||||
task_path: Path of the task creating the writes.
|
||||
config (CheckpointConfig): Configuration of the related checkpoint.
|
||||
writes (List[Tuple[str, Any]]): List of writes to store.
|
||||
task_id (str): Identifier for the task creating the writes.
|
||||
task_path (str): Path of the task creating the writes.
|
||||
|
||||
Raises:
|
||||
NotImplementedError: Implement this method in your custom checkpoint saver.
|
||||
"""
|
||||
raise NotImplementedError
|
||||
|
||||
def delete_thread(
|
||||
self,
|
||||
thread_id: str,
|
||||
) -> None:
|
||||
"""Delete all checkpoints and writes associated with a specific thread ID.
|
||||
|
||||
Args:
|
||||
thread_id: The thread ID whose checkpoints should be deleted.
|
||||
"""
|
||||
raise NotImplementedError
|
||||
|
||||
async def aget(self, config: RunnableConfig) -> Checkpoint | None:
|
||||
"""Asynchronously fetch a checkpoint using the given configuration.
|
||||
|
||||
Args:
|
||||
config: Configuration specifying which checkpoint to retrieve.
|
||||
|
||||
Returns:
|
||||
The requested checkpoint, or `None` if not found.
|
||||
"""
|
||||
if value := await self.aget_tuple(config):
|
||||
return value.checkpoint
|
||||
|
||||
async def aget_tuple(self, config: RunnableConfig) -> CheckpointTuple | None:
|
||||
"""Asynchronously fetch a checkpoint tuple using the given configuration.
|
||||
|
||||
Args:
|
||||
config: Configuration specifying which checkpoint to retrieve.
|
||||
|
||||
Returns:
|
||||
The requested checkpoint tuple, or `None` if not found.
|
||||
|
||||
Raises:
|
||||
NotImplementedError: Implement this method in your custom checkpoint saver.
|
||||
"""
|
||||
raise NotImplementedError
|
||||
|
||||
async def alist(
|
||||
self,
|
||||
config: RunnableConfig | None,
|
||||
*,
|
||||
filter: dict[str, Any] | None = None,
|
||||
before: RunnableConfig | None = None,
|
||||
limit: int | None = None,
|
||||
) -> AsyncIterator[CheckpointTuple]:
|
||||
"""Asynchronously list checkpoints that match the given criteria.
|
||||
|
||||
Args:
|
||||
config: Base configuration for filtering checkpoints.
|
||||
filter: Additional filtering criteria for metadata.
|
||||
before: List checkpoints created before this configuration.
|
||||
limit: Maximum number of checkpoints to return.
|
||||
|
||||
Returns:
|
||||
Async iterator of matching checkpoint tuples.
|
||||
|
||||
Raises:
|
||||
NotImplementedError: Implement this method in your custom checkpoint saver.
|
||||
"""
|
||||
raise NotImplementedError
|
||||
yield
|
||||
|
||||
async def aput(
|
||||
self,
|
||||
config: RunnableConfig,
|
||||
checkpoint: Checkpoint,
|
||||
metadata: CheckpointMetadata,
|
||||
new_versions: ChannelVersions,
|
||||
) -> RunnableConfig:
|
||||
"""Asynchronously store a checkpoint with its configuration and metadata.
|
||||
|
||||
Args:
|
||||
config: Configuration for the checkpoint.
|
||||
checkpoint: The checkpoint to store.
|
||||
metadata: Additional metadata for the checkpoint.
|
||||
new_versions: New channel versions as of this write.
|
||||
|
||||
Returns:
|
||||
RunnableConfig: Updated configuration after storing the checkpoint.
|
||||
|
||||
Raises:
|
||||
NotImplementedError: Implement this method in your custom checkpoint saver.
|
||||
"""
|
||||
raise NotImplementedError
|
||||
|
||||
async def aput_writes(
|
||||
self,
|
||||
config: RunnableConfig,
|
||||
writes: Sequence[tuple[str, Any]],
|
||||
task_id: str,
|
||||
task_path: str = "",
|
||||
) -> None:
|
||||
"""Asynchronously store intermediate writes linked to a checkpoint.
|
||||
|
||||
Args:
|
||||
config: Configuration of the related checkpoint.
|
||||
writes: List of writes to store.
|
||||
task_id: Identifier for the task creating the writes.
|
||||
task_path: Path of the task creating the writes.
|
||||
|
||||
Raises:
|
||||
NotImplementedError: Implement this method in your custom checkpoint saver.
|
||||
"""
|
||||
raise NotImplementedError
|
||||
|
||||
async def adelete_thread(
|
||||
self,
|
||||
thread_id: str,
|
||||
) -> None:
|
||||
"""Delete all checkpoints and writes associated with a specific thread ID.
|
||||
|
||||
Args:
|
||||
thread_id: The thread ID whose checkpoints should be deleted.
|
||||
"""
|
||||
raise NotImplementedError
|
||||
|
||||
def get_next_version(self, current: V | None, channel: None) -> V:
|
||||
def get_next_version(self, current: Optional[V], channel: ChannelProtocol) -> V:
|
||||
"""Generate the next version ID for a channel.
|
||||
|
||||
Default is to use integer versions, incrementing by `1`.
|
||||
|
||||
If you override, you can use `str`/`int`/`float` versions, as long as they are monotonically increasing.
|
||||
Default is to use integer versions, incrementing by 1. If you override, you can use str/int/float versions,
|
||||
as long as they are monotonically increasing.
|
||||
|
||||
Args:
|
||||
current: The current version identifier (`int`, `float`, or `str`).
|
||||
channel: Deprecated argument, kept for backwards compatibility.
|
||||
current (Optional[V]): The current version identifier (int, float, or str).
|
||||
channel (BaseChannel): The channel being versioned.
|
||||
|
||||
Returns:
|
||||
V: The next version identifier, which must be increasing.
|
||||
@@ -383,42 +324,30 @@ class EmptyChannelError(Exception):
|
||||
pass
|
||||
|
||||
|
||||
def get_checkpoint_id(config: RunnableConfig) -> str | None:
|
||||
"""Get checkpoint ID."""
|
||||
return config["configurable"].get("checkpoint_id")
|
||||
def get_checkpoint_id(config: CheckpointConfig) -> Optional[str]:
|
||||
"""Get checkpoint ID in a backwards-compatible manner (fallback on thread_ts)."""
|
||||
return config["configurable"].get(
|
||||
"checkpoint_id", config["configurable"].get("thread_ts")
|
||||
)
|
||||
|
||||
|
||||
def get_checkpoint_metadata(
|
||||
config: RunnableConfig, metadata: CheckpointMetadata
|
||||
config: CheckpointConfig, metadata: CheckpointMetadata
|
||||
) -> CheckpointMetadata:
|
||||
"""Get checkpoint metadata in a backwards-compatible manner."""
|
||||
metadata = {
|
||||
k: v.replace("\u0000", "") if isinstance(v, str) else v
|
||||
for k, v in metadata.items()
|
||||
}
|
||||
metadata = metadata.copy()
|
||||
for obj in (config.get("metadata"), config.get("configurable")):
|
||||
if not obj:
|
||||
continue
|
||||
for key, v in obj.items():
|
||||
for key in obj:
|
||||
if key in metadata or key in EXCLUDED_METADATA_KEYS or key.startswith("__"):
|
||||
continue
|
||||
elif isinstance(v, str):
|
||||
metadata[key] = v.replace("\u0000", "")
|
||||
elif isinstance(v, (int, bool, float)):
|
||||
metadata[key] = v
|
||||
v = obj[key]
|
||||
if isinstance(v, (str, int, bool, float)):
|
||||
metadata[key] = v # type: ignore[literal-required]
|
||||
return metadata
|
||||
|
||||
|
||||
def get_serializable_checkpoint_metadata(
|
||||
config: RunnableConfig, metadata: CheckpointMetadata
|
||||
) -> CheckpointMetadata:
|
||||
"""Get checkpoint metadata in a backwards-compatible manner."""
|
||||
checkpoint_metadata = get_checkpoint_metadata(config, metadata)
|
||||
if "writes" in checkpoint_metadata:
|
||||
checkpoint_metadata.pop("writes")
|
||||
return checkpoint_metadata
|
||||
|
||||
|
||||
"""
|
||||
Mapping from error type to error index.
|
||||
Regular writes just map to their index in the list of writes being saved.
|
||||
@@ -429,66 +358,7 @@ Each Checkpointer implementation should use this mapping in put_writes.
|
||||
WRITES_IDX_MAP = {ERROR: -1, SCHEDULED: -2, INTERRUPT: -3, RESUME: -4}
|
||||
|
||||
EXCLUDED_METADATA_KEYS = {
|
||||
"thread_id",
|
||||
"checkpoint_id",
|
||||
"checkpoint_ns",
|
||||
"checkpoint_map",
|
||||
"langgraph_step",
|
||||
"langgraph_node",
|
||||
"langgraph_triggers",
|
||||
"langgraph_path",
|
||||
"langgraph_checkpoint_ns",
|
||||
}
|
||||
|
||||
# --- below are deprecated utilities used by past versions of LangGraph ---
|
||||
|
||||
LATEST_VERSION = 2
|
||||
|
||||
|
||||
def empty_checkpoint() -> Checkpoint:
|
||||
from datetime import datetime, timezone
|
||||
|
||||
return Checkpoint(
|
||||
v=LATEST_VERSION,
|
||||
id=str(uuid6(clock_seq=-2)),
|
||||
ts=datetime.now(timezone.utc).isoformat(),
|
||||
channel_values={},
|
||||
channel_versions={},
|
||||
versions_seen={},
|
||||
pending_sends=[],
|
||||
updated_channels=None,
|
||||
)
|
||||
|
||||
|
||||
def create_checkpoint(
|
||||
checkpoint: Checkpoint,
|
||||
channels: Mapping[str, ChannelProtocol] | None,
|
||||
step: int,
|
||||
*,
|
||||
id: str | None = None,
|
||||
) -> Checkpoint:
|
||||
"""Create a checkpoint for the given channels."""
|
||||
from datetime import datetime, timezone
|
||||
|
||||
ts = datetime.now(timezone.utc).isoformat()
|
||||
if channels is None:
|
||||
values = checkpoint["channel_values"]
|
||||
else:
|
||||
values = {}
|
||||
for k, v in channels.items():
|
||||
if k not in checkpoint["channel_versions"]:
|
||||
continue
|
||||
try:
|
||||
values[k] = v.checkpoint()
|
||||
except EmptyChannelError:
|
||||
pass
|
||||
return Checkpoint(
|
||||
v=LATEST_VERSION,
|
||||
ts=ts,
|
||||
id=id or str(uuid6(clock_seq=step)),
|
||||
channel_values=values,
|
||||
channel_versions=checkpoint["channel_versions"],
|
||||
versions_seen=checkpoint["versions_seen"],
|
||||
pending_sends=checkpoint.get("pending_sends", []),
|
||||
updated_channels=None,
|
||||
)
|
||||
@@ -3,11 +3,10 @@ https://github.com/oittaa/uuid6-python/blob/main/src/uuid6/__init__.py#L95
|
||||
Bundled in to avoid install issues with uuid6 package
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import random
|
||||
import time
|
||||
import uuid
|
||||
from typing import Optional, Tuple
|
||||
|
||||
_last_v6_timestamp = None
|
||||
|
||||
@@ -19,12 +18,12 @@ class UUID(uuid.UUID):
|
||||
|
||||
def __init__(
|
||||
self,
|
||||
hex: str | None = None,
|
||||
bytes: bytes | None = None,
|
||||
bytes_le: bytes | None = None,
|
||||
fields: tuple[int, int, int, int, int, int] | None = None,
|
||||
int: int | None = None,
|
||||
version: int | None = None,
|
||||
hex: Optional[str] = None,
|
||||
bytes: Optional[bytes] = None,
|
||||
bytes_le: Optional[bytes] = None,
|
||||
fields: Optional[Tuple[int, int, int, int, int, int]] = None,
|
||||
int: Optional[int] = None,
|
||||
version: Optional[int] = None,
|
||||
*,
|
||||
is_safe: uuid.SafeUUID = uuid.SafeUUID.unknown,
|
||||
) -> None:
|
||||
@@ -76,7 +75,7 @@ def _subsec_decode(value: int) -> int:
|
||||
return -(-value * 10**6 // 2**20)
|
||||
|
||||
|
||||
def uuid6(node: int | None = None, clock_seq: int | None = None) -> UUID:
|
||||
def uuid6(node: Optional[int] = None, clock_seq: Optional[int] = None) -> UUID:
|
||||
r"""UUID version 6 is a field-compatible version of UUIDv1, reordered for
|
||||
improved DB locality. It is expected that UUIDv6 will primarily be
|
||||
used in contexts where there are existing v1 UUIDs. Systems that do
|
||||
|
||||
@@ -1,165 +1,136 @@
|
||||
from __future__ import annotations
|
||||
|
||||
import logging
|
||||
import os
|
||||
import pickle
|
||||
import random
|
||||
import shutil
|
||||
from collections import defaultdict
|
||||
from collections.abc import AsyncIterator, Iterator, Sequence
|
||||
from contextlib import AbstractAsyncContextManager, AbstractContextManager, ExitStack
|
||||
from collections.abc import Iterator, Sequence
|
||||
from contextlib import AbstractContextManager, ExitStack
|
||||
from types import TracebackType
|
||||
from typing import Any
|
||||
|
||||
from langchain_core.runnables import RunnableConfig
|
||||
from typing import Any, Optional
|
||||
|
||||
from langgraph.checkpoint.base import (
|
||||
WRITES_IDX_MAP,
|
||||
BaseCheckpointSaver,
|
||||
ChannelVersions,
|
||||
Checkpoint,
|
||||
CheckpointConfig,
|
||||
CheckpointMetadata,
|
||||
CheckpointTuple,
|
||||
SerializerProtocol,
|
||||
get_checkpoint_id,
|
||||
get_checkpoint_metadata,
|
||||
)
|
||||
from langgraph.checkpoint.serde.types import TASKS, ChannelProtocol
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
class InMemorySaver(
|
||||
BaseCheckpointSaver[str], AbstractContextManager, AbstractAsyncContextManager
|
||||
):
|
||||
class InMemorySaver(BaseCheckpointSaver[str], AbstractContextManager):
|
||||
"""An in-memory checkpoint saver.
|
||||
|
||||
This checkpoint saver stores checkpoints in memory using a `defaultdict`.
|
||||
This checkpoint saver stores checkpoints in memory using a defaultdict.
|
||||
|
||||
Note:
|
||||
Only use `InMemorySaver` for debugging or testing purposes.
|
||||
For production use cases we recommend installing [langgraph-checkpoint-postgres](https://pypi.org/project/langgraph-checkpoint-postgres/) and using `PostgresSaver` / `AsyncPostgresSaver`.
|
||||
|
||||
If you are using LangSmith Deployment, no checkpointer needs to be specified. The correct managed checkpointer will be used automatically.
|
||||
|
||||
Args:
|
||||
serde: The serializer to use for serializing and deserializing checkpoints.
|
||||
serde (Optional[SerializerProtocol]): The serializer to use for serializing and deserializing checkpoints. Defaults to None.
|
||||
|
||||
Example:
|
||||
```python
|
||||
import asyncio
|
||||
Examples:
|
||||
|
||||
from langgraph.checkpoint.memory import InMemorySaver
|
||||
from langgraph.graph import StateGraph
|
||||
import asyncio
|
||||
|
||||
builder = StateGraph(int)
|
||||
builder.add_node("add_one", lambda x: x + 1)
|
||||
builder.set_entry_point("add_one")
|
||||
builder.set_finish_point("add_one")
|
||||
from langgraph.checkpoint.memory import InMemorySaver
|
||||
from langgraph.graph import StateGraph
|
||||
|
||||
memory = InMemorySaver()
|
||||
graph = builder.compile(checkpointer=memory)
|
||||
coro = graph.ainvoke(1, {"configurable": {"thread_id": "thread-1"}})
|
||||
asyncio.run(coro) # Output: 2
|
||||
```
|
||||
builder = StateGraph(int)
|
||||
builder.add_node("add_one", lambda x: x + 1)
|
||||
builder.set_entry_point("add_one")
|
||||
builder.set_finish_point("add_one")
|
||||
|
||||
memory = InMemorySaver()
|
||||
graph = builder.compile(checkpointer=memory)
|
||||
coro = graph.ainvoke(1, {"configurable": {"thread_id": "thread-1"}})
|
||||
asyncio.run(coro) # Output: 2
|
||||
"""
|
||||
|
||||
# thread ID -> checkpoint NS -> checkpoint ID -> checkpoint mapping
|
||||
storage: defaultdict[
|
||||
str,
|
||||
dict[str, dict[str, tuple[tuple[str, bytes], tuple[str, bytes], str | None]]],
|
||||
dict[
|
||||
str, dict[str, tuple[tuple[str, bytes], tuple[str, bytes], Optional[str]]]
|
||||
],
|
||||
]
|
||||
# (thread ID, checkpoint NS, checkpoint ID) -> (task ID, write idx)
|
||||
writes: defaultdict[
|
||||
tuple[str, str, str],
|
||||
dict[tuple[str, int], tuple[str, str, tuple[str, bytes], str]],
|
||||
]
|
||||
blobs: dict[
|
||||
tuple[
|
||||
str, str, str, str | int | float
|
||||
], # thread id, checkpoint ns, channel, version
|
||||
tuple[str, bytes],
|
||||
]
|
||||
|
||||
def __init__(
|
||||
self,
|
||||
*,
|
||||
serde: SerializerProtocol | None = None,
|
||||
serde: Optional[SerializerProtocol] = None,
|
||||
factory: type[defaultdict] = defaultdict,
|
||||
) -> None:
|
||||
super().__init__(serde=serde)
|
||||
self.storage = factory(lambda: defaultdict(dict))
|
||||
self.writes = factory(dict)
|
||||
self.blobs = factory()
|
||||
self.stack = ExitStack()
|
||||
if factory is not defaultdict:
|
||||
self.stack.enter_context(self.storage) # type: ignore[arg-type]
|
||||
self.stack.enter_context(self.writes) # type: ignore[arg-type]
|
||||
self.stack.enter_context(self.blobs) # type: ignore[arg-type]
|
||||
|
||||
def __enter__(self) -> InMemorySaver:
|
||||
self.stack.__enter__()
|
||||
return self
|
||||
def __enter__(self) -> "InMemorySaver":
|
||||
return self.stack.__enter__()
|
||||
|
||||
def __exit__(
|
||||
self,
|
||||
exc_type: type[BaseException] | None,
|
||||
exc_value: BaseException | None,
|
||||
traceback: TracebackType | None,
|
||||
) -> bool | None:
|
||||
exc_type: Optional[type[BaseException]],
|
||||
exc_value: Optional[BaseException],
|
||||
traceback: Optional[TracebackType],
|
||||
) -> Optional[bool]:
|
||||
return self.stack.__exit__(exc_type, exc_value, traceback)
|
||||
|
||||
async def __aenter__(self) -> InMemorySaver:
|
||||
self.stack.__enter__()
|
||||
return self
|
||||
|
||||
async def __aexit__(
|
||||
self,
|
||||
__exc_type: type[BaseException] | None,
|
||||
__exc_value: BaseException | None,
|
||||
__traceback: TracebackType | None,
|
||||
) -> bool | None:
|
||||
return self.stack.__exit__(__exc_type, __exc_value, __traceback)
|
||||
|
||||
def _load_blobs(
|
||||
self, thread_id: str, checkpoint_ns: str, versions: ChannelVersions
|
||||
) -> dict[str, Any]:
|
||||
channel_values: dict[str, Any] = {}
|
||||
for k, v in versions.items():
|
||||
kk = (thread_id, checkpoint_ns, k, v)
|
||||
if kk in self.blobs:
|
||||
vv = self.blobs[kk]
|
||||
if vv[0] != "empty":
|
||||
channel_values[k] = self.serde.loads_typed(vv)
|
||||
return channel_values
|
||||
|
||||
def get_tuple(self, config: RunnableConfig) -> CheckpointTuple | None:
|
||||
def get_tuple(self, config: CheckpointConfig) -> Optional[CheckpointTuple]:
|
||||
"""Get a checkpoint tuple from the in-memory storage.
|
||||
|
||||
This method retrieves a checkpoint tuple from the in-memory storage based on the
|
||||
provided config. If the config contains a `checkpoint_id` key, the checkpoint with
|
||||
provided config. If the config contains a "checkpoint_id" key, the checkpoint with
|
||||
the matching thread ID and timestamp is retrieved. Otherwise, the latest checkpoint
|
||||
for the given thread ID is retrieved.
|
||||
|
||||
Args:
|
||||
config: The config to use for retrieving the checkpoint.
|
||||
config (CheckpointConfig): The config to use for retrieving the checkpoint.
|
||||
|
||||
Returns:
|
||||
The retrieved checkpoint tuple, or None if no matching checkpoint was found.
|
||||
Optional[CheckpointTuple]: The retrieved checkpoint tuple, or None if no matching checkpoint was found.
|
||||
"""
|
||||
thread_id: str = config["configurable"]["thread_id"]
|
||||
checkpoint_ns: str = config["configurable"].get("checkpoint_ns", "")
|
||||
thread_id = config["configurable"]["thread_id"]
|
||||
checkpoint_ns = config["configurable"].get("checkpoint_ns", "")
|
||||
if checkpoint_id := get_checkpoint_id(config):
|
||||
if saved := self.storage[thread_id][checkpoint_ns].get(checkpoint_id):
|
||||
checkpoint, metadata, parent_checkpoint_id = saved
|
||||
writes = self.writes[(thread_id, checkpoint_ns, checkpoint_id)].values()
|
||||
checkpoint_: Checkpoint = self.serde.loads_typed(checkpoint)
|
||||
if parent_checkpoint_id:
|
||||
sends = sorted(
|
||||
(
|
||||
(*w, k[1])
|
||||
for k, w in self.writes[
|
||||
(thread_id, checkpoint_ns, parent_checkpoint_id)
|
||||
].items()
|
||||
if w[1] == TASKS
|
||||
),
|
||||
key=lambda w: (w[3], w[0], w[4]),
|
||||
)
|
||||
else:
|
||||
sends = []
|
||||
return CheckpointTuple(
|
||||
config=config,
|
||||
checkpoint={
|
||||
**checkpoint_,
|
||||
"channel_values": self._load_blobs(
|
||||
thread_id, checkpoint_ns, checkpoint_["channel_versions"]
|
||||
),
|
||||
**self.serde.loads_typed(checkpoint),
|
||||
"pending_sends": [self.serde.loads_typed(s[2]) for s in sends],
|
||||
},
|
||||
metadata=self.serde.loads_typed(metadata),
|
||||
pending_writes=[
|
||||
@@ -182,7 +153,19 @@ class InMemorySaver(
|
||||
checkpoint_id = max(checkpoints.keys())
|
||||
checkpoint, metadata, parent_checkpoint_id = checkpoints[checkpoint_id]
|
||||
writes = self.writes[(thread_id, checkpoint_ns, checkpoint_id)].values()
|
||||
checkpoint_ = self.serde.loads_typed(checkpoint)
|
||||
if parent_checkpoint_id:
|
||||
sends = sorted(
|
||||
(
|
||||
(*w, k[1])
|
||||
for k, w in self.writes[
|
||||
(thread_id, checkpoint_ns, parent_checkpoint_id)
|
||||
].items()
|
||||
if w[1] == TASKS
|
||||
),
|
||||
key=lambda w: (w[3], w[0], w[4]),
|
||||
)
|
||||
else:
|
||||
sends = []
|
||||
return CheckpointTuple(
|
||||
config={
|
||||
"configurable": {
|
||||
@@ -192,10 +175,8 @@ class InMemorySaver(
|
||||
}
|
||||
},
|
||||
checkpoint={
|
||||
**checkpoint_,
|
||||
"channel_values": self._load_blobs(
|
||||
thread_id, checkpoint_ns, checkpoint_["channel_versions"]
|
||||
),
|
||||
**self.serde.loads_typed(checkpoint),
|
||||
"pending_sends": [self.serde.loads_typed(s[2]) for s in sends],
|
||||
},
|
||||
metadata=self.serde.loads_typed(metadata),
|
||||
pending_writes=[
|
||||
@@ -216,11 +197,11 @@ class InMemorySaver(
|
||||
|
||||
def list(
|
||||
self,
|
||||
config: RunnableConfig | None,
|
||||
config: Optional[CheckpointConfig],
|
||||
*,
|
||||
filter: dict[str, Any] | None = None,
|
||||
before: RunnableConfig | None = None,
|
||||
limit: int | None = None,
|
||||
filter: Optional[dict[str, Any]] = None,
|
||||
before: Optional[CheckpointConfig] = None,
|
||||
limit: Optional[int] = None,
|
||||
) -> Iterator[CheckpointTuple]:
|
||||
"""List checkpoints from the in-memory storage.
|
||||
|
||||
@@ -228,13 +209,13 @@ class InMemorySaver(
|
||||
on the provided criteria.
|
||||
|
||||
Args:
|
||||
config: Base configuration for filtering checkpoints.
|
||||
filter: Additional filtering criteria for metadata.
|
||||
before: List checkpoints created before this configuration.
|
||||
limit: Maximum number of checkpoints to return.
|
||||
config (Optional[CheckpointConfig]): Base configuration for filtering checkpoints.
|
||||
filter (Optional[Dict[str, Any]]): Additional filtering criteria for metadata.
|
||||
before (Optional[CheckpointConfig]): List checkpoints created before this configuration.
|
||||
limit (Optional[int]): Maximum number of checkpoints to return.
|
||||
|
||||
Yields:
|
||||
An iterator of matching checkpoint tuples.
|
||||
Iterator[CheckpointTuple]: An iterator of matching checkpoint tuples.
|
||||
"""
|
||||
thread_ids = (config["configurable"]["thread_id"],) if config else self.storage
|
||||
config_checkpoint_ns = (
|
||||
@@ -288,7 +269,19 @@ class InMemorySaver(
|
||||
(thread_id, checkpoint_ns, checkpoint_id)
|
||||
].values()
|
||||
|
||||
checkpoint_: Checkpoint = self.serde.loads_typed(checkpoint)
|
||||
if parent_checkpoint_id:
|
||||
sends = sorted(
|
||||
(
|
||||
(*w, k[1])
|
||||
for k, w in self.writes[
|
||||
(thread_id, checkpoint_ns, parent_checkpoint_id)
|
||||
].items()
|
||||
if w[1] == TASKS
|
||||
),
|
||||
key=lambda w: (w[3], w[0], w[4]),
|
||||
)
|
||||
else:
|
||||
sends = []
|
||||
|
||||
yield CheckpointTuple(
|
||||
config={
|
||||
@@ -299,12 +292,10 @@ class InMemorySaver(
|
||||
}
|
||||
},
|
||||
checkpoint={
|
||||
**checkpoint_,
|
||||
"channel_values": self._load_blobs(
|
||||
thread_id,
|
||||
checkpoint_ns,
|
||||
checkpoint_["channel_versions"],
|
||||
),
|
||||
**self.serde.loads_typed(checkpoint),
|
||||
"pending_sends": [
|
||||
self.serde.loads_typed(s[2]) for s in sends
|
||||
],
|
||||
},
|
||||
metadata=metadata,
|
||||
parent_config=(
|
||||
@@ -325,33 +316,29 @@ class InMemorySaver(
|
||||
|
||||
def put(
|
||||
self,
|
||||
config: RunnableConfig,
|
||||
config: CheckpointConfig,
|
||||
checkpoint: Checkpoint,
|
||||
metadata: CheckpointMetadata,
|
||||
new_versions: ChannelVersions,
|
||||
) -> RunnableConfig:
|
||||
) -> CheckpointConfig:
|
||||
"""Save a checkpoint to the in-memory storage.
|
||||
|
||||
This method saves a checkpoint to the in-memory storage. The checkpoint is associated
|
||||
with the provided config.
|
||||
|
||||
Args:
|
||||
config: The config to associate with the checkpoint.
|
||||
checkpoint: The checkpoint to save.
|
||||
metadata: Additional metadata to save with the checkpoint.
|
||||
new_versions: New versions as of this write
|
||||
config (CheckpointConfig): The config to associate with the checkpoint.
|
||||
checkpoint (Checkpoint): The checkpoint to save.
|
||||
metadata (CheckpointMetadata): Additional metadata to save with the checkpoint.
|
||||
new_versions (dict): New versions as of this write
|
||||
|
||||
Returns:
|
||||
RunnableConfig: The updated config containing the saved checkpoint's timestamp.
|
||||
CheckpointConfig: The updated config containing the saved checkpoint's timestamp.
|
||||
"""
|
||||
c = checkpoint.copy()
|
||||
c.pop("pending_sends") # type: ignore[misc]
|
||||
thread_id = config["configurable"]["thread_id"]
|
||||
checkpoint_ns = config["configurable"]["checkpoint_ns"]
|
||||
values: dict[str, Any] = c.pop("channel_values") # type: ignore[misc]
|
||||
for k, v in new_versions.items():
|
||||
self.blobs[(thread_id, checkpoint_ns, k, v)] = (
|
||||
self.serde.dumps_typed(values[k]) if k in values else ("empty", b"")
|
||||
)
|
||||
self.storage[thread_id][checkpoint_ns].update(
|
||||
{
|
||||
checkpoint["id"]: (
|
||||
@@ -371,7 +358,7 @@ class InMemorySaver(
|
||||
|
||||
def put_writes(
|
||||
self,
|
||||
config: RunnableConfig,
|
||||
config: CheckpointConfig,
|
||||
writes: Sequence[tuple[str, Any]],
|
||||
task_id: str,
|
||||
task_path: str = "",
|
||||
@@ -382,13 +369,13 @@ class InMemorySaver(
|
||||
with the provided config.
|
||||
|
||||
Args:
|
||||
config: The config to associate with the writes.
|
||||
writes: The writes to save.
|
||||
task_id: Identifier for the task creating the writes.
|
||||
task_path: Path of the task creating the writes.
|
||||
config (CheckpointConfig): The config to associate with the writes.
|
||||
writes (list[tuple[str, Any]]): The writes to save.
|
||||
task_id (str): Identifier for the task creating the writes.
|
||||
task_path (str): Path of the task creating the writes.
|
||||
|
||||
Returns:
|
||||
RunnableConfig: The updated config containing the saved writes' timestamp.
|
||||
CheckpointConfig: The updated config containing the saved writes' timestamp.
|
||||
"""
|
||||
thread_id = config["configurable"]["thread_id"]
|
||||
checkpoint_ns = config["configurable"].get("checkpoint_ns", "")
|
||||
@@ -407,115 +394,7 @@ class InMemorySaver(
|
||||
task_path,
|
||||
)
|
||||
|
||||
def delete_thread(self, thread_id: str) -> None:
|
||||
"""Delete all checkpoints and writes associated with a thread ID.
|
||||
|
||||
Args:
|
||||
thread_id: The thread ID to delete.
|
||||
|
||||
Returns:
|
||||
None
|
||||
"""
|
||||
if thread_id in self.storage:
|
||||
del self.storage[thread_id]
|
||||
for k in list(self.writes.keys()):
|
||||
if k[0] == thread_id:
|
||||
del self.writes[k]
|
||||
for k in list(self.blobs.keys()):
|
||||
if k[0] == thread_id:
|
||||
del self.blobs[k]
|
||||
|
||||
async def aget_tuple(self, config: RunnableConfig) -> CheckpointTuple | None:
|
||||
"""Asynchronous version of `get_tuple`.
|
||||
|
||||
This method is an asynchronous wrapper around `get_tuple` that runs the synchronous
|
||||
method in a separate thread using asyncio.
|
||||
|
||||
Args:
|
||||
config: The config to use for retrieving the checkpoint.
|
||||
|
||||
Returns:
|
||||
The retrieved checkpoint tuple, or None if no matching checkpoint was found.
|
||||
"""
|
||||
return self.get_tuple(config)
|
||||
|
||||
async def alist(
|
||||
self,
|
||||
config: RunnableConfig | None,
|
||||
*,
|
||||
filter: dict[str, Any] | None = None,
|
||||
before: RunnableConfig | None = None,
|
||||
limit: int | None = None,
|
||||
) -> AsyncIterator[CheckpointTuple]:
|
||||
"""Asynchronous version of `list`.
|
||||
|
||||
This method is an asynchronous wrapper around `list` that runs the synchronous
|
||||
method in a separate thread using asyncio.
|
||||
|
||||
Args:
|
||||
config: The config to use for listing the checkpoints.
|
||||
|
||||
Yields:
|
||||
An asynchronous iterator of checkpoint tuples.
|
||||
"""
|
||||
for item in self.list(config, filter=filter, before=before, limit=limit):
|
||||
yield item
|
||||
|
||||
async def aput(
|
||||
self,
|
||||
config: RunnableConfig,
|
||||
checkpoint: Checkpoint,
|
||||
metadata: CheckpointMetadata,
|
||||
new_versions: ChannelVersions,
|
||||
) -> RunnableConfig:
|
||||
"""Asynchronous version of `put`.
|
||||
|
||||
Args:
|
||||
config: The config to associate with the checkpoint.
|
||||
checkpoint: The checkpoint to save.
|
||||
metadata: Additional metadata to save with the checkpoint.
|
||||
new_versions: New versions as of this write
|
||||
|
||||
Returns:
|
||||
RunnableConfig: The updated config containing the saved checkpoint's timestamp.
|
||||
"""
|
||||
return self.put(config, checkpoint, metadata, new_versions)
|
||||
|
||||
async def aput_writes(
|
||||
self,
|
||||
config: RunnableConfig,
|
||||
writes: Sequence[tuple[str, Any]],
|
||||
task_id: str,
|
||||
task_path: str = "",
|
||||
) -> None:
|
||||
"""Asynchronous version of `put_writes`.
|
||||
|
||||
This method is an asynchronous wrapper around `put_writes` that runs the synchronous
|
||||
method in a separate thread using asyncio.
|
||||
|
||||
Args:
|
||||
config: The config to associate with the writes.
|
||||
writes: The writes to save, each as a (channel, value) pair.
|
||||
task_id: Identifier for the task creating the writes.
|
||||
task_path: Path of the task creating the writes.
|
||||
|
||||
Returns:
|
||||
None
|
||||
"""
|
||||
return self.put_writes(config, writes, task_id, task_path)
|
||||
|
||||
async def adelete_thread(self, thread_id: str) -> None:
|
||||
"""Delete all checkpoints and writes associated with a thread ID.
|
||||
|
||||
Args:
|
||||
thread_id: The thread ID to delete.
|
||||
|
||||
Returns:
|
||||
None
|
||||
"""
|
||||
return self.delete_thread(thread_id)
|
||||
|
||||
def get_next_version(self, current: str | None, channel: None) -> str:
|
||||
def get_next_version(self, current: Optional[str], channel: ChannelProtocol) -> str:
|
||||
if current is None:
|
||||
current_v = 0
|
||||
elif isinstance(current, int):
|
||||
@@ -574,7 +453,7 @@ class PersistentDict(defaultdict):
|
||||
self.sync()
|
||||
self.clear()
|
||||
|
||||
def __enter__(self) -> PersistentDict:
|
||||
def __enter__(self) -> "PersistentDict":
|
||||
return self
|
||||
|
||||
def __exit__(self, *exc_info: Any) -> None:
|
||||
@@ -598,6 +477,6 @@ class PersistentDict(defaultdict):
|
||||
except EOFError:
|
||||
return
|
||||
except Exception:
|
||||
logger.error(f"Failed to load file: {fileobj.name}")
|
||||
logging.error(f"Failed to load file: {fileobj.name}")
|
||||
raise
|
||||
raise ValueError("File not in a supported format")
|
||||
@@ -1,35 +1,36 @@
|
||||
from __future__ import annotations
|
||||
|
||||
from typing import Any, Protocol, runtime_checkable
|
||||
from typing import Any, Protocol
|
||||
|
||||
|
||||
class UntypedSerializerProtocol(Protocol):
|
||||
"""Protocol for serialization and deserialization of objects."""
|
||||
|
||||
def dumps(self, obj: Any) -> bytes: ...
|
||||
|
||||
def loads(self, data: bytes) -> Any: ...
|
||||
|
||||
|
||||
@runtime_checkable
|
||||
class SerializerProtocol(Protocol):
|
||||
"""Protocol for serialization and deserialization of objects.
|
||||
|
||||
- `dumps_typed`: Serialize an object to a tuple `(type, bytes)`.
|
||||
- `loads_typed`: Deserialize an object from a tuple `(type, bytes)`.
|
||||
- `dumps`: Serialize an object to bytes.
|
||||
- `dumps_typed`: Serialize an object to a tuple (type, bytes).
|
||||
- `loads`: Deserialize an object from bytes.
|
||||
- `loads_typed`: Deserialize an object from a tuple (type, bytes).
|
||||
|
||||
Valid implementations include the `pickle`, `json` and `orjson` modules.
|
||||
"""
|
||||
|
||||
def dumps(self, obj: Any) -> bytes: ...
|
||||
|
||||
def dumps_typed(self, obj: Any) -> tuple[str, bytes]: ...
|
||||
|
||||
def loads(self, data: bytes) -> Any: ...
|
||||
|
||||
def loads_typed(self, data: tuple[str, bytes]) -> Any: ...
|
||||
|
||||
|
||||
class SerializerCompat(SerializerProtocol):
|
||||
def __init__(self, serde: UntypedSerializerProtocol) -> None:
|
||||
def __init__(self, serde: SerializerProtocol) -> None:
|
||||
self.serde = serde
|
||||
|
||||
def dumps(self, obj: Any) -> bytes:
|
||||
return self.serde.dumps(obj)
|
||||
|
||||
def loads(self, data: bytes) -> Any:
|
||||
return self.serde.loads(data)
|
||||
|
||||
def dumps_typed(self, obj: Any) -> tuple[str, bytes]:
|
||||
return type(obj).__name__, self.serde.dumps(obj)
|
||||
|
||||
@@ -37,28 +38,10 @@ class SerializerCompat(SerializerProtocol):
|
||||
return self.serde.loads(data[1])
|
||||
|
||||
|
||||
def maybe_add_typed_methods(
|
||||
serde: SerializerProtocol | UntypedSerializerProtocol,
|
||||
) -> SerializerProtocol:
|
||||
def maybe_add_typed_methods(serde: SerializerProtocol) -> SerializerProtocol:
|
||||
"""Wrap serde old serde implementations in a class with loads_typed and dumps_typed for backwards compatibility."""
|
||||
|
||||
if not isinstance(serde, SerializerProtocol):
|
||||
if not hasattr(serde, "loads_typed") or not hasattr(serde, "dumps_typed"):
|
||||
return SerializerCompat(serde)
|
||||
|
||||
return serde
|
||||
|
||||
|
||||
class CipherProtocol(Protocol):
|
||||
"""Protocol for encryption and decryption of data.
|
||||
|
||||
- `encrypt`: Encrypt plaintext.
|
||||
- `decrypt`: Decrypt ciphertext.
|
||||
"""
|
||||
|
||||
def encrypt(self, plaintext: bytes) -> tuple[str, bytes]:
|
||||
"""Encrypt plaintext. Returns a tuple `(cipher name, ciphertext)`."""
|
||||
...
|
||||
|
||||
def decrypt(self, ciphername: str, ciphertext: bytes) -> bytes:
|
||||
"""Decrypt ciphertext. Returns the plaintext."""
|
||||
...
|
||||
@@ -1,80 +0,0 @@
|
||||
import os
|
||||
from typing import Any
|
||||
|
||||
from langgraph.checkpoint.serde.base import CipherProtocol, SerializerProtocol
|
||||
from langgraph.checkpoint.serde.jsonplus import JsonPlusSerializer
|
||||
|
||||
|
||||
class EncryptedSerializer(SerializerProtocol):
|
||||
"""Serializer that encrypts and decrypts data using an encryption protocol."""
|
||||
|
||||
def __init__(
|
||||
self, cipher: CipherProtocol, serde: SerializerProtocol = JsonPlusSerializer()
|
||||
) -> None:
|
||||
self.cipher = cipher
|
||||
self.serde = serde
|
||||
|
||||
def dumps_typed(self, obj: Any) -> tuple[str, bytes]:
|
||||
"""Serialize an object to a tuple `(type, bytes)` and encrypt the bytes."""
|
||||
# serialize data
|
||||
typ, data = self.serde.dumps_typed(obj)
|
||||
# encrypt data
|
||||
ciphername, ciphertext = self.cipher.encrypt(data)
|
||||
# add cipher name to type
|
||||
return f"{typ}+{ciphername}", ciphertext
|
||||
|
||||
def loads_typed(self, data: tuple[str, bytes]) -> Any:
|
||||
enc_cipher, ciphertext = data
|
||||
# unencrypted data
|
||||
if "+" not in enc_cipher:
|
||||
return self.serde.loads_typed(data)
|
||||
# extract cipher name
|
||||
typ, ciphername = enc_cipher.split("+", 1)
|
||||
# decrypt data
|
||||
decrypted_data = self.cipher.decrypt(ciphername, ciphertext)
|
||||
# deserialize data
|
||||
return self.serde.loads_typed((typ, decrypted_data))
|
||||
|
||||
@classmethod
|
||||
def from_pycryptodome_aes(
|
||||
cls, serde: SerializerProtocol = JsonPlusSerializer(), **kwargs: Any
|
||||
) -> "EncryptedSerializer":
|
||||
"""Create an `EncryptedSerializer` using AES encryption."""
|
||||
try:
|
||||
from Crypto.Cipher import AES # type: ignore
|
||||
except ImportError:
|
||||
raise ImportError(
|
||||
"Pycryptodome is not installed. Please install it with `pip install pycryptodome`."
|
||||
) from None
|
||||
|
||||
# check if AES key is provided
|
||||
if "key" in kwargs:
|
||||
key: bytes = kwargs.pop("key")
|
||||
else:
|
||||
key_str = os.getenv("LANGGRAPH_AES_KEY")
|
||||
if key_str is None:
|
||||
raise ValueError("LANGGRAPH_AES_KEY environment variable is not set.")
|
||||
key = key_str.encode()
|
||||
if len(key) not in (16, 24, 32):
|
||||
raise ValueError("LANGGRAPH_AES_KEY must be 16, 24, or 32 bytes long.")
|
||||
|
||||
# set default mode to EAX if not provided
|
||||
if kwargs.get("mode") is None:
|
||||
kwargs["mode"] = AES.MODE_EAX
|
||||
|
||||
class PycryptodomeAesCipher(CipherProtocol):
|
||||
def encrypt(self, plaintext: bytes) -> tuple[str, bytes]:
|
||||
cipher = AES.new(key, **kwargs)
|
||||
ciphertext, tag = cipher.encrypt_and_digest(plaintext)
|
||||
return "aes", cipher.nonce + tag + ciphertext
|
||||
|
||||
def decrypt(self, ciphername: str, ciphertext: bytes) -> bytes:
|
||||
assert ciphername == "aes", f"Unsupported cipher: {ciphername}"
|
||||
nonce = ciphertext[:16]
|
||||
tag = ciphertext[16:32]
|
||||
actual_ciphertext = ciphertext[32:]
|
||||
|
||||
cipher = AES.new(key, **kwargs, nonce=nonce)
|
||||
return cipher.decrypt_and_verify(actual_ciphertext, tag)
|
||||
|
||||
return cls(PycryptodomeAesCipher(), serde)
|
||||
@@ -1,16 +1,11 @@
|
||||
from __future__ import annotations
|
||||
|
||||
import dataclasses
|
||||
import decimal
|
||||
import importlib
|
||||
import json
|
||||
import logging
|
||||
import pathlib
|
||||
import pickle
|
||||
import re
|
||||
import sys
|
||||
from collections import deque
|
||||
from collections.abc import Callable, Sequence
|
||||
from collections.abc import Sequence
|
||||
from datetime import date, datetime, time, timedelta, timezone
|
||||
from enum import Enum
|
||||
from inspect import isclass
|
||||
@@ -22,59 +17,25 @@ from ipaddress import (
|
||||
IPv6Interface,
|
||||
IPv6Network,
|
||||
)
|
||||
from typing import Any, Literal
|
||||
from typing import Any, Callable, Optional, Union
|
||||
from uuid import UUID
|
||||
from zoneinfo import ZoneInfo
|
||||
|
||||
import ormsgpack
|
||||
from langchain_core.load.load import Reviver
|
||||
import msgpack # type: ignore[import-untyped]
|
||||
from zoneinfo import ZoneInfo
|
||||
|
||||
from langgraph.checkpoint.serde.base import SerializerProtocol
|
||||
from langgraph.checkpoint.serde.types import SendProtocol
|
||||
from langgraph.store.base import Item
|
||||
|
||||
LC_REVIVER = Reviver()
|
||||
EMPTY_BYTES = b""
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
class JsonPlusSerializer(SerializerProtocol):
|
||||
"""Serializer that uses ormsgpack, with optional fallbacks.
|
||||
|
||||
!!! warning
|
||||
|
||||
Security note: This serializer is intended for use within the `BaseCheckpointSaver`
|
||||
class and called within the Pregel loop. It should not be used on untrusted
|
||||
python objects. If an attacker can write directly to your checkpoint database,
|
||||
they may be able to trigger code execution when data is deserialized.
|
||||
"""
|
||||
|
||||
def __init__(
|
||||
self,
|
||||
*,
|
||||
pickle_fallback: bool = False,
|
||||
allowed_json_modules: Sequence[tuple[str, ...]] | Literal[True] | None = None,
|
||||
__unpack_ext_hook__: Callable[[int, bytes], Any] | None = None,
|
||||
) -> None:
|
||||
self.pickle_fallback = pickle_fallback
|
||||
self._allowed_modules = (
|
||||
{mod_and_name for mod_and_name in allowed_json_modules}
|
||||
if allowed_json_modules and allowed_json_modules is not True
|
||||
else (allowed_json_modules if allowed_json_modules is True else None)
|
||||
)
|
||||
self._unpack_ext_hook = (
|
||||
__unpack_ext_hook__
|
||||
if __unpack_ext_hook__ is not None
|
||||
else _msgpack_ext_hook
|
||||
)
|
||||
|
||||
def _encode_constructor_args(
|
||||
self,
|
||||
constructor: Callable | type[Any],
|
||||
constructor: Union[Callable, type[Any]],
|
||||
*,
|
||||
method: None | str | Sequence[None | str] = None,
|
||||
args: Sequence[Any] | None = None,
|
||||
kwargs: dict[str, Any] | None = None,
|
||||
method: Union[None, str, Sequence[Union[None, str]]] = None,
|
||||
args: Optional[Sequence[Any]] = None,
|
||||
kwargs: Optional[dict[str, Any]] = None,
|
||||
) -> dict[str, Any]:
|
||||
out = {
|
||||
"lc": 2,
|
||||
@@ -89,123 +50,160 @@ class JsonPlusSerializer(SerializerProtocol):
|
||||
out["kwargs"] = kwargs
|
||||
return out
|
||||
|
||||
def _default(self, obj: Any) -> Union[str, dict[str, Any]]:
|
||||
if hasattr(obj, "model_dump") and callable(obj.model_dump):
|
||||
return self._encode_constructor_args(
|
||||
obj.__class__, method=(None, "model_construct"), kwargs=obj.model_dump()
|
||||
)
|
||||
elif hasattr(obj, "dict") and callable(obj.dict):
|
||||
return self._encode_constructor_args(
|
||||
obj.__class__, method=(None, "construct"), kwargs=obj.dict()
|
||||
)
|
||||
elif hasattr(obj, "_asdict") and callable(obj._asdict):
|
||||
return self._encode_constructor_args(obj.__class__, kwargs=obj._asdict())
|
||||
elif isinstance(obj, pathlib.Path):
|
||||
return self._encode_constructor_args(pathlib.Path, args=obj.parts)
|
||||
elif isinstance(obj, re.Pattern):
|
||||
return self._encode_constructor_args(
|
||||
re.compile, args=(obj.pattern, obj.flags)
|
||||
)
|
||||
elif isinstance(obj, UUID):
|
||||
return self._encode_constructor_args(UUID, args=(obj.hex,))
|
||||
elif isinstance(obj, decimal.Decimal):
|
||||
return self._encode_constructor_args(decimal.Decimal, args=(str(obj),))
|
||||
elif isinstance(obj, (set, frozenset, deque)):
|
||||
return self._encode_constructor_args(type(obj), args=(tuple(obj),))
|
||||
elif isinstance(obj, (IPv4Address, IPv4Interface, IPv4Network)):
|
||||
return self._encode_constructor_args(obj.__class__, args=(str(obj),))
|
||||
elif isinstance(obj, (IPv6Address, IPv6Interface, IPv6Network)):
|
||||
return self._encode_constructor_args(obj.__class__, args=(str(obj),))
|
||||
|
||||
elif isinstance(obj, datetime):
|
||||
return self._encode_constructor_args(
|
||||
datetime, method="fromisoformat", args=(obj.isoformat(),)
|
||||
)
|
||||
elif isinstance(obj, timezone):
|
||||
return self._encode_constructor_args(
|
||||
timezone,
|
||||
args=obj.__getinitargs__(), # type: ignore[attr-defined]
|
||||
)
|
||||
elif isinstance(obj, ZoneInfo):
|
||||
return self._encode_constructor_args(ZoneInfo, args=(obj.key,))
|
||||
elif isinstance(obj, timedelta):
|
||||
return self._encode_constructor_args(
|
||||
timedelta, args=(obj.days, obj.seconds, obj.microseconds)
|
||||
)
|
||||
elif isinstance(obj, date):
|
||||
return self._encode_constructor_args(
|
||||
date, args=(obj.year, obj.month, obj.day)
|
||||
)
|
||||
elif isinstance(obj, time):
|
||||
return self._encode_constructor_args(
|
||||
time,
|
||||
args=(obj.hour, obj.minute, obj.second, obj.microsecond, obj.tzinfo),
|
||||
kwargs={"fold": obj.fold},
|
||||
)
|
||||
elif dataclasses.is_dataclass(obj):
|
||||
return self._encode_constructor_args(
|
||||
obj.__class__,
|
||||
kwargs={
|
||||
field.name: getattr(obj, field.name)
|
||||
for field in dataclasses.fields(obj)
|
||||
},
|
||||
)
|
||||
elif isinstance(obj, Enum):
|
||||
return self._encode_constructor_args(obj.__class__, args=(obj.value,))
|
||||
elif isinstance(obj, SendProtocol):
|
||||
return self._encode_constructor_args(
|
||||
obj.__class__, kwargs={"node": obj.node, "arg": obj.arg}
|
||||
)
|
||||
elif isinstance(obj, (bytes, bytearray)):
|
||||
return self._encode_constructor_args(
|
||||
obj.__class__, method="fromhex", args=(obj.hex(),)
|
||||
)
|
||||
elif isinstance(obj, BaseException):
|
||||
return repr(obj)
|
||||
else:
|
||||
raise TypeError(
|
||||
f"Object of type {obj.__class__.__name__} is not JSON serializable"
|
||||
)
|
||||
|
||||
def _reviver(self, value: dict[str, Any]) -> Any:
|
||||
if self._allowed_modules and (
|
||||
if (
|
||||
value.get("lc", None) == 2
|
||||
and value.get("type", None) == "constructor"
|
||||
and value.get("id", None) is not None
|
||||
):
|
||||
try:
|
||||
return self._revive_lc2(value)
|
||||
except InvalidModuleError as e:
|
||||
logger.warning(
|
||||
"Object %s is not in the deserialization allowlist.\n%s",
|
||||
value["id"],
|
||||
e.message,
|
||||
)
|
||||
# Get module and class name
|
||||
[*module, name] = value["id"]
|
||||
# Import module
|
||||
mod = importlib.import_module(".".join(module))
|
||||
# Import class
|
||||
cls = getattr(mod, name)
|
||||
# Instantiate class
|
||||
method = value.get("method")
|
||||
if isinstance(method, str):
|
||||
methods = [getattr(cls, method)]
|
||||
elif isinstance(method, list):
|
||||
methods = [
|
||||
cls if method is None else getattr(cls, method)
|
||||
for method in method
|
||||
]
|
||||
else:
|
||||
methods = [cls]
|
||||
args = value.get("args")
|
||||
kwargs = value.get("kwargs")
|
||||
for method in methods:
|
||||
try:
|
||||
if isclass(method) and issubclass(method, BaseException):
|
||||
return None
|
||||
if args and kwargs:
|
||||
return method(*args, **kwargs)
|
||||
elif args:
|
||||
return method(*args)
|
||||
elif kwargs:
|
||||
return method(**kwargs)
|
||||
else:
|
||||
return method()
|
||||
except Exception:
|
||||
continue
|
||||
except Exception:
|
||||
return None
|
||||
|
||||
return LC_REVIVER(value)
|
||||
return value
|
||||
|
||||
def _revive_lc2(self, value: dict[str, Any]) -> Any:
|
||||
self._check_allowed_modules(value)
|
||||
|
||||
[*module, name] = value["id"]
|
||||
try:
|
||||
mod = importlib.import_module(".".join(module))
|
||||
cls = getattr(mod, name)
|
||||
method = value.get("method")
|
||||
if isinstance(method, str):
|
||||
methods = [getattr(cls, method)]
|
||||
elif isinstance(method, list):
|
||||
methods = [cls if m is None else getattr(cls, m) for m in method]
|
||||
else:
|
||||
methods = [cls]
|
||||
args = value.get("args")
|
||||
kwargs = value.get("kwargs")
|
||||
for method in methods:
|
||||
try:
|
||||
if isclass(method) and issubclass(method, BaseException):
|
||||
return None
|
||||
if args and kwargs:
|
||||
return method(*args, **kwargs)
|
||||
elif args:
|
||||
return method(*args)
|
||||
elif kwargs:
|
||||
return method(**kwargs)
|
||||
else:
|
||||
return method()
|
||||
except Exception:
|
||||
continue
|
||||
except Exception:
|
||||
return None
|
||||
|
||||
def _check_allowed_modules(self, value: dict[str, Any]) -> None:
|
||||
needed = tuple(value["id"])
|
||||
method = value.get("method")
|
||||
if isinstance(method, list):
|
||||
method_display = ",".join(m or "<init>" for m in method)
|
||||
elif isinstance(method, str):
|
||||
method_display = method
|
||||
else:
|
||||
method_display = "<init>"
|
||||
|
||||
dotted = ".".join(needed)
|
||||
if not self._allowed_modules:
|
||||
raise InvalidModuleError(
|
||||
f"Refused to deserialize JSON constructor: {dotted} (method: {method_display}). "
|
||||
"No allowed_json_modules configured.\n\n"
|
||||
"Unblock with ONE of:\n"
|
||||
f" • JsonPlusSerializer(allowed_json_modules=[{needed!r}, ...])\n"
|
||||
" • (DANGEROUS) JsonPlusSerializer(allowed_json_modules=True)\n\n"
|
||||
"Note: Prefix allowlists are intentionally unsupported; prefer exact symbols "
|
||||
"or plain-JSON representations revived without import-time side effects."
|
||||
)
|
||||
|
||||
if self._allowed_modules is True:
|
||||
return
|
||||
if needed in self._allowed_modules:
|
||||
return
|
||||
|
||||
raise InvalidModuleError(
|
||||
f"Refused to deserialize JSON constructor: {dotted} (method: {method_display}). "
|
||||
"Symbol is not in the deserialization allowlist.\n\n"
|
||||
"Add exactly this symbol to unblock:\n"
|
||||
f" JsonPlusSerializer(allowed_json_modules=[{needed!r}, ...])\n"
|
||||
"Or, as a last resort (DANGEROUS):\n"
|
||||
" JsonPlusSerializer(allowed_json_modules=True)"
|
||||
def dumps(self, obj: Any) -> bytes:
|
||||
return json.dumps(obj, default=self._default, ensure_ascii=False).encode(
|
||||
"utf-8", "ignore"
|
||||
)
|
||||
|
||||
def dumps_typed(self, obj: Any) -> tuple[str, bytes]:
|
||||
if obj is None:
|
||||
return "null", EMPTY_BYTES
|
||||
elif isinstance(obj, bytes):
|
||||
if isinstance(obj, bytes):
|
||||
return "bytes", obj
|
||||
elif isinstance(obj, bytearray):
|
||||
return "bytearray", obj
|
||||
else:
|
||||
try:
|
||||
return "msgpack", _msgpack_enc(obj)
|
||||
except ormsgpack.MsgpackEncodeError as exc:
|
||||
if self.pickle_fallback:
|
||||
return "pickle", pickle.dumps(obj)
|
||||
raise exc
|
||||
except UnicodeEncodeError:
|
||||
return "json", self.dumps(obj)
|
||||
|
||||
def loads(self, data: bytes) -> Any:
|
||||
return json.loads(data, object_hook=self._reviver)
|
||||
|
||||
def loads_typed(self, data: tuple[str, bytes]) -> Any:
|
||||
type_, data_ = data
|
||||
if type_ == "null":
|
||||
return None
|
||||
elif type_ == "bytes":
|
||||
if type_ == "bytes":
|
||||
return data_
|
||||
elif type_ == "bytearray":
|
||||
return bytearray(data_)
|
||||
elif type_ == "json":
|
||||
return json.loads(data_, object_hook=self._reviver)
|
||||
return self.loads(data_)
|
||||
elif type_ == "msgpack":
|
||||
return ormsgpack.unpackb(
|
||||
data_, ext_hook=self._unpack_ext_hook, option=ormsgpack.OPT_NON_STR_KEYS
|
||||
return msgpack.unpackb(
|
||||
data_, ext_hook=_msgpack_ext_hook, strict_map_key=False
|
||||
)
|
||||
elif self.pickle_fallback and type_ == "pickle":
|
||||
return pickle.loads(data_)
|
||||
else:
|
||||
raise NotImplementedError(f"Unknown serialization type: {type_}")
|
||||
|
||||
@@ -218,12 +216,11 @@ EXT_CONSTRUCTOR_KW_ARGS = 2
|
||||
EXT_METHOD_SINGLE_ARG = 3
|
||||
EXT_PYDANTIC_V1 = 4
|
||||
EXT_PYDANTIC_V2 = 5
|
||||
EXT_NUMPY_ARRAY = 6
|
||||
|
||||
|
||||
def _msgpack_default(obj: Any) -> str | ormsgpack.Ext:
|
||||
def _msgpack_default(obj: Any) -> Union[str, msgpack.ExtType]:
|
||||
if hasattr(obj, "model_dump") and callable(obj.model_dump): # pydantic v2
|
||||
return ormsgpack.Ext(
|
||||
return msgpack.ExtType(
|
||||
EXT_PYDANTIC_V2,
|
||||
_msgpack_enc(
|
||||
(
|
||||
@@ -235,7 +232,7 @@ def _msgpack_default(obj: Any) -> str | ormsgpack.Ext:
|
||||
),
|
||||
)
|
||||
elif hasattr(obj, "get_secret_value") and callable(obj.get_secret_value):
|
||||
return ormsgpack.Ext(
|
||||
return msgpack.ExtType(
|
||||
EXT_CONSTRUCTOR_SINGLE_ARG,
|
||||
_msgpack_enc(
|
||||
(
|
||||
@@ -246,7 +243,7 @@ def _msgpack_default(obj: Any) -> str | ormsgpack.Ext:
|
||||
),
|
||||
)
|
||||
elif hasattr(obj, "dict") and callable(obj.dict): # pydantic v1
|
||||
return ormsgpack.Ext(
|
||||
return msgpack.ExtType(
|
||||
EXT_PYDANTIC_V1,
|
||||
_msgpack_enc(
|
||||
(
|
||||
@@ -257,7 +254,7 @@ def _msgpack_default(obj: Any) -> str | ormsgpack.Ext:
|
||||
),
|
||||
)
|
||||
elif hasattr(obj, "_asdict") and callable(obj._asdict): # namedtuple
|
||||
return ormsgpack.Ext(
|
||||
return msgpack.ExtType(
|
||||
EXT_CONSTRUCTOR_KW_ARGS,
|
||||
_msgpack_enc(
|
||||
(
|
||||
@@ -268,56 +265,56 @@ def _msgpack_default(obj: Any) -> str | ormsgpack.Ext:
|
||||
),
|
||||
)
|
||||
elif isinstance(obj, pathlib.Path):
|
||||
return ormsgpack.Ext(
|
||||
return msgpack.ExtType(
|
||||
EXT_CONSTRUCTOR_POS_ARGS,
|
||||
_msgpack_enc(
|
||||
(obj.__class__.__module__, obj.__class__.__name__, obj.parts),
|
||||
),
|
||||
)
|
||||
elif isinstance(obj, re.Pattern):
|
||||
return ormsgpack.Ext(
|
||||
return msgpack.ExtType(
|
||||
EXT_CONSTRUCTOR_POS_ARGS,
|
||||
_msgpack_enc(
|
||||
("re", "compile", (obj.pattern, obj.flags)),
|
||||
),
|
||||
)
|
||||
elif isinstance(obj, UUID):
|
||||
return ormsgpack.Ext(
|
||||
return msgpack.ExtType(
|
||||
EXT_CONSTRUCTOR_SINGLE_ARG,
|
||||
_msgpack_enc(
|
||||
(obj.__class__.__module__, obj.__class__.__name__, obj.hex),
|
||||
),
|
||||
)
|
||||
elif isinstance(obj, decimal.Decimal):
|
||||
return ormsgpack.Ext(
|
||||
return msgpack.ExtType(
|
||||
EXT_CONSTRUCTOR_SINGLE_ARG,
|
||||
_msgpack_enc(
|
||||
(obj.__class__.__module__, obj.__class__.__name__, str(obj)),
|
||||
),
|
||||
)
|
||||
elif isinstance(obj, (set, frozenset, deque)):
|
||||
return ormsgpack.Ext(
|
||||
return msgpack.ExtType(
|
||||
EXT_CONSTRUCTOR_SINGLE_ARG,
|
||||
_msgpack_enc(
|
||||
(obj.__class__.__module__, obj.__class__.__name__, tuple(obj)),
|
||||
),
|
||||
)
|
||||
elif isinstance(obj, (IPv4Address, IPv4Interface, IPv4Network)):
|
||||
return ormsgpack.Ext(
|
||||
return msgpack.ExtType(
|
||||
EXT_CONSTRUCTOR_SINGLE_ARG,
|
||||
_msgpack_enc(
|
||||
(obj.__class__.__module__, obj.__class__.__name__, str(obj)),
|
||||
),
|
||||
)
|
||||
elif isinstance(obj, (IPv6Address, IPv6Interface, IPv6Network)):
|
||||
return ormsgpack.Ext(
|
||||
return msgpack.ExtType(
|
||||
EXT_CONSTRUCTOR_SINGLE_ARG,
|
||||
_msgpack_enc(
|
||||
(obj.__class__.__module__, obj.__class__.__name__, str(obj)),
|
||||
),
|
||||
)
|
||||
elif isinstance(obj, datetime):
|
||||
return ormsgpack.Ext(
|
||||
return msgpack.ExtType(
|
||||
EXT_METHOD_SINGLE_ARG,
|
||||
_msgpack_enc(
|
||||
(
|
||||
@@ -329,7 +326,7 @@ def _msgpack_default(obj: Any) -> str | ormsgpack.Ext:
|
||||
),
|
||||
)
|
||||
elif isinstance(obj, timedelta):
|
||||
return ormsgpack.Ext(
|
||||
return msgpack.ExtType(
|
||||
EXT_CONSTRUCTOR_POS_ARGS,
|
||||
_msgpack_enc(
|
||||
(
|
||||
@@ -340,7 +337,7 @@ def _msgpack_default(obj: Any) -> str | ormsgpack.Ext:
|
||||
),
|
||||
)
|
||||
elif isinstance(obj, date):
|
||||
return ormsgpack.Ext(
|
||||
return msgpack.ExtType(
|
||||
EXT_CONSTRUCTOR_POS_ARGS,
|
||||
_msgpack_enc(
|
||||
(
|
||||
@@ -351,7 +348,7 @@ def _msgpack_default(obj: Any) -> str | ormsgpack.Ext:
|
||||
),
|
||||
)
|
||||
elif isinstance(obj, time):
|
||||
return ormsgpack.Ext(
|
||||
return msgpack.ExtType(
|
||||
EXT_CONSTRUCTOR_KW_ARGS,
|
||||
_msgpack_enc(
|
||||
(
|
||||
@@ -369,7 +366,7 @@ def _msgpack_default(obj: Any) -> str | ormsgpack.Ext:
|
||||
),
|
||||
)
|
||||
elif isinstance(obj, timezone):
|
||||
return ormsgpack.Ext(
|
||||
return msgpack.ExtType(
|
||||
EXT_CONSTRUCTOR_POS_ARGS,
|
||||
_msgpack_enc(
|
||||
(
|
||||
@@ -380,21 +377,21 @@ def _msgpack_default(obj: Any) -> str | ormsgpack.Ext:
|
||||
),
|
||||
)
|
||||
elif isinstance(obj, ZoneInfo):
|
||||
return ormsgpack.Ext(
|
||||
return msgpack.ExtType(
|
||||
EXT_CONSTRUCTOR_SINGLE_ARG,
|
||||
_msgpack_enc(
|
||||
(obj.__class__.__module__, obj.__class__.__name__, obj.key),
|
||||
),
|
||||
)
|
||||
elif isinstance(obj, Enum):
|
||||
return ormsgpack.Ext(
|
||||
return msgpack.ExtType(
|
||||
EXT_CONSTRUCTOR_SINGLE_ARG,
|
||||
_msgpack_enc(
|
||||
(obj.__class__.__module__, obj.__class__.__name__, obj.value),
|
||||
),
|
||||
)
|
||||
elif isinstance(obj, SendProtocol):
|
||||
return ormsgpack.Ext(
|
||||
return msgpack.ExtType(
|
||||
EXT_CONSTRUCTOR_POS_ARGS,
|
||||
_msgpack_enc(
|
||||
(obj.__class__.__module__, obj.__class__.__name__, (obj.node, obj.arg)),
|
||||
@@ -402,7 +399,7 @@ def _msgpack_default(obj: Any) -> str | ormsgpack.Ext:
|
||||
)
|
||||
elif dataclasses.is_dataclass(obj):
|
||||
# doesn't use dataclasses.asdict to avoid deepcopy and recursion
|
||||
return ormsgpack.Ext(
|
||||
return msgpack.ExtType(
|
||||
EXT_CONSTRUCTOR_KW_ARGS,
|
||||
_msgpack_enc(
|
||||
(
|
||||
@@ -416,7 +413,7 @@ def _msgpack_default(obj: Any) -> str | ormsgpack.Ext:
|
||||
),
|
||||
)
|
||||
elif isinstance(obj, Item):
|
||||
return ormsgpack.Ext(
|
||||
return msgpack.ExtType(
|
||||
EXT_CONSTRUCTOR_KW_ARGS,
|
||||
_msgpack_enc(
|
||||
(
|
||||
@@ -426,21 +423,6 @@ def _msgpack_default(obj: Any) -> str | ormsgpack.Ext:
|
||||
),
|
||||
),
|
||||
)
|
||||
elif (np_mod := sys.modules.get("numpy")) is not None and isinstance(
|
||||
obj, np_mod.ndarray
|
||||
):
|
||||
order = "F" if obj.flags.f_contiguous and not obj.flags.c_contiguous else "C"
|
||||
if obj.flags.c_contiguous:
|
||||
mv = memoryview(obj)
|
||||
try:
|
||||
meta = (obj.dtype.str, obj.shape, order, mv)
|
||||
return ormsgpack.Ext(EXT_NUMPY_ARRAY, _msgpack_enc(meta))
|
||||
finally:
|
||||
mv.release()
|
||||
else:
|
||||
buf = obj.tobytes(order="A")
|
||||
meta = (obj.dtype.str, obj.shape, order, buf)
|
||||
return ormsgpack.Ext(EXT_NUMPY_ARRAY, _msgpack_enc(meta))
|
||||
|
||||
elif isinstance(obj, BaseException):
|
||||
return repr(obj)
|
||||
@@ -451,8 +433,8 @@ def _msgpack_default(obj: Any) -> str | ormsgpack.Ext:
|
||||
def _msgpack_ext_hook(code: int, data: bytes) -> Any:
|
||||
if code == EXT_CONSTRUCTOR_SINGLE_ARG:
|
||||
try:
|
||||
tup = ormsgpack.unpackb(
|
||||
data, ext_hook=_msgpack_ext_hook, option=ormsgpack.OPT_NON_STR_KEYS
|
||||
tup = msgpack.unpackb(
|
||||
data, ext_hook=_msgpack_ext_hook, strict_map_key=False
|
||||
)
|
||||
# module, name, arg
|
||||
return getattr(importlib.import_module(tup[0]), tup[1])(tup[2])
|
||||
@@ -460,8 +442,8 @@ def _msgpack_ext_hook(code: int, data: bytes) -> Any:
|
||||
return
|
||||
elif code == EXT_CONSTRUCTOR_POS_ARGS:
|
||||
try:
|
||||
tup = ormsgpack.unpackb(
|
||||
data, ext_hook=_msgpack_ext_hook, option=ormsgpack.OPT_NON_STR_KEYS
|
||||
tup = msgpack.unpackb(
|
||||
data, ext_hook=_msgpack_ext_hook, strict_map_key=False
|
||||
)
|
||||
# module, name, args
|
||||
return getattr(importlib.import_module(tup[0]), tup[1])(*tup[2])
|
||||
@@ -469,8 +451,8 @@ def _msgpack_ext_hook(code: int, data: bytes) -> Any:
|
||||
return
|
||||
elif code == EXT_CONSTRUCTOR_KW_ARGS:
|
||||
try:
|
||||
tup = ormsgpack.unpackb(
|
||||
data, ext_hook=_msgpack_ext_hook, option=ormsgpack.OPT_NON_STR_KEYS
|
||||
tup = msgpack.unpackb(
|
||||
data, ext_hook=_msgpack_ext_hook, strict_map_key=False
|
||||
)
|
||||
# module, name, args
|
||||
return getattr(importlib.import_module(tup[0]), tup[1])(**tup[2])
|
||||
@@ -478,8 +460,8 @@ def _msgpack_ext_hook(code: int, data: bytes) -> Any:
|
||||
return
|
||||
elif code == EXT_METHOD_SINGLE_ARG:
|
||||
try:
|
||||
tup = ormsgpack.unpackb(
|
||||
data, ext_hook=_msgpack_ext_hook, option=ormsgpack.OPT_NON_STR_KEYS
|
||||
tup = msgpack.unpackb(
|
||||
data, ext_hook=_msgpack_ext_hook, strict_map_key=False
|
||||
)
|
||||
# module, name, arg, method
|
||||
return getattr(getattr(importlib.import_module(tup[0]), tup[1]), tup[3])(
|
||||
@@ -489,8 +471,8 @@ def _msgpack_ext_hook(code: int, data: bytes) -> Any:
|
||||
return
|
||||
elif code == EXT_PYDANTIC_V1:
|
||||
try:
|
||||
tup = ormsgpack.unpackb(
|
||||
data, ext_hook=_msgpack_ext_hook, option=ormsgpack.OPT_NON_STR_KEYS
|
||||
tup = msgpack.unpackb(
|
||||
data, ext_hook=_msgpack_ext_hook, strict_map_key=False
|
||||
)
|
||||
# module, name, kwargs
|
||||
cls = getattr(importlib.import_module(tup[0]), tup[1])
|
||||
@@ -499,16 +481,11 @@ def _msgpack_ext_hook(code: int, data: bytes) -> Any:
|
||||
except Exception:
|
||||
return cls.construct(**tup[2])
|
||||
except Exception:
|
||||
# for pydantic objects we can't find/reconstruct
|
||||
# let's return the kwargs dict instead
|
||||
try:
|
||||
return tup[2]
|
||||
except NameError:
|
||||
return
|
||||
return
|
||||
elif code == EXT_PYDANTIC_V2:
|
||||
try:
|
||||
tup = ormsgpack.unpackb(
|
||||
data, ext_hook=_msgpack_ext_hook, option=ormsgpack.OPT_NON_STR_KEYS
|
||||
tup = msgpack.unpackb(
|
||||
data, ext_hook=_msgpack_ext_hook, strict_map_key=False
|
||||
)
|
||||
# module, name, kwargs, method
|
||||
cls = getattr(importlib.import_module(tup[0]), tup[1])
|
||||
@@ -516,135 +493,9 @@ def _msgpack_ext_hook(code: int, data: bytes) -> Any:
|
||||
return cls(**tup[2])
|
||||
except Exception:
|
||||
return cls.model_construct(**tup[2])
|
||||
except Exception:
|
||||
# for pydantic objects we can't find/reconstruct
|
||||
# let's return the kwargs dict instead
|
||||
try:
|
||||
return tup[2]
|
||||
except NameError:
|
||||
return
|
||||
elif code == EXT_NUMPY_ARRAY:
|
||||
try:
|
||||
import numpy as _np
|
||||
|
||||
dtype_str, shape, order, buf = ormsgpack.unpackb(
|
||||
data, ext_hook=_msgpack_ext_hook, option=ormsgpack.OPT_NON_STR_KEYS
|
||||
)
|
||||
arr = _np.frombuffer(buf, dtype=_np.dtype(dtype_str))
|
||||
return arr.reshape(shape, order=order)
|
||||
except Exception:
|
||||
return
|
||||
|
||||
|
||||
def _msgpack_ext_hook_to_json(code: int, data: bytes) -> Any:
|
||||
if code == EXT_CONSTRUCTOR_SINGLE_ARG:
|
||||
try:
|
||||
tup = ormsgpack.unpackb(
|
||||
data,
|
||||
ext_hook=_msgpack_ext_hook_to_json,
|
||||
option=ormsgpack.OPT_NON_STR_KEYS,
|
||||
)
|
||||
if tup[0] == "uuid" and tup[1] == "UUID":
|
||||
hex_ = tup[2]
|
||||
return (
|
||||
f"{hex_[:8]}-{hex_[8:12]}-{hex_[12:16]}-{hex_[16:20]}-{hex_[20:]}"
|
||||
)
|
||||
# module, name, arg
|
||||
return tup[2]
|
||||
except Exception:
|
||||
return
|
||||
elif code == EXT_CONSTRUCTOR_POS_ARGS:
|
||||
try:
|
||||
tup = ormsgpack.unpackb(
|
||||
data,
|
||||
ext_hook=_msgpack_ext_hook_to_json,
|
||||
option=ormsgpack.OPT_NON_STR_KEYS,
|
||||
)
|
||||
if tup[0] == "langgraph.types" and tup[1] == "Send":
|
||||
from langgraph.types import Send # type: ignore
|
||||
|
||||
return Send(*tup[2])
|
||||
# module, name, args
|
||||
return tup[2]
|
||||
except Exception:
|
||||
return
|
||||
elif code == EXT_CONSTRUCTOR_KW_ARGS:
|
||||
try:
|
||||
tup = ormsgpack.unpackb(
|
||||
data,
|
||||
ext_hook=_msgpack_ext_hook_to_json,
|
||||
option=ormsgpack.OPT_NON_STR_KEYS,
|
||||
)
|
||||
# module, name, args
|
||||
return tup[2]
|
||||
except Exception:
|
||||
return
|
||||
elif code == EXT_METHOD_SINGLE_ARG:
|
||||
try:
|
||||
tup = ormsgpack.unpackb(
|
||||
data,
|
||||
ext_hook=_msgpack_ext_hook_to_json,
|
||||
option=ormsgpack.OPT_NON_STR_KEYS,
|
||||
)
|
||||
# module, name, arg, method
|
||||
return tup[2]
|
||||
except Exception:
|
||||
return
|
||||
elif code == EXT_PYDANTIC_V1:
|
||||
try:
|
||||
tup = ormsgpack.unpackb(
|
||||
data,
|
||||
ext_hook=_msgpack_ext_hook_to_json,
|
||||
option=ormsgpack.OPT_NON_STR_KEYS,
|
||||
)
|
||||
# module, name, kwargs
|
||||
return tup[2]
|
||||
except Exception:
|
||||
# for pydantic objects we can't find/reconstruct
|
||||
# let's return the kwargs dict instead
|
||||
return
|
||||
elif code == EXT_PYDANTIC_V2:
|
||||
try:
|
||||
tup = ormsgpack.unpackb(
|
||||
data,
|
||||
ext_hook=_msgpack_ext_hook_to_json,
|
||||
option=ormsgpack.OPT_NON_STR_KEYS,
|
||||
)
|
||||
# module, name, kwargs, method
|
||||
return tup[2]
|
||||
except Exception:
|
||||
return
|
||||
elif code == EXT_NUMPY_ARRAY:
|
||||
try:
|
||||
import numpy as _np
|
||||
|
||||
dtype_str, shape, order, buf = ormsgpack.unpackb(
|
||||
data,
|
||||
ext_hook=_msgpack_ext_hook_to_json,
|
||||
option=ormsgpack.OPT_NON_STR_KEYS,
|
||||
)
|
||||
arr = _np.frombuffer(buf, dtype=_np.dtype(dtype_str))
|
||||
return arr.reshape(shape, order=order).tolist()
|
||||
except Exception:
|
||||
return
|
||||
|
||||
|
||||
class InvalidModuleError(Exception):
|
||||
"""Exception raised when a module is not in the allowlist."""
|
||||
|
||||
def __init__(self, message: str):
|
||||
self.message = message
|
||||
|
||||
|
||||
_option = (
|
||||
ormsgpack.OPT_NON_STR_KEYS
|
||||
| ormsgpack.OPT_PASSTHROUGH_DATACLASS
|
||||
| ormsgpack.OPT_PASSTHROUGH_DATETIME
|
||||
| ormsgpack.OPT_PASSTHROUGH_ENUM
|
||||
| ormsgpack.OPT_PASSTHROUGH_UUID
|
||||
| ormsgpack.OPT_REPLACE_SURROGATES
|
||||
)
|
||||
|
||||
|
||||
def _msgpack_enc(data: Any) -> bytes:
|
||||
return ormsgpack.packb(data, default=_msgpack_default, option=_option)
|
||||
return msgpack.packb(data, default=_msgpack_default)
|
||||
@@ -1,7 +1,8 @@
|
||||
from collections.abc import Sequence
|
||||
from typing import (
|
||||
Any,
|
||||
Optional,
|
||||
Protocol,
|
||||
Sequence,
|
||||
TypeVar,
|
||||
runtime_checkable,
|
||||
)
|
||||
@@ -27,9 +28,9 @@ class ChannelProtocol(Protocol[Value, Update, C]):
|
||||
@property
|
||||
def UpdateType(self) -> Any: ...
|
||||
|
||||
def checkpoint(self) -> C | None: ...
|
||||
def checkpoint(self) -> Optional[C]: ...
|
||||
|
||||
def from_checkpoint(self, checkpoint: C | None) -> Self: ...
|
||||
def from_checkpoint(self, checkpoint: Optional[C]) -> Self: ...
|
||||
|
||||
def update(self, values: Sequence[Update]) -> bool: ...
|
||||
|
||||
|
||||
File diff suppressed because it is too large.
Load diff
@@ -1,365 +0,0 @@
|
||||
"""Utilities for batching operations in a background task."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import asyncio
|
||||
import functools
|
||||
import weakref
|
||||
from collections.abc import Callable, Iterable
|
||||
from typing import Any, Literal, TypeVar
|
||||
|
||||
from langgraph.store.base import (
|
||||
NOT_PROVIDED,
|
||||
BaseStore,
|
||||
GetOp,
|
||||
Item,
|
||||
ListNamespacesOp,
|
||||
MatchCondition,
|
||||
NamespacePath,
|
||||
NotProvided,
|
||||
Op,
|
||||
PutOp,
|
||||
Result,
|
||||
SearchItem,
|
||||
SearchOp,
|
||||
_ensure_refresh,
|
||||
_ensure_ttl,
|
||||
_validate_namespace,
|
||||
)
|
||||
|
||||
F = TypeVar("F", bound=Callable)
|
||||
|
||||
|
||||
def _check_loop(func: F) -> F:
|
||||
@functools.wraps(func)
|
||||
def wrapper(store: AsyncBatchedBaseStore, *args: Any, **kwargs: Any) -> Any:
|
||||
method_name: str = func.__name__
|
||||
try:
|
||||
current_loop = asyncio.get_running_loop()
|
||||
if current_loop is store._loop:
|
||||
replacement_str = (
|
||||
f"Specifically, replace `store.{method_name}(...)` with `await store.a{method_name}(...)"
|
||||
if method_name
|
||||
else "For example, replace `store.get(...)` with `await store.aget(...)`"
|
||||
)
|
||||
raise asyncio.InvalidStateError(
|
||||
f"Synchronous calls to {store.__class__.__name__} detected in the main event loop. "
|
||||
"This can lead to deadlocks or performance issues. "
|
||||
"Please use the asynchronous interface for main thread operations. "
|
||||
f"{replacement_str} "
|
||||
)
|
||||
except RuntimeError:
|
||||
pass
|
||||
return func(store, *args, **kwargs)
|
||||
|
||||
return wrapper
|
||||
|
||||
|
||||
class AsyncBatchedBaseStore(BaseStore):
|
||||
"""Efficiently batch operations in a background task."""
|
||||
|
||||
__slots__ = ("_loop", "_aqueue", "_task")
|
||||
|
||||
def __init__(self) -> None:
|
||||
super().__init__()
|
||||
self._loop = asyncio.get_running_loop()
|
||||
self._aqueue: asyncio.Queue[tuple[asyncio.Future, Op]] = asyncio.Queue()
|
||||
self._task: asyncio.Task | None = None
|
||||
self._ensure_task()
|
||||
|
||||
def __del__(self) -> None:
|
||||
try:
|
||||
if self._task:
|
||||
self._task.cancel()
|
||||
except RuntimeError:
|
||||
pass
|
||||
|
||||
def _ensure_task(self) -> None:
|
||||
"""Ensure the background processing loop is running."""
|
||||
if self._task is None or self._task.done():
|
||||
self._task = self._loop.create_task(_run(self._aqueue, weakref.ref(self)))
|
||||
|
||||
async def aget(
|
||||
self,
|
||||
namespace: tuple[str, ...],
|
||||
key: str,
|
||||
*,
|
||||
refresh_ttl: bool | None = None,
|
||||
) -> Item | None:
|
||||
self._ensure_task()
|
||||
fut = self._loop.create_future()
|
||||
self._aqueue.put_nowait(
|
||||
(
|
||||
fut,
|
||||
GetOp(
|
||||
namespace,
|
||||
key,
|
||||
refresh_ttl=_ensure_refresh(self.ttl_config, refresh_ttl),
|
||||
),
|
||||
)
|
||||
)
|
||||
return await fut
|
||||
|
||||
async def asearch(
|
||||
self,
|
||||
namespace_prefix: tuple[str, ...],
|
||||
/,
|
||||
*,
|
||||
query: str | None = None,
|
||||
filter: dict[str, Any] | None = None,
|
||||
limit: int = 10,
|
||||
offset: int = 0,
|
||||
refresh_ttl: bool | None = None,
|
||||
) -> list[SearchItem]:
|
||||
self._ensure_task()
|
||||
fut = self._loop.create_future()
|
||||
self._aqueue.put_nowait(
|
||||
(
|
||||
fut,
|
||||
SearchOp(
|
||||
namespace_prefix,
|
||||
filter,
|
||||
limit,
|
||||
offset,
|
||||
query,
|
||||
refresh_ttl=_ensure_refresh(self.ttl_config, refresh_ttl),
|
||||
),
|
||||
)
|
||||
)
|
||||
return await fut
|
||||
|
||||
async def aput(
|
||||
self,
|
||||
namespace: tuple[str, ...],
|
||||
key: str,
|
||||
value: dict[str, Any],
|
||||
index: Literal[False] | list[str] | None = None,
|
||||
*,
|
||||
ttl: float | None | NotProvided = NOT_PROVIDED,
|
||||
) -> None:
|
||||
self._ensure_task()
|
||||
_validate_namespace(namespace)
|
||||
fut = self._loop.create_future()
|
||||
self._aqueue.put_nowait(
|
||||
(
|
||||
fut,
|
||||
PutOp(
|
||||
namespace, key, value, index, ttl=_ensure_ttl(self.ttl_config, ttl)
|
||||
),
|
||||
)
|
||||
)
|
||||
return await fut
|
||||
|
||||
async def adelete(
|
||||
self,
|
||||
namespace: tuple[str, ...],
|
||||
key: str,
|
||||
) -> None:
|
||||
self._ensure_task()
|
||||
fut = self._loop.create_future()
|
||||
self._aqueue.put_nowait((fut, PutOp(namespace, key, None)))
|
||||
return await fut
|
||||
|
||||
async def alist_namespaces(
|
||||
self,
|
||||
*,
|
||||
prefix: NamespacePath | None = None,
|
||||
suffix: NamespacePath | None = None,
|
||||
max_depth: int | None = None,
|
||||
limit: int = 100,
|
||||
offset: int = 0,
|
||||
) -> list[tuple[str, ...]]:
|
||||
self._ensure_task()
|
||||
fut = self._loop.create_future()
|
||||
match_conditions = []
|
||||
if prefix:
|
||||
match_conditions.append(MatchCondition(match_type="prefix", path=prefix))
|
||||
if suffix:
|
||||
match_conditions.append(MatchCondition(match_type="suffix", path=suffix))
|
||||
|
||||
op = ListNamespacesOp(
|
||||
match_conditions=tuple(match_conditions),
|
||||
max_depth=max_depth,
|
||||
limit=limit,
|
||||
offset=offset,
|
||||
)
|
||||
self._aqueue.put_nowait((fut, op))
|
||||
return await fut
|
||||
|
||||
@_check_loop
|
||||
def batch(self, ops: Iterable[Op]) -> list[Result]:
|
||||
return asyncio.run_coroutine_threadsafe(self.abatch(ops), self._loop).result()
|
||||
|
||||
@_check_loop
|
||||
def get(
|
||||
self,
|
||||
namespace: tuple[str, ...],
|
||||
key: str,
|
||||
*,
|
||||
refresh_ttl: bool | None = None,
|
||||
) -> Item | None:
|
||||
return asyncio.run_coroutine_threadsafe(
|
||||
self.aget(namespace, key=key, refresh_ttl=refresh_ttl), self._loop
|
||||
).result()
|
||||
|
||||
@_check_loop
|
||||
def search(
|
||||
self,
|
||||
namespace_prefix: tuple[str, ...],
|
||||
/,
|
||||
*,
|
||||
query: str | None = None,
|
||||
filter: dict[str, Any] | None = None,
|
||||
limit: int = 10,
|
||||
offset: int = 0,
|
||||
refresh_ttl: bool | None = None,
|
||||
) -> list[SearchItem]:
|
||||
return asyncio.run_coroutine_threadsafe(
|
||||
self.asearch(
|
||||
namespace_prefix,
|
||||
query=query,
|
||||
filter=filter,
|
||||
limit=limit,
|
||||
offset=offset,
|
||||
refresh_ttl=refresh_ttl,
|
||||
),
|
||||
self._loop,
|
||||
).result()
|
||||
|
||||
@_check_loop
|
||||
def put(
|
||||
self,
|
||||
namespace: tuple[str, ...],
|
||||
key: str,
|
||||
value: dict[str, Any],
|
||||
index: Literal[False] | list[str] | None = None,
|
||||
*,
|
||||
ttl: float | None | NotProvided = NOT_PROVIDED,
|
||||
) -> None:
|
||||
_validate_namespace(namespace)
|
||||
asyncio.run_coroutine_threadsafe(
|
||||
self.aput(
|
||||
namespace,
|
||||
key=key,
|
||||
value=value,
|
||||
index=index,
|
||||
ttl=_ensure_ttl(self.ttl_config, ttl),
|
||||
),
|
||||
self._loop,
|
||||
).result()
|
||||
|
||||
@_check_loop
|
||||
def delete(
|
||||
self,
|
||||
namespace: tuple[str, ...],
|
||||
key: str,
|
||||
) -> None:
|
||||
asyncio.run_coroutine_threadsafe(
|
||||
self.adelete(namespace, key=key), self._loop
|
||||
).result()
|
||||
|
||||
@_check_loop
|
||||
def list_namespaces(
|
||||
self,
|
||||
*,
|
||||
prefix: NamespacePath | None = None,
|
||||
suffix: NamespacePath | None = None,
|
||||
max_depth: int | None = None,
|
||||
limit: int = 100,
|
||||
offset: int = 0,
|
||||
) -> list[tuple[str, ...]]:
|
||||
return asyncio.run_coroutine_threadsafe(
|
||||
self.alist_namespaces(
|
||||
prefix=prefix,
|
||||
suffix=suffix,
|
||||
max_depth=max_depth,
|
||||
limit=limit,
|
||||
offset=offset,
|
||||
),
|
||||
self._loop,
|
||||
).result()
|
||||
|
||||
|
||||
def _dedupe_ops(values: list[Op]) -> tuple[list[int] | None, list[Op]]:
|
||||
"""Dedupe operations while preserving order for results.
|
||||
|
||||
Args:
|
||||
values: List of operations to dedupe
|
||||
|
||||
Returns:
|
||||
Tuple of (listen indices, deduped operations)
|
||||
where listen indices map deduped operation results back to original positions
|
||||
"""
|
||||
if len(values) <= 1:
|
||||
return None, list(values)
|
||||
|
||||
dedupped: list[Op] = []
|
||||
listen: list[int] = []
|
||||
puts: dict[tuple[tuple[str, ...], str], int] = {}
|
||||
|
||||
for op in values:
|
||||
if isinstance(op, (GetOp, SearchOp, ListNamespacesOp)):
|
||||
try:
|
||||
listen.append(dedupped.index(op))
|
||||
except ValueError:
|
||||
listen.append(len(dedupped))
|
||||
dedupped.append(op)
|
||||
elif isinstance(op, PutOp):
|
||||
putkey = (op.namespace, op.key)
|
||||
if putkey in puts:
|
||||
# Overwrite previous put
|
||||
ix = puts[putkey]
|
||||
dedupped[ix] = op
|
||||
listen.append(ix)
|
||||
else:
|
||||
puts[putkey] = len(dedupped)
|
||||
listen.append(len(dedupped))
|
||||
dedupped.append(op)
|
||||
|
||||
else: # Any new ops will be treated regularly
|
||||
listen.append(len(dedupped))
|
||||
dedupped.append(op)
|
||||
|
||||
return listen, dedupped
|
||||
|
||||
|
||||
async def _run(
|
||||
aqueue: asyncio.Queue[tuple[asyncio.Future, Op]],
|
||||
store: weakref.ReferenceType[BaseStore],
|
||||
) -> None:
|
||||
while item := await aqueue.get():
|
||||
# check if store is still alive
|
||||
if s := store():
|
||||
try:
|
||||
# accumulate operations scheduled in same tick
|
||||
items = [item]
|
||||
try:
|
||||
while item := aqueue.get_nowait():
|
||||
items.append(item)
|
||||
except asyncio.QueueEmpty:
|
||||
pass
|
||||
# get the operations to run
|
||||
futs = [item[0] for item in items]
|
||||
values = [item[1] for item in items]
|
||||
# action each operation
|
||||
try:
|
||||
listen, dedupped = _dedupe_ops(values)
|
||||
results = await s.abatch(dedupped)
|
||||
if listen is not None:
|
||||
results = [results[ix] for ix in listen]
|
||||
|
||||
# set the results of each operation
|
||||
for fut, result in zip(futs, results, strict=False):
|
||||
# guard against future being done (e.g. cancelled)
|
||||
if not fut.done():
|
||||
fut.set_result(result)
|
||||
except Exception as e:
|
||||
for fut in futs:
|
||||
# guard against future being done (e.g. cancelled)
|
||||
if not fut.done():
|
||||
fut.set_exception(e)
|
||||
finally:
|
||||
# remove strong ref to store
|
||||
del s
|
||||
else:
|
||||
break
|
||||
@@ -6,15 +6,9 @@ with LangChain-compatible tools while maintaining support for both synchronous a
|
||||
asynchronous operations.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import asyncio
|
||||
import functools
|
||||
import json
|
||||
from collections.abc import Awaitable, Callable, Sequence
|
||||
from typing import Any
|
||||
|
||||
from langchain_core.embeddings import Embeddings
|
||||
from typing import Any, Callable, Optional, Sequence, Union
|
||||
|
||||
EmbeddingsFunc = Callable[[Sequence[str]], list[list[float]]]
|
||||
"""Type for synchronous embedding functions.
|
||||
@@ -24,16 +18,10 @@ where each embedding is a list of floats. The dimensionality of the embeddings
|
||||
should be consistent for all inputs.
|
||||
"""
|
||||
|
||||
AEmbeddingsFunc = Callable[[Sequence[str]], Awaitable[list[list[float]]]]
|
||||
"""Type for asynchronous embedding functions.
|
||||
|
||||
Similar to EmbeddingsFunc, but returns an awaitable that resolves to the embeddings.
|
||||
"""
|
||||
|
||||
|
||||
def ensure_embeddings(
|
||||
embed: Embeddings | EmbeddingsFunc | AEmbeddingsFunc | str | None,
|
||||
) -> Embeddings:
|
||||
embed: Union[EmbeddingsFunc, str, None],
|
||||
) -> "Embeddings":
|
||||
"""Ensure that an embedding function conforms to LangChain's Embeddings interface.
|
||||
|
||||
This function wraps arbitrary embedding functions to make them compatible with
|
||||
@@ -49,9 +37,7 @@ def ensure_embeddings(
|
||||
An Embeddings instance that wraps the provided function(s).
|
||||
|
||||
??? example "Examples"
|
||||
|
||||
Wrap a synchronous embedding function:
|
||||
|
||||
```python
|
||||
def my_embed_fn(texts):
|
||||
return [[0.1, 0.2] for _ in texts]
|
||||
@@ -60,18 +46,7 @@ def ensure_embeddings(
|
||||
result = embeddings.embed_query("hello") # Returns [0.1, 0.2]
|
||||
```
|
||||
|
||||
Wrap an asynchronous embedding function:
|
||||
|
||||
```python
|
||||
async def my_async_fn(texts):
|
||||
return [[0.1, 0.2] for _ in texts]
|
||||
|
||||
embeddings = ensure_embeddings(my_async_fn)
|
||||
result = await embeddings.aembed_query("hello") # Returns [0.1, 0.2]
|
||||
```
|
||||
|
||||
Initialize embeddings using a provider string:
|
||||
|
||||
```python
|
||||
# Requires langchain>=0.3.9 and langgraph-checkpoint>=2.0.11
|
||||
embeddings = ensure_embeddings("openai:text-embedding-3-small")
|
||||
@@ -103,10 +78,10 @@ def ensure_embeddings(
|
||||
|
||||
if isinstance(embed, Embeddings):
|
||||
return embed
|
||||
return EmbeddingsLambda(embed)
|
||||
return Embeddings(embed)
|
||||
|
||||
|
||||
class EmbeddingsLambda(Embeddings):
|
||||
class Embeddings:
|
||||
"""Wrapper to convert embedding functions into LangChain's Embeddings interface.
|
||||
|
||||
This class allows arbitrary embedding functions to be used with LangChain-compatible
|
||||
@@ -123,9 +98,7 @@ class EmbeddingsLambda(Embeddings):
|
||||
will raise an error. If sync, it will be used for both sync and async operations.
|
||||
|
||||
??? example "Examples"
|
||||
|
||||
With a sync function:
|
||||
|
||||
```python
|
||||
def my_embed_fn(texts):
|
||||
# Return 2D embeddings for each text
|
||||
@@ -135,27 +108,14 @@ class EmbeddingsLambda(Embeddings):
|
||||
result = embeddings.embed_query("hello") # Returns [0.1, 0.2]
|
||||
await embeddings.aembed_query("hello") # Also returns [0.1, 0.2]
|
||||
```
|
||||
|
||||
With an async function:
|
||||
|
||||
```python
|
||||
async def my_async_fn(texts):
|
||||
return [[0.1, 0.2] for _ in texts]
|
||||
|
||||
embeddings = EmbeddingsLambda(my_async_fn)
|
||||
await embeddings.aembed_query("hello") # Returns [0.1, 0.2]
|
||||
# Note: embed_query() would raise an error
|
||||
```
|
||||
"""
|
||||
|
||||
def __init__(
|
||||
self,
|
||||
func: EmbeddingsFunc | AEmbeddingsFunc,
|
||||
func: EmbeddingsFunc,
|
||||
) -> None:
|
||||
if func is None:
|
||||
raise ValueError("func must be provided")
|
||||
if _is_async_callable(func):
|
||||
self.afunc = func
|
||||
else:
|
||||
self.func = func
|
||||
|
||||
@@ -168,16 +128,8 @@ class EmbeddingsLambda(Embeddings):
|
||||
Returns:
|
||||
list of embeddings, one per input text. Each embedding is a list of floats.
|
||||
|
||||
Raises:
|
||||
ValueError: If the instance was initialized with only an async function.
|
||||
"""
|
||||
func = getattr(self, "func", None)
|
||||
if func is None:
|
||||
raise ValueError(
|
||||
"EmbeddingsLambda was initialized with an async function but no sync function. "
|
||||
"Use aembed_documents for async operation or provide a sync function."
|
||||
)
|
||||
return func(texts)
|
||||
return self.func(texts)
|
||||
|
||||
def embed_query(self, text: str) -> list[float]:
|
||||
"""Embed a single piece of text.
|
||||
@@ -194,43 +146,8 @@ class EmbeddingsLambda(Embeddings):
|
||||
"""
|
||||
return self.embed_documents([text])[0]
|
||||
|
||||
async def aembed_documents(self, texts: list[str]) -> list[list[float]]:
|
||||
"""Asynchronously embed a list of texts into vectors.
|
||||
|
||||
Args:
|
||||
texts: list of texts to convert to embeddings.
|
||||
|
||||
Returns:
|
||||
list of embeddings, one per input text. Each embedding is a list of floats.
|
||||
|
||||
Note:
|
||||
If no async function was provided, this falls back to the sync implementation.
|
||||
"""
|
||||
afunc = getattr(self, "afunc", None)
|
||||
if afunc is None:
|
||||
return await super().aembed_documents(texts)
|
||||
return await afunc(texts)
|
||||
|
||||
async def aembed_query(self, text: str) -> list[float]:
|
||||
"""Asynchronously embed a single piece of text.
|
||||
|
||||
Args:
|
||||
text: Text to convert to an embedding.
|
||||
|
||||
Returns:
|
||||
Embedding vector as a list of floats.
|
||||
|
||||
Note:
|
||||
This is equivalent to calling aembed_documents with a single text
|
||||
and taking the first result.
|
||||
"""
|
||||
afunc = getattr(self, "afunc", None)
|
||||
if afunc is None:
|
||||
return await super().aembed_query(text)
|
||||
return (await afunc([text]))[0]
|
||||
|
||||
|
||||
def get_text_at_path(obj: Any, path: str | list[str]) -> list[str]:
|
||||
def get_text_at_path(obj: Any, path: Union[str, list[str]]) -> list[str]:
|
||||
"""Extract text from an object using a path expression or pre-tokenized path.
|
||||
|
||||
Args:
|
||||
@@ -245,7 +162,7 @@ def get_text_at_path(obj: Any, path: str | list[str]) -> list[str]:
|
||||
- Nested paths in multi-field: "{field1,nested.field2}"
|
||||
"""
|
||||
if not path or path == "$":
|
||||
return [json.dumps(obj, sort_keys=True, ensure_ascii=False)]
|
||||
return [json.dumps(obj, sort_keys=True)]
|
||||
|
||||
tokens = tokenize_path(path) if isinstance(path, str) else path
|
||||
|
||||
@@ -256,7 +173,7 @@ def get_text_at_path(obj: Any, path: str | list[str]) -> list[str]:
|
||||
elif obj is None:
|
||||
return []
|
||||
elif isinstance(obj, (list, dict)):
|
||||
return [json.dumps(obj, sort_keys=True, ensure_ascii=False)]
|
||||
return [json.dumps(obj, sort_keys=True)]
|
||||
return []
|
||||
|
||||
token = tokens[pos]
|
||||
@@ -288,7 +205,7 @@ def get_text_at_path(obj: Any, path: str | list[str]) -> list[str]:
|
||||
for field in fields:
|
||||
nested_tokens = tokenize_path(field)
|
||||
if nested_tokens:
|
||||
current_obj: dict | None = obj
|
||||
current_obj: Optional[dict] = obj
|
||||
for nested_token in nested_tokens:
|
||||
if (
|
||||
isinstance(current_obj, dict)
|
||||
@@ -302,11 +219,7 @@ def get_text_at_path(obj: Any, path: str | list[str]) -> list[str]:
|
||||
if isinstance(current_obj, (str, int, float, bool)):
|
||||
results.append(str(current_obj))
|
||||
elif isinstance(current_obj, (list, dict)):
|
||||
results.append(
|
||||
json.dumps(
|
||||
current_obj, sort_keys=True, ensure_ascii=False
|
||||
)
|
||||
)
|
||||
results.append(json.dumps(current_obj, sort_keys=True))
|
||||
|
||||
# Handle wildcard
|
||||
elif token == "*":
|
||||
@@ -396,28 +309,8 @@ def tokenize_path(path: str) -> list[str]:
|
||||
return tokens
|
||||
|
||||
|
||||
def _is_async_callable(
|
||||
func: Any,
|
||||
) -> bool:
|
||||
"""Check if a function is async.
|
||||
|
||||
This includes both async def functions and classes with async __call__ methods.
|
||||
|
||||
Args:
|
||||
func: Function or callable object to check.
|
||||
|
||||
Returns:
|
||||
True if the function is async, False otherwise.
|
||||
"""
|
||||
return (
|
||||
asyncio.iscoroutinefunction(func)
|
||||
or hasattr(func, "__call__") # noqa: B004
|
||||
and asyncio.iscoroutinefunction(func.__call__)
|
||||
)
|
||||
|
||||
|
||||
@functools.lru_cache
|
||||
def _get_init_embeddings() -> Callable[[str], Embeddings] | None:
|
||||
def _get_init_embeddings() -> Optional[Callable[[str], Embeddings]]:
|
||||
try:
|
||||
from langchain.embeddings import init_embeddings # type: ignore
|
||||
|
||||
@@ -429,5 +322,5 @@ def _get_init_embeddings() -> Callable[[str], Embeddings] | None:
|
||||
__all__ = [
|
||||
"ensure_embeddings",
|
||||
"EmbeddingsFunc",
|
||||
"AEmbeddingsFunc",
|
||||
"Embeddings",
|
||||
]
|
||||
@@ -59,35 +59,6 @@
|
||||
results = store.search(("docs",), query="python programming")
|
||||
```
|
||||
|
||||
Async vector search using OpenAI SDK:
|
||||
```python
|
||||
from openai import AsyncOpenAI
|
||||
from langgraph.store.memory import InMemoryStore
|
||||
|
||||
client = AsyncOpenAI()
|
||||
|
||||
async def aembed_texts(texts: list[str]) -> list[list[float]]:
|
||||
response = await client.embeddings.create(
|
||||
model="text-embedding-3-small",
|
||||
input=texts
|
||||
)
|
||||
return [e.embedding for e in response.data]
|
||||
|
||||
store = InMemoryStore(
|
||||
index={
|
||||
"dims": 1536,
|
||||
"embed": aembed_texts
|
||||
}
|
||||
)
|
||||
|
||||
# Store documents
|
||||
await store.aput(("docs",), "doc1", {"text": "Python tutorial"})
|
||||
await store.aput(("docs",), "doc2", {"text": "TypeScript guide"})
|
||||
|
||||
# Search by similarity
|
||||
results = await store.asearch(("docs",), query="python programming")
|
||||
```
|
||||
|
||||
Warning:
|
||||
This store keeps all data in memory. Data is lost when the process exits.
|
||||
For persistence, use a database-backed store like PostgresStore.
|
||||
@@ -99,22 +70,17 @@ Tip:
|
||||
```
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import asyncio
|
||||
import concurrent.futures as cf
|
||||
import functools
|
||||
import logging
|
||||
from collections import defaultdict
|
||||
from collections.abc import Iterable
|
||||
from datetime import datetime, timezone
|
||||
from importlib import util
|
||||
from typing import Any
|
||||
|
||||
from langchain_core.embeddings import Embeddings
|
||||
from typing import Any, Iterable, Optional
|
||||
|
||||
from langgraph.store.base import (
|
||||
BaseStore,
|
||||
Embeddings,
|
||||
GetOp,
|
||||
IndexConfig,
|
||||
Item,
|
||||
@@ -180,7 +146,7 @@ class InMemoryStore(BaseStore):
|
||||
"embeddings",
|
||||
)
|
||||
|
||||
def __init__(self, *, index: IndexConfig | None = None) -> None:
|
||||
def __init__(self, *, index: Optional[IndexConfig] = None) -> None:
|
||||
# Both _data and _vectors are wrapped in the In-memory API
|
||||
# Do not change their names
|
||||
self._data: dict[tuple[str, ...], dict[str, Item]] = defaultdict(dict)
|
||||
@@ -191,7 +157,7 @@ class InMemoryStore(BaseStore):
|
||||
self.index_config = index
|
||||
if self.index_config:
|
||||
self.index_config = self.index_config.copy()
|
||||
self.embeddings: Embeddings | None = ensure_embeddings(
|
||||
self.embeddings: Optional[Embeddings] = ensure_embeddings(
|
||||
self.index_config.get("embed"),
|
||||
)
|
||||
self.index_config["__tokenized_fields"] = [
|
||||
@@ -218,21 +184,6 @@ class InMemoryStore(BaseStore):
|
||||
self._apply_put_ops(put_ops)
|
||||
return results
|
||||
|
||||
async def abatch(self, ops: Iterable[Op]) -> list[Result]:
|
||||
# The batch/abatch methods are treated as internal.
|
||||
# Users should access via put/search/get/list_namespaces/etc.
|
||||
results, put_ops, search_ops = self._prepare_ops(ops)
|
||||
if search_ops:
|
||||
queryinmem_store = await self._aembed_search_queries(search_ops)
|
||||
self._batch_search(search_ops, queryinmem_store, results)
|
||||
|
||||
to_embed = self._extract_texts(put_ops)
|
||||
if to_embed and self.index_config and self.embeddings:
|
||||
embeddings = await self.embeddings.aembed_documents(list(to_embed))
|
||||
self._insertinmem_store(to_embed, embeddings)
|
||||
self._apply_put_ops(put_ops)
|
||||
return results
|
||||
|
||||
# Helpers
|
||||
|
||||
def _filter_items(self, op: SearchOp) -> list[tuple[Item, list[list[float]]]]:
|
||||
@@ -284,21 +235,6 @@ class InMemoryStore(BaseStore):
|
||||
|
||||
return queryinmem_store
|
||||
|
||||
async def _aembed_search_queries(
|
||||
self,
|
||||
search_ops: dict[int, tuple[SearchOp, list[tuple[Item, list[list[float]]]]]],
|
||||
) -> dict[str, list[float]]:
|
||||
queryinmem_store = {}
|
||||
if self.index_config and self.embeddings and search_ops:
|
||||
queries = {op.query for (op, _) in search_ops.values() if op.query}
|
||||
|
||||
if queries:
|
||||
coros = [self.embeddings.aembed_query(q) for q in list(queries)]
|
||||
results = await asyncio.gather(*coros)
|
||||
queryinmem_store = dict(zip(queries, results, strict=False))
|
||||
|
||||
return queryinmem_store
|
||||
|
||||
def _batch_search(
|
||||
self,
|
||||
ops: dict[int, tuple[SearchOp, list[tuple[Item, list[list[float]]]]]],
|
||||
@@ -323,13 +259,11 @@ class InMemoryStore(BaseStore):
|
||||
|
||||
scores = _cosine_similarity(query_embedding, flat_vectors)
|
||||
sorted_results = sorted(
|
||||
zip(scores, flat_items, strict=False),
|
||||
key=lambda x: x[0],
|
||||
reverse=True,
|
||||
zip(scores, flat_items), key=lambda x: x[0], reverse=True
|
||||
)
|
||||
# max pooling
|
||||
seen: set[tuple[tuple[str, ...], str]] = set()
|
||||
kept: list[tuple[float | None, Item]] = []
|
||||
kept: list[tuple[Optional[float], Item]] = []
|
||||
for score, item in sorted_results:
|
||||
key = (item.namespace, item.key)
|
||||
if key in seen:
|
||||
@@ -454,7 +388,7 @@ class InMemoryStore(BaseStore):
|
||||
f"Number of embeddings ({len(embeddings)}) does not"
|
||||
f" match number of indices ({len(indices)})"
|
||||
)
|
||||
for embedding, (ns, key, path) in zip(embeddings, indices, strict=False):
|
||||
for embedding, (ns, key, path) in zip(embeddings, indices):
|
||||
self._vectors[ns][key][path] = embedding
|
||||
|
||||
def _handle_list_namespaces(self, op: ListNamespacesOp) -> list[tuple[str, ...]]:
|
||||
@@ -498,7 +432,7 @@ def _cosine_similarity(X: list[float], Y: list[list[float]]) -> list[float]:
|
||||
if not Y:
|
||||
return []
|
||||
if _check_numpy():
|
||||
import numpy as np
|
||||
import numpy as np # type: ignore[import-not-found]
|
||||
|
||||
X_arr = np.array(X) if not isinstance(X, np.ndarray) else X
|
||||
Y_arr = np.array(Y) if not isinstance(Y, np.ndarray) else Y
|
||||
@@ -513,7 +447,7 @@ def _cosine_similarity(X: list[float], Y: list[list[float]]) -> list[float]:
|
||||
|
||||
similarities = []
|
||||
for y in Y:
|
||||
dot_product = sum(a * b for a, b in zip(X, y, strict=False))
|
||||
dot_product = sum(a * b for a, b in zip(X, y))
|
||||
norm1 = sum(a * a for a in X) ** 0.5
|
||||
norm2 = sum(a * a for a in y) ** 0.5
|
||||
similarity = dot_product / (norm1 * norm2) if norm1 > 0 and norm2 > 0 else 0.0
|
||||
@@ -531,14 +465,14 @@ def _does_match(match_condition: MatchCondition, key: tuple[str, ...]) -> bool:
|
||||
return False
|
||||
|
||||
if match_type == "prefix":
|
||||
for k_elem, p_elem in zip(key, path, strict=False):
|
||||
for k_elem, p_elem in zip(key, path):
|
||||
if p_elem == "*":
|
||||
continue # Wildcard matches any element
|
||||
if k_elem != p_elem:
|
||||
return False
|
||||
return True
|
||||
elif match_type == "suffix":
|
||||
for k_elem, p_elem in zip(reversed(key), reversed(path), strict=False):
|
||||
for k_elem, p_elem in zip(reversed(key), reversed(path)):
|
||||
if p_elem == "*":
|
||||
continue # Wildcard matches any element
|
||||
if k_elem != p_elem:
|
||||
@@ -565,10 +499,7 @@ def _compare_values(item_value: Any, filter_value: Any) -> bool:
|
||||
return (
|
||||
isinstance(item_value, (list, tuple))
|
||||
and len(item_value) == len(filter_value)
|
||||
and all(
|
||||
_compare_values(iv, fv)
|
||||
for iv, fv in zip(item_value, filter_value, strict=False)
|
||||
)
|
||||
and all(_compare_values(iv, fv) for iv, fv in zip(item_value, filter_value))
|
||||
)
|
||||
else:
|
||||
return item_value == filter_value
|
||||
|
||||
Generated
+927
@@ -0,0 +1,927 @@
|
||||
# This file is automatically @generated by Poetry 1.8.2 and should not be changed by hand.
|
||||
|
||||
[[package]]
|
||||
name = "annotated-types"
|
||||
version = "0.7.0"
|
||||
description = "Reusable constraint types to use with typing.Annotated"
|
||||
optional = false
|
||||
python-versions = ">=3.8"
|
||||
files = [
|
||||
{file = "annotated_types-0.7.0-py3-none-any.whl", hash = "sha256:1f02e8b43a8fbbc3f3e0d4f0f4bfc8131bcb4eebe8849b8e5c773f3a1c582a53"},
|
||||
{file = "annotated_types-0.7.0.tar.gz", hash = "sha256:aff07c09a53a08bc8cfccb9c85b05f1aa9a2a6f23728d790723543408344ce89"},
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "certifi"
|
||||
version = "2024.7.4"
|
||||
description = "Python package for providing Mozilla's CA Bundle."
|
||||
optional = false
|
||||
python-versions = ">=3.6"
|
||||
files = [
|
||||
{file = "certifi-2024.7.4-py3-none-any.whl", hash = "sha256:c198e21b1289c2ab85ee4e67bb4b4ef3ead0892059901a8d5b622f24a1101e90"},
|
||||
{file = "certifi-2024.7.4.tar.gz", hash = "sha256:5a1e7645bc0ec61a09e26c36f6106dd4cf40c6db3a1fb6352b0244e7fb057c7b"},
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "charset-normalizer"
|
||||
version = "3.3.2"
|
||||
description = "The Real First Universal Charset Detector. Open, modern and actively maintained alternative to Chardet."
|
||||
optional = false
|
||||
python-versions = ">=3.7.0"
|
||||
files = [
|
||||
{file = "charset-normalizer-3.3.2.tar.gz", hash = "sha256:f30c3cb33b24454a82faecaf01b19c18562b1e89558fb6c56de4d9118a032fd5"},
|
||||
{file = "charset_normalizer-3.3.2-cp310-cp310-macosx_10_9_universal2.whl", hash = "sha256:25baf083bf6f6b341f4121c2f3c548875ee6f5339300e08be3f2b2ba1721cdd3"},
|
||||
{file = "charset_normalizer-3.3.2-cp310-cp310-macosx_10_9_x86_64.whl", hash = "sha256:06435b539f889b1f6f4ac1758871aae42dc3a8c0e24ac9e60c2384973ad73027"},
|
||||
{file = "charset_normalizer-3.3.2-cp310-cp310-macosx_11_0_arm64.whl", hash = "sha256:9063e24fdb1e498ab71cb7419e24622516c4a04476b17a2dab57e8baa30d6e03"},
|
||||
{file = "charset_normalizer-3.3.2-cp310-cp310-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:6897af51655e3691ff853668779c7bad41579facacf5fd7253b0133308cf000d"},
|
||||
{file = "charset_normalizer-3.3.2-cp310-cp310-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", hash = "sha256:1d3193f4a680c64b4b6a9115943538edb896edc190f0b222e73761716519268e"},
|
||||
{file = "charset_normalizer-3.3.2-cp310-cp310-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:cd70574b12bb8a4d2aaa0094515df2463cb429d8536cfb6c7ce983246983e5a6"},
|
||||
{file = "charset_normalizer-3.3.2-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:8465322196c8b4d7ab6d1e049e4c5cb460d0394da4a27d23cc242fbf0034b6b5"},
|
||||
{file = "charset_normalizer-3.3.2-cp310-cp310-manylinux_2_5_i686.manylinux1_i686.manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:a9a8e9031d613fd2009c182b69c7b2c1ef8239a0efb1df3f7c8da66d5dd3d537"},
|
||||
{file = "charset_normalizer-3.3.2-cp310-cp310-musllinux_1_1_aarch64.whl", hash = "sha256:beb58fe5cdb101e3a055192ac291b7a21e3b7ef4f67fa1d74e331a7f2124341c"},
|
||||
{file = "charset_normalizer-3.3.2-cp310-cp310-musllinux_1_1_i686.whl", hash = "sha256:e06ed3eb3218bc64786f7db41917d4e686cc4856944f53d5bdf83a6884432e12"},
|
||||
{file = "charset_normalizer-3.3.2-cp310-cp310-musllinux_1_1_ppc64le.whl", hash = "sha256:2e81c7b9c8979ce92ed306c249d46894776a909505d8f5a4ba55b14206e3222f"},
|
||||
{file = "charset_normalizer-3.3.2-cp310-cp310-musllinux_1_1_s390x.whl", hash = "sha256:572c3763a264ba47b3cf708a44ce965d98555f618ca42c926a9c1616d8f34269"},
|
||||
{file = "charset_normalizer-3.3.2-cp310-cp310-musllinux_1_1_x86_64.whl", hash = "sha256:fd1abc0d89e30cc4e02e4064dc67fcc51bd941eb395c502aac3ec19fab46b519"},
|
||||
{file = "charset_normalizer-3.3.2-cp310-cp310-win32.whl", hash = "sha256:3d47fa203a7bd9c5b6cee4736ee84ca03b8ef23193c0d1ca99b5089f72645c73"},
|
||||
{file = "charset_normalizer-3.3.2-cp310-cp310-win_amd64.whl", hash = "sha256:10955842570876604d404661fbccbc9c7e684caf432c09c715ec38fbae45ae09"},
|
||||
{file = "charset_normalizer-3.3.2-cp311-cp311-macosx_10_9_universal2.whl", hash = "sha256:802fe99cca7457642125a8a88a084cef28ff0cf9407060f7b93dca5aa25480db"},
|
||||
{file = "charset_normalizer-3.3.2-cp311-cp311-macosx_10_9_x86_64.whl", hash = "sha256:573f6eac48f4769d667c4442081b1794f52919e7edada77495aaed9236d13a96"},
|
||||
{file = "charset_normalizer-3.3.2-cp311-cp311-macosx_11_0_arm64.whl", hash = "sha256:549a3a73da901d5bc3ce8d24e0600d1fa85524c10287f6004fbab87672bf3e1e"},
|
||||
{file = "charset_normalizer-3.3.2-cp311-cp311-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:f27273b60488abe721a075bcca6d7f3964f9f6f067c8c4c605743023d7d3944f"},
|
||||
{file = "charset_normalizer-3.3.2-cp311-cp311-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", hash = "sha256:1ceae2f17a9c33cb48e3263960dc5fc8005351ee19db217e9b1bb15d28c02574"},
|
||||
{file = "charset_normalizer-3.3.2-cp311-cp311-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:65f6f63034100ead094b8744b3b97965785388f308a64cf8d7c34f2f2e5be0c4"},
|
||||
{file = "charset_normalizer-3.3.2-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:753f10e867343b4511128c6ed8c82f7bec3bd026875576dfd88483c5c73b2fd8"},
|
||||
{file = "charset_normalizer-3.3.2-cp311-cp311-manylinux_2_5_i686.manylinux1_i686.manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:4a78b2b446bd7c934f5dcedc588903fb2f5eec172f3d29e52a9096a43722adfc"},
|
||||
{file = "charset_normalizer-3.3.2-cp311-cp311-musllinux_1_1_aarch64.whl", hash = "sha256:e537484df0d8f426ce2afb2d0f8e1c3d0b114b83f8850e5f2fbea0e797bd82ae"},
|
||||
{file = "charset_normalizer-3.3.2-cp311-cp311-musllinux_1_1_i686.whl", hash = "sha256:eb6904c354526e758fda7167b33005998fb68c46fbc10e013ca97f21ca5c8887"},
|
||||
{file = "charset_normalizer-3.3.2-cp311-cp311-musllinux_1_1_ppc64le.whl", hash = "sha256:deb6be0ac38ece9ba87dea880e438f25ca3eddfac8b002a2ec3d9183a454e8ae"},
|
||||
{file = "charset_normalizer-3.3.2-cp311-cp311-musllinux_1_1_s390x.whl", hash = "sha256:4ab2fe47fae9e0f9dee8c04187ce5d09f48eabe611be8259444906793ab7cbce"},
|
||||
{file = "charset_normalizer-3.3.2-cp311-cp311-musllinux_1_1_x86_64.whl", hash = "sha256:80402cd6ee291dcb72644d6eac93785fe2c8b9cb30893c1af5b8fdd753b9d40f"},
|
||||
{file = "charset_normalizer-3.3.2-cp311-cp311-win32.whl", hash = "sha256:7cd13a2e3ddeed6913a65e66e94b51d80a041145a026c27e6bb76c31a853c6ab"},
|
||||
{file = "charset_normalizer-3.3.2-cp311-cp311-win_amd64.whl", hash = "sha256:663946639d296df6a2bb2aa51b60a2454ca1cb29835324c640dafb5ff2131a77"},
|
||||
{file = "charset_normalizer-3.3.2-cp312-cp312-macosx_10_9_universal2.whl", hash = "sha256:0b2b64d2bb6d3fb9112bafa732def486049e63de9618b5843bcdd081d8144cd8"},
|
||||
{file = "charset_normalizer-3.3.2-cp312-cp312-macosx_10_9_x86_64.whl", hash = "sha256:ddbb2551d7e0102e7252db79ba445cdab71b26640817ab1e3e3648dad515003b"},
|
||||
{file = "charset_normalizer-3.3.2-cp312-cp312-macosx_11_0_arm64.whl", hash = "sha256:55086ee1064215781fff39a1af09518bc9255b50d6333f2e4c74ca09fac6a8f6"},
|
||||
{file = "charset_normalizer-3.3.2-cp312-cp312-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:8f4a014bc36d3c57402e2977dada34f9c12300af536839dc38c0beab8878f38a"},
|
||||
{file = "charset_normalizer-3.3.2-cp312-cp312-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", hash = "sha256:a10af20b82360ab00827f916a6058451b723b4e65030c5a18577c8b2de5b3389"},
|
||||
{file = "charset_normalizer-3.3.2-cp312-cp312-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:8d756e44e94489e49571086ef83b2bb8ce311e730092d2c34ca8f7d925cb20aa"},
|
||||
{file = "charset_normalizer-3.3.2-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:90d558489962fd4918143277a773316e56c72da56ec7aa3dc3dbbe20fdfed15b"},
|
||||
{file = "charset_normalizer-3.3.2-cp312-cp312-manylinux_2_5_i686.manylinux1_i686.manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:6ac7ffc7ad6d040517be39eb591cac5ff87416c2537df6ba3cba3bae290c0fed"},
|
||||
{file = "charset_normalizer-3.3.2-cp312-cp312-musllinux_1_1_aarch64.whl", hash = "sha256:7ed9e526742851e8d5cc9e6cf41427dfc6068d4f5a3bb03659444b4cabf6bc26"},
|
||||
{file = "charset_normalizer-3.3.2-cp312-cp312-musllinux_1_1_i686.whl", hash = "sha256:8bdb58ff7ba23002a4c5808d608e4e6c687175724f54a5dade5fa8c67b604e4d"},
|
||||
{file = "charset_normalizer-3.3.2-cp312-cp312-musllinux_1_1_ppc64le.whl", hash = "sha256:6b3251890fff30ee142c44144871185dbe13b11bab478a88887a639655be1068"},
|
||||
{file = "charset_normalizer-3.3.2-cp312-cp312-musllinux_1_1_s390x.whl", hash = "sha256:b4a23f61ce87adf89be746c8a8974fe1c823c891d8f86eb218bb957c924bb143"},
|
||||
{file = "charset_normalizer-3.3.2-cp312-cp312-musllinux_1_1_x86_64.whl", hash = "sha256:efcb3f6676480691518c177e3b465bcddf57cea040302f9f4e6e191af91174d4"},
|
||||
{file = "charset_normalizer-3.3.2-cp312-cp312-win32.whl", hash = "sha256:d965bba47ddeec8cd560687584e88cf699fd28f192ceb452d1d7ee807c5597b7"},
|
||||
{file = "charset_normalizer-3.3.2-cp312-cp312-win_amd64.whl", hash = "sha256:96b02a3dc4381e5494fad39be677abcb5e6634bf7b4fa83a6dd3112607547001"},
|
||||
{file = "charset_normalizer-3.3.2-cp37-cp37m-macosx_10_9_x86_64.whl", hash = "sha256:95f2a5796329323b8f0512e09dbb7a1860c46a39da62ecb2324f116fa8fdc85c"},
|
||||
{file = "charset_normalizer-3.3.2-cp37-cp37m-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:c002b4ffc0be611f0d9da932eb0f704fe2602a9a949d1f738e4c34c75b0863d5"},
|
||||
{file = "charset_normalizer-3.3.2-cp37-cp37m-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", hash = "sha256:a981a536974bbc7a512cf44ed14938cf01030a99e9b3a06dd59578882f06f985"},
|
||||
{file = "charset_normalizer-3.3.2-cp37-cp37m-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:3287761bc4ee9e33561a7e058c72ac0938c4f57fe49a09eae428fd88aafe7bb6"},
|
||||
{file = "charset_normalizer-3.3.2-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:42cb296636fcc8b0644486d15c12376cb9fa75443e00fb25de0b8602e64c1714"},
|
||||
{file = "charset_normalizer-3.3.2-cp37-cp37m-manylinux_2_5_i686.manylinux1_i686.manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:0a55554a2fa0d408816b3b5cedf0045f4b8e1a6065aec45849de2d6f3f8e9786"},
|
||||
{file = "charset_normalizer-3.3.2-cp37-cp37m-musllinux_1_1_aarch64.whl", hash = "sha256:c083af607d2515612056a31f0a8d9e0fcb5876b7bfc0abad3ecd275bc4ebc2d5"},
|
||||
{file = "charset_normalizer-3.3.2-cp37-cp37m-musllinux_1_1_i686.whl", hash = "sha256:87d1351268731db79e0f8e745d92493ee2841c974128ef629dc518b937d9194c"},
|
||||
{file = "charset_normalizer-3.3.2-cp37-cp37m-musllinux_1_1_ppc64le.whl", hash = "sha256:bd8f7df7d12c2db9fab40bdd87a7c09b1530128315d047a086fa3ae3435cb3a8"},
|
||||
{file = "charset_normalizer-3.3.2-cp37-cp37m-musllinux_1_1_s390x.whl", hash = "sha256:c180f51afb394e165eafe4ac2936a14bee3eb10debc9d9e4db8958fe36afe711"},
|
||||
{file = "charset_normalizer-3.3.2-cp37-cp37m-musllinux_1_1_x86_64.whl", hash = "sha256:8c622a5fe39a48f78944a87d4fb8a53ee07344641b0562c540d840748571b811"},
|
||||
{file = "charset_normalizer-3.3.2-cp37-cp37m-win32.whl", hash = "sha256:db364eca23f876da6f9e16c9da0df51aa4f104a972735574842618b8c6d999d4"},
|
||||
{file = "charset_normalizer-3.3.2-cp37-cp37m-win_amd64.whl", hash = "sha256:86216b5cee4b06df986d214f664305142d9c76df9b6512be2738aa72a2048f99"},
|
||||
{file = "charset_normalizer-3.3.2-cp38-cp38-macosx_10_9_universal2.whl", hash = "sha256:6463effa3186ea09411d50efc7d85360b38d5f09b870c48e4600f63af490e56a"},
|
||||
{file = "charset_normalizer-3.3.2-cp38-cp38-macosx_10_9_x86_64.whl", hash = "sha256:6c4caeef8fa63d06bd437cd4bdcf3ffefe6738fb1b25951440d80dc7df8c03ac"},
|
||||
{file = "charset_normalizer-3.3.2-cp38-cp38-macosx_11_0_arm64.whl", hash = "sha256:37e55c8e51c236f95b033f6fb391d7d7970ba5fe7ff453dad675e88cf303377a"},
|
||||
{file = "charset_normalizer-3.3.2-cp38-cp38-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:fb69256e180cb6c8a894fee62b3afebae785babc1ee98b81cdf68bbca1987f33"},
|
||||
{file = "charset_normalizer-3.3.2-cp38-cp38-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", hash = "sha256:ae5f4161f18c61806f411a13b0310bea87f987c7d2ecdbdaad0e94eb2e404238"},
|
||||
{file = "charset_normalizer-3.3.2-cp38-cp38-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:b2b0a0c0517616b6869869f8c581d4eb2dd83a4d79e0ebcb7d373ef9956aeb0a"},
|
||||
{file = "charset_normalizer-3.3.2-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:45485e01ff4d3630ec0d9617310448a8702f70e9c01906b0d0118bdf9d124cf2"},
|
||||
{file = "charset_normalizer-3.3.2-cp38-cp38-manylinux_2_5_i686.manylinux1_i686.manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:eb00ed941194665c332bf8e078baf037d6c35d7c4f3102ea2d4f16ca94a26dc8"},
|
||||
{file = "charset_normalizer-3.3.2-cp38-cp38-musllinux_1_1_aarch64.whl", hash = "sha256:2127566c664442652f024c837091890cb1942c30937add288223dc895793f898"},
|
||||
{file = "charset_normalizer-3.3.2-cp38-cp38-musllinux_1_1_i686.whl", hash = "sha256:a50aebfa173e157099939b17f18600f72f84eed3049e743b68ad15bd69b6bf99"},
|
||||
{file = "charset_normalizer-3.3.2-cp38-cp38-musllinux_1_1_ppc64le.whl", hash = "sha256:4d0d1650369165a14e14e1e47b372cfcb31d6ab44e6e33cb2d4e57265290044d"},
|
||||
{file = "charset_normalizer-3.3.2-cp38-cp38-musllinux_1_1_s390x.whl", hash = "sha256:923c0c831b7cfcb071580d3f46c4baf50f174be571576556269530f4bbd79d04"},
|
||||
{file = "charset_normalizer-3.3.2-cp38-cp38-musllinux_1_1_x86_64.whl", hash = "sha256:06a81e93cd441c56a9b65d8e1d043daeb97a3d0856d177d5c90ba85acb3db087"},
|
||||
{file = "charset_normalizer-3.3.2-cp38-cp38-win32.whl", hash = "sha256:6ef1d82a3af9d3eecdba2321dc1b3c238245d890843e040e41e470ffa64c3e25"},
|
||||
{file = "charset_normalizer-3.3.2-cp38-cp38-win_amd64.whl", hash = "sha256:eb8821e09e916165e160797a6c17edda0679379a4be5c716c260e836e122f54b"},
|
||||
{file = "charset_normalizer-3.3.2-cp39-cp39-macosx_10_9_universal2.whl", hash = "sha256:c235ebd9baae02f1b77bcea61bce332cb4331dc3617d254df3323aa01ab47bd4"},
|
||||
{file = "charset_normalizer-3.3.2-cp39-cp39-macosx_10_9_x86_64.whl", hash = "sha256:5b4c145409bef602a690e7cfad0a15a55c13320ff7a3ad7ca59c13bb8ba4d45d"},
|
||||
{file = "charset_normalizer-3.3.2-cp39-cp39-macosx_11_0_arm64.whl", hash = "sha256:68d1f8a9e9e37c1223b656399be5d6b448dea850bed7d0f87a8311f1ff3dabb0"},
|
||||
{file = "charset_normalizer-3.3.2-cp39-cp39-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:22afcb9f253dac0696b5a4be4a1c0f8762f8239e21b99680099abd9b2b1b2269"},
|
||||
{file = "charset_normalizer-3.3.2-cp39-cp39-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", hash = "sha256:e27ad930a842b4c5eb8ac0016b0a54f5aebbe679340c26101df33424142c143c"},
|
||||
{file = "charset_normalizer-3.3.2-cp39-cp39-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:1f79682fbe303db92bc2b1136016a38a42e835d932bab5b3b1bfcfbf0640e519"},
|
||||
{file = "charset_normalizer-3.3.2-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:b261ccdec7821281dade748d088bb6e9b69e6d15b30652b74cbbac25e280b796"},
|
||||
{file = "charset_normalizer-3.3.2-cp39-cp39-manylinux_2_5_i686.manylinux1_i686.manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:122c7fa62b130ed55f8f285bfd56d5f4b4a5b503609d181f9ad85e55c89f4185"},
|
||||
{file = "charset_normalizer-3.3.2-cp39-cp39-musllinux_1_1_aarch64.whl", hash = "sha256:d0eccceffcb53201b5bfebb52600a5fb483a20b61da9dbc885f8b103cbe7598c"},
|
||||
{file = "charset_normalizer-3.3.2-cp39-cp39-musllinux_1_1_i686.whl", hash = "sha256:9f96df6923e21816da7e0ad3fd47dd8f94b2a5ce594e00677c0013018b813458"},
|
||||
{file = "charset_normalizer-3.3.2-cp39-cp39-musllinux_1_1_ppc64le.whl", hash = "sha256:7f04c839ed0b6b98b1a7501a002144b76c18fb1c1850c8b98d458ac269e26ed2"},
|
||||
{file = "charset_normalizer-3.3.2-cp39-cp39-musllinux_1_1_s390x.whl", hash = "sha256:34d1c8da1e78d2e001f363791c98a272bb734000fcef47a491c1e3b0505657a8"},
|
||||
{file = "charset_normalizer-3.3.2-cp39-cp39-musllinux_1_1_x86_64.whl", hash = "sha256:ff8fa367d09b717b2a17a052544193ad76cd49979c805768879cb63d9ca50561"},
|
||||
{file = "charset_normalizer-3.3.2-cp39-cp39-win32.whl", hash = "sha256:aed38f6e4fb3f5d6bf81bfa990a07806be9d83cf7bacef998ab1a9bd660a581f"},
|
||||
{file = "charset_normalizer-3.3.2-cp39-cp39-win_amd64.whl", hash = "sha256:b01b88d45a6fcb69667cd6d2f7a9aeb4bf53760d7fc536bf679ec94fe9f3ff3d"},
|
||||
{file = "charset_normalizer-3.3.2-py3-none-any.whl", hash = "sha256:3e4d1f6587322d2788836a99c69062fbb091331ec940e02d12d179c1d53e25fc"},
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "codespell"
|
||||
version = "2.3.0"
|
||||
description = "Codespell"
|
||||
optional = false
|
||||
python-versions = ">=3.8"
|
||||
files = [
|
||||
{file = "codespell-2.3.0-py3-none-any.whl", hash = "sha256:a9c7cef2501c9cfede2110fd6d4e5e62296920efe9abfb84648df866e47f58d1"},
|
||||
{file = "codespell-2.3.0.tar.gz", hash = "sha256:360c7d10f75e65f67bad720af7007e1060a5d395670ec11a7ed1fed9dd17471f"},
|
||||
]
|
||||
|
||||
[package.extras]
|
||||
dev = ["Pygments", "build", "chardet", "pre-commit", "pytest", "pytest-cov", "pytest-dependency", "ruff", "tomli", "twine"]
|
||||
hard-encoding-detection = ["chardet"]
|
||||
toml = ["tomli"]
|
||||
types = ["chardet (>=5.1.0)", "mypy", "pytest", "pytest-cov", "pytest-dependency"]
|
||||
|
||||
[[package]]
|
||||
name = "colorama"
|
||||
version = "0.4.6"
|
||||
description = "Cross-platform colored terminal text."
|
||||
optional = false
|
||||
python-versions = "!=3.0.*,!=3.1.*,!=3.2.*,!=3.3.*,!=3.4.*,!=3.5.*,!=3.6.*,>=2.7"
|
||||
files = [
|
||||
{file = "colorama-0.4.6-py2.py3-none-any.whl", hash = "sha256:4f1d9991f5acc0ca119f9d443620b77f9d6b33703e51011c16baf57afb285fc6"},
|
||||
{file = "colorama-0.4.6.tar.gz", hash = "sha256:08695f5cb7ed6e0531a20572697297273c47b8cae5a63ffc6d6ed5c201be6e44"},
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "dataclasses-json"
|
||||
version = "0.6.7"
|
||||
description = "Easily serialize dataclasses to and from JSON."
|
||||
optional = false
|
||||
python-versions = "<4.0,>=3.7"
|
||||
files = [
|
||||
{file = "dataclasses_json-0.6.7-py3-none-any.whl", hash = "sha256:0dbf33f26c8d5305befd61b39d2b3414e8a407bedc2834dea9b8d642666fb40a"},
|
||||
{file = "dataclasses_json-0.6.7.tar.gz", hash = "sha256:b6b3e528266ea45b9535223bc53ca645f5208833c29229e847b3f26a1cc55fc0"},
|
||||
]
|
||||
|
||||
[package.dependencies]
|
||||
marshmallow = ">=3.18.0,<4.0.0"
|
||||
typing-inspect = ">=0.4.0,<1"
|
||||
|
||||
[[package]]
|
||||
name = "exceptiongroup"
|
||||
version = "1.2.2"
|
||||
description = "Backport of PEP 654 (exception groups)"
|
||||
optional = false
|
||||
python-versions = ">=3.7"
|
||||
files = [
|
||||
{file = "exceptiongroup-1.2.2-py3-none-any.whl", hash = "sha256:3111b9d131c238bec2f8f516e123e14ba243563fb135d3fe885990585aa7795b"},
|
||||
{file = "exceptiongroup-1.2.2.tar.gz", hash = "sha256:47c2edf7c6738fafb49fd34290706d1a1a2f4d1c6df275526b62cbb4aa5393cc"},
|
||||
]
|
||||
|
||||
[package.extras]
|
||||
test = ["pytest (>=6)"]
|
||||
|
||||
[[package]]
|
||||
name = "idna"
|
||||
version = "3.7"
|
||||
description = "Internationalized Domain Names in Applications (IDNA)"
|
||||
optional = false
|
||||
python-versions = ">=3.5"
|
||||
files = [
|
||||
{file = "idna-3.7-py3-none-any.whl", hash = "sha256:82fee1fc78add43492d3a1898bfa6d8a904cc97d8427f683ed8e798d07761aa0"},
|
||||
{file = "idna-3.7.tar.gz", hash = "sha256:028ff3aadf0609c1fd278d8ea3089299412a7a8b9bd005dd08b9f8285bcb5cfc"},
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "iniconfig"
|
||||
version = "2.0.0"
|
||||
description = "brain-dead simple config-ini parsing"
|
||||
optional = false
|
||||
python-versions = ">=3.7"
|
||||
files = [
|
||||
{file = "iniconfig-2.0.0-py3-none-any.whl", hash = "sha256:b6a85871a79d2e3b22d2d1b94ac2824226a63c6b741c88f7ae975f18b6778374"},
|
||||
{file = "iniconfig-2.0.0.tar.gz", hash = "sha256:2d91e135bf72d31a410b17c16da610a82cb55f6b0477d1a902134b24a455b8b3"},
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "jsonpatch"
|
||||
version = "1.33"
|
||||
description = "Apply JSON-Patches (RFC 6902)"
|
||||
optional = false
|
||||
python-versions = ">=2.7, !=3.0.*, !=3.1.*, !=3.2.*, !=3.3.*, !=3.4.*, !=3.5.*, !=3.6.*"
|
||||
files = [
|
||||
{file = "jsonpatch-1.33-py2.py3-none-any.whl", hash = "sha256:0ae28c0cd062bbd8b8ecc26d7d164fbbea9652a1a3693f3b956c1eae5145dade"},
|
||||
{file = "jsonpatch-1.33.tar.gz", hash = "sha256:9fcd4009c41e6d12348b4a0ff2563ba56a2923a7dfee731d004e212e1ee5030c"},
|
||||
]
|
||||
|
||||
[package.dependencies]
|
||||
jsonpointer = ">=1.9"
|
||||
|
||||
[[package]]
|
||||
name = "jsonpointer"
|
||||
version = "3.0.0"
|
||||
description = "Identify specific nodes in a JSON document (RFC 6901)"
|
||||
optional = false
|
||||
python-versions = ">=3.7"
|
||||
files = [
|
||||
{file = "jsonpointer-3.0.0-py2.py3-none-any.whl", hash = "sha256:13e088adc14fca8b6aa8177c044e12701e6ad4b28ff10e65f2267a90109c9942"},
|
||||
{file = "jsonpointer-3.0.0.tar.gz", hash = "sha256:2b2d729f2091522d61c3b31f82e11870f60b68f43fbc705cb76bf4b832af59ef"},
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "langchain-core"
|
||||
version = "0.2.38"
|
||||
description = "Building applications with LLMs through composability"
|
||||
optional = false
|
||||
python-versions = "<4.0,>=3.8.1"
|
||||
files = [
|
||||
{file = "langchain_core-0.2.38-py3-none-any.whl", hash = "sha256:8a5729bc7e68b4af089af20eff44fe4e7ca21d0e0c87ec21cef7621981fd1a4a"},
|
||||
{file = "langchain_core-0.2.38.tar.gz", hash = "sha256:eb69dbedd344f2ee1f15bcea6c71a05884b867588fadc42d04632e727c1238f3"},
|
||||
]
|
||||
|
||||
[package.dependencies]
|
||||
jsonpatch = ">=1.33,<2.0"
|
||||
langsmith = ">=0.1.75,<0.2.0"
|
||||
packaging = ">=23.2,<25"
|
||||
pydantic = [
|
||||
{version = ">=1,<3", markers = "python_full_version < \"3.12.4\""},
|
||||
{version = ">=2.7.4,<3.0.0", markers = "python_full_version >= \"3.12.4\""},
|
||||
]
|
||||
PyYAML = ">=5.3"
|
||||
tenacity = ">=8.1.0,<8.4.0 || >8.4.0,<9.0.0"
|
||||
typing-extensions = ">=4.7"
|
||||
|
||||
[[package]]
|
||||
name = "langsmith"
|
||||
version = "0.1.93"
|
||||
description = "Client library to connect to the LangSmith LLM Tracing and Evaluation Platform."
|
||||
optional = false
|
||||
python-versions = "<4.0,>=3.8.1"
|
||||
files = [
|
||||
{file = "langsmith-0.1.93-py3-none-any.whl", hash = "sha256:811210b9d5f108f36431bd7b997eb9476a9ecf5a2abd7ddbb606c1cdcf0f43ce"},
|
||||
{file = "langsmith-0.1.93.tar.gz", hash = "sha256:285b6ad3a54f50fa8eb97b5f600acc57d0e37e139dd8cf2111a117d0435ba9b4"},
|
||||
]
|
||||
|
||||
[package.dependencies]
|
||||
orjson = ">=3.9.14,<4.0.0"
|
||||
pydantic = [
|
||||
{version = ">=1,<3", markers = "python_full_version < \"3.12.4\""},
|
||||
{version = ">=2.7.4,<3.0.0", markers = "python_full_version >= \"3.12.4\""},
|
||||
]
|
||||
requests = ">=2,<3"
|
||||
|
||||
[[package]]
|
||||
name = "marshmallow"
|
||||
version = "3.21.3"
|
||||
description = "A lightweight library for converting complex datatypes to and from native Python datatypes."
|
||||
optional = false
|
||||
python-versions = ">=3.8"
|
||||
files = [
|
||||
{file = "marshmallow-3.21.3-py3-none-any.whl", hash = "sha256:86ce7fb914aa865001a4b2092c4c2872d13bc347f3d42673272cabfdbad386f1"},
|
||||
{file = "marshmallow-3.21.3.tar.gz", hash = "sha256:4f57c5e050a54d66361e826f94fba213eb10b67b2fdb02c3e0343ce207ba1662"},
|
||||
]
|
||||
|
||||
[package.dependencies]
|
||||
packaging = ">=17.0"
|
||||
|
||||
[package.extras]
|
||||
dev = ["marshmallow[tests]", "pre-commit (>=3.5,<4.0)", "tox"]
|
||||
docs = ["alabaster (==0.7.16)", "autodocsumm (==0.2.12)", "sphinx (==7.3.7)", "sphinx-issues (==4.1.0)", "sphinx-version-warning (==1.1.2)"]
|
||||
tests = ["pytest", "pytz", "simplejson"]
|
||||
|
||||
[[package]]
|
||||
name = "msgpack"
|
||||
version = "1.1.0"
|
||||
description = "MessagePack serializer"
|
||||
optional = false
|
||||
python-versions = ">=3.8"
|
||||
files = [
|
||||
{file = "msgpack-1.1.0-cp310-cp310-macosx_10_9_universal2.whl", hash = "sha256:7ad442d527a7e358a469faf43fda45aaf4ac3249c8310a82f0ccff9164e5dccd"},
|
||||
{file = "msgpack-1.1.0-cp310-cp310-macosx_10_9_x86_64.whl", hash = "sha256:74bed8f63f8f14d75eec75cf3d04ad581da6b914001b474a5d3cd3372c8cc27d"},
|
||||
{file = "msgpack-1.1.0-cp310-cp310-macosx_11_0_arm64.whl", hash = "sha256:914571a2a5b4e7606997e169f64ce53a8b1e06f2cf2c3a7273aa106236d43dd5"},
|
||||
{file = "msgpack-1.1.0-cp310-cp310-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:c921af52214dcbb75e6bdf6a661b23c3e6417f00c603dd2070bccb5c3ef499f5"},
|
||||
{file = "msgpack-1.1.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:d8ce0b22b890be5d252de90d0e0d119f363012027cf256185fc3d474c44b1b9e"},
|
||||
{file = "msgpack-1.1.0-cp310-cp310-manylinux_2_5_i686.manylinux1_i686.manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:73322a6cc57fcee3c0c57c4463d828e9428275fb85a27aa2aa1a92fdc42afd7b"},
|
||||
{file = "msgpack-1.1.0-cp310-cp310-musllinux_1_2_aarch64.whl", hash = "sha256:e1f3c3d21f7cf67bcf2da8e494d30a75e4cf60041d98b3f79875afb5b96f3a3f"},
|
||||
{file = "msgpack-1.1.0-cp310-cp310-musllinux_1_2_i686.whl", hash = "sha256:64fc9068d701233effd61b19efb1485587560b66fe57b3e50d29c5d78e7fef68"},
|
||||
{file = "msgpack-1.1.0-cp310-cp310-musllinux_1_2_x86_64.whl", hash = "sha256:42f754515e0f683f9c79210a5d1cad631ec3d06cea5172214d2176a42e67e19b"},
|
||||
{file = "msgpack-1.1.0-cp310-cp310-win32.whl", hash = "sha256:3df7e6b05571b3814361e8464f9304c42d2196808e0119f55d0d3e62cd5ea044"},
|
||||
{file = "msgpack-1.1.0-cp310-cp310-win_amd64.whl", hash = "sha256:685ec345eefc757a7c8af44a3032734a739f8c45d1b0ac45efc5d8977aa4720f"},
|
||||
{file = "msgpack-1.1.0-cp311-cp311-macosx_10_9_universal2.whl", hash = "sha256:3d364a55082fb2a7416f6c63ae383fbd903adb5a6cf78c5b96cc6316dc1cedc7"},
|
||||
{file = "msgpack-1.1.0-cp311-cp311-macosx_10_9_x86_64.whl", hash = "sha256:79ec007767b9b56860e0372085f8504db5d06bd6a327a335449508bbee9648fa"},
|
||||
{file = "msgpack-1.1.0-cp311-cp311-macosx_11_0_arm64.whl", hash = "sha256:6ad622bf7756d5a497d5b6836e7fc3752e2dd6f4c648e24b1803f6048596f701"},
|
||||
{file = "msgpack-1.1.0-cp311-cp311-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:8e59bca908d9ca0de3dc8684f21ebf9a690fe47b6be93236eb40b99af28b6ea6"},
|
||||
{file = "msgpack-1.1.0-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:5e1da8f11a3dd397f0a32c76165cf0c4eb95b31013a94f6ecc0b280c05c91b59"},
|
||||
{file = "msgpack-1.1.0-cp311-cp311-manylinux_2_5_i686.manylinux1_i686.manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:452aff037287acb1d70a804ffd022b21fa2bb7c46bee884dbc864cc9024128a0"},
|
||||
{file = "msgpack-1.1.0-cp311-cp311-musllinux_1_2_aarch64.whl", hash = "sha256:8da4bf6d54ceed70e8861f833f83ce0814a2b72102e890cbdfe4b34764cdd66e"},
|
||||
{file = "msgpack-1.1.0-cp311-cp311-musllinux_1_2_i686.whl", hash = "sha256:41c991beebf175faf352fb940bf2af9ad1fb77fd25f38d9142053914947cdbf6"},
|
||||
{file = "msgpack-1.1.0-cp311-cp311-musllinux_1_2_x86_64.whl", hash = "sha256:a52a1f3a5af7ba1c9ace055b659189f6c669cf3657095b50f9602af3a3ba0fe5"},
|
||||
{file = "msgpack-1.1.0-cp311-cp311-win32.whl", hash = "sha256:58638690ebd0a06427c5fe1a227bb6b8b9fdc2bd07701bec13c2335c82131a88"},
|
||||
{file = "msgpack-1.1.0-cp311-cp311-win_amd64.whl", hash = "sha256:fd2906780f25c8ed5d7b323379f6138524ba793428db5d0e9d226d3fa6aa1788"},
|
||||
{file = "msgpack-1.1.0-cp312-cp312-macosx_10_9_universal2.whl", hash = "sha256:d46cf9e3705ea9485687aa4001a76e44748b609d260af21c4ceea7f2212a501d"},
|
||||
{file = "msgpack-1.1.0-cp312-cp312-macosx_10_9_x86_64.whl", hash = "sha256:5dbad74103df937e1325cc4bfeaf57713be0b4f15e1c2da43ccdd836393e2ea2"},
|
||||
{file = "msgpack-1.1.0-cp312-cp312-macosx_11_0_arm64.whl", hash = "sha256:58dfc47f8b102da61e8949708b3eafc3504509a5728f8b4ddef84bd9e16ad420"},
|
||||
{file = "msgpack-1.1.0-cp312-cp312-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:4676e5be1b472909b2ee6356ff425ebedf5142427842aa06b4dfd5117d1ca8a2"},
|
||||
{file = "msgpack-1.1.0-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:17fb65dd0bec285907f68b15734a993ad3fc94332b5bb21b0435846228de1f39"},
|
||||
{file = "msgpack-1.1.0-cp312-cp312-manylinux_2_5_i686.manylinux1_i686.manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:a51abd48c6d8ac89e0cfd4fe177c61481aca2d5e7ba42044fd218cfd8ea9899f"},
|
||||
{file = "msgpack-1.1.0-cp312-cp312-musllinux_1_2_aarch64.whl", hash = "sha256:2137773500afa5494a61b1208619e3871f75f27b03bcfca7b3a7023284140247"},
|
||||
{file = "msgpack-1.1.0-cp312-cp312-musllinux_1_2_i686.whl", hash = "sha256:398b713459fea610861c8a7b62a6fec1882759f308ae0795b5413ff6a160cf3c"},
|
||||
{file = "msgpack-1.1.0-cp312-cp312-musllinux_1_2_x86_64.whl", hash = "sha256:06f5fd2f6bb2a7914922d935d3b8bb4a7fff3a9a91cfce6d06c13bc42bec975b"},
|
||||
{file = "msgpack-1.1.0-cp312-cp312-win32.whl", hash = "sha256:ad33e8400e4ec17ba782f7b9cf868977d867ed784a1f5f2ab46e7ba53b6e1e1b"},
|
||||
{file = "msgpack-1.1.0-cp312-cp312-win_amd64.whl", hash = "sha256:115a7af8ee9e8cddc10f87636767857e7e3717b7a2e97379dc2054712693e90f"},
|
||||
{file = "msgpack-1.1.0-cp313-cp313-macosx_10_13_universal2.whl", hash = "sha256:071603e2f0771c45ad9bc65719291c568d4edf120b44eb36324dcb02a13bfddf"},
|
||||
{file = "msgpack-1.1.0-cp313-cp313-macosx_10_13_x86_64.whl", hash = "sha256:0f92a83b84e7c0749e3f12821949d79485971f087604178026085f60ce109330"},
|
||||
{file = "msgpack-1.1.0-cp313-cp313-macosx_11_0_arm64.whl", hash = "sha256:4a1964df7b81285d00a84da4e70cb1383f2e665e0f1f2a7027e683956d04b734"},
|
||||
{file = "msgpack-1.1.0-cp313-cp313-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:59caf6a4ed0d164055ccff8fe31eddc0ebc07cf7326a2aaa0dbf7a4001cd823e"},
|
||||
{file = "msgpack-1.1.0-cp313-cp313-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:0907e1a7119b337971a689153665764adc34e89175f9a34793307d9def08e6ca"},
|
||||
{file = "msgpack-1.1.0-cp313-cp313-manylinux_2_5_i686.manylinux1_i686.manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:65553c9b6da8166e819a6aa90ad15288599b340f91d18f60b2061f402b9a4915"},
|
||||
{file = "msgpack-1.1.0-cp313-cp313-musllinux_1_2_aarch64.whl", hash = "sha256:7a946a8992941fea80ed4beae6bff74ffd7ee129a90b4dd5cf9c476a30e9708d"},
|
||||
{file = "msgpack-1.1.0-cp313-cp313-musllinux_1_2_i686.whl", hash = "sha256:4b51405e36e075193bc051315dbf29168d6141ae2500ba8cd80a522964e31434"},
|
||||
{file = "msgpack-1.1.0-cp313-cp313-musllinux_1_2_x86_64.whl", hash = "sha256:b4c01941fd2ff87c2a934ee6055bda4ed353a7846b8d4f341c428109e9fcde8c"},
|
||||
{file = "msgpack-1.1.0-cp313-cp313-win32.whl", hash = "sha256:7c9a35ce2c2573bada929e0b7b3576de647b0defbd25f5139dcdaba0ae35a4cc"},
|
||||
{file = "msgpack-1.1.0-cp313-cp313-win_amd64.whl", hash = "sha256:bce7d9e614a04d0883af0b3d4d501171fbfca038f12c77fa838d9f198147a23f"},
|
||||
{file = "msgpack-1.1.0-cp38-cp38-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:c40ffa9a15d74e05ba1fe2681ea33b9caffd886675412612d93ab17b58ea2fec"},
|
||||
{file = "msgpack-1.1.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:f1ba6136e650898082d9d5a5217d5906d1e138024f836ff48691784bbe1adf96"},
|
||||
{file = "msgpack-1.1.0-cp38-cp38-manylinux_2_5_i686.manylinux1_i686.manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:e0856a2b7e8dcb874be44fea031d22e5b3a19121be92a1e098f46068a11b0870"},
|
||||
{file = "msgpack-1.1.0-cp38-cp38-musllinux_1_2_aarch64.whl", hash = "sha256:471e27a5787a2e3f974ba023f9e265a8c7cfd373632247deb225617e3100a3c7"},
|
||||
{file = "msgpack-1.1.0-cp38-cp38-musllinux_1_2_i686.whl", hash = "sha256:646afc8102935a388ffc3914b336d22d1c2d6209c773f3eb5dd4d6d3b6f8c1cb"},
|
||||
{file = "msgpack-1.1.0-cp38-cp38-musllinux_1_2_x86_64.whl", hash = "sha256:13599f8829cfbe0158f6456374e9eea9f44eee08076291771d8ae93eda56607f"},
|
||||
{file = "msgpack-1.1.0-cp38-cp38-win32.whl", hash = "sha256:8a84efb768fb968381e525eeeb3d92857e4985aacc39f3c47ffd00eb4509315b"},
|
||||
{file = "msgpack-1.1.0-cp38-cp38-win_amd64.whl", hash = "sha256:879a7b7b0ad82481c52d3c7eb99bf6f0645dbdec5134a4bddbd16f3506947feb"},
|
||||
{file = "msgpack-1.1.0-cp39-cp39-macosx_10_9_universal2.whl", hash = "sha256:53258eeb7a80fc46f62fd59c876957a2d0e15e6449a9e71842b6d24419d88ca1"},
|
||||
{file = "msgpack-1.1.0-cp39-cp39-macosx_10_9_x86_64.whl", hash = "sha256:7e7b853bbc44fb03fbdba34feb4bd414322180135e2cb5164f20ce1c9795ee48"},
|
||||
{file = "msgpack-1.1.0-cp39-cp39-macosx_11_0_arm64.whl", hash = "sha256:f3e9b4936df53b970513eac1758f3882c88658a220b58dcc1e39606dccaaf01c"},
|
||||
{file = "msgpack-1.1.0-cp39-cp39-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:46c34e99110762a76e3911fc923222472c9d681f1094096ac4102c18319e6468"},
|
||||
{file = "msgpack-1.1.0-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:8a706d1e74dd3dea05cb54580d9bd8b2880e9264856ce5068027eed09680aa74"},
|
||||
{file = "msgpack-1.1.0-cp39-cp39-manylinux_2_5_i686.manylinux1_i686.manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:534480ee5690ab3cbed89d4c8971a5c631b69a8c0883ecfea96c19118510c846"},
|
||||
{file = "msgpack-1.1.0-cp39-cp39-musllinux_1_2_aarch64.whl", hash = "sha256:8cf9e8c3a2153934a23ac160cc4cba0ec035f6867c8013cc6077a79823370346"},
|
||||
{file = "msgpack-1.1.0-cp39-cp39-musllinux_1_2_i686.whl", hash = "sha256:3180065ec2abbe13a4ad37688b61b99d7f9e012a535b930e0e683ad6bc30155b"},
|
||||
{file = "msgpack-1.1.0-cp39-cp39-musllinux_1_2_x86_64.whl", hash = "sha256:c5a91481a3cc573ac8c0d9aace09345d989dc4a0202b7fcb312c88c26d4e71a8"},
|
||||
{file = "msgpack-1.1.0-cp39-cp39-win32.whl", hash = "sha256:f80bc7d47f76089633763f952e67f8214cb7b3ee6bfa489b3cb6a84cfac114cd"},
|
||||
{file = "msgpack-1.1.0-cp39-cp39-win_amd64.whl", hash = "sha256:4d1b7ff2d6146e16e8bd665ac726a89c74163ef8cd39fa8c1087d4e52d3a2325"},
|
||||
{file = "msgpack-1.1.0.tar.gz", hash = "sha256:dd432ccc2c72b914e4cb77afce64aab761c1137cc698be3984eee260bcb2896e"},
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "mypy"
|
||||
version = "1.11.0"
|
||||
description = "Optional static typing for Python"
|
||||
optional = false
|
||||
python-versions = ">=3.8"
|
||||
files = [
|
||||
{file = "mypy-1.11.0-cp310-cp310-macosx_10_9_x86_64.whl", hash = "sha256:a3824187c99b893f90c845bab405a585d1ced4ff55421fdf5c84cb7710995229"},
|
||||
{file = "mypy-1.11.0-cp310-cp310-macosx_11_0_arm64.whl", hash = "sha256:96f8dbc2c85046c81bcddc246232d500ad729cb720da4e20fce3b542cab91287"},
|
||||
{file = "mypy-1.11.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:1a5d8d8dd8613a3e2be3eae829ee891b6b2de6302f24766ff06cb2875f5be9c6"},
|
||||
{file = "mypy-1.11.0-cp310-cp310-musllinux_1_1_x86_64.whl", hash = "sha256:72596a79bbfb195fd41405cffa18210af3811beb91ff946dbcb7368240eed6be"},
|
||||
{file = "mypy-1.11.0-cp310-cp310-win_amd64.whl", hash = "sha256:35ce88b8ed3a759634cb4eb646d002c4cef0a38f20565ee82b5023558eb90c00"},
|
||||
{file = "mypy-1.11.0-cp311-cp311-macosx_10_9_x86_64.whl", hash = "sha256:98790025861cb2c3db8c2f5ad10fc8c336ed2a55f4daf1b8b3f877826b6ff2eb"},
|
||||
{file = "mypy-1.11.0-cp311-cp311-macosx_11_0_arm64.whl", hash = "sha256:25bcfa75b9b5a5f8d67147a54ea97ed63a653995a82798221cca2a315c0238c1"},
|
||||
{file = "mypy-1.11.0-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:0bea2a0e71c2a375c9fa0ede3d98324214d67b3cbbfcbd55ac8f750f85a414e3"},
|
||||
{file = "mypy-1.11.0-cp311-cp311-musllinux_1_1_x86_64.whl", hash = "sha256:d2b3d36baac48e40e3064d2901f2fbd2a2d6880ec6ce6358825c85031d7c0d4d"},
|
||||
{file = "mypy-1.11.0-cp311-cp311-win_amd64.whl", hash = "sha256:d8e2e43977f0e09f149ea69fd0556623919f816764e26d74da0c8a7b48f3e18a"},
|
||||
{file = "mypy-1.11.0-cp312-cp312-macosx_10_9_x86_64.whl", hash = "sha256:1d44c1e44a8be986b54b09f15f2c1a66368eb43861b4e82573026e04c48a9e20"},
|
||||
{file = "mypy-1.11.0-cp312-cp312-macosx_11_0_arm64.whl", hash = "sha256:cea3d0fb69637944dd321f41bc896e11d0fb0b0aa531d887a6da70f6e7473aba"},
|
||||
{file = "mypy-1.11.0-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:a83ec98ae12d51c252be61521aa5731f5512231d0b738b4cb2498344f0b840cd"},
|
||||
{file = "mypy-1.11.0-cp312-cp312-musllinux_1_1_x86_64.whl", hash = "sha256:c7b73a856522417beb78e0fb6d33ef89474e7a622db2653bc1285af36e2e3e3d"},
|
||||
{file = "mypy-1.11.0-cp312-cp312-win_amd64.whl", hash = "sha256:f2268d9fcd9686b61ab64f077be7ffbc6fbcdfb4103e5dd0cc5eaab53a8886c2"},
|
||||
{file = "mypy-1.11.0-cp38-cp38-macosx_10_9_x86_64.whl", hash = "sha256:940bfff7283c267ae6522ef926a7887305945f716a7704d3344d6d07f02df850"},
|
||||
{file = "mypy-1.11.0-cp38-cp38-macosx_11_0_arm64.whl", hash = "sha256:14f9294528b5f5cf96c721f231c9f5b2733164e02c1c018ed1a0eff8a18005ac"},
|
||||
{file = "mypy-1.11.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:d7b54c27783991399046837df5c7c9d325d921394757d09dbcbf96aee4649fe9"},
|
||||
{file = "mypy-1.11.0-cp38-cp38-musllinux_1_1_x86_64.whl", hash = "sha256:65f190a6349dec29c8d1a1cd4aa71284177aee5949e0502e6379b42873eddbe7"},
|
||||
{file = "mypy-1.11.0-cp38-cp38-win_amd64.whl", hash = "sha256:dbe286303241fea8c2ea5466f6e0e6a046a135a7e7609167b07fd4e7baf151bf"},
|
||||
{file = "mypy-1.11.0-cp39-cp39-macosx_10_9_x86_64.whl", hash = "sha256:104e9c1620c2675420abd1f6c44bab7dd33cc85aea751c985006e83dcd001095"},
|
||||
{file = "mypy-1.11.0-cp39-cp39-macosx_11_0_arm64.whl", hash = "sha256:f006e955718ecd8d159cee9932b64fba8f86ee6f7728ca3ac66c3a54b0062abe"},
|
||||
{file = "mypy-1.11.0-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:becc9111ca572b04e7e77131bc708480cc88a911adf3d0239f974c034b78085c"},
|
||||
{file = "mypy-1.11.0-cp39-cp39-musllinux_1_1_x86_64.whl", hash = "sha256:6801319fe76c3f3a3833f2b5af7bd2c17bb93c00026a2a1b924e6762f5b19e13"},
|
||||
{file = "mypy-1.11.0-cp39-cp39-win_amd64.whl", hash = "sha256:c1a184c64521dc549324ec6ef7cbaa6b351912be9cb5edb803c2808a0d7e85ac"},
|
||||
{file = "mypy-1.11.0-py3-none-any.whl", hash = "sha256:56913ec8c7638b0091ef4da6fcc9136896914a9d60d54670a75880c3e5b99ace"},
|
||||
{file = "mypy-1.11.0.tar.gz", hash = "sha256:93743608c7348772fdc717af4aeee1997293a1ad04bc0ea6efa15bf65385c538"},
|
||||
]
|
||||
|
||||
[package.dependencies]
|
||||
mypy-extensions = ">=1.0.0"
|
||||
tomli = {version = ">=1.1.0", markers = "python_version < \"3.11\""}
|
||||
typing-extensions = ">=4.6.0"
|
||||
|
||||
[package.extras]
|
||||
dmypy = ["psutil (>=4.0)"]
|
||||
install-types = ["pip"]
|
||||
mypyc = ["setuptools (>=50)"]
|
||||
reports = ["lxml"]
|
||||
|
||||
[[package]]
|
||||
name = "mypy-extensions"
|
||||
version = "1.0.0"
|
||||
description = "Type system extensions for programs checked with the mypy type checker."
|
||||
optional = false
|
||||
python-versions = ">=3.5"
|
||||
files = [
|
||||
{file = "mypy_extensions-1.0.0-py3-none-any.whl", hash = "sha256:4392f6c0eb8a5668a69e23d168ffa70f0be9ccfd32b5cc2d26a34ae5b844552d"},
|
||||
{file = "mypy_extensions-1.0.0.tar.gz", hash = "sha256:75dbf8955dc00442a438fc4d0666508a9a97b6bd41aa2f0ffe9d2f2725af0782"},
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "orjson"
|
||||
version = "3.10.6"
|
||||
description = "Fast, correct Python JSON library supporting dataclasses, datetimes, and numpy"
|
||||
optional = false
|
||||
python-versions = ">=3.8"
|
||||
files = [
|
||||
{file = "orjson-3.10.6-cp310-cp310-macosx_10_15_x86_64.macosx_11_0_arm64.macosx_10_15_universal2.whl", hash = "sha256:fb0ee33124db6eaa517d00890fc1a55c3bfe1cf78ba4a8899d71a06f2d6ff5c7"},
|
||||
{file = "orjson-3.10.6-cp310-cp310-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:9c1c4b53b24a4c06547ce43e5fee6ec4e0d8fe2d597f4647fc033fd205707365"},
|
||||
{file = "orjson-3.10.6-cp310-cp310-manylinux_2_17_armv7l.manylinux2014_armv7l.whl", hash = "sha256:eadc8fd310edb4bdbd333374f2c8fec6794bbbae99b592f448d8214a5e4050c0"},
|
||||
{file = "orjson-3.10.6-cp310-cp310-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", hash = "sha256:61272a5aec2b2661f4fa2b37c907ce9701e821b2c1285d5c3ab0207ebd358d38"},
|
||||
{file = "orjson-3.10.6-cp310-cp310-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:57985ee7e91d6214c837936dc1608f40f330a6b88bb13f5a57ce5257807da143"},
|
||||
{file = "orjson-3.10.6-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:633a3b31d9d7c9f02d49c4ab4d0a86065c4a6f6adc297d63d272e043472acab5"},
|
||||
{file = "orjson-3.10.6-cp310-cp310-musllinux_1_2_aarch64.whl", hash = "sha256:1c680b269d33ec444afe2bdc647c9eb73166fa47a16d9a75ee56a374f4a45f43"},
|
||||
{file = "orjson-3.10.6-cp310-cp310-musllinux_1_2_x86_64.whl", hash = "sha256:f759503a97a6ace19e55461395ab0d618b5a117e8d0fbb20e70cfd68a47327f2"},
|
||||
{file = "orjson-3.10.6-cp310-none-win32.whl", hash = "sha256:95a0cce17f969fb5391762e5719575217bd10ac5a189d1979442ee54456393f3"},
|
||||
{file = "orjson-3.10.6-cp310-none-win_amd64.whl", hash = "sha256:df25d9271270ba2133cc88ee83c318372bdc0f2cd6f32e7a450809a111efc45c"},
|
||||
{file = "orjson-3.10.6-cp311-cp311-macosx_10_15_x86_64.macosx_11_0_arm64.macosx_10_15_universal2.whl", hash = "sha256:b1ec490e10d2a77c345def52599311849fc063ae0e67cf4f84528073152bb2ba"},
|
||||
{file = "orjson-3.10.6-cp311-cp311-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:55d43d3feb8f19d07e9f01e5b9be4f28801cf7c60d0fa0d279951b18fae1932b"},
|
||||
{file = "orjson-3.10.6-cp311-cp311-manylinux_2_17_armv7l.manylinux2014_armv7l.whl", hash = "sha256:ac3045267e98fe749408eee1593a142e02357c5c99be0802185ef2170086a863"},
|
||||
{file = "orjson-3.10.6-cp311-cp311-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", hash = "sha256:c27bc6a28ae95923350ab382c57113abd38f3928af3c80be6f2ba7eb8d8db0b0"},
|
||||
{file = "orjson-3.10.6-cp311-cp311-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:d27456491ca79532d11e507cadca37fb8c9324a3976294f68fb1eff2dc6ced5a"},
|
||||
{file = "orjson-3.10.6-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:05ac3d3916023745aa3b3b388e91b9166be1ca02b7c7e41045da6d12985685f0"},
|
||||
{file = "orjson-3.10.6-cp311-cp311-musllinux_1_2_aarch64.whl", hash = "sha256:1335d4ef59ab85cab66fe73fd7a4e881c298ee7f63ede918b7faa1b27cbe5212"},
|
||||
{file = "orjson-3.10.6-cp311-cp311-musllinux_1_2_x86_64.whl", hash = "sha256:4bbc6d0af24c1575edc79994c20e1b29e6fb3c6a570371306db0993ecf144dc5"},
|
||||
{file = "orjson-3.10.6-cp311-none-win32.whl", hash = "sha256:450e39ab1f7694465060a0550b3f6d328d20297bf2e06aa947b97c21e5241fbd"},
|
||||
{file = "orjson-3.10.6-cp311-none-win_amd64.whl", hash = "sha256:227df19441372610b20e05bdb906e1742ec2ad7a66ac8350dcfd29a63014a83b"},
|
||||
{file = "orjson-3.10.6-cp312-cp312-macosx_10_15_x86_64.macosx_11_0_arm64.macosx_10_15_universal2.whl", hash = "sha256:ea2977b21f8d5d9b758bb3f344a75e55ca78e3ff85595d248eee813ae23ecdfb"},
|
||||
{file = "orjson-3.10.6-cp312-cp312-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:b6f3d167d13a16ed263b52dbfedff52c962bfd3d270b46b7518365bcc2121eed"},
|
||||
{file = "orjson-3.10.6-cp312-cp312-manylinux_2_17_armv7l.manylinux2014_armv7l.whl", hash = "sha256:f710f346e4c44a4e8bdf23daa974faede58f83334289df80bc9cd12fe82573c7"},
|
||||
{file = "orjson-3.10.6-cp312-cp312-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", hash = "sha256:7275664f84e027dcb1ad5200b8b18373e9c669b2a9ec33d410c40f5ccf4b257e"},
|
||||
{file = "orjson-3.10.6-cp312-cp312-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:0943e4c701196b23c240b3d10ed8ecd674f03089198cf503105b474a4f77f21f"},
|
||||
{file = "orjson-3.10.6-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:446dee5a491b5bc7d8f825d80d9637e7af43f86a331207b9c9610e2f93fee22a"},
|
||||
{file = "orjson-3.10.6-cp312-cp312-musllinux_1_2_aarch64.whl", hash = "sha256:64c81456d2a050d380786413786b057983892db105516639cb5d3ee3c7fd5148"},
|
||||
{file = "orjson-3.10.6-cp312-cp312-musllinux_1_2_x86_64.whl", hash = "sha256:960db0e31c4e52fa0fc3ecbaea5b2d3b58f379e32a95ae6b0ebeaa25b93dfd34"},
|
||||
{file = "orjson-3.10.6-cp312-none-win32.whl", hash = "sha256:a6ea7afb5b30b2317e0bee03c8d34c8181bc5a36f2afd4d0952f378972c4efd5"},
|
||||
{file = "orjson-3.10.6-cp312-none-win_amd64.whl", hash = "sha256:874ce88264b7e655dde4aeaacdc8fd772a7962faadfb41abe63e2a4861abc3dc"},
|
||||
{file = "orjson-3.10.6-cp313-none-win32.whl", hash = "sha256:efdf2c5cde290ae6b83095f03119bdc00303d7a03b42b16c54517baa3c4ca3d0"},
|
||||
{file = "orjson-3.10.6-cp313-none-win_amd64.whl", hash = "sha256:8e190fe7888e2e4392f52cafb9626113ba135ef53aacc65cd13109eb9746c43e"},
|
||||
{file = "orjson-3.10.6-cp38-cp38-macosx_10_15_x86_64.macosx_11_0_arm64.macosx_10_15_universal2.whl", hash = "sha256:66680eae4c4e7fc193d91cfc1353ad6d01b4801ae9b5314f17e11ba55e934183"},
|
||||
{file = "orjson-3.10.6-cp38-cp38-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:caff75b425db5ef8e8f23af93c80f072f97b4fb3afd4af44482905c9f588da28"},
|
||||
{file = "orjson-3.10.6-cp38-cp38-manylinux_2_17_armv7l.manylinux2014_armv7l.whl", hash = "sha256:3722fddb821b6036fd2a3c814f6bd9b57a89dc6337b9924ecd614ebce3271394"},
|
||||
{file = "orjson-3.10.6-cp38-cp38-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", hash = "sha256:c2c116072a8533f2fec435fde4d134610f806bdac20188c7bd2081f3e9e0133f"},
|
||||
{file = "orjson-3.10.6-cp38-cp38-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:6eeb13218c8cf34c61912e9df2de2853f1d009de0e46ea09ccdf3d757896af0a"},
|
||||
{file = "orjson-3.10.6-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:965a916373382674e323c957d560b953d81d7a8603fbeee26f7b8248638bd48b"},
|
||||
{file = "orjson-3.10.6-cp38-cp38-musllinux_1_2_aarch64.whl", hash = "sha256:03c95484d53ed8e479cade8628c9cea00fd9d67f5554764a1110e0d5aa2de96e"},
|
||||
{file = "orjson-3.10.6-cp38-cp38-musllinux_1_2_x86_64.whl", hash = "sha256:e060748a04cccf1e0a6f2358dffea9c080b849a4a68c28b1b907f272b5127e9b"},
|
||||
{file = "orjson-3.10.6-cp38-none-win32.whl", hash = "sha256:738dbe3ef909c4b019d69afc19caf6b5ed0e2f1c786b5d6215fbb7539246e4c6"},
|
||||
{file = "orjson-3.10.6-cp38-none-win_amd64.whl", hash = "sha256:d40f839dddf6a7d77114fe6b8a70218556408c71d4d6e29413bb5f150a692ff7"},
|
||||
{file = "orjson-3.10.6-cp39-cp39-macosx_10_15_x86_64.macosx_11_0_arm64.macosx_10_15_universal2.whl", hash = "sha256:697a35a083c4f834807a6232b3e62c8b280f7a44ad0b759fd4dce748951e70db"},
|
||||
{file = "orjson-3.10.6-cp39-cp39-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:fd502f96bf5ea9a61cbc0b2b5900d0dd68aa0da197179042bdd2be67e51a1e4b"},
|
||||
{file = "orjson-3.10.6-cp39-cp39-manylinux_2_17_armv7l.manylinux2014_armv7l.whl", hash = "sha256:f215789fb1667cdc874c1b8af6a84dc939fd802bf293a8334fce185c79cd359b"},
|
||||
{file = "orjson-3.10.6-cp39-cp39-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", hash = "sha256:a2debd8ddce948a8c0938c8c93ade191d2f4ba4649a54302a7da905a81f00b56"},
|
||||
{file = "orjson-3.10.6-cp39-cp39-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:5410111d7b6681d4b0d65e0f58a13be588d01b473822483f77f513c7f93bd3b2"},
|
||||
{file = "orjson-3.10.6-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:bb1f28a137337fdc18384079fa5726810681055b32b92253fa15ae5656e1dddb"},
|
||||
{file = "orjson-3.10.6-cp39-cp39-musllinux_1_2_aarch64.whl", hash = "sha256:bf2fbbce5fe7cd1aa177ea3eab2b8e6a6bc6e8592e4279ed3db2d62e57c0e1b2"},
|
||||
{file = "orjson-3.10.6-cp39-cp39-musllinux_1_2_x86_64.whl", hash = "sha256:79b9b9e33bd4c517445a62b90ca0cc279b0f1f3970655c3df9e608bc3f91741a"},
|
||||
{file = "orjson-3.10.6-cp39-none-win32.whl", hash = "sha256:30b0a09a2014e621b1adf66a4f705f0809358350a757508ee80209b2d8dae219"},
|
||||
{file = "orjson-3.10.6-cp39-none-win_amd64.whl", hash = "sha256:49e3bc615652617d463069f91b867a4458114c5b104e13b7ae6872e5f79d0844"},
|
||||
{file = "orjson-3.10.6.tar.gz", hash = "sha256:e54b63d0a7c6c54a5f5f726bc93a2078111ef060fec4ecbf34c5db800ca3b3a7"},
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "packaging"
|
||||
version = "24.1"
|
||||
description = "Core utilities for Python packages"
|
||||
optional = false
|
||||
python-versions = ">=3.8"
|
||||
files = [
|
||||
{file = "packaging-24.1-py3-none-any.whl", hash = "sha256:5b8f2217dbdbd2f7f384c41c628544e6d52f2d0f53c6d0c3ea61aa5d1d7ff124"},
|
||||
{file = "packaging-24.1.tar.gz", hash = "sha256:026ed72c8ed3fcce5bf8950572258698927fd1dbda10a5e981cdf0ac37f4f002"},
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "pluggy"
|
||||
version = "1.5.0"
|
||||
description = "plugin and hook calling mechanisms for python"
|
||||
optional = false
|
||||
python-versions = ">=3.8"
|
||||
files = [
|
||||
{file = "pluggy-1.5.0-py3-none-any.whl", hash = "sha256:44e1ad92c8ca002de6377e165f3e0f1be63266ab4d554740532335b9d75ea669"},
|
||||
{file = "pluggy-1.5.0.tar.gz", hash = "sha256:2cffa88e94fdc978c4c574f15f9e59b7f4201d439195c3715ca9e2486f1d0cf1"},
|
||||
]
|
||||
|
||||
[package.extras]
|
||||
dev = ["pre-commit", "tox"]
|
||||
testing = ["pytest", "pytest-benchmark"]
|
||||
|
||||
[[package]]
|
||||
name = "pydantic"
|
||||
version = "2.8.2"
|
||||
description = "Data validation using Python type hints"
|
||||
optional = false
|
||||
python-versions = ">=3.8"
|
||||
files = [
|
||||
{file = "pydantic-2.8.2-py3-none-any.whl", hash = "sha256:73ee9fddd406dc318b885c7a2eab8a6472b68b8fb5ba8150949fc3db939f23c8"},
|
||||
{file = "pydantic-2.8.2.tar.gz", hash = "sha256:6f62c13d067b0755ad1c21a34bdd06c0c12625a22b0fc09c6b149816604f7c2a"},
|
||||
]
|
||||
|
||||
[package.dependencies]
|
||||
annotated-types = ">=0.4.0"
|
||||
pydantic-core = "2.20.1"
|
||||
typing-extensions = [
|
||||
{version = ">=4.6.1", markers = "python_version < \"3.13\""},
|
||||
{version = ">=4.12.2", markers = "python_version >= \"3.13\""},
|
||||
]
|
||||
|
||||
[package.extras]
|
||||
email = ["email-validator (>=2.0.0)"]
|
||||
|
||||
[[package]]
|
||||
name = "pydantic-core"
|
||||
version = "2.20.1"
|
||||
description = "Core functionality for Pydantic validation and serialization"
|
||||
optional = false
|
||||
python-versions = ">=3.8"
|
||||
files = [
|
||||
{file = "pydantic_core-2.20.1-cp310-cp310-macosx_10_12_x86_64.whl", hash = "sha256:3acae97ffd19bf091c72df4d726d552c473f3576409b2a7ca36b2f535ffff4a3"},
|
||||
{file = "pydantic_core-2.20.1-cp310-cp310-macosx_11_0_arm64.whl", hash = "sha256:41f4c96227a67a013e7de5ff8f20fb496ce573893b7f4f2707d065907bffdbd6"},
|
||||
{file = "pydantic_core-2.20.1-cp310-cp310-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:5f239eb799a2081495ea659d8d4a43a8f42cd1fe9ff2e7e436295c38a10c286a"},
|
||||
{file = "pydantic_core-2.20.1-cp310-cp310-manylinux_2_17_armv7l.manylinux2014_armv7l.whl", hash = "sha256:53e431da3fc53360db73eedf6f7124d1076e1b4ee4276b36fb25514544ceb4a3"},
|
||||
{file = "pydantic_core-2.20.1-cp310-cp310-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", hash = "sha256:f1f62b2413c3a0e846c3b838b2ecd6c7a19ec6793b2a522745b0869e37ab5bc1"},
|
||||
{file = "pydantic_core-2.20.1-cp310-cp310-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:5d41e6daee2813ecceea8eda38062d69e280b39df793f5a942fa515b8ed67953"},
|
||||
{file = "pydantic_core-2.20.1-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:3d482efec8b7dc6bfaedc0f166b2ce349df0011f5d2f1f25537ced4cfc34fd98"},
|
||||
{file = "pydantic_core-2.20.1-cp310-cp310-manylinux_2_5_i686.manylinux1_i686.whl", hash = "sha256:e93e1a4b4b33daed65d781a57a522ff153dcf748dee70b40c7258c5861e1768a"},
|
||||
{file = "pydantic_core-2.20.1-cp310-cp310-musllinux_1_1_aarch64.whl", hash = "sha256:e7c4ea22b6739b162c9ecaaa41d718dfad48a244909fe7ef4b54c0b530effc5a"},
|
||||
{file = "pydantic_core-2.20.1-cp310-cp310-musllinux_1_1_x86_64.whl", hash = "sha256:4f2790949cf385d985a31984907fecb3896999329103df4e4983a4a41e13e840"},
|
||||
{file = "pydantic_core-2.20.1-cp310-none-win32.whl", hash = "sha256:5e999ba8dd90e93d57410c5e67ebb67ffcaadcea0ad973240fdfd3a135506250"},
|
||||
{file = "pydantic_core-2.20.1-cp310-none-win_amd64.whl", hash = "sha256:512ecfbefef6dac7bc5eaaf46177b2de58cdf7acac8793fe033b24ece0b9566c"},
|
||||
{file = "pydantic_core-2.20.1-cp311-cp311-macosx_10_12_x86_64.whl", hash = "sha256:d2a8fa9d6d6f891f3deec72f5cc668e6f66b188ab14bb1ab52422fe8e644f312"},
|
||||
{file = "pydantic_core-2.20.1-cp311-cp311-macosx_11_0_arm64.whl", hash = "sha256:175873691124f3d0da55aeea1d90660a6ea7a3cfea137c38afa0a5ffabe37b88"},
|
||||
{file = "pydantic_core-2.20.1-cp311-cp311-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:37eee5b638f0e0dcd18d21f59b679686bbd18917b87db0193ae36f9c23c355fc"},
|
||||
{file = "pydantic_core-2.20.1-cp311-cp311-manylinux_2_17_armv7l.manylinux2014_armv7l.whl", hash = "sha256:25e9185e2d06c16ee438ed39bf62935ec436474a6ac4f9358524220f1b236e43"},
|
||||
{file = "pydantic_core-2.20.1-cp311-cp311-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", hash = "sha256:150906b40ff188a3260cbee25380e7494ee85048584998c1e66df0c7a11c17a6"},
|
||||
{file = "pydantic_core-2.20.1-cp311-cp311-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:8ad4aeb3e9a97286573c03df758fc7627aecdd02f1da04516a86dc159bf70121"},
|
||||
{file = "pydantic_core-2.20.1-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:d3f3ed29cd9f978c604708511a1f9c2fdcb6c38b9aae36a51905b8811ee5cbf1"},
|
||||
{file = "pydantic_core-2.20.1-cp311-cp311-manylinux_2_5_i686.manylinux1_i686.whl", hash = "sha256:b0dae11d8f5ded51699c74d9548dcc5938e0804cc8298ec0aa0da95c21fff57b"},
|
||||
{file = "pydantic_core-2.20.1-cp311-cp311-musllinux_1_1_aarch64.whl", hash = "sha256:faa6b09ee09433b87992fb5a2859efd1c264ddc37280d2dd5db502126d0e7f27"},
|
||||
{file = "pydantic_core-2.20.1-cp311-cp311-musllinux_1_1_x86_64.whl", hash = "sha256:9dc1b507c12eb0481d071f3c1808f0529ad41dc415d0ca11f7ebfc666e66a18b"},
|
||||
{file = "pydantic_core-2.20.1-cp311-none-win32.whl", hash = "sha256:fa2fddcb7107e0d1808086ca306dcade7df60a13a6c347a7acf1ec139aa6789a"},
|
||||
{file = "pydantic_core-2.20.1-cp311-none-win_amd64.whl", hash = "sha256:40a783fb7ee353c50bd3853e626f15677ea527ae556429453685ae32280c19c2"},
|
||||
{file = "pydantic_core-2.20.1-cp312-cp312-macosx_10_12_x86_64.whl", hash = "sha256:595ba5be69b35777474fa07f80fc260ea71255656191adb22a8c53aba4479231"},
|
||||
{file = "pydantic_core-2.20.1-cp312-cp312-macosx_11_0_arm64.whl", hash = "sha256:a4f55095ad087474999ee28d3398bae183a66be4823f753cd7d67dd0153427c9"},
|
||||
{file = "pydantic_core-2.20.1-cp312-cp312-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:f9aa05d09ecf4c75157197f27cdc9cfaeb7c5f15021c6373932bf3e124af029f"},
|
||||
{file = "pydantic_core-2.20.1-cp312-cp312-manylinux_2_17_armv7l.manylinux2014_armv7l.whl", hash = "sha256:e97fdf088d4b31ff4ba35db26d9cc472ac7ef4a2ff2badeabf8d727b3377fc52"},
|
||||
{file = "pydantic_core-2.20.1-cp312-cp312-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", hash = "sha256:bc633a9fe1eb87e250b5c57d389cf28998e4292336926b0b6cdaee353f89a237"},
|
||||
{file = "pydantic_core-2.20.1-cp312-cp312-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:d573faf8eb7e6b1cbbcb4f5b247c60ca8be39fe2c674495df0eb4318303137fe"},
|
||||
{file = "pydantic_core-2.20.1-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:26dc97754b57d2fd00ac2b24dfa341abffc380b823211994c4efac7f13b9e90e"},
|
||||
{file = "pydantic_core-2.20.1-cp312-cp312-manylinux_2_5_i686.manylinux1_i686.whl", hash = "sha256:33499e85e739a4b60c9dac710c20a08dc73cb3240c9a0e22325e671b27b70d24"},
|
||||
{file = "pydantic_core-2.20.1-cp312-cp312-musllinux_1_1_aarch64.whl", hash = "sha256:bebb4d6715c814597f85297c332297c6ce81e29436125ca59d1159b07f423eb1"},
|
||||
{file = "pydantic_core-2.20.1-cp312-cp312-musllinux_1_1_x86_64.whl", hash = "sha256:516d9227919612425c8ef1c9b869bbbee249bc91912c8aaffb66116c0b447ebd"},
|
||||
{file = "pydantic_core-2.20.1-cp312-none-win32.whl", hash = "sha256:469f29f9093c9d834432034d33f5fe45699e664f12a13bf38c04967ce233d688"},
|
||||
{file = "pydantic_core-2.20.1-cp312-none-win_amd64.whl", hash = "sha256:035ede2e16da7281041f0e626459bcae33ed998cca6a0a007a5ebb73414ac72d"},
|
||||
{file = "pydantic_core-2.20.1-cp313-cp313-macosx_10_12_x86_64.whl", hash = "sha256:0827505a5c87e8aa285dc31e9ec7f4a17c81a813d45f70b1d9164e03a813a686"},
|
||||
{file = "pydantic_core-2.20.1-cp313-cp313-macosx_11_0_arm64.whl", hash = "sha256:19c0fa39fa154e7e0b7f82f88ef85faa2a4c23cc65aae2f5aea625e3c13c735a"},
|
||||
{file = "pydantic_core-2.20.1-cp313-cp313-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:4aa223cd1e36b642092c326d694d8bf59b71ddddc94cdb752bbbb1c5c91d833b"},
|
||||
{file = "pydantic_core-2.20.1-cp313-cp313-manylinux_2_17_armv7l.manylinux2014_armv7l.whl", hash = "sha256:c336a6d235522a62fef872c6295a42ecb0c4e1d0f1a3e500fe949415761b8a19"},
|
||||
{file = "pydantic_core-2.20.1-cp313-cp313-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", hash = "sha256:7eb6a0587eded33aeefea9f916899d42b1799b7b14b8f8ff2753c0ac1741edac"},
|
||||
{file = "pydantic_core-2.20.1-cp313-cp313-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:70c8daf4faca8da5a6d655f9af86faf6ec2e1768f4b8b9d0226c02f3d6209703"},
|
||||
{file = "pydantic_core-2.20.1-cp313-cp313-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:e9fa4c9bf273ca41f940bceb86922a7667cd5bf90e95dbb157cbb8441008482c"},
|
||||
{file = "pydantic_core-2.20.1-cp313-cp313-manylinux_2_5_i686.manylinux1_i686.whl", hash = "sha256:11b71d67b4725e7e2a9f6e9c0ac1239bbc0c48cce3dc59f98635efc57d6dac83"},
|
||||
{file = "pydantic_core-2.20.1-cp313-cp313-musllinux_1_1_aarch64.whl", hash = "sha256:270755f15174fb983890c49881e93f8f1b80f0b5e3a3cc1394a255706cabd203"},
|
||||
{file = "pydantic_core-2.20.1-cp313-cp313-musllinux_1_1_x86_64.whl", hash = "sha256:c81131869240e3e568916ef4c307f8b99583efaa60a8112ef27a366eefba8ef0"},
|
||||
{file = "pydantic_core-2.20.1-cp313-none-win32.whl", hash = "sha256:b91ced227c41aa29c672814f50dbb05ec93536abf8f43cd14ec9521ea09afe4e"},
|
||||
{file = "pydantic_core-2.20.1-cp313-none-win_amd64.whl", hash = "sha256:65db0f2eefcaad1a3950f498aabb4875c8890438bc80b19362cf633b87a8ab20"},
|
||||
{file = "pydantic_core-2.20.1-cp38-cp38-macosx_10_12_x86_64.whl", hash = "sha256:4745f4ac52cc6686390c40eaa01d48b18997cb130833154801a442323cc78f91"},
|
||||
{file = "pydantic_core-2.20.1-cp38-cp38-macosx_11_0_arm64.whl", hash = "sha256:a8ad4c766d3f33ba8fd692f9aa297c9058970530a32c728a2c4bfd2616d3358b"},
|
||||
{file = "pydantic_core-2.20.1-cp38-cp38-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:41e81317dd6a0127cabce83c0c9c3fbecceae981c8391e6f1dec88a77c8a569a"},
|
||||
{file = "pydantic_core-2.20.1-cp38-cp38-manylinux_2_17_armv7l.manylinux2014_armv7l.whl", hash = "sha256:04024d270cf63f586ad41fff13fde4311c4fc13ea74676962c876d9577bcc78f"},
|
||||
{file = "pydantic_core-2.20.1-cp38-cp38-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", hash = "sha256:eaad4ff2de1c3823fddf82f41121bdf453d922e9a238642b1dedb33c4e4f98ad"},
|
||||
{file = "pydantic_core-2.20.1-cp38-cp38-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:26ab812fa0c845df815e506be30337e2df27e88399b985d0bb4e3ecfe72df31c"},
|
||||
{file = "pydantic_core-2.20.1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:3c5ebac750d9d5f2706654c638c041635c385596caf68f81342011ddfa1e5598"},
|
||||
{file = "pydantic_core-2.20.1-cp38-cp38-manylinux_2_5_i686.manylinux1_i686.whl", hash = "sha256:2aafc5a503855ea5885559eae883978c9b6d8c8993d67766ee73d82e841300dd"},
|
||||
{file = "pydantic_core-2.20.1-cp38-cp38-musllinux_1_1_aarch64.whl", hash = "sha256:4868f6bd7c9d98904b748a2653031fc9c2f85b6237009d475b1008bfaeb0a5aa"},
|
||||
{file = "pydantic_core-2.20.1-cp38-cp38-musllinux_1_1_x86_64.whl", hash = "sha256:aa2f457b4af386254372dfa78a2eda2563680d982422641a85f271c859df1987"},
|
||||
{file = "pydantic_core-2.20.1-cp38-none-win32.whl", hash = "sha256:225b67a1f6d602de0ce7f6c1c3ae89a4aa25d3de9be857999e9124f15dab486a"},
|
||||
{file = "pydantic_core-2.20.1-cp38-none-win_amd64.whl", hash = "sha256:6b507132dcfc0dea440cce23ee2182c0ce7aba7054576efc65634f080dbe9434"},
|
||||
{file = "pydantic_core-2.20.1-cp39-cp39-macosx_10_12_x86_64.whl", hash = "sha256:b03f7941783b4c4a26051846dea594628b38f6940a2fdc0df00b221aed39314c"},
|
||||
{file = "pydantic_core-2.20.1-cp39-cp39-macosx_11_0_arm64.whl", hash = "sha256:1eedfeb6089ed3fad42e81a67755846ad4dcc14d73698c120a82e4ccf0f1f9f6"},
|
||||
{file = "pydantic_core-2.20.1-cp39-cp39-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:635fee4e041ab9c479e31edda27fcf966ea9614fff1317e280d99eb3e5ab6fe2"},
|
||||
{file = "pydantic_core-2.20.1-cp39-cp39-manylinux_2_17_armv7l.manylinux2014_armv7l.whl", hash = "sha256:77bf3ac639c1ff567ae3b47f8d4cc3dc20f9966a2a6dd2311dcc055d3d04fb8a"},
|
||||
{file = "pydantic_core-2.20.1-cp39-cp39-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", hash = "sha256:7ed1b0132f24beeec5a78b67d9388656d03e6a7c837394f99257e2d55b461611"},
|
||||
{file = "pydantic_core-2.20.1-cp39-cp39-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:c6514f963b023aeee506678a1cf821fe31159b925c4b76fe2afa94cc70b3222b"},
|
||||
{file = "pydantic_core-2.20.1-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:10d4204d8ca33146e761c79f83cc861df20e7ae9f6487ca290a97702daf56006"},
|
||||
{file = "pydantic_core-2.20.1-cp39-cp39-manylinux_2_5_i686.manylinux1_i686.whl", hash = "sha256:2d036c7187b9422ae5b262badb87a20a49eb6c5238b2004e96d4da1231badef1"},
|
||||
{file = "pydantic_core-2.20.1-cp39-cp39-musllinux_1_1_aarch64.whl", hash = "sha256:9ebfef07dbe1d93efb94b4700f2d278494e9162565a54f124c404a5656d7ff09"},
|
||||
{file = "pydantic_core-2.20.1-cp39-cp39-musllinux_1_1_x86_64.whl", hash = "sha256:6b9d9bb600328a1ce523ab4f454859e9d439150abb0906c5a1983c146580ebab"},
|
||||
{file = "pydantic_core-2.20.1-cp39-none-win32.whl", hash = "sha256:784c1214cb6dd1e3b15dd8b91b9a53852aed16671cc3fbe4786f4f1db07089e2"},
|
||||
{file = "pydantic_core-2.20.1-cp39-none-win_amd64.whl", hash = "sha256:d2fe69c5434391727efa54b47a1e7986bb0186e72a41b203df8f5b0a19a4f669"},
|
||||
{file = "pydantic_core-2.20.1-pp310-pypy310_pp73-macosx_10_12_x86_64.whl", hash = "sha256:a45f84b09ac9c3d35dfcf6a27fd0634d30d183205230a0ebe8373a0e8cfa0906"},
|
||||
{file = "pydantic_core-2.20.1-pp310-pypy310_pp73-macosx_11_0_arm64.whl", hash = "sha256:d02a72df14dfdbaf228424573a07af10637bd490f0901cee872c4f434a735b94"},
|
||||
{file = "pydantic_core-2.20.1-pp310-pypy310_pp73-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:d2b27e6af28f07e2f195552b37d7d66b150adbaa39a6d327766ffd695799780f"},
|
||||
{file = "pydantic_core-2.20.1-pp310-pypy310_pp73-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:084659fac3c83fd674596612aeff6041a18402f1e1bc19ca39e417d554468482"},
|
||||
{file = "pydantic_core-2.20.1-pp310-pypy310_pp73-manylinux_2_5_i686.manylinux1_i686.whl", hash = "sha256:242b8feb3c493ab78be289c034a1f659e8826e2233786e36f2893a950a719bb6"},
|
||||
{file = "pydantic_core-2.20.1-pp310-pypy310_pp73-musllinux_1_1_aarch64.whl", hash = "sha256:38cf1c40a921d05c5edc61a785c0ddb4bed67827069f535d794ce6bcded919fc"},
|
||||
{file = "pydantic_core-2.20.1-pp310-pypy310_pp73-musllinux_1_1_x86_64.whl", hash = "sha256:e0bbdd76ce9aa5d4209d65f2b27fc6e5ef1312ae6c5333c26db3f5ade53a1e99"},
|
||||
{file = "pydantic_core-2.20.1-pp310-pypy310_pp73-win_amd64.whl", hash = "sha256:254ec27fdb5b1ee60684f91683be95e5133c994cc54e86a0b0963afa25c8f8a6"},
|
||||
{file = "pydantic_core-2.20.1-pp39-pypy39_pp73-macosx_10_12_x86_64.whl", hash = "sha256:407653af5617f0757261ae249d3fba09504d7a71ab36ac057c938572d1bc9331"},
|
||||
{file = "pydantic_core-2.20.1-pp39-pypy39_pp73-macosx_11_0_arm64.whl", hash = "sha256:c693e916709c2465b02ca0ad7b387c4f8423d1db7b4649c551f27a529181c5ad"},
|
||||
{file = "pydantic_core-2.20.1-pp39-pypy39_pp73-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:5b5ff4911aea936a47d9376fd3ab17e970cc543d1b68921886e7f64bd28308d1"},
|
||||
{file = "pydantic_core-2.20.1-pp39-pypy39_pp73-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:177f55a886d74f1808763976ac4efd29b7ed15c69f4d838bbd74d9d09cf6fa86"},
|
||||
{file = "pydantic_core-2.20.1-pp39-pypy39_pp73-manylinux_2_5_i686.manylinux1_i686.whl", hash = "sha256:964faa8a861d2664f0c7ab0c181af0bea66098b1919439815ca8803ef136fc4e"},
|
||||
{file = "pydantic_core-2.20.1-pp39-pypy39_pp73-musllinux_1_1_aarch64.whl", hash = "sha256:4dd484681c15e6b9a977c785a345d3e378d72678fd5f1f3c0509608da24f2ac0"},
|
||||
{file = "pydantic_core-2.20.1-pp39-pypy39_pp73-musllinux_1_1_x86_64.whl", hash = "sha256:f6d6cff3538391e8486a431569b77921adfcdef14eb18fbf19b7c0a5294d4e6a"},
|
||||
{file = "pydantic_core-2.20.1-pp39-pypy39_pp73-win_amd64.whl", hash = "sha256:a6d511cc297ff0883bc3708b465ff82d7560193169a8b93260f74ecb0a5e08a7"},
|
||||
{file = "pydantic_core-2.20.1.tar.gz", hash = "sha256:26ca695eeee5f9f1aeeb211ffc12f10bcb6f71e2989988fda61dabd65db878d4"},
|
||||
]
|
||||
|
||||
[package.dependencies]
|
||||
typing-extensions = ">=4.6.0,<4.7.0 || >4.7.0"
|
||||
|
||||
[[package]]
|
||||
name = "pytest"
|
||||
version = "7.4.4"
|
||||
description = "pytest: simple powerful testing with Python"
|
||||
optional = false
|
||||
python-versions = ">=3.7"
|
||||
files = [
|
||||
{file = "pytest-7.4.4-py3-none-any.whl", hash = "sha256:b090cdf5ed60bf4c45261be03239c2c1c22df034fbffe691abe93cd80cea01d8"},
|
||||
{file = "pytest-7.4.4.tar.gz", hash = "sha256:2cf0005922c6ace4a3e2ec8b4080eb0d9753fdc93107415332f50ce9e7994280"},
|
||||
]
|
||||
|
||||
[package.dependencies]
|
||||
colorama = {version = "*", markers = "sys_platform == \"win32\""}
|
||||
exceptiongroup = {version = ">=1.0.0rc8", markers = "python_version < \"3.11\""}
|
||||
iniconfig = "*"
|
||||
packaging = "*"
|
||||
pluggy = ">=0.12,<2.0"
|
||||
tomli = {version = ">=1.0.0", markers = "python_version < \"3.11\""}
|
||||
|
||||
[package.extras]
|
||||
testing = ["argcomplete", "attrs (>=19.2.0)", "hypothesis (>=3.56)", "mock", "nose", "pygments (>=2.7.2)", "requests", "setuptools", "xmlschema"]
|
||||
|
||||
[[package]]
|
||||
name = "pytest-asyncio"
|
||||
version = "0.21.2"
|
||||
description = "Pytest support for asyncio"
|
||||
optional = false
|
||||
python-versions = ">=3.7"
|
||||
files = [
|
||||
{file = "pytest_asyncio-0.21.2-py3-none-any.whl", hash = "sha256:ab664c88bb7998f711d8039cacd4884da6430886ae8bbd4eded552ed2004f16b"},
|
||||
{file = "pytest_asyncio-0.21.2.tar.gz", hash = "sha256:d67738fc232b94b326b9d060750beb16e0074210b98dd8b58a5239fa2a154f45"},
|
||||
]
|
||||
|
||||
[package.dependencies]
|
||||
pytest = ">=7.0.0"
|
||||
|
||||
[package.extras]
|
||||
docs = ["sphinx (>=5.3)", "sphinx-rtd-theme (>=1.0)"]
|
||||
testing = ["coverage (>=6.2)", "flaky (>=3.5.0)", "hypothesis (>=5.7.1)", "mypy (>=0.931)", "pytest-trio (>=0.7.0)"]
|
||||
|
||||
[[package]]
|
||||
name = "pytest-mock"
|
||||
version = "3.14.0"
|
||||
description = "Thin-wrapper around the mock package for easier use with pytest"
|
||||
optional = false
|
||||
python-versions = ">=3.8"
|
||||
files = [
|
||||
{file = "pytest-mock-3.14.0.tar.gz", hash = "sha256:2719255a1efeceadbc056d6bf3df3d1c5015530fb40cf347c0f9afac88410bd0"},
|
||||
{file = "pytest_mock-3.14.0-py3-none-any.whl", hash = "sha256:0b72c38033392a5f4621342fe11e9219ac11ec9d375f8e2a0c164539e0d70f6f"},
|
||||
]
|
||||
|
||||
[package.dependencies]
|
||||
pytest = ">=6.2.5"
|
||||
|
||||
[package.extras]
|
||||
dev = ["pre-commit", "pytest-asyncio", "tox"]
|
||||
|
||||
[[package]]
|
||||
name = "pytest-watcher"
|
||||
version = "0.4.2"
|
||||
description = "Automatically rerun your tests on file modifications"
|
||||
optional = false
|
||||
python-versions = "<4.0.0,>=3.7.0"
|
||||
files = [
|
||||
{file = "pytest_watcher-0.4.2-py3-none-any.whl", hash = "sha256:a43949ba67dd8d7e1fd0de5eea44a999081f0aec9f93b4e744264b4c6a3d9bbe"},
|
||||
{file = "pytest_watcher-0.4.2.tar.gz", hash = "sha256:7b292f025ca19617cd7567c228c6187b5087f2da9e4d2cf6e144e5764a0471b0"},
|
||||
]
|
||||
|
||||
[package.dependencies]
|
||||
tomli = {version = ">=2.0.1,<3.0.0", markers = "python_version < \"3.11\""}
|
||||
watchdog = ">=2.0.0"
|
||||
|
||||
[[package]]
|
||||
name = "pyyaml"
|
||||
version = "6.0.1"
|
||||
description = "YAML parser and emitter for Python"
|
||||
optional = false
|
||||
python-versions = ">=3.6"
|
||||
files = [
|
||||
{file = "PyYAML-6.0.1-cp310-cp310-macosx_10_9_x86_64.whl", hash = "sha256:d858aa552c999bc8a8d57426ed01e40bef403cd8ccdd0fc5f6f04a00414cac2a"},
|
||||
{file = "PyYAML-6.0.1-cp310-cp310-macosx_11_0_arm64.whl", hash = "sha256:fd66fc5d0da6d9815ba2cebeb4205f95818ff4b79c3ebe268e75d961704af52f"},
|
||||
{file = "PyYAML-6.0.1-cp310-cp310-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:69b023b2b4daa7548bcfbd4aa3da05b3a74b772db9e23b982788168117739938"},
|
||||
{file = "PyYAML-6.0.1-cp310-cp310-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:81e0b275a9ecc9c0c0c07b4b90ba548307583c125f54d5b6946cfee6360c733d"},
|
||||
{file = "PyYAML-6.0.1-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:ba336e390cd8e4d1739f42dfe9bb83a3cc2e80f567d8805e11b46f4a943f5515"},
|
||||
{file = "PyYAML-6.0.1-cp310-cp310-musllinux_1_1_x86_64.whl", hash = "sha256:326c013efe8048858a6d312ddd31d56e468118ad4cdeda36c719bf5bb6192290"},
|
||||
{file = "PyYAML-6.0.1-cp310-cp310-win32.whl", hash = "sha256:bd4af7373a854424dabd882decdc5579653d7868b8fb26dc7d0e99f823aa5924"},
|
||||
{file = "PyYAML-6.0.1-cp310-cp310-win_amd64.whl", hash = "sha256:fd1592b3fdf65fff2ad0004b5e363300ef59ced41c2e6b3a99d4089fa8c5435d"},
|
||||
{file = "PyYAML-6.0.1-cp311-cp311-macosx_10_9_x86_64.whl", hash = "sha256:6965a7bc3cf88e5a1c3bd2e0b5c22f8d677dc88a455344035f03399034eb3007"},
|
||||
{file = "PyYAML-6.0.1-cp311-cp311-macosx_11_0_arm64.whl", hash = "sha256:f003ed9ad21d6a4713f0a9b5a7a0a79e08dd0f221aff4525a2be4c346ee60aab"},
|
||||
{file = "PyYAML-6.0.1-cp311-cp311-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:42f8152b8dbc4fe7d96729ec2b99c7097d656dc1213a3229ca5383f973a5ed6d"},
|
||||
{file = "PyYAML-6.0.1-cp311-cp311-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:062582fca9fabdd2c8b54a3ef1c978d786e0f6b3a1510e0ac93ef59e0ddae2bc"},
|
||||
{file = "PyYAML-6.0.1-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:d2b04aac4d386b172d5b9692e2d2da8de7bfb6c387fa4f801fbf6fb2e6ba4673"},
|
||||
{file = "PyYAML-6.0.1-cp311-cp311-musllinux_1_1_x86_64.whl", hash = "sha256:e7d73685e87afe9f3b36c799222440d6cf362062f78be1013661b00c5c6f678b"},
|
||||
{file = "PyYAML-6.0.1-cp311-cp311-win32.whl", hash = "sha256:1635fd110e8d85d55237ab316b5b011de701ea0f29d07611174a1b42f1444741"},
|
||||
{file = "PyYAML-6.0.1-cp311-cp311-win_amd64.whl", hash = "sha256:bf07ee2fef7014951eeb99f56f39c9bb4af143d8aa3c21b1677805985307da34"},
|
||||
{file = "PyYAML-6.0.1-cp312-cp312-macosx_10_9_x86_64.whl", hash = "sha256:855fb52b0dc35af121542a76b9a84f8d1cd886ea97c84703eaa6d88e37a2ad28"},
|
||||
{file = "PyYAML-6.0.1-cp312-cp312-macosx_11_0_arm64.whl", hash = "sha256:40df9b996c2b73138957fe23a16a4f0ba614f4c0efce1e9406a184b6d07fa3a9"},
|
||||
{file = "PyYAML-6.0.1-cp312-cp312-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:a08c6f0fe150303c1c6b71ebcd7213c2858041a7e01975da3a99aed1e7a378ef"},
|
||||
{file = "PyYAML-6.0.1-cp312-cp312-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:6c22bec3fbe2524cde73d7ada88f6566758a8f7227bfbf93a408a9d86bcc12a0"},
|
||||
{file = "PyYAML-6.0.1-cp312-cp312-musllinux_1_1_x86_64.whl", hash = "sha256:8d4e9c88387b0f5c7d5f281e55304de64cf7f9c0021a3525bd3b1c542da3b0e4"},
|
||||
{file = "PyYAML-6.0.1-cp312-cp312-win32.whl", hash = "sha256:d483d2cdf104e7c9fa60c544d92981f12ad66a457afae824d146093b8c294c54"},
|
||||
{file = "PyYAML-6.0.1-cp312-cp312-win_amd64.whl", hash = "sha256:0d3304d8c0adc42be59c5f8a4d9e3d7379e6955ad754aa9d6ab7a398b59dd1df"},
|
||||
{file = "PyYAML-6.0.1-cp36-cp36m-macosx_10_9_x86_64.whl", hash = "sha256:50550eb667afee136e9a77d6dc71ae76a44df8b3e51e41b77f6de2932bfe0f47"},
|
||||
{file = "PyYAML-6.0.1-cp36-cp36m-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:1fe35611261b29bd1de0070f0b2f47cb6ff71fa6595c077e42bd0c419fa27b98"},
|
||||
{file = "PyYAML-6.0.1-cp36-cp36m-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:704219a11b772aea0d8ecd7058d0082713c3562b4e271b849ad7dc4a5c90c13c"},
|
||||
{file = "PyYAML-6.0.1-cp36-cp36m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:afd7e57eddb1a54f0f1a974bc4391af8bcce0b444685d936840f125cf046d5bd"},
|
||||
{file = "PyYAML-6.0.1-cp36-cp36m-win32.whl", hash = "sha256:fca0e3a251908a499833aa292323f32437106001d436eca0e6e7833256674585"},
|
||||
{file = "PyYAML-6.0.1-cp36-cp36m-win_amd64.whl", hash = "sha256:f22ac1c3cac4dbc50079e965eba2c1058622631e526bd9afd45fedd49ba781fa"},
|
||||
{file = "PyYAML-6.0.1-cp37-cp37m-macosx_10_9_x86_64.whl", hash = "sha256:b1275ad35a5d18c62a7220633c913e1b42d44b46ee12554e5fd39c70a243d6a3"},
|
||||
{file = "PyYAML-6.0.1-cp37-cp37m-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:18aeb1bf9a78867dc38b259769503436b7c72f7a1f1f4c93ff9a17de54319b27"},
|
||||
{file = "PyYAML-6.0.1-cp37-cp37m-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:596106435fa6ad000c2991a98fa58eeb8656ef2325d7e158344fb33864ed87e3"},
|
||||
{file = "PyYAML-6.0.1-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:baa90d3f661d43131ca170712d903e6295d1f7a0f595074f151c0aed377c9b9c"},
|
||||
{file = "PyYAML-6.0.1-cp37-cp37m-win32.whl", hash = "sha256:9046c58c4395dff28dd494285c82ba00b546adfc7ef001486fbf0324bc174fba"},
|
||||
{file = "PyYAML-6.0.1-cp37-cp37m-win_amd64.whl", hash = "sha256:4fb147e7a67ef577a588a0e2c17b6db51dda102c71de36f8549b6816a96e1867"},
|
||||
{file = "PyYAML-6.0.1-cp38-cp38-macosx_10_9_x86_64.whl", hash = "sha256:1d4c7e777c441b20e32f52bd377e0c409713e8bb1386e1099c2415f26e479595"},
|
||||
{file = "PyYAML-6.0.1-cp38-cp38-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:a0cd17c15d3bb3fa06978b4e8958dcdc6e0174ccea823003a106c7d4d7899ac5"},
|
||||
{file = "PyYAML-6.0.1-cp38-cp38-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:28c119d996beec18c05208a8bd78cbe4007878c6dd15091efb73a30e90539696"},
|
||||
{file = "PyYAML-6.0.1-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:7e07cbde391ba96ab58e532ff4803f79c4129397514e1413a7dc761ccd755735"},
|
||||
{file = "PyYAML-6.0.1-cp38-cp38-musllinux_1_1_x86_64.whl", hash = "sha256:49a183be227561de579b4a36efbb21b3eab9651dd81b1858589f796549873dd6"},
|
||||
{file = "PyYAML-6.0.1-cp38-cp38-win32.whl", hash = "sha256:184c5108a2aca3c5b3d3bf9395d50893a7ab82a38004c8f61c258d4428e80206"},
|
||||
{file = "PyYAML-6.0.1-cp38-cp38-win_amd64.whl", hash = "sha256:1e2722cc9fbb45d9b87631ac70924c11d3a401b2d7f410cc0e3bbf249f2dca62"},
|
||||
{file = "PyYAML-6.0.1-cp39-cp39-macosx_10_9_x86_64.whl", hash = "sha256:9eb6caa9a297fc2c2fb8862bc5370d0303ddba53ba97e71f08023b6cd73d16a8"},
|
||||
{file = "PyYAML-6.0.1-cp39-cp39-macosx_11_0_arm64.whl", hash = "sha256:c8098ddcc2a85b61647b2590f825f3db38891662cfc2fc776415143f599bb859"},
|
||||
{file = "PyYAML-6.0.1-cp39-cp39-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:5773183b6446b2c99bb77e77595dd486303b4faab2b086e7b17bc6bef28865f6"},
|
||||
{file = "PyYAML-6.0.1-cp39-cp39-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:b786eecbdf8499b9ca1d697215862083bd6d2a99965554781d0d8d1ad31e13a0"},
|
||||
{file = "PyYAML-6.0.1-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:bc1bf2925a1ecd43da378f4db9e4f799775d6367bdb94671027b73b393a7c42c"},
|
||||
{file = "PyYAML-6.0.1-cp39-cp39-musllinux_1_1_x86_64.whl", hash = "sha256:04ac92ad1925b2cff1db0cfebffb6ffc43457495c9b3c39d3fcae417d7125dc5"},
|
||||
{file = "PyYAML-6.0.1-cp39-cp39-win32.whl", hash = "sha256:faca3bdcf85b2fc05d06ff3fbc1f83e1391b3e724afa3feba7d13eeab355484c"},
|
||||
{file = "PyYAML-6.0.1-cp39-cp39-win_amd64.whl", hash = "sha256:510c9deebc5c0225e8c96813043e62b680ba2f9c50a08d3724c7f28a747d1486"},
|
||||
{file = "PyYAML-6.0.1.tar.gz", hash = "sha256:bfdf460b1736c775f2ba9f6a92bca30bc2095067b8a9d77876d1fad6cc3b4a43"},
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "requests"
|
||||
version = "2.32.3"
|
||||
description = "Python HTTP for Humans."
|
||||
optional = false
|
||||
python-versions = ">=3.8"
|
||||
files = [
|
||||
{file = "requests-2.32.3-py3-none-any.whl", hash = "sha256:70761cfe03c773ceb22aa2f671b4757976145175cdfca038c02654d061d6dcc6"},
|
||||
{file = "requests-2.32.3.tar.gz", hash = "sha256:55365417734eb18255590a9ff9eb97e9e1da868d4ccd6402399eaf68af20a760"},
|
||||
]
|
||||
|
||||
[package.dependencies]
|
||||
certifi = ">=2017.4.17"
|
||||
charset-normalizer = ">=2,<4"
|
||||
idna = ">=2.5,<4"
|
||||
urllib3 = ">=1.21.1,<3"
|
||||
|
||||
[package.extras]
|
||||
socks = ["PySocks (>=1.5.6,!=1.5.7)"]
|
||||
use-chardet-on-py3 = ["chardet (>=3.0.2,<6)"]
|
||||
|
||||
[[package]]
|
||||
name = "ruff"
|
||||
version = "0.6.2"
|
||||
description = "An extremely fast Python linter and code formatter, written in Rust."
|
||||
optional = false
|
||||
python-versions = ">=3.7"
|
||||
files = [
|
||||
{file = "ruff-0.6.2-py3-none-linux_armv6l.whl", hash = "sha256:5c8cbc6252deb3ea840ad6a20b0f8583caab0c5ef4f9cca21adc5a92b8f79f3c"},
|
||||
{file = "ruff-0.6.2-py3-none-macosx_10_12_x86_64.whl", hash = "sha256:17002fe241e76544448a8e1e6118abecbe8cd10cf68fde635dad480dba594570"},
|
||||
{file = "ruff-0.6.2-py3-none-macosx_11_0_arm64.whl", hash = "sha256:3dbeac76ed13456f8158b8f4fe087bf87882e645c8e8b606dd17b0b66c2c1158"},
|
||||
{file = "ruff-0.6.2-py3-none-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:094600ee88cda325988d3f54e3588c46de5c18dae09d683ace278b11f9d4d534"},
|
||||
{file = "ruff-0.6.2-py3-none-manylinux_2_17_armv7l.manylinux2014_armv7l.whl", hash = "sha256:316d418fe258c036ba05fbf7dfc1f7d3d4096db63431546163b472285668132b"},
|
||||
{file = "ruff-0.6.2-py3-none-manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:d72b8b3abf8a2d51b7b9944a41307d2f442558ccb3859bbd87e6ae9be1694a5d"},
|
||||
{file = "ruff-0.6.2-py3-none-manylinux_2_17_ppc64.manylinux2014_ppc64.whl", hash = "sha256:2aed7e243be68487aa8982e91c6e260982d00da3f38955873aecd5a9204b1d66"},
|
||||
{file = "ruff-0.6.2-py3-none-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl", hash = "sha256:d371f7fc9cec83497fe7cf5eaf5b76e22a8efce463de5f775a1826197feb9df8"},
|
||||
{file = "ruff-0.6.2-py3-none-manylinux_2_17_s390x.manylinux2014_s390x.whl", hash = "sha256:a8f310d63af08f583363dfb844ba8f9417b558199c58a5999215082036d795a1"},
|
||||
{file = "ruff-0.6.2-py3-none-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:7db6880c53c56addb8638fe444818183385ec85eeada1d48fc5abe045301b2f1"},
|
||||
{file = "ruff-0.6.2-py3-none-musllinux_1_2_aarch64.whl", hash = "sha256:1175d39faadd9a50718f478d23bfc1d4da5743f1ab56af81a2b6caf0a2394f23"},
|
||||
{file = "ruff-0.6.2-py3-none-musllinux_1_2_armv7l.whl", hash = "sha256:5b939f9c86d51635fe486585389f54582f0d65b8238e08c327c1534844b3bb9a"},
|
||||
{file = "ruff-0.6.2-py3-none-musllinux_1_2_i686.whl", hash = "sha256:d0d62ca91219f906caf9b187dea50d17353f15ec9bb15aae4a606cd697b49b4c"},
|
||||
{file = "ruff-0.6.2-py3-none-musllinux_1_2_x86_64.whl", hash = "sha256:7438a7288f9d67ed3c8ce4d059e67f7ed65e9fe3aa2ab6f5b4b3610e57e3cb56"},
|
||||
{file = "ruff-0.6.2-py3-none-win32.whl", hash = "sha256:279d5f7d86696df5f9549b56b9b6a7f6c72961b619022b5b7999b15db392a4da"},
|
||||
{file = "ruff-0.6.2-py3-none-win_amd64.whl", hash = "sha256:d9f3469c7dd43cd22eb1c3fc16926fb8258d50cb1b216658a07be95dd117b0f2"},
|
||||
{file = "ruff-0.6.2-py3-none-win_arm64.whl", hash = "sha256:f28fcd2cd0e02bdf739297516d5643a945cc7caf09bd9bcb4d932540a5ea4fa9"},
|
||||
{file = "ruff-0.6.2.tar.gz", hash = "sha256:239ee6beb9e91feb8e0ec384204a763f36cb53fb895a1a364618c6abb076b3be"},
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "tenacity"
|
||||
version = "8.5.0"
|
||||
description = "Retry code until it succeeds"
|
||||
optional = false
|
||||
python-versions = ">=3.8"
|
||||
files = [
|
||||
{file = "tenacity-8.5.0-py3-none-any.whl", hash = "sha256:b594c2a5945830c267ce6b79a166228323ed52718f30302c1359836112346687"},
|
||||
{file = "tenacity-8.5.0.tar.gz", hash = "sha256:8bc6c0c8a09b31e6cad13c47afbed1a567518250a9a171418582ed8d9c20ca78"},
|
||||
]
|
||||
|
||||
[package.extras]
|
||||
doc = ["reno", "sphinx"]
|
||||
test = ["pytest", "tornado (>=4.5)", "typeguard"]
|
||||
|
||||
[[package]]
|
||||
name = "tomli"
|
||||
version = "2.0.1"
|
||||
description = "A lil' TOML parser"
|
||||
optional = false
|
||||
python-versions = ">=3.7"
|
||||
files = [
|
||||
{file = "tomli-2.0.1-py3-none-any.whl", hash = "sha256:939de3e7a6161af0c887ef91b7d41a53e7c5a1ca976325f429cb46ea9bc30ecc"},
|
||||
{file = "tomli-2.0.1.tar.gz", hash = "sha256:de526c12914f0c550d15924c62d72abc48d6fe7364aa87328337a31007fe8a4f"},
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "typing-extensions"
|
||||
version = "4.12.2"
|
||||
description = "Backported and Experimental Type Hints for Python 3.8+"
|
||||
optional = false
|
||||
python-versions = ">=3.8"
|
||||
files = [
|
||||
{file = "typing_extensions-4.12.2-py3-none-any.whl", hash = "sha256:04e5ca0351e0f3f85c6853954072df659d0d13fac324d0072316b67d7794700d"},
|
||||
{file = "typing_extensions-4.12.2.tar.gz", hash = "sha256:1a7ead55c7e559dd4dee8856e3a88b41225abfe1ce8df57b7c13915fe121ffb8"},
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "typing-inspect"
|
||||
version = "0.9.0"
|
||||
description = "Runtime inspection utilities for typing module."
|
||||
optional = false
|
||||
python-versions = "*"
|
||||
files = [
|
||||
{file = "typing_inspect-0.9.0-py3-none-any.whl", hash = "sha256:9ee6fc59062311ef8547596ab6b955e1b8aa46242d854bfc78f4f6b0eff35f9f"},
|
||||
{file = "typing_inspect-0.9.0.tar.gz", hash = "sha256:b23fc42ff6f6ef6954e4852c1fb512cdd18dbea03134f91f856a95ccc9461f78"},
|
||||
]
|
||||
|
||||
[package.dependencies]
|
||||
mypy-extensions = ">=0.3.0"
|
||||
typing-extensions = ">=3.7.4"
|
||||
|
||||
[[package]]
|
||||
name = "urllib3"
|
||||
version = "2.2.2"
|
||||
description = "HTTP library with thread-safe connection pooling, file post, and more."
|
||||
optional = false
|
||||
python-versions = ">=3.8"
|
||||
files = [
|
||||
{file = "urllib3-2.2.2-py3-none-any.whl", hash = "sha256:a448b2f64d686155468037e1ace9f2d2199776e17f0a46610480d311f73e3472"},
|
||||
{file = "urllib3-2.2.2.tar.gz", hash = "sha256:dd505485549a7a552833da5e6063639d0d177c04f23bc3864e41e5dc5f612168"},
|
||||
]
|
||||
|
||||
[package.extras]
|
||||
brotli = ["brotli (>=1.0.9)", "brotlicffi (>=0.8.0)"]
|
||||
h2 = ["h2 (>=4,<5)"]
|
||||
socks = ["pysocks (>=1.5.6,!=1.5.7,<2.0)"]
|
||||
zstd = ["zstandard (>=0.18.0)"]
|
||||
|
||||
[[package]]
|
||||
name = "watchdog"
|
||||
version = "4.0.1"
|
||||
description = "Filesystem events monitoring"
|
||||
optional = false
|
||||
python-versions = ">=3.8"
|
||||
files = [
|
||||
{file = "watchdog-4.0.1-cp310-cp310-macosx_10_9_universal2.whl", hash = "sha256:da2dfdaa8006eb6a71051795856bedd97e5b03e57da96f98e375682c48850645"},
|
||||
{file = "watchdog-4.0.1-cp310-cp310-macosx_10_9_x86_64.whl", hash = "sha256:e93f451f2dfa433d97765ca2634628b789b49ba8b504fdde5837cdcf25fdb53b"},
|
||||
{file = "watchdog-4.0.1-cp310-cp310-macosx_11_0_arm64.whl", hash = "sha256:ef0107bbb6a55f5be727cfc2ef945d5676b97bffb8425650dadbb184be9f9a2b"},
|
||||
{file = "watchdog-4.0.1-cp311-cp311-macosx_10_9_universal2.whl", hash = "sha256:17e32f147d8bf9657e0922c0940bcde863b894cd871dbb694beb6704cfbd2fb5"},
|
||||
{file = "watchdog-4.0.1-cp311-cp311-macosx_10_9_x86_64.whl", hash = "sha256:03e70d2df2258fb6cb0e95bbdbe06c16e608af94a3ffbd2b90c3f1e83eb10767"},
|
||||
{file = "watchdog-4.0.1-cp311-cp311-macosx_11_0_arm64.whl", hash = "sha256:123587af84260c991dc5f62a6e7ef3d1c57dfddc99faacee508c71d287248459"},
|
||||
{file = "watchdog-4.0.1-cp312-cp312-macosx_10_9_universal2.whl", hash = "sha256:093b23e6906a8b97051191a4a0c73a77ecc958121d42346274c6af6520dec175"},
|
||||
{file = "watchdog-4.0.1-cp312-cp312-macosx_10_9_x86_64.whl", hash = "sha256:611be3904f9843f0529c35a3ff3fd617449463cb4b73b1633950b3d97fa4bfb7"},
|
||||
{file = "watchdog-4.0.1-cp312-cp312-macosx_11_0_arm64.whl", hash = "sha256:62c613ad689ddcb11707f030e722fa929f322ef7e4f18f5335d2b73c61a85c28"},
|
||||
{file = "watchdog-4.0.1-cp38-cp38-macosx_10_9_universal2.whl", hash = "sha256:d4925e4bf7b9bddd1c3de13c9b8a2cdb89a468f640e66fbfabaf735bd85b3e35"},
|
||||
{file = "watchdog-4.0.1-cp38-cp38-macosx_10_9_x86_64.whl", hash = "sha256:cad0bbd66cd59fc474b4a4376bc5ac3fc698723510cbb64091c2a793b18654db"},
|
||||
{file = "watchdog-4.0.1-cp38-cp38-macosx_11_0_arm64.whl", hash = "sha256:a3c2c317a8fb53e5b3d25790553796105501a235343f5d2bf23bb8649c2c8709"},
|
||||
{file = "watchdog-4.0.1-cp39-cp39-macosx_10_9_universal2.whl", hash = "sha256:c9904904b6564d4ee8a1ed820db76185a3c96e05560c776c79a6ce5ab71888ba"},
|
||||
{file = "watchdog-4.0.1-cp39-cp39-macosx_10_9_x86_64.whl", hash = "sha256:667f3c579e813fcbad1b784db7a1aaa96524bed53437e119f6a2f5de4db04235"},
|
||||
{file = "watchdog-4.0.1-cp39-cp39-macosx_11_0_arm64.whl", hash = "sha256:d10a681c9a1d5a77e75c48a3b8e1a9f2ae2928eda463e8d33660437705659682"},
|
||||
{file = "watchdog-4.0.1-pp310-pypy310_pp73-macosx_10_9_x86_64.whl", hash = "sha256:0144c0ea9997b92615af1d94afc0c217e07ce2c14912c7b1a5731776329fcfc7"},
|
||||
{file = "watchdog-4.0.1-pp310-pypy310_pp73-macosx_11_0_arm64.whl", hash = "sha256:998d2be6976a0ee3a81fb8e2777900c28641fb5bfbd0c84717d89bca0addcdc5"},
|
||||
{file = "watchdog-4.0.1-pp38-pypy38_pp73-macosx_10_9_x86_64.whl", hash = "sha256:e7921319fe4430b11278d924ef66d4daa469fafb1da679a2e48c935fa27af193"},
|
||||
{file = "watchdog-4.0.1-pp38-pypy38_pp73-macosx_11_0_arm64.whl", hash = "sha256:f0de0f284248ab40188f23380b03b59126d1479cd59940f2a34f8852db710625"},
|
||||
{file = "watchdog-4.0.1-pp39-pypy39_pp73-macosx_10_9_x86_64.whl", hash = "sha256:bca36be5707e81b9e6ce3208d92d95540d4ca244c006b61511753583c81c70dd"},
|
||||
{file = "watchdog-4.0.1-pp39-pypy39_pp73-macosx_11_0_arm64.whl", hash = "sha256:ab998f567ebdf6b1da7dc1e5accfaa7c6992244629c0fdaef062f43249bd8dee"},
|
||||
{file = "watchdog-4.0.1-py3-none-manylinux2014_aarch64.whl", hash = "sha256:dddba7ca1c807045323b6af4ff80f5ddc4d654c8bce8317dde1bd96b128ed253"},
|
||||
{file = "watchdog-4.0.1-py3-none-manylinux2014_armv7l.whl", hash = "sha256:4513ec234c68b14d4161440e07f995f231be21a09329051e67a2118a7a612d2d"},
|
||||
{file = "watchdog-4.0.1-py3-none-manylinux2014_i686.whl", hash = "sha256:4107ac5ab936a63952dea2a46a734a23230aa2f6f9db1291bf171dac3ebd53c6"},
|
||||
{file = "watchdog-4.0.1-py3-none-manylinux2014_ppc64.whl", hash = "sha256:6e8c70d2cd745daec2a08734d9f63092b793ad97612470a0ee4cbb8f5f705c57"},
|
||||
{file = "watchdog-4.0.1-py3-none-manylinux2014_ppc64le.whl", hash = "sha256:f27279d060e2ab24c0aa98363ff906d2386aa6c4dc2f1a374655d4e02a6c5e5e"},
|
||||
{file = "watchdog-4.0.1-py3-none-manylinux2014_s390x.whl", hash = "sha256:f8affdf3c0f0466e69f5b3917cdd042f89c8c63aebdb9f7c078996f607cdb0f5"},
|
||||
{file = "watchdog-4.0.1-py3-none-manylinux2014_x86_64.whl", hash = "sha256:ac7041b385f04c047fcc2951dc001671dee1b7e0615cde772e84b01fbf68ee84"},
|
||||
{file = "watchdog-4.0.1-py3-none-win32.whl", hash = "sha256:206afc3d964f9a233e6ad34618ec60b9837d0582b500b63687e34011e15bb429"},
|
||||
{file = "watchdog-4.0.1-py3-none-win_amd64.whl", hash = "sha256:7577b3c43e5909623149f76b099ac49a1a01ca4e167d1785c76eb52fa585745a"},
|
||||
{file = "watchdog-4.0.1-py3-none-win_ia64.whl", hash = "sha256:d7b9f5f3299e8dd230880b6c55504a1f69cf1e4316275d1b215ebdd8187ec88d"},
|
||||
{file = "watchdog-4.0.1.tar.gz", hash = "sha256:eebaacf674fa25511e8867028d281e602ee6500045b57f43b08778082f7f8b44"},
|
||||
]
|
||||
|
||||
[package.extras]
|
||||
watchmedo = ["PyYAML (>=3.10)"]
|
||||
|
||||
[metadata]
|
||||
lock-version = "2.0"
|
||||
python-versions = "^3.9.0,<4.0"
|
||||
content-hash = "8861f12053a7b4594cd8a218f31b78f861d7e5391017bc9209cc5bccbd6f769c"
|
||||
@@ -1,56 +1,43 @@
|
||||
[build-system]
|
||||
requires = ["hatchling"]
|
||||
build-backend = "hatchling.build"
|
||||
|
||||
[project]
|
||||
[tool.poetry]
|
||||
name = "langgraph-checkpoint"
|
||||
version = "4.0.0"
|
||||
version = "2.0.16"
|
||||
description = "Library with base interfaces for LangGraph checkpoint savers."
|
||||
authors = []
|
||||
requires-python = ">=3.10"
|
||||
readme = "README.md"
|
||||
license = "MIT"
|
||||
license-files = ['LICENSE']
|
||||
dependencies = [
|
||||
"langchain-core>=0.2.38",
|
||||
"ormsgpack>=1.12.0",
|
||||
]
|
||||
readme = "README.md"
|
||||
repository = "https://www.github.com/langchain-ai/langgraph"
|
||||
packages = [{ include = "langgraph" }]
|
||||
|
||||
[project.urls]
|
||||
Source = "https://github.com/langchain-ai/langgraph/tree/main/libs/checkpoint"
|
||||
Twitter = "https://x.com/LangChainAI"
|
||||
Slack = "https://www.langchain.com/join-community"
|
||||
Reddit = "https://www.reddit.com/r/LangChain/"
|
||||
[tool.poetry.dependencies]
|
||||
python = "^3.9.0,<4.0"
|
||||
langchain-core = ">=0.2.38,<0.4"
|
||||
msgpack = "^1.1.0"
|
||||
|
||||
[dependency-groups]
|
||||
test = [
|
||||
"pytest",
|
||||
"pytest-asyncio",
|
||||
"pytest-mock",
|
||||
"pytest-watcher",
|
||||
"dataclasses-json",
|
||||
"numpy",
|
||||
"pandas",
|
||||
"pandas-stubs>=2.2.2.240807",
|
||||
"redis",
|
||||
]
|
||||
lint = [
|
||||
"ruff",
|
||||
"codespell",
|
||||
"mypy",
|
||||
]
|
||||
dev = [
|
||||
{include-group = "test"},
|
||||
{include-group = "lint"},
|
||||
]
|
||||
|
||||
[tool.hatch.build.targets.wheel]
|
||||
include = ["langgraph"]
|
||||
[tool.poetry.group.dev.dependencies]
|
||||
ruff = "^0.6.2"
|
||||
codespell = "^2.2.0"
|
||||
pytest = "^7.2.1"
|
||||
pytest-asyncio = "^0.21.1"
|
||||
pytest-mock = "^3.11.1"
|
||||
pytest-watcher = "^0.4.1"
|
||||
mypy = "^1.10.0"
|
||||
dataclasses-json = "^0.6.7"
|
||||
|
||||
[tool.pytest.ini_options]
|
||||
# --strict-markers will raise errors on unknown marks.
|
||||
# https://docs.pytest.org/en/7.1.x/how-to/mark.html#raising-errors-on-unknown-marks
|
||||
#
|
||||
# https://docs.pytest.org/en/7.1.x/reference/reference.html
|
||||
# --strict-config any warnings encountered while parsing the `pytest`
|
||||
# section of the configuration file raise errors.
|
||||
addopts = "--strict-markers --strict-config --durations=5 -vv"
|
||||
asyncio_mode = "auto"
|
||||
|
||||
|
||||
[build-system]
|
||||
requires = ["poetry-core"]
|
||||
build-backend = "poetry.core.masonry.api"
|
||||
|
||||
[tool.ruff]
|
||||
lint.select = [
|
||||
"E", # pycodestyle
|
||||
@@ -58,10 +45,8 @@ lint.select = [
|
||||
"UP", # pyupgrade
|
||||
"B", # flake8-bugbear
|
||||
"I", # isort
|
||||
"UP", # pyupgrade
|
||||
]
|
||||
lint.ignore = ["E501", "B008"]
|
||||
target-version = "py310"
|
||||
lint.ignore = ["E501", "B008", "UP007", "UP006"]
|
||||
|
||||
[tool.pytest-watcher]
|
||||
now = true
|
||||
|
||||
@@ -5,7 +5,7 @@ import random
|
||||
from collections import Counter, defaultdict
|
||||
from typing import Any
|
||||
|
||||
from langchain_core.embeddings import Embeddings
|
||||
from langgraph.store.base.embed import Embeddings
|
||||
|
||||
|
||||
class CharacterEmbeddings(Embeddings):
|
||||
|
||||
@@ -1,5 +1,4 @@
|
||||
import dataclasses
|
||||
import json
|
||||
import pathlib
|
||||
import re
|
||||
import sys
|
||||
@@ -9,21 +8,14 @@ from datetime import date, datetime, time, timezone
|
||||
from decimal import Decimal
|
||||
from enum import Enum
|
||||
from ipaddress import IPv4Address
|
||||
from zoneinfo import ZoneInfo
|
||||
|
||||
import dataclasses_json
|
||||
import numpy as np
|
||||
import pandas as pd
|
||||
import pytest
|
||||
from pydantic import BaseModel, SecretStr
|
||||
from pydantic.v1 import BaseModel as BaseModelV1
|
||||
from pydantic.v1 import SecretStr as SecretStrV1
|
||||
from zoneinfo import ZoneInfo
|
||||
|
||||
from langgraph.checkpoint.serde.jsonplus import (
|
||||
InvalidModuleError,
|
||||
JsonPlusSerializer,
|
||||
_msgpack_ext_hook_to_json,
|
||||
)
|
||||
from langgraph.checkpoint.serde.jsonplus import JsonPlusSerializer
|
||||
from langgraph.store.base import Item
|
||||
|
||||
|
||||
@@ -62,15 +54,22 @@ class MyDataclass:
|
||||
pass
|
||||
|
||||
|
||||
@dataclasses.dataclass(slots=True)
|
||||
class MyDataclassWSlots:
|
||||
foo: str
|
||||
bar: int
|
||||
inner: InnerDataclass
|
||||
if sys.version_info < (3, 10):
|
||||
|
||||
def something(self) -> None:
|
||||
class MyDataclassWSlots(MyDataclass):
|
||||
pass
|
||||
|
||||
else:
|
||||
|
||||
@dataclasses.dataclass(slots=True)
|
||||
class MyDataclassWSlots:
|
||||
foo: str
|
||||
bar: int
|
||||
inner: InnerDataclass
|
||||
|
||||
def something(self) -> None:
|
||||
pass
|
||||
|
||||
|
||||
class MyEnum(Enum):
|
||||
FOO = "foo"
|
||||
@@ -105,12 +104,15 @@ def test_serde_jsonplus() -> None:
|
||||
"time": current_time,
|
||||
"uid": uid,
|
||||
"timestamp": current_timestamp,
|
||||
"my_rich_dict": {(1, 2, 3): 45},
|
||||
"my_slotted_class": MyDataclassWSlots("bar", 2, InnerDataclass("hello")),
|
||||
"my_dataclass": MyDataclass("foo", 1, InnerDataclass("hello")),
|
||||
"my_enum": MyEnum.FOO,
|
||||
"my_pydantic": MyPydantic(foo="foo", bar=1, inner=InnerPydantic(hello="hello")),
|
||||
"my_pydantic_v1": MyPydanticV1(
|
||||
foo="foo", bar=1, inner=InnerPydanticV1(hello="hello")
|
||||
),
|
||||
"my_secret_str": SecretStr("meow"),
|
||||
"my_secret_str_v1": SecretStrV1("meow"),
|
||||
"person": Person(name="foo"),
|
||||
"a_bool": True,
|
||||
"a_none": None,
|
||||
@@ -128,16 +130,10 @@ def test_serde_jsonplus() -> None:
|
||||
key="my-key",
|
||||
namespace=("a", "name", " "),
|
||||
created_at=datetime(2024, 9, 24, 17, 29, 10, 128397),
|
||||
updated_at=datetime(2024, 9, 24, 17, 29, 11, 128397),
|
||||
updated_at=datetime(2024, 9, 24, 17, 29, 10, 128397),
|
||||
),
|
||||
}
|
||||
|
||||
if sys.version_info < (3, 14):
|
||||
to_serialize["my_pydantic_v1"] = MyPydanticV1(
|
||||
foo="foo", bar=1, inner=InnerPydanticV1(hello="hello")
|
||||
)
|
||||
to_serialize["my_secret_str_v1"] = SecretStrV1("meow")
|
||||
|
||||
serde = JsonPlusSerializer()
|
||||
|
||||
dumped = serde.dumps_typed(to_serialize)
|
||||
@@ -149,140 +145,23 @@ def test_serde_jsonplus() -> None:
|
||||
assert serde.loads_typed(serde.dumps_typed(value)) == value
|
||||
|
||||
surrogates = [
|
||||
"Hello??",
|
||||
"Python??",
|
||||
"Surrogate??",
|
||||
"Example??",
|
||||
"String??",
|
||||
"With??",
|
||||
"Surrogates??",
|
||||
"Embedded??",
|
||||
"In??",
|
||||
"The??",
|
||||
"Text??",
|
||||
"Hello\ud83d\ude00",
|
||||
"Python\ud83d\udc0d",
|
||||
"Surrogate\ud834\udd1e",
|
||||
"Example\ud83c\udf89",
|
||||
"String\ud83c\udfa7",
|
||||
"With\ud83c\udf08",
|
||||
"Surrogates\ud83d\ude0e",
|
||||
"Embedded\ud83d\udcbb",
|
||||
"In\ud83c\udf0e",
|
||||
"The\ud83d\udcd6",
|
||||
"Text\ud83d\udcac",
|
||||
"收花🙄·到",
|
||||
]
|
||||
serde = JsonPlusSerializer(pickle_fallback=False)
|
||||
|
||||
assert serde.loads_typed(serde.dumps_typed(surrogates)) == surrogates
|
||||
|
||||
|
||||
def test_serde_jsonplus_json_mode() -> None:
|
||||
uid = uuid.UUID(int=1)
|
||||
deque_instance = deque([1, 2, 3])
|
||||
tzn = ZoneInfo("America/New_York")
|
||||
ip4 = IPv4Address("192.168.0.1")
|
||||
current_date = date(2024, 4, 19)
|
||||
current_time = time(23, 4, 57, 51022, timezone.max)
|
||||
current_timestamp = datetime(2024, 4, 19, 23, 4, 57, 51022, timezone.max)
|
||||
|
||||
to_serialize = {
|
||||
"path": pathlib.Path("foo", "bar"),
|
||||
"re": re.compile(r"foo", re.DOTALL),
|
||||
"decimal": Decimal("1.10101"),
|
||||
"set": {1, 2, frozenset({1, 2})},
|
||||
"frozen_set": frozenset({1, 2, 3}),
|
||||
"ip4": ip4,
|
||||
"deque": deque_instance,
|
||||
"tzn": tzn,
|
||||
"date": current_date,
|
||||
"time": current_time,
|
||||
"uid": uid,
|
||||
"timestamp": current_timestamp,
|
||||
"my_slotted_class": MyDataclassWSlots("bar", 2, InnerDataclass("hello")),
|
||||
"my_dataclass": MyDataclass("foo", 1, InnerDataclass("hello")),
|
||||
"my_enum": MyEnum.FOO,
|
||||
"my_pydantic": MyPydantic(foo="foo", bar=1, inner=InnerPydantic(hello="hello")),
|
||||
"my_secret_str": SecretStr("meow"),
|
||||
"person": Person(name="foo"),
|
||||
"a_bool": True,
|
||||
"a_none": None,
|
||||
"a_str": "foo",
|
||||
"a_str_nuc": "foo\u0000",
|
||||
"a_str_uc": "foo ⛰️",
|
||||
"a_str_ucuc": "foo \u26f0\ufe0f\u0000",
|
||||
"a_str_ucucuc": "foo \\u26f0\\ufe0f",
|
||||
"an_int": 1,
|
||||
"a_float": 1.1,
|
||||
"a_bytes": b"my bytes",
|
||||
"a_bytearray": bytearray([42]),
|
||||
"my_item": Item(
|
||||
value={},
|
||||
key="my-key",
|
||||
namespace=("a", "name", " "),
|
||||
created_at=datetime(2024, 9, 24, 17, 29, 10, 128397),
|
||||
updated_at=datetime(2024, 9, 24, 17, 29, 11, 128397),
|
||||
),
|
||||
}
|
||||
|
||||
if sys.version_info < (3, 14):
|
||||
to_serialize["my_pydantic_v1"] = MyPydanticV1(
|
||||
foo="foo", bar=1, inner=InnerPydanticV1(hello="hello")
|
||||
)
|
||||
to_serialize["my_secret_str_v1"] = SecretStrV1("meow")
|
||||
|
||||
serde = JsonPlusSerializer(__unpack_ext_hook__=_msgpack_ext_hook_to_json)
|
||||
|
||||
dumped = serde.dumps_typed(to_serialize)
|
||||
|
||||
assert dumped[0] == "msgpack"
|
||||
result = serde.loads_typed(dumped)
|
||||
|
||||
expected_result = {
|
||||
"path": ["foo", "bar"],
|
||||
"re": ["foo", 48],
|
||||
"decimal": "1.10101",
|
||||
"set": [1, 2, [1, 2]],
|
||||
"frozen_set": [1, 2, 3],
|
||||
"ip4": "192.168.0.1",
|
||||
"deque": [1, 2, 3],
|
||||
"tzn": "America/New_York",
|
||||
"date": [2024, 4, 19],
|
||||
"time": {
|
||||
"hour": 23,
|
||||
"minute": 4,
|
||||
"second": 57,
|
||||
"microsecond": 51022,
|
||||
"tzinfo": [[0, 86340, 0]],
|
||||
"fold": 0,
|
||||
},
|
||||
"uid": "00000000-0000-0000-0000-000000000001",
|
||||
"timestamp": "2024-04-19T23:04:57.051022+23:59",
|
||||
"my_slotted_class": {"foo": "bar", "bar": 2, "inner": {"hello": "hello"}},
|
||||
"my_dataclass": {"foo": "foo", "bar": 1, "inner": {"hello": "hello"}},
|
||||
"my_enum": "foo",
|
||||
"my_pydantic": {"foo": "foo", "bar": 1, "inner": {"hello": "hello"}},
|
||||
"my_secret_str": "meow",
|
||||
"person": {"name": "foo"},
|
||||
"a_bool": True,
|
||||
"a_none": None,
|
||||
"a_str": "foo",
|
||||
"a_str_nuc": "foo\x00",
|
||||
"a_str_uc": "foo ⛰️",
|
||||
"a_str_ucuc": "foo ⛰️\x00",
|
||||
"a_str_ucucuc": "foo \\u26f0\\ufe0f",
|
||||
"an_int": 1,
|
||||
"a_float": 1.1,
|
||||
"a_bytes": b"my bytes",
|
||||
"a_bytearray": b"*",
|
||||
"my_item": {
|
||||
"namespace": ["a", "name", " "],
|
||||
"key": "my-key",
|
||||
"value": {},
|
||||
"created_at": "2024-09-24T17:29:10.128397",
|
||||
"updated_at": "2024-09-24T17:29:11.128397",
|
||||
},
|
||||
}
|
||||
|
||||
if sys.version_info < (3, 14):
|
||||
expected_result["my_pydantic_v1"] = {
|
||||
"foo": "foo",
|
||||
"bar": 1,
|
||||
"inner": {"hello": "hello"},
|
||||
}
|
||||
expected_result["my_secret_str_v1"] = "meow"
|
||||
|
||||
assert result == expected_result
|
||||
assert serde.loads_typed(serde.dumps_typed(surrogates)) == [
|
||||
v.encode("utf-8", "ignore").decode() for v in surrogates
|
||||
]
|
||||
|
||||
|
||||
def test_serde_jsonplus_bytes() -> None:
|
||||
@@ -295,20 +174,6 @@ def test_serde_jsonplus_bytes() -> None:
|
||||
assert serde.loads_typed(dumped) == some_bytes
|
||||
|
||||
|
||||
def test_deserde_invalid_module() -> None:
|
||||
serde = JsonPlusSerializer()
|
||||
load = {
|
||||
"lc": 2,
|
||||
"type": "constructor",
|
||||
"id": ["pprint", "pprint"],
|
||||
"kwargs": {"object": "HELLO"},
|
||||
}
|
||||
with pytest.raises(InvalidModuleError):
|
||||
serde._revive_lc2(load)
|
||||
serde = JsonPlusSerializer(allowed_json_modules=[("pprint", "pprint")])
|
||||
serde.loads_typed(("json", json.dumps(load).encode("utf-8")))
|
||||
|
||||
|
||||
def test_serde_jsonplus_bytearray() -> None:
|
||||
serde = JsonPlusSerializer()
|
||||
|
||||
@@ -319,198 +184,19 @@ def test_serde_jsonplus_bytearray() -> None:
|
||||
assert serde.loads_typed(dumped) == some_bytearray
|
||||
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
"arr",
|
||||
[
|
||||
np.arange(9, dtype=np.int32).reshape(3, 3),
|
||||
np.asfortranarray(np.arange(9, dtype=np.float64).reshape(3, 3)),
|
||||
np.arange(12, dtype=np.int16)[::2].reshape(3, 2),
|
||||
],
|
||||
)
|
||||
def test_serde_jsonplus_numpy_array(arr: np.ndarray) -> None:
|
||||
def test_loads_cannot_find() -> None:
|
||||
serde = JsonPlusSerializer()
|
||||
|
||||
dumped = serde.dumps_typed(arr)
|
||||
assert dumped[0] == "msgpack"
|
||||
result = serde.loads_typed(dumped)
|
||||
assert isinstance(result, np.ndarray)
|
||||
assert result.dtype == arr.dtype
|
||||
assert np.array_equal(result, arr)
|
||||
dumped = (
|
||||
"json",
|
||||
b'{"lc": 2, "type": "constructor", "id": ["tests", "test_jsonplus", "MyPydanticccc"], "method": null, "args": [], "kwargs": {"foo": "foo", "bar": 1}}',
|
||||
)
|
||||
|
||||
assert serde.loads_typed(dumped) is None, "Should return None if cannot find class"
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
"arr",
|
||||
[
|
||||
np.arange(6, dtype=np.float32).reshape(2, 3),
|
||||
np.asfortranarray(np.arange(4, dtype=np.complex128).reshape(2, 2)),
|
||||
],
|
||||
)
|
||||
def test_serde_jsonplus_numpy_array_json_hook(arr: np.ndarray) -> None:
|
||||
serde = JsonPlusSerializer(__unpack_ext_hook__=_msgpack_ext_hook_to_json)
|
||||
dumped = serde.dumps_typed(arr)
|
||||
assert dumped[0] == "msgpack"
|
||||
result = serde.loads_typed(dumped)
|
||||
assert isinstance(result, list)
|
||||
assert result == arr.tolist()
|
||||
dumped = (
|
||||
"json",
|
||||
b'{"lc": 2, "type": "constructor", "id": ["tests", "test_jsonpluss", "MyPydantic"], "method": null, "args": [], "kwargs": {"foo": "foo", "bar": 1}}',
|
||||
)
|
||||
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
"df",
|
||||
[
|
||||
pd.DataFrame(),
|
||||
pd.DataFrame({"int_col": [1, 2, 3]}),
|
||||
pd.DataFrame({"float_col": [1.1, 2.2, 3.3]}),
|
||||
pd.DataFrame({"str_col": ["a", "b", "c"]}),
|
||||
pd.DataFrame({"bool_col": [True, False, True]}),
|
||||
pd.DataFrame(
|
||||
{
|
||||
"datetime_col": [
|
||||
datetime(2024, 1, 1),
|
||||
datetime(2024, 1, 2),
|
||||
datetime(2024, 1, 3),
|
||||
]
|
||||
}
|
||||
),
|
||||
pd.DataFrame(
|
||||
{
|
||||
"int_col": [1, 2, 3],
|
||||
"float_col": [1.1, 2.2, 3.3],
|
||||
"str_col": ["a", "b", "c"],
|
||||
}
|
||||
),
|
||||
pd.DataFrame(
|
||||
{
|
||||
"int_col": [1, 2, None],
|
||||
"float_col": [1.1, None, 3.3],
|
||||
"str_col": ["a", None, "c"],
|
||||
}
|
||||
),
|
||||
pytest.param(
|
||||
pd.DataFrame({"cat_col": pd.Categorical(["a", "b", "a", "c"])}),
|
||||
marks=pytest.mark.skipif(
|
||||
sys.version_info >= (3, 14), reason="NotImplementedError on Python 3.14"
|
||||
),
|
||||
),
|
||||
pd.DataFrame(
|
||||
{
|
||||
"int8": pd.array([1, 2, 3], dtype="int8"),
|
||||
"int16": pd.array([10, 20, 30], dtype="int16"),
|
||||
"int32": pd.array([100, 200, 300], dtype="int32"),
|
||||
"int64": pd.array([1000, 2000, 3000], dtype="int64"),
|
||||
"float32": pd.array([1.1, 2.2, 3.3], dtype="float32"),
|
||||
"float64": pd.array([10.1, 20.2, 30.3], dtype="float64"),
|
||||
}
|
||||
),
|
||||
pd.DataFrame({"value": [1, 2, 3]}, index=["x", "y", "z"]),
|
||||
pd.DataFrame(
|
||||
[[1, 2, 3, 4]],
|
||||
columns=pd.MultiIndex.from_tuples(
|
||||
[("A", "X"), ("A", "Y"), ("B", "X"), ("B", "Y")]
|
||||
),
|
||||
),
|
||||
pd.DataFrame(
|
||||
{"value": [1, 2, 3]}, index=pd.date_range("2024-01-01", periods=3, freq="D")
|
||||
),
|
||||
pd.DataFrame(
|
||||
{
|
||||
"col1": range(1000),
|
||||
"col2": [f"str_{i}" for i in range(1000)],
|
||||
"col3": np.random.rand(1000),
|
||||
}
|
||||
),
|
||||
pytest.param(
|
||||
pd.DataFrame(
|
||||
{
|
||||
"tz_datetime": pd.date_range(
|
||||
"2024-01-01", periods=3, freq="D", tz="UTC"
|
||||
)
|
||||
}
|
||||
),
|
||||
marks=pytest.mark.skipif(
|
||||
sys.version_info >= (3, 14), reason="NotImplementedError on Python 3.14"
|
||||
),
|
||||
),
|
||||
pd.DataFrame({"timedelta": pd.to_timedelta([1, 2, 3], unit="D")}),
|
||||
pytest.param(
|
||||
pd.DataFrame({"period": pd.period_range("2024-01", periods=3, freq="M")}),
|
||||
marks=pytest.mark.skipif(
|
||||
sys.version_info >= (3, 14), reason="NotImplementedError on Python 3.14"
|
||||
),
|
||||
),
|
||||
pd.DataFrame({"interval": pd.interval_range(start=0, end=3, periods=3)}),
|
||||
pd.DataFrame({"unicode": ["Hello 🌍", "Python 🐍", "Data 📊"]}),
|
||||
pd.DataFrame({"mixed": [1, "string", [1, 2, 3], {"key": "value"}]}),
|
||||
pd.DataFrame({"a": [1], "b": ["test"], "c": [3.14]}),
|
||||
pd.DataFrame({"single": [42]}),
|
||||
pd.DataFrame(
|
||||
{
|
||||
"small": [sys.float_info.min, 0, sys.float_info.max],
|
||||
"large_int": [-(2**63), 0, 2**63 - 1],
|
||||
}
|
||||
),
|
||||
pd.DataFrame({"special_strings": ["", "null", "None", "NaN", "inf", "-inf"]}),
|
||||
pd.DataFrame({"bytes_col": [b"hello", b"world", b"\x00\x01\x02"]}),
|
||||
],
|
||||
)
|
||||
def test_serde_jsonplus_pandas_dataframe(df: pd.DataFrame) -> None:
|
||||
serde = JsonPlusSerializer(pickle_fallback=True)
|
||||
|
||||
dumped = serde.dumps_typed(df)
|
||||
assert dumped[0] == "pickle"
|
||||
result = serde.loads_typed(dumped)
|
||||
assert result.equals(df)
|
||||
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
"series",
|
||||
[
|
||||
pd.Series([]),
|
||||
pd.Series([1, 2, 3]),
|
||||
pd.Series([1.1, 2.2, 3.3]),
|
||||
pd.Series(["a", "b", "c"]),
|
||||
pd.Series([True, False, True]),
|
||||
pd.Series([datetime(2024, 1, 1), datetime(2024, 1, 2), datetime(2024, 1, 3)]),
|
||||
pd.Series([1, 2, None]),
|
||||
pd.Series([1.1, None, 3.3]),
|
||||
pd.Series(["a", None, "c"]),
|
||||
pytest.param(
|
||||
pd.Series(pd.Categorical(["a", "b", "a", "c"])),
|
||||
marks=pytest.mark.skipif(
|
||||
sys.version_info >= (3, 14), reason="NotImplementedError on Python 3.14"
|
||||
),
|
||||
),
|
||||
pd.Series([1, 2, 3], dtype="int8"),
|
||||
pd.Series([10, 20, 30], dtype="int16"),
|
||||
pd.Series([100, 200, 300], dtype="int32"),
|
||||
pd.Series([1000, 2000, 3000], dtype="int64"),
|
||||
pd.Series([1.1, 2.2, 3.3], dtype="float32"),
|
||||
pd.Series([10.1, 20.2, 30.3], dtype="float64"),
|
||||
pd.Series([1, 2, 3], index=["x", "y", "z"]),
|
||||
pd.Series([1, 2, 3], index=pd.date_range("2024-01-01", periods=3, freq="D")),
|
||||
pd.Series(range(1000)),
|
||||
pd.Series(pd.date_range("2024-01-01", periods=3, freq="D", tz="UTC")),
|
||||
pd.Series(pd.to_timedelta([1, 2, 3], unit="D")),
|
||||
pd.Series(pd.period_range("2024-01", periods=3, freq="M")),
|
||||
pd.Series(pd.interval_range(start=0, end=3, periods=3)),
|
||||
pd.Series(["Hello 🌍", "Python 🐍", "Data 📊"]),
|
||||
pd.Series([1, "string", [1, 2, 3], {"key": "value"}]),
|
||||
pd.Series([42], name="single"),
|
||||
pd.Series([sys.float_info.min, 0, sys.float_info.max]),
|
||||
pd.Series([-(2**63), 0, 2**63 - 1]),
|
||||
pd.Series(["", "null", "None", "NaN", "inf", "-inf"]),
|
||||
pd.Series([b"hello", b"world", b"\x00\x01\x02"]),
|
||||
pd.Series([1, 2, 3], name="named_series"),
|
||||
pd.Series(
|
||||
[10, 20],
|
||||
index=pd.MultiIndex.from_tuples([("a", 1), ("b", 2)], names=["x", "y"]),
|
||||
),
|
||||
],
|
||||
)
|
||||
def test_serde_jsonplus_pandas_series(series: pd.Series) -> None:
|
||||
serde = JsonPlusSerializer(pickle_fallback=True)
|
||||
dumped = serde.dumps_typed(series)
|
||||
|
||||
assert dumped[0] == "pickle"
|
||||
result = serde.loads_typed(dumped)
|
||||
|
||||
assert result.equals(series)
|
||||
assert serde.loads_typed(dumped) is None, "Should return None if cannot find module"
|
||||
@@ -22,7 +22,8 @@ class TestMemorySaver:
|
||||
"configurable": {
|
||||
"thread_id": "thread-1",
|
||||
"checkpoint_ns": "",
|
||||
"checkpoint_id": "1",
|
||||
# for backwards compatibility testing
|
||||
"thread_ts": "1",
|
||||
}
|
||||
}
|
||||
self.config_2: RunnableConfig = {
|
||||
@@ -67,37 +68,21 @@ class TestMemorySaver:
|
||||
},
|
||||
"metadata": {"run_id": "my_run_id"},
|
||||
}
|
||||
self.memory_saver.put(
|
||||
config, self.chkpnt_2, self.metadata_2, self.chkpnt_2["channel_versions"]
|
||||
)
|
||||
self.memory_saver.put(config, self.chkpnt_2, self.metadata_2, {})
|
||||
checkpoint = self.memory_saver.get_tuple(config)
|
||||
assert checkpoint is not None
|
||||
assert checkpoint.metadata == {
|
||||
**self.metadata_2,
|
||||
"thread_id": "thread-2",
|
||||
"run_id": "my_run_id",
|
||||
}
|
||||
|
||||
async def test_search(self) -> None:
|
||||
def test_search(self) -> None:
|
||||
# set up test
|
||||
# save checkpoints
|
||||
self.memory_saver.put(
|
||||
self.config_1,
|
||||
self.chkpnt_1,
|
||||
self.metadata_1,
|
||||
self.chkpnt_1["channel_versions"],
|
||||
)
|
||||
self.memory_saver.put(
|
||||
self.config_2,
|
||||
self.chkpnt_2,
|
||||
self.metadata_2,
|
||||
self.chkpnt_2["channel_versions"],
|
||||
)
|
||||
self.memory_saver.put(
|
||||
self.config_3,
|
||||
self.chkpnt_3,
|
||||
self.metadata_3,
|
||||
self.chkpnt_3["channel_versions"],
|
||||
)
|
||||
self.memory_saver.put(self.config_1, self.chkpnt_1, self.metadata_1, {})
|
||||
self.memory_saver.put(self.config_2, self.chkpnt_2, self.metadata_2, {})
|
||||
self.memory_saver.put(self.config_3, self.chkpnt_3, self.metadata_3, {})
|
||||
|
||||
# call method / assertions
|
||||
query_1 = {"source": "input"} # search by 1 key
|
||||
@@ -110,11 +95,18 @@ class TestMemorySaver:
|
||||
|
||||
search_results_1 = list(self.memory_saver.list(None, filter=query_1))
|
||||
assert len(search_results_1) == 1
|
||||
assert search_results_1[0].metadata == self.metadata_1
|
||||
assert search_results_1[0].metadata == {
|
||||
"thread_id": "thread-1",
|
||||
"thread_ts": "1",
|
||||
**self.metadata_1,
|
||||
}
|
||||
|
||||
search_results_2 = list(self.memory_saver.list(None, filter=query_2))
|
||||
assert len(search_results_2) == 1
|
||||
assert search_results_2[0].metadata == self.metadata_2
|
||||
assert search_results_2[0].metadata == {
|
||||
"thread_id": "thread-2",
|
||||
**self.metadata_2,
|
||||
}
|
||||
|
||||
search_results_3 = list(self.memory_saver.list(None, filter=query_3))
|
||||
assert len(search_results_3) == 3
|
||||
@@ -134,68 +126,8 @@ class TestMemorySaver:
|
||||
|
||||
# TODO: test before and limit params
|
||||
|
||||
async def test_asearch(self) -> None:
|
||||
# set up test
|
||||
# save checkpoints
|
||||
self.memory_saver.put(
|
||||
self.config_1,
|
||||
self.chkpnt_1,
|
||||
self.metadata_1,
|
||||
self.chkpnt_1["channel_versions"],
|
||||
)
|
||||
self.memory_saver.put(
|
||||
self.config_2,
|
||||
self.chkpnt_2,
|
||||
self.metadata_2,
|
||||
self.chkpnt_2["channel_versions"],
|
||||
)
|
||||
self.memory_saver.put(
|
||||
self.config_3,
|
||||
self.chkpnt_3,
|
||||
self.metadata_3,
|
||||
self.chkpnt_3["channel_versions"],
|
||||
)
|
||||
|
||||
# call method / assertions
|
||||
query_1 = {"source": "input"} # search by 1 key
|
||||
query_2 = {
|
||||
"step": 1,
|
||||
"writes": {"foo": "bar"},
|
||||
} # search by multiple keys
|
||||
query_3: dict[str, Any] = {} # search by no keys, return all checkpoints
|
||||
query_4 = {"source": "update", "step": 1} # no match
|
||||
def test_memory_saver() -> None:
|
||||
from langgraph.checkpoint.memory import MemorySaver
|
||||
|
||||
search_results_1 = [
|
||||
c async for c in self.memory_saver.alist(None, filter=query_1)
|
||||
]
|
||||
assert len(search_results_1) == 1
|
||||
assert search_results_1[0].metadata == self.metadata_1
|
||||
|
||||
search_results_2 = [
|
||||
c async for c in self.memory_saver.alist(None, filter=query_2)
|
||||
]
|
||||
assert len(search_results_2) == 1
|
||||
assert search_results_2[0].metadata == self.metadata_2
|
||||
|
||||
search_results_3 = [
|
||||
c async for c in self.memory_saver.alist(None, filter=query_3)
|
||||
]
|
||||
assert len(search_results_3) == 3
|
||||
|
||||
search_results_4 = [
|
||||
c async for c in self.memory_saver.alist(None, filter=query_4)
|
||||
]
|
||||
assert len(search_results_4) == 0
|
||||
|
||||
|
||||
async def test_memory_saver() -> None:
|
||||
from langgraph.checkpoint.memory import InMemorySaver
|
||||
|
||||
memory_saver = InMemorySaver()
|
||||
assert isinstance(memory_saver, InMemorySaver)
|
||||
|
||||
async with memory_saver as async_memory_saver:
|
||||
assert async_memory_saver is memory_saver
|
||||
|
||||
with memory_saver as sync_memory_saver:
|
||||
assert sync_memory_saver is memory_saver
|
||||
assert isinstance(MemorySaver(), InMemorySaver)
|
||||
@@ -1,316 +0,0 @@
|
||||
"""Unit tests for Redis cache implementation."""
|
||||
|
||||
import time
|
||||
|
||||
import pytest
|
||||
import redis
|
||||
|
||||
from langgraph.cache.base import FullKey
|
||||
from langgraph.cache.redis import RedisCache
|
||||
|
||||
|
||||
class TestRedisCache:
|
||||
@pytest.fixture(autouse=True)
|
||||
def setup(self) -> None:
|
||||
"""Set up test Redis client and cache."""
|
||||
self.client = redis.Redis(
|
||||
host="localhost", port=6379, db=0, decode_responses=False
|
||||
)
|
||||
try:
|
||||
self.client.ping()
|
||||
except redis.ConnectionError:
|
||||
pytest.skip("Redis server not available")
|
||||
|
||||
self.cache: RedisCache = RedisCache(self.client, prefix="test:cache:")
|
||||
|
||||
# Clean up before each test
|
||||
self.client.flushdb()
|
||||
|
||||
def teardown_method(self) -> None:
|
||||
"""Clean up after each test."""
|
||||
try:
|
||||
self.client.flushdb()
|
||||
except Exception:
|
||||
pass
|
||||
|
||||
def test_basic_set_and_get(self) -> None:
|
||||
"""Test basic set and get operations."""
|
||||
keys: list[FullKey] = [(("graph", "node"), "key1")]
|
||||
values = {keys[0]: ({"result": 42}, None)}
|
||||
|
||||
# Set value
|
||||
self.cache.set(values)
|
||||
|
||||
# Get value
|
||||
result = self.cache.get(keys)
|
||||
assert len(result) == 1
|
||||
assert result[keys[0]] == {"result": 42}
|
||||
|
||||
def test_batch_operations(self) -> None:
|
||||
"""Test batch set and get operations."""
|
||||
keys: list[FullKey] = [
|
||||
(("graph", "node1"), "key1"),
|
||||
(("graph", "node2"), "key2"),
|
||||
(("other", "node"), "key3"),
|
||||
]
|
||||
values = {
|
||||
keys[0]: ({"result": 1}, None),
|
||||
keys[1]: ({"result": 2}, 60), # With TTL
|
||||
keys[2]: ({"result": 3}, None),
|
||||
}
|
||||
|
||||
# Set values
|
||||
self.cache.set(values)
|
||||
|
||||
# Get all values
|
||||
result = self.cache.get(keys)
|
||||
assert len(result) == 3
|
||||
assert result[keys[0]] == {"result": 1}
|
||||
assert result[keys[1]] == {"result": 2}
|
||||
assert result[keys[2]] == {"result": 3}
|
||||
|
||||
def test_ttl_behavior(self) -> None:
|
||||
"""Test TTL (time-to-live) functionality."""
|
||||
key: FullKey = (("graph", "node"), "ttl_key")
|
||||
values = {key: ({"data": "expires_soon"}, 1)} # 1 second TTL
|
||||
|
||||
# Set with TTL
|
||||
self.cache.set(values)
|
||||
|
||||
# Should be available immediately
|
||||
result = self.cache.get([key])
|
||||
assert len(result) == 1
|
||||
assert result[key] == {"data": "expires_soon"}
|
||||
|
||||
# Wait for expiration
|
||||
time.sleep(1.1)
|
||||
|
||||
# Should be expired
|
||||
result = self.cache.get([key])
|
||||
assert len(result) == 0
|
||||
|
||||
def test_namespace_isolation(self) -> None:
|
||||
"""Test that different namespaces are isolated."""
|
||||
key1: FullKey = (("graph1", "node"), "same_key")
|
||||
key2: FullKey = (("graph2", "node"), "same_key")
|
||||
|
||||
values = {key1: ({"graph": 1}, None), key2: ({"graph": 2}, None)}
|
||||
|
||||
self.cache.set(values)
|
||||
|
||||
result = self.cache.get([key1, key2])
|
||||
assert result[key1] == {"graph": 1}
|
||||
assert result[key2] == {"graph": 2}
|
||||
|
||||
def test_clear_all(self) -> None:
|
||||
"""Test clearing all cached values."""
|
||||
keys: list[FullKey] = [
|
||||
(("graph", "node1"), "key1"),
|
||||
(("graph", "node2"), "key2"),
|
||||
]
|
||||
values = {keys[0]: ({"result": 1}, None), keys[1]: ({"result": 2}, None)}
|
||||
|
||||
self.cache.set(values)
|
||||
|
||||
# Verify data exists
|
||||
result = self.cache.get(keys)
|
||||
assert len(result) == 2
|
||||
|
||||
# Clear all
|
||||
self.cache.clear()
|
||||
|
||||
# Verify data is gone
|
||||
result = self.cache.get(keys)
|
||||
assert len(result) == 0
|
||||
|
||||
def test_clear_by_namespace(self) -> None:
|
||||
"""Test clearing cached values by namespace."""
|
||||
keys: list[FullKey] = [
|
||||
(("graph1", "node"), "key1"),
|
||||
(("graph2", "node"), "key2"),
|
||||
(("graph1", "other"), "key3"),
|
||||
]
|
||||
values = {
|
||||
keys[0]: ({"result": 1}, None),
|
||||
keys[1]: ({"result": 2}, None),
|
||||
keys[2]: ({"result": 3}, None),
|
||||
}
|
||||
|
||||
self.cache.set(values)
|
||||
|
||||
# Clear only graph1 namespace
|
||||
self.cache.clear([("graph1", "node"), ("graph1", "other")])
|
||||
|
||||
# graph1 should be cleared, graph2 should remain
|
||||
result = self.cache.get(keys)
|
||||
assert len(result) == 1
|
||||
assert result[keys[1]] == {"result": 2}
|
||||
|
||||
def test_empty_operations(self) -> None:
|
||||
"""Test behavior with empty keys/values."""
|
||||
# Empty get
|
||||
result = self.cache.get([])
|
||||
assert result == {}
|
||||
|
||||
# Empty set
|
||||
self.cache.set({}) # Should not raise error
|
||||
|
||||
def test_nonexistent_keys(self) -> None:
|
||||
"""Test getting keys that don't exist."""
|
||||
keys: list[FullKey] = [(("graph", "node"), "nonexistent")]
|
||||
result = self.cache.get(keys)
|
||||
assert len(result) == 0
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_async_operations(self) -> None:
|
||||
"""Test async set and get operations with sync Redis client."""
|
||||
# Create sync Redis client and cache (like main integration tests)
|
||||
client = redis.Redis(host="localhost", port=6379, db=1, decode_responses=False)
|
||||
try:
|
||||
client.ping()
|
||||
except Exception:
|
||||
pytest.skip("Redis not available")
|
||||
|
||||
cache: RedisCache = RedisCache(client, prefix="test:async:")
|
||||
|
||||
keys: list[FullKey] = [(("graph", "node"), "async_key")]
|
||||
values = {keys[0]: ({"async": True}, None)}
|
||||
|
||||
# Async set (delegates to sync)
|
||||
await cache.aset(values)
|
||||
|
||||
# Async get (delegates to sync)
|
||||
result = await cache.aget(keys)
|
||||
assert len(result) == 1
|
||||
assert result[keys[0]] == {"async": True}
|
||||
|
||||
# Cleanup
|
||||
client.flushdb()
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_async_clear(self) -> None:
|
||||
"""Test async clear operations with sync Redis client."""
|
||||
# Create sync Redis client and cache (like main integration tests)
|
||||
client = redis.Redis(host="localhost", port=6379, db=1, decode_responses=False)
|
||||
try:
|
||||
client.ping()
|
||||
except Exception:
|
||||
pytest.skip("Redis not available")
|
||||
|
||||
cache: RedisCache = RedisCache(client, prefix="test:async:")
|
||||
|
||||
keys: list[FullKey] = [(("graph", "node"), "key")]
|
||||
values = {keys[0]: ({"data": "test"}, None)}
|
||||
|
||||
await cache.aset(values)
|
||||
|
||||
# Verify data exists
|
||||
result = await cache.aget(keys)
|
||||
assert len(result) == 1
|
||||
|
||||
# Clear all (delegates to sync)
|
||||
await cache.aclear()
|
||||
|
||||
# Verify data is gone
|
||||
result = await cache.aget(keys)
|
||||
assert len(result) == 0
|
||||
|
||||
# Cleanup
|
||||
client.flushdb()
|
||||
|
||||
def test_redis_unavailable_get(self) -> None:
|
||||
"""Test behavior when Redis is unavailable during get operations."""
|
||||
# Create cache with non-existent Redis server
|
||||
bad_client = redis.Redis(
|
||||
host="nonexistent", port=9999, socket_connect_timeout=0.1
|
||||
)
|
||||
cache: RedisCache = RedisCache(bad_client, prefix="test:cache:")
|
||||
|
||||
keys: list[FullKey] = [(("graph", "node"), "key")]
|
||||
result = cache.get(keys)
|
||||
|
||||
# Should return empty dict when Redis unavailable
|
||||
assert result == {}
|
||||
|
||||
def test_redis_unavailable_set(self) -> None:
|
||||
"""Test behavior when Redis is unavailable during set operations."""
|
||||
# Create cache with non-existent Redis server
|
||||
bad_client = redis.Redis(
|
||||
host="nonexistent", port=9999, socket_connect_timeout=0.1
|
||||
)
|
||||
cache: RedisCache = RedisCache(bad_client, prefix="test:cache:")
|
||||
|
||||
keys: list[FullKey] = [(("graph", "node"), "key")]
|
||||
values = {keys[0]: ({"data": "test"}, None)}
|
||||
|
||||
# Should not raise exception when Redis unavailable
|
||||
cache.set(values) # Should silently fail
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_redis_unavailable_async(self) -> None:
|
||||
"""Test async behavior when Redis is unavailable."""
|
||||
# Create sync cache with non-existent Redis server (like main integration tests)
|
||||
bad_client = redis.Redis(
|
||||
host="nonexistent", port=9999, socket_connect_timeout=0.1
|
||||
)
|
||||
cache: RedisCache = RedisCache(bad_client, prefix="test:cache:")
|
||||
|
||||
keys: list[FullKey] = [(("graph", "node"), "key")]
|
||||
values = {keys[0]: ({"data": "test"}, None)}
|
||||
|
||||
# Should return empty dict for get (delegates to sync)
|
||||
result = await cache.aget(keys)
|
||||
assert result == {}
|
||||
|
||||
# Should not raise exception for set (delegates to sync)
|
||||
await cache.aset(values) # Should silently fail
|
||||
|
||||
def test_corrupted_data_handling(self) -> None:
|
||||
"""Test handling of corrupted data in Redis."""
|
||||
# Set some valid data first
|
||||
keys: list[FullKey] = [(("graph", "node"), "valid_key")]
|
||||
values = {keys[0]: ({"data": "valid"}, None)}
|
||||
self.cache.set(values)
|
||||
|
||||
# Manually insert corrupted data
|
||||
corrupted_key = self.cache._make_key(("graph", "node"), "corrupted_key")
|
||||
self.client.set(corrupted_key, b"invalid:data:format:too:many:colons")
|
||||
|
||||
# Should skip corrupted entry and return only valid ones
|
||||
all_keys: list[FullKey] = [keys[0], (("graph", "node"), "corrupted_key")]
|
||||
result = self.cache.get(all_keys)
|
||||
|
||||
assert len(result) == 1
|
||||
assert result[keys[0]] == {"data": "valid"}
|
||||
|
||||
def test_key_parsing_edge_cases(self) -> None:
|
||||
"""Test key parsing with edge cases."""
|
||||
# Test empty namespace
|
||||
key1: FullKey = ((), "empty_ns")
|
||||
values = {key1: ({"data": "empty_ns"}, None)}
|
||||
self.cache.set(values)
|
||||
result = self.cache.get([key1])
|
||||
assert result[key1] == {"data": "empty_ns"}
|
||||
|
||||
# Test namespace with special characters
|
||||
key2: FullKey = (
|
||||
("graph:with:colons", "node-with-dashes"),
|
||||
"key_with_underscores",
|
||||
)
|
||||
values = {key2: ({"data": "special_chars"}, None)}
|
||||
self.cache.set(values)
|
||||
result = self.cache.get([key2])
|
||||
assert result[key2] == {"data": "special_chars"}
|
||||
|
||||
def test_large_data_serialization(self) -> None:
|
||||
"""Test handling of large data objects."""
|
||||
# Create a large data structure
|
||||
large_data = {"large_list": list(range(1000)), "nested": {"data": "x" * 1000}}
|
||||
key: FullKey = (("graph", "node"), "large_key")
|
||||
values = {key: (large_data, None)}
|
||||
|
||||
self.cache.set(values)
|
||||
result = self.cache.get([key])
|
||||
|
||||
assert len(result) == 1
|
||||
assert result[key] == large_data
|
||||
@@ -1,75 +1,17 @@
|
||||
# mypy: disable-error-code="operator"
|
||||
import asyncio
|
||||
import json
|
||||
from collections.abc import Iterable
|
||||
from datetime import datetime
|
||||
from typing import Any
|
||||
|
||||
import pytest
|
||||
from pytest_mock import MockerFixture
|
||||
|
||||
from langgraph.store.base import (
|
||||
GetOp,
|
||||
InvalidNamespaceError,
|
||||
Item,
|
||||
Op,
|
||||
PutOp,
|
||||
Result,
|
||||
get_text_at_path,
|
||||
)
|
||||
from langgraph.store.base.batch import AsyncBatchedBaseStore
|
||||
from langgraph.store.memory import InMemoryStore
|
||||
from tests.embed_test_utils import CharacterEmbeddings
|
||||
|
||||
|
||||
class MockAsyncBatchedStore(AsyncBatchedBaseStore):
|
||||
def __init__(self, **kwargs: Any) -> None:
|
||||
super().__init__()
|
||||
self._store = InMemoryStore(**kwargs)
|
||||
|
||||
def batch(self, ops: Iterable[Op]) -> list[Result]:
|
||||
return self._store.batch(ops)
|
||||
|
||||
async def abatch(self, ops: Iterable[Op]) -> list[Result]:
|
||||
return self._store.batch(ops)
|
||||
|
||||
|
||||
async def test_async_batch_store_resilience() -> None:
|
||||
"""Test that AsyncBatchedBaseStore recovers gracefully from task cancellation."""
|
||||
doc = {"foo": "bar"}
|
||||
async_store = MockAsyncBatchedStore()
|
||||
|
||||
await async_store.aput(("foo", "langgraph", "foo"), "bar", doc)
|
||||
|
||||
# Store the original task reference
|
||||
original_task = async_store._task
|
||||
assert original_task is not None
|
||||
assert not original_task.done()
|
||||
|
||||
# Cancel the background task
|
||||
original_task.cancel()
|
||||
await asyncio.sleep(0.01)
|
||||
assert original_task.cancelled()
|
||||
|
||||
# Perform a new operation - this should trigger _ensure_task() to create a new task
|
||||
result = await async_store.asearch(("foo", "langgraph", "foo"))
|
||||
assert len(result) > 0
|
||||
assert result[0].value == doc
|
||||
|
||||
# Verify a new task was created
|
||||
new_task = async_store._task
|
||||
assert new_task is not None
|
||||
assert new_task is not original_task
|
||||
assert not new_task.done()
|
||||
|
||||
# Test that operations continue to work with the new task
|
||||
doc2 = {"baz": "qux"}
|
||||
await async_store.aput(("test", "namespace"), "key", doc2)
|
||||
result2 = await async_store.aget(("test", "namespace"), "key")
|
||||
assert result2 is not None
|
||||
assert result2.value == doc2
|
||||
|
||||
|
||||
def test_get_text_at_path() -> None:
|
||||
nested_data = {
|
||||
"name": "test",
|
||||
@@ -138,96 +80,6 @@ def test_get_text_at_path() -> None:
|
||||
assert get_text_at_path(nested_data, "nested[{invalid}]") == []
|
||||
|
||||
|
||||
async def test_async_batch_store(mocker: MockerFixture) -> None:
|
||||
abatch = mocker.stub()
|
||||
|
||||
class MockStore(AsyncBatchedBaseStore):
|
||||
def batch(self, ops: Iterable[Op]) -> list[Result]:
|
||||
raise NotImplementedError
|
||||
|
||||
async def abatch(self, ops: Iterable[Op]) -> list[Result]:
|
||||
assert all(isinstance(op, GetOp) for op in ops)
|
||||
abatch(ops)
|
||||
return [
|
||||
Item(
|
||||
value={},
|
||||
key=getattr(op, "key", ""),
|
||||
namespace=getattr(op, "namespace", ()),
|
||||
created_at=datetime(2024, 9, 24, 17, 29, 10, 128397),
|
||||
updated_at=datetime(2024, 9, 24, 17, 29, 10, 128397),
|
||||
)
|
||||
for op in ops
|
||||
]
|
||||
|
||||
store = MockStore()
|
||||
|
||||
# concurrent calls are batched
|
||||
results = await asyncio.gather(
|
||||
store.aget(namespace=("a",), key="b"),
|
||||
store.aget(namespace=("c",), key="d"),
|
||||
)
|
||||
assert results == [
|
||||
Item(
|
||||
value={},
|
||||
key="b",
|
||||
namespace=("a",),
|
||||
created_at=datetime(2024, 9, 24, 17, 29, 10, 128397),
|
||||
updated_at=datetime(2024, 9, 24, 17, 29, 10, 128397),
|
||||
),
|
||||
Item(
|
||||
value={},
|
||||
key="d",
|
||||
namespace=("c",),
|
||||
created_at=datetime(2024, 9, 24, 17, 29, 10, 128397),
|
||||
updated_at=datetime(2024, 9, 24, 17, 29, 10, 128397),
|
||||
),
|
||||
]
|
||||
assert abatch.call_count == 1
|
||||
assert [tuple(c.args[0]) for c in abatch.call_args_list] == [
|
||||
(
|
||||
GetOp(("a",), "b", refresh_ttl=True),
|
||||
GetOp(("c",), "d", refresh_ttl=True),
|
||||
),
|
||||
]
|
||||
|
||||
|
||||
async def test_async_batch_store_handles_cancellation() -> None:
|
||||
class MockStore(AsyncBatchedBaseStore):
|
||||
def batch(self, ops: Iterable[Op]) -> list[Result]:
|
||||
raise NotImplementedError
|
||||
|
||||
async def abatch(self, ops: Iterable[Op]) -> list[Result]:
|
||||
assert all(isinstance(op, GetOp) for op in ops)
|
||||
return [
|
||||
Item(
|
||||
value={},
|
||||
key=getattr(op, "key", ""),
|
||||
namespace=getattr(op, "namespace", ()),
|
||||
created_at=datetime(2024, 9, 24, 17, 29, 10, 128397),
|
||||
updated_at=datetime(2024, 9, 24, 17, 29, 10, 128397),
|
||||
)
|
||||
for op in ops
|
||||
]
|
||||
|
||||
store = MockStore()
|
||||
|
||||
# Simulate cancellation
|
||||
task = asyncio.create_task(store.aget(namespace=("a",), key="b"))
|
||||
await asyncio.sleep(0)
|
||||
task.cancel()
|
||||
await asyncio.sleep(0)
|
||||
|
||||
# Cancelling individual queries against the store should not break the store
|
||||
result = await store.aget(namespace=("c",), key="d")
|
||||
assert result == Item(
|
||||
value={},
|
||||
key="d",
|
||||
namespace=("c",),
|
||||
created_at=datetime(2024, 9, 24, 17, 29, 10, 128397),
|
||||
updated_at=datetime(2024, 9, 24, 17, 29, 10, 128397),
|
||||
)
|
||||
|
||||
|
||||
def test_list_namespaces_basic() -> None:
|
||||
store = InMemoryStore()
|
||||
|
||||
@@ -480,105 +332,6 @@ async def test_cannot_put_empty_namespace() -> None:
|
||||
store.delete(("langgraph", "foo"), "bar")
|
||||
assert store.get(("langgraph", "foo"), "bar") is None
|
||||
|
||||
async_store = MockAsyncBatchedStore()
|
||||
doc = {"foo": "bar"}
|
||||
|
||||
with pytest.raises(InvalidNamespaceError):
|
||||
await async_store.aput((), "foo", doc)
|
||||
|
||||
with pytest.raises(InvalidNamespaceError):
|
||||
await async_store.aput(("the", "thing.about"), "foo", doc)
|
||||
|
||||
with pytest.raises(InvalidNamespaceError):
|
||||
await async_store.aput(("some", "fun", ""), "foo", doc)
|
||||
|
||||
with pytest.raises(InvalidNamespaceError):
|
||||
await async_store.aput(("langgraph", "foo"), "bar", doc)
|
||||
|
||||
await async_store.aput(("foo", "langgraph", "foo"), "bar", doc)
|
||||
val = await async_store.aget(("foo", "langgraph", "foo"), "bar")
|
||||
assert val is not None
|
||||
assert val.value == doc
|
||||
assert (await async_store.asearch(("foo", "langgraph", "foo")))[0].value == doc
|
||||
assert (await async_store.asearch(("foo", "langgraph", "foo"), query="bar"))[
|
||||
0
|
||||
].value == doc
|
||||
await async_store.adelete(("foo", "langgraph", "foo"), "bar")
|
||||
assert (await async_store.aget(("foo", "langgraph", "foo"), "bar")) is None
|
||||
|
||||
await async_store.abatch([PutOp(("valid", "namespace"), "key", doc)])
|
||||
val = await async_store.aget(("valid", "namespace"), "key")
|
||||
assert val is not None
|
||||
assert val.value == doc
|
||||
assert (await async_store.asearch(("valid", "namespace")))[0].value == doc
|
||||
await async_store.adelete(("valid", "namespace"), "key")
|
||||
assert (await async_store.aget(("valid", "namespace"), "key")) is None
|
||||
|
||||
|
||||
async def test_async_batch_store_deduplication(mocker: MockerFixture) -> None:
|
||||
abatch = mocker.spy(InMemoryStore, "batch")
|
||||
store = MockAsyncBatchedStore()
|
||||
|
||||
same_doc = {"value": "same"}
|
||||
diff_doc = {"value": "different"}
|
||||
await asyncio.gather(
|
||||
store.aput(namespace=("test",), key="same", value=same_doc),
|
||||
store.aput(namespace=("test",), key="different", value=diff_doc),
|
||||
)
|
||||
abatch.reset_mock()
|
||||
|
||||
results = await asyncio.gather(
|
||||
store.aget(namespace=("test",), key="same"),
|
||||
store.aget(namespace=("test",), key="same"),
|
||||
store.aget(namespace=("test",), key="different"),
|
||||
)
|
||||
|
||||
assert len(results) == 3
|
||||
assert results[0] == results[1]
|
||||
assert results[0] != results[2]
|
||||
assert results[0].value == same_doc # type: ignore
|
||||
assert results[2].value == diff_doc # type: ignore
|
||||
assert len(abatch.call_args_list) == 1
|
||||
ops = list(abatch.call_args_list[0].args[1])
|
||||
assert len(ops) == 2
|
||||
assert GetOp(("test",), "same", refresh_ttl=True) in ops
|
||||
assert GetOp(("test",), "different", refresh_ttl=True) in ops
|
||||
|
||||
abatch.reset_mock()
|
||||
|
||||
doc1 = {"value": 1}
|
||||
doc2 = {"value": 2}
|
||||
results = await asyncio.gather(
|
||||
store.aput(namespace=("test",), key="key", value=doc1),
|
||||
store.aput(namespace=("test",), key="key", value=doc2),
|
||||
)
|
||||
assert len(abatch.call_args_list) == 1
|
||||
ops = list(abatch.call_args_list[0].args[1])
|
||||
assert len(ops) == 1
|
||||
assert ops[0] == PutOp(("test",), "key", doc2)
|
||||
assert len(results) == 2
|
||||
assert all(result is None for result in results)
|
||||
|
||||
result = await store.aget(namespace=("test",), key="key")
|
||||
assert result is not None
|
||||
assert result.value == doc2
|
||||
|
||||
abatch.reset_mock()
|
||||
|
||||
results = await asyncio.gather(
|
||||
store.asearch(("test",), filter={"value": 2}),
|
||||
store.asearch(("test",), filter={"value": 2}),
|
||||
)
|
||||
assert len(abatch.call_args_list) == 1
|
||||
ops = list(abatch.call_args_list[0].args[1])
|
||||
assert len(ops) == 1
|
||||
assert len(results) == 2
|
||||
assert results[0] == results[1]
|
||||
assert len(results[0]) == 1
|
||||
assert results[0][0].value == doc2
|
||||
|
||||
abatch.reset_mock()
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def fake_embeddings() -> CharacterEmbeddings:
|
||||
@@ -795,88 +548,6 @@ async def test_async_vector_search_with_filters(
|
||||
assert results[0].key == "doc3"
|
||||
|
||||
|
||||
async def test_async_batched_vector_search_concurrent(
|
||||
fake_embeddings: CharacterEmbeddings,
|
||||
) -> None:
|
||||
"""Test concurrent vector search operations using async batched store."""
|
||||
store = MockAsyncBatchedStore(
|
||||
index={"dims": fake_embeddings.dims, "embed": fake_embeddings}
|
||||
)
|
||||
|
||||
colors = ["red", "blue", "green", "yellow", "purple"]
|
||||
items = ["apple", "car", "house", "book", "phone"]
|
||||
scores = [3.0, 3.5, 4.0, 4.5, 5.0]
|
||||
|
||||
docs = []
|
||||
for i in range(50):
|
||||
color = colors[i % len(colors)]
|
||||
item = items[i % len(items)]
|
||||
score = scores[i % len(scores)]
|
||||
docs.append(
|
||||
(
|
||||
f"doc{i}",
|
||||
{"text": f"{color} {item}", "color": color, "score": score, "index": i},
|
||||
)
|
||||
)
|
||||
coros = [
|
||||
*[store.aput(("test",), key, value) for key, value in docs],
|
||||
*[store.adelete(("test",), key) for key, value in docs],
|
||||
*[store.aput(("test",), key, value) for key, value in docs],
|
||||
]
|
||||
await asyncio.gather(*coros)
|
||||
|
||||
# Prepare multiple search queries with different filters
|
||||
search_queries: list[tuple[str, dict[str, Any]]] = [
|
||||
("apple", {"color": "red"}),
|
||||
("car", {"color": "blue"}),
|
||||
("house", {"color": "green"}),
|
||||
("phone", {"score": {"$gt": 4.99}}),
|
||||
("book", {"score": {"$lte": 3.5}}),
|
||||
("apple", {"score": {"$gte": 3.0}, "color": "red"}),
|
||||
("car", {"score": {"$lt": 5.1}, "color": "blue"}),
|
||||
("house", {"index": {"$gt": 25}}),
|
||||
("phone", {"index": {"$lte": 10}}),
|
||||
]
|
||||
|
||||
all_results = await asyncio.gather(
|
||||
*[
|
||||
store.asearch(("test",), query=query, filter=filter_)
|
||||
for query, filter_ in search_queries
|
||||
]
|
||||
)
|
||||
|
||||
for results, (query, filter_) in zip(all_results, search_queries, strict=False):
|
||||
assert len(results) > 0, f"No results for query '{query}' with filter {filter_}"
|
||||
|
||||
for result in results:
|
||||
if "color" in filter_:
|
||||
assert result.value["color"] == filter_["color"]
|
||||
|
||||
if "score" in filter_:
|
||||
score = result.value["score"]
|
||||
for op, value in filter_["score"].items():
|
||||
if op == "$gt":
|
||||
assert score > value
|
||||
elif op == "$gte":
|
||||
assert score >= value
|
||||
elif op == "$lt":
|
||||
assert score < value
|
||||
elif op == "$lte":
|
||||
assert score <= value
|
||||
|
||||
if "index" in filter_:
|
||||
index = result.value["index"]
|
||||
for op, value in filter_["index"].items():
|
||||
if op == "$gt":
|
||||
assert index > value
|
||||
elif op == "$gte":
|
||||
assert index >= value
|
||||
elif op == "$lt":
|
||||
assert index < value
|
||||
elif op == "$lte":
|
||||
assert index <= value
|
||||
|
||||
|
||||
def test_vector_search_pagination(fake_embeddings: CharacterEmbeddings) -> None:
|
||||
"""Test pagination with vector search."""
|
||||
store = InMemoryStore(
|
||||
@@ -950,8 +621,8 @@ async def test_embed_with_path(fake_embeddings: CharacterEmbeddings) -> None:
|
||||
assert results[0].key != results[1].key
|
||||
ascore = results[0].score
|
||||
bscore = results[1].score
|
||||
assert ascore == bscore
|
||||
assert ascore is not None and bscore is not None
|
||||
assert ascore == pytest.approx(bscore, abs=1e-5)
|
||||
|
||||
results = await store.asearch(("test",), query="uuu")
|
||||
assert len(results) == 2
|
||||
@@ -1021,27 +692,3 @@ async def test_embed_with_path(fake_embeddings: CharacterEmbeddings) -> None:
|
||||
assert len(results) == 3
|
||||
doc5_result = next(r for r in results if r.key == "doc5")
|
||||
assert doc5_result.score is None
|
||||
|
||||
|
||||
def test_non_ascii(fake_embeddings: CharacterEmbeddings) -> None:
|
||||
"""Test support for non-ascii characters"""
|
||||
store = InMemoryStore(
|
||||
index={"dims": fake_embeddings.dims, "embed": fake_embeddings}
|
||||
)
|
||||
store.put(("user_123", "memories"), "1", {"text": "这是中文"}) # Chinese
|
||||
store.put(("user_123", "memories"), "2", {"text": "これは日本語です"}) # Japanese
|
||||
store.put(("user_123", "memories"), "3", {"text": "이건 한국어야"}) # Korean
|
||||
store.put(("user_123", "memories"), "4", {"text": "Это русский"}) # Russian
|
||||
store.put(("user_123", "memories"), "5", {"text": "यह रूसी है"}) # Hindi
|
||||
|
||||
result1 = store.search(("user_123", "memories"), query="这是中文")
|
||||
result2 = store.search(("user_123", "memories"), query="これは日本語です")
|
||||
result3 = store.search(("user_123", "memories"), query="이건 한국어야")
|
||||
result4 = store.search(("user_123", "memories"), query="Это русский")
|
||||
result5 = store.search(("user_123", "memories"), query="यह रूसी है")
|
||||
|
||||
assert result1[0].key == "1"
|
||||
assert result2[0].key == "2"
|
||||
assert result3[0].key == "3"
|
||||
assert result4[0].key == "4"
|
||||
assert result5[0].key == "5"
|
||||
Generated
-1443
File diff suppressed because it is too large.
Load diff
@@ -1 +0,0 @@
|
||||
.langgraph_api/
|
||||
@@ -1,21 +0,0 @@
|
||||
MIT License
|
||||
|
||||
Copyright (c) 2024 LangChain, Inc.
|
||||
|
||||
Permission is hereby granted, free of charge, to any person obtaining a copy
|
||||
of this software and associated documentation files (the "Software"), to deal
|
||||
in the Software without restriction, including without limitation the rights
|
||||
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
||||
copies of the Software, and to permit persons to whom the Software is
|
||||
furnished to do so, subject to the following conditions:
|
||||
|
||||
The above copyright notice and this permission notice shall be included in all
|
||||
copies or substantial portions of the Software.
|
||||
|
||||
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
||||
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
||||
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
||||
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
||||
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
||||
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
||||
SOFTWARE.
|
||||
@@ -1,40 +0,0 @@
|
||||
.PHONY: test lint format test-integration update-schema bump-version
|
||||
|
||||
######################
|
||||
# TESTING AND COVERAGE
|
||||
######################
|
||||
|
||||
TEST?= "tests/unit_tests"
|
||||
test:
|
||||
uv run pytest $(TEST)
|
||||
test-integration:
|
||||
uv run pytest tests/integration_tests
|
||||
|
||||
######################
|
||||
# LINTING AND FORMATTING
|
||||
######################
|
||||
|
||||
# Define a variable for Python and notebook files.
|
||||
PYTHON_FILES=.
|
||||
MYPY_CACHE=.mypy_cache
|
||||
lint format: PYTHON_FILES=.
|
||||
lint_diff format_diff: PYTHON_FILES=$(shell git diff --name-only --relative --diff-filter=d main . | grep -E '\.py$$|\.ipynb$$')
|
||||
lint_package: PYTHON_FILES=langgraph_cli
|
||||
lint_tests: PYTHON_FILES=tests
|
||||
lint_tests: MYPY_CACHE=.mypy_cache_test
|
||||
|
||||
lint lint_diff lint_package lint_tests:
|
||||
uv run ruff check .
|
||||
[ "$(PYTHON_FILES)" = "" ] || uv run ruff format $(PYTHON_FILES) --diff
|
||||
[ "$(PYTHON_FILES)" = "" ] || uv run ruff check --select I $(PYTHON_FILES)
|
||||
[ "$(PYTHON_FILES)" = "" ] || mkdir -p $(MYPY_CACHE) || uv run mypy $(PYTHON_FILES) --cache-dir $(MYPY_CACHE)
|
||||
|
||||
format format_diff:
|
||||
uv run ruff format $(PYTHON_FILES)
|
||||
uv run ruff check --select I --fix $(PYTHON_FILES)
|
||||
|
||||
update-schema:
|
||||
uv run python generate_schema.py
|
||||
|
||||
bump-version:
|
||||
uv run hatch version patch
|
||||
@@ -1,105 +0,0 @@
|
||||
# LangGraph CLI
|
||||
|
||||
The official command-line interface for LangGraph, providing tools to create, develop, and deploy LangGraph applications.
|
||||
|
||||
## Installation
|
||||
|
||||
Install via pip:
|
||||
```bash
|
||||
pip install langgraph-cli
|
||||
```
|
||||
|
||||
For development mode with hot reloading:
|
||||
```bash
|
||||
pip install "langgraph-cli[inmem]"
|
||||
```
|
||||
|
||||
## Commands
|
||||
|
||||
### `langgraph new` 🌱
|
||||
Create a new LangGraph project from a template
|
||||
```bash
|
||||
langgraph new [PATH] --template TEMPLATE_NAME
|
||||
```
|
||||
|
||||
### `langgraph dev` 🏃♀️
|
||||
Run LangGraph API server in development mode with hot reloading
|
||||
```bash
|
||||
langgraph dev [OPTIONS]
|
||||
--host TEXT Host to bind to (default: 127.0.0.1)
|
||||
--port INTEGER Port to bind to (default: 2024)
|
||||
--no-reload Disable auto-reload
|
||||
--debug-port INTEGER Enable remote debugging
|
||||
--no-browser Skip opening browser window
|
||||
-c, --config FILE Config file path (default: langgraph.json)
|
||||
```
|
||||
|
||||
### `langgraph up` 🚀
|
||||
Launch LangGraph API server in Docker
|
||||
```bash
|
||||
langgraph up [OPTIONS]
|
||||
-p, --port INTEGER Port to expose (default: 8123)
|
||||
--wait Wait for services to start
|
||||
--watch Restart on file changes
|
||||
--verbose Show detailed logs
|
||||
-c, --config FILE Config file path
|
||||
-d, --docker-compose Additional services file
|
||||
```
|
||||
|
||||
### `langgraph build`
|
||||
Build a Docker image for your LangGraph application
|
||||
```bash
|
||||
langgraph build -t IMAGE_TAG [OPTIONS]
|
||||
--platform TEXT Target platforms (e.g., linux/amd64,linux/arm64)
|
||||
--pull / --no-pull Use latest/local base image
|
||||
-c, --config FILE Config file path
|
||||
```
|
||||
|
||||
### `langgraph dockerfile`
|
||||
Generate a Dockerfile for custom deployments
|
||||
```bash
|
||||
langgraph dockerfile SAVE_PATH [OPTIONS]
|
||||
-c, --config FILE Config file path
|
||||
```
|
||||
|
||||
## Configuration
|
||||
|
||||
The CLI uses a `langgraph.json` configuration file with these key settings:
|
||||
|
||||
```json
|
||||
{
|
||||
"dependencies": ["langchain_openai", "./your_package"], // Required: Package dependencies
|
||||
"graphs": {
|
||||
"my_graph": "./your_package/file.py:graph" // Required: Graph definitions
|
||||
},
|
||||
"env": "./.env", // Optional: Environment variables
|
||||
"python_version": "3.11", // Optional: Python version (3.11/3.12)
|
||||
"pip_config_file": "./pip.conf", // Optional: pip configuration
|
||||
"dockerfile_lines": [] // Optional: Additional Dockerfile commands
|
||||
}
|
||||
```
|
||||
|
||||
See the [full documentation](https://langchain-ai.github.io/langgraph/cloud/reference/cli/) for detailed configuration options.
|
||||
|
||||
## Development
|
||||
|
||||
To develop the CLI itself:
|
||||
|
||||
1. Clone the repository
|
||||
2. Navigate to the CLI directory: `cd libs/cli`
|
||||
3. Install development dependencies: `uv pip install`
|
||||
4. Make your changes to the CLI code
|
||||
5. Test your changes:
|
||||
```bash
|
||||
# Run CLI commands directly
|
||||
uv run langgraph --help
|
||||
|
||||
# Or use the examples
|
||||
cd examples
|
||||
uv pip install
|
||||
uv run langgraph dev # or other commands
|
||||
```
|
||||
|
||||
## License
|
||||
|
||||
This project is licensed under the terms specified in the repository's LICENSE file.
|
||||
Loaded 100 of 407 files, more files were not shown because too many files have changed in this diff.
Show more
Reference in new issue
Block a user