This commit is contained in:
William Fu-Hinthorn
2024-11-25 13:24:51 -08:00
18 changed files with 210 additions and 106 deletions
+8
View File
@@ -14,6 +14,13 @@ A **deployment** is an instance of a LangGraph API. A single deployment can have
See the [how-to guide](../cloud/deployment/cloud.md#create-new-deployment) for creating a new deployment.
## Resource Allocation
| **Deployment Type** | **CPU** | **Memory** | **Scaling** |
|---------------------|---------|------------|---------------------|
| Development | 1 CPU | 1 GB | Up to 1 container |
| Production | 1 CPU | 2 GB | Up to 10 containers |
## Revision
A revision is an iteration of a [deployment](#deployment). When a new deployment is created, an initial revision is automatically created. To deploy new code changes or update environment variable configurations for a deployment, a new revision must be created. When a revision is created, a new container image is built automatically.
@@ -33,6 +40,7 @@ A high-level diagram of a Cloud SaaS deployment.
![diagram](img/langgraph_cloud_architecture.png)
## Related
- [Deployment Options](./deployment_options.md)
+5 -1
View File
@@ -7,7 +7,7 @@
## Versions
There are two versions of the self hosted deployment: [Self-Hosted Enterprise](./deployment_options.md#self-hosted-enterprise) and [Self-Hosted Lite](./deployment_options.md#self-hosted-lite).
There are two versions of the self-hosted deployment: [Self-Hosted Enterprise](./deployment_options.md#self-hosted-enterprise) and [Self-Hosted Lite](./deployment_options.md#self-hosted-lite).
### Self-Hosted Lite
@@ -34,6 +34,10 @@ To use the Self-Hosted Enterprise version, you must acquire a license key that y
For step-by-step instructions, see [How to set up a self-hosted deployment of LangGraph](../how-tos/deploy-self-hosted.md).
## Helm Chart
If you would like to deploy LangGraph Cloud on Kubernetes, you can use this [Helm chart](https://github.com/langchain-ai/helm/blob/main/charts/langgraph-cloud/README.md).
## Related
- [How to set up a self-hosted deployment of LangGraph](../how-tos/deploy-self-hosted.md).
+4
View File
@@ -17,6 +17,10 @@ You will need to do the following:
2. Build a docker image with the [LangGraph Server](../concepts/langgraph_server.md) using the [LangGraph CLI](../concepts/langgraph_cli.md).
3. Deploy a web server that will run the docker image and pass in the necessary environment variables.
## Helm Chart
If you would like to deploy LangGraph Cloud on Kubernetes, you can use this [Helm chart](https://github.com/langchain-ai/helm/blob/main/charts/langgraph-cloud/README.md).
## Environment Variables
You will eventually need to pass in the following environment variables to the LangGraph Deploy server:
+1 -1
View File
@@ -2043,7 +2043,7 @@
"\n",
"So far, we've relied on a simple state (it's just a list of messages!). You can go far with this simple state, but if you want to define complex behavior without relying on the message list, you can add additional fields to the state. In this section, we will extend our chat bot with a new node to illustrate this.\n",
"\n",
"In the examples above, we involved a human deterministically: the graph __always__ interrupted whenever an tool was invoked. Suppose we wanted our chat bot to have the choice of relying on a human.\n",
"In the examples above, we involved a human deterministically: the graph __always__ interrupted whenever a tool was invoked. Suppose we wanted our chat bot to have the choice of relying on a human.\n",
"\n",
"One way to do this is to create a passthrough \"human\" node, before which the graph will always stop. We will only execute this node if the LLM invokes a \"human\" tool. For our convenience, we will include an \"ask_human\" flag in our graph state that we will flip if the LLM calls this tool.\n",
"\n",
@@ -4,12 +4,13 @@ This is a quick start guide to help you get a LangGraph app up and running local
!!! info "Requirements"
- Python >= 3.11
- [LangGraph CLI](https://langchain-ai.github.io/langgraph/cloud/reference/cli/): Requires langchain-cli[inmem] >= 0.1.58
## Install the LangGraph CLI
```bash
pip install "langgraph-cli[inmem]==0.1.58" python-dot-env
pip install "langgraph-cli[inmem]==0.1.58" python-dotenv
```
## 🌱 Create a LangGraph App
@@ -32,6 +33,14 @@ Create a new app from the `react-agent` template. This template is a simple agen
If you use `langgraph new` without specifying a template, you will be presented with an interactive menu that will allow you to choose from a list of available templates.
## Install Dependencies
In the root of your new LangGraph app, install the dependencies:
```shell
pip install .
```
## Create a `.env` file
You will find a `.env.example` in the root of your new LangGraph app. Create
@@ -1,6 +1,7 @@
import threading
from collections.abc import Iterator, Sequence
from contextlib import contextmanager
from typing import Any, Iterator, Optional, Sequence
from typing import Any, Optional
from langchain_core.runnables import RunnableConfig
from psycopg import Capabilities, Connection, Cursor, Pipeline
@@ -21,6 +22,8 @@ from langgraph.checkpoint.postgres import _internal
from langgraph.checkpoint.postgres.base import BasePostgresSaver
from langgraph.checkpoint.serde.base import SerializerProtocol
Conn = _internal.Conn # For backward compatibility
class PostgresSaver(BasePostgresSaver):
lock: threading.Lock
@@ -61,9 +64,9 @@ class PostgresSaver(BasePostgresSaver):
) as conn:
if pipeline:
with conn.pipeline() as pipe:
yield PostgresSaver(conn, pipe)
yield cls(conn, pipe)
else:
yield PostgresSaver(conn)
yield cls(conn)
def setup(self) -> None:
"""Set up the checkpoint database asynchronously.
@@ -376,19 +379,23 @@ class PostgresSaver(BasePostgresSaver):
# a connection not in pipeline mode can only be used by one
# thread/coroutine at a time, so we acquire a lock
if self.supports_pipeline:
with self.lock, conn.pipeline(), conn.cursor(
binary=True, row_factory=dict_row
) as cur:
with (
self.lock,
conn.pipeline(),
conn.cursor(binary=True, row_factory=dict_row) as cur,
):
yield cur
else:
# Use connection's transaction context manager when pipeline mode not supported
with self.lock, conn.transaction(), conn.cursor(
binary=True, row_factory=dict_row
) as cur:
with (
self.lock,
conn.transaction(),
conn.cursor(binary=True, row_factory=dict_row) as cur,
):
yield cur
else:
with self.lock, conn.cursor(binary=True, row_factory=dict_row) as cur:
yield cur
__all__ = ["PostgresSaver", "Conn"]
__all__ = ["PostgresSaver", "BasePostgresSaver", "Conn"]
@@ -1,7 +1,8 @@
"""Shared async utility functions for the Postgres checkpoint & storage classes."""
from collections.abc import AsyncIterator
from contextlib import asynccontextmanager
from typing import AsyncIterator, Union
from typing import Union
from psycopg import AsyncConnection
from psycopg.rows import DictRow
@@ -1,7 +1,8 @@
"""Shared utility functions for the Postgres checkpoint & storage classes."""
from collections.abc import Iterator
from contextlib import contextmanager
from typing import Iterator, Union
from typing import Union
from psycopg import Connection
from psycopg.rows import DictRow
@@ -1,6 +1,7 @@
import asyncio
from collections.abc import AsyncIterator, Iterator, Sequence
from contextlib import asynccontextmanager
from typing import Any, AsyncIterator, Iterator, Optional, Sequence
from typing import Any, Optional
from langchain_core.runnables import RunnableConfig
from psycopg import AsyncConnection, AsyncCursor, AsyncPipeline, Capabilities
@@ -21,6 +22,8 @@ from langgraph.checkpoint.postgres import _ainternal
from langgraph.checkpoint.postgres.base import BasePostgresSaver
from langgraph.checkpoint.serde.base import SerializerProtocol
Conn = _ainternal.Conn # For backward compatibility
class AsyncPostgresSaver(BasePostgresSaver):
lock: asyncio.Lock
@@ -66,9 +69,9 @@ class AsyncPostgresSaver(BasePostgresSaver):
) as conn:
if pipeline:
async with conn.pipeline() as pipe:
yield AsyncPostgresSaver(conn=conn, pipe=pipe, serde=serde)
yield cls(conn=conn, pipe=pipe, serde=serde)
else:
yield AsyncPostgresSaver(conn=conn, serde=serde)
yield cls(conn=conn, serde=serde)
async def setup(self) -> None:
"""Set up the checkpoint database asynchronously.
@@ -143,15 +146,17 @@ class AsyncPostgresSaver(BasePostgresSaver):
value["pending_sends"],
),
self._load_metadata(value["metadata"]),
{
"configurable": {
"thread_id": value["thread_id"],
"checkpoint_ns": value["checkpoint_ns"],
"checkpoint_id": value["parent_checkpoint_id"],
(
{
"configurable": {
"thread_id": value["thread_id"],
"checkpoint_ns": value["checkpoint_ns"],
"checkpoint_id": value["parent_checkpoint_id"],
}
}
}
if value["parent_checkpoint_id"]
else None,
if value["parent_checkpoint_id"]
else None
),
await asyncio.to_thread(self._load_writes, value["pending_writes"]),
)
@@ -202,15 +207,17 @@ class AsyncPostgresSaver(BasePostgresSaver):
value["pending_sends"],
),
self._load_metadata(value["metadata"]),
{
"configurable": {
"thread_id": thread_id,
"checkpoint_ns": checkpoint_ns,
"checkpoint_id": value["parent_checkpoint_id"],
(
{
"configurable": {
"thread_id": thread_id,
"checkpoint_ns": checkpoint_ns,
"checkpoint_id": value["parent_checkpoint_id"],
}
}
}
if value["parent_checkpoint_id"]
else None,
if value["parent_checkpoint_id"]
else None
),
await asyncio.to_thread(self._load_writes, value["pending_writes"]),
)
@@ -332,20 +339,25 @@ class AsyncPostgresSaver(BasePostgresSaver):
# a connection not in pipeline mode can only be used by one
# thread/coroutine at a time, so we acquire a lock
if self.supports_pipeline:
async with self.lock, conn.pipeline(), conn.cursor(
binary=True, row_factory=dict_row
) as cur:
async with (
self.lock,
conn.pipeline(),
conn.cursor(binary=True, row_factory=dict_row) as cur,
):
yield cur
else:
# Use connection's transaction context manager when pipeline mode not supported
async with self.lock, conn.transaction(), conn.cursor(
binary=True, row_factory=dict_row
) as cur:
async with (
self.lock,
conn.transaction(),
conn.cursor(binary=True, row_factory=dict_row) as cur,
):
yield cur
else:
async with self.lock, conn.cursor(
binary=True, row_factory=dict_row
) as cur:
async with (
self.lock,
conn.cursor(binary=True, row_factory=dict_row) as cur,
):
yield cur
def list(
@@ -374,7 +386,7 @@ class AsyncPostgresSaver(BasePostgresSaver):
while True:
try:
yield asyncio.run_coroutine_threadsafe(
anext(aiter_),
anext(aiter_), # noqa: F821
self.loop,
).result()
except StopAsyncIteration:
@@ -453,3 +465,6 @@ class AsyncPostgresSaver(BasePostgresSaver):
return asyncio.run_coroutine_threadsafe(
self.aput_writes(config, writes, task_id), self.loop
).result()
__all__ = ["AsyncPostgresSaver", "Conn"]
@@ -1,5 +1,6 @@
import random
from typing import Any, List, Optional, Sequence, Tuple, cast
from collections.abc import Sequence
from typing import Any, Optional, cast
from langchain_core.runnables import RunnableConfig
from psycopg.types.json import Jsonb
@@ -249,7 +250,7 @@ class BasePostgresSaver(BaseCheckpointSaver[str]):
config: Optional[RunnableConfig],
filter: MetadataInput,
before: Optional[RunnableConfig] = None,
) -> Tuple[str, List[Any]]:
) -> tuple[str, list[Any]]:
"""Return WHERE clause predicates for alist() given config, filter, before.
This method returns a tuple of a string and a tuple of values. The string
@@ -1,13 +1,11 @@
import asyncio
import logging
from collections.abc import AsyncIterator, Iterable, Sequence
from contextlib import asynccontextmanager
from typing import (
Any,
AsyncIterator,
Callable,
Iterable,
Optional,
Sequence,
Union,
cast,
)
@@ -32,6 +30,7 @@ from langgraph.store.base.batch import AsyncBatchedBaseStore
from langgraph.store.postgres.base import (
BasePostgresStore,
EmbeddingConfig,
PoolConfig,
Row,
_decode_ns_bytes,
_group_ops,
@@ -241,14 +240,18 @@ class AsyncPostgresStore(AsyncBatchedBaseStore, BasePostgresStore[_ainternal.Con
# a connection not in pipeline mode can only be used by one
# thread/coroutine at a time, so we acquire a lock
if self.supports_pipeline:
async with self.lock, conn.pipeline(), conn.cursor(
binary=True, row_factory=dict_row
) as cur:
async with (
self.lock,
conn.pipeline(),
conn.cursor(binary=True, row_factory=dict_row) as cur,
):
yield cur
else:
async with self.lock, conn.transaction(), conn.cursor(
binary=True, row_factory=dict_row
) as cur:
async with (
self.lock,
conn.transaction(),
conn.cursor(binary=True, row_factory=dict_row) as cur,
):
yield cur
else:
async with conn.cursor(binary=True, row_factory=dict_row) as cur:
@@ -264,9 +267,7 @@ class AsyncPostgresStore(AsyncBatchedBaseStore, BasePostgresStore[_ainternal.Con
conn_string: str,
*,
pipeline: bool = False,
min_size: int = 1,
max_size: Optional[int] = None,
use_pool: bool = False,
pool_config: Optional[PoolConfig] = None,
embedding: Optional[EmbeddingConfig] = None,
) -> AsyncIterator["AsyncPostgresStore"]:
"""Create a new AsyncPostgresStore instance from a connection string.
@@ -274,26 +275,29 @@ class AsyncPostgresStore(AsyncBatchedBaseStore, BasePostgresStore[_ainternal.Con
Args:
conn_string (str): The Postgres connection info string.
pipeline (bool): Whether to use AsyncPipeline (only for single connections)
min_size (int): Minimum number of connections when using a pool
max_size (Optional[int]): Maximum number of connections when using a pool
use_pool (bool): Whether to use a connection pool
embedding (Optional[EmbeddingConfig]): Configuration for vector embeddings
pool_config (Optional[PoolConfig]): Configuration for the connection pool.
If provided, will create a connection pool and use it instead of a single connection.
This overrides the `pipeline` argument.
embedding (Optional[EmbeddingConfig]): The embedding config.
Returns:
AsyncPostgresStore: A new AsyncPostgresStore instance.
"""
if use_pool:
if pool_config is not None:
pc = pool_config.copy()
async with cast(
AsyncConnectionPool[AsyncConnection[DictRow]],
AsyncConnectionPool(
conn_string,
min_size=min_size,
max_size=max_size,
min_size=pc.pop("min_size", 1),
max_size=pc.pop("max_size", None),
kwargs={
"autocommit": True,
"prepare_threshold": 0,
"row_factory": dict_row,
**(pc.pop("kwargs", None) or {}),
},
**cast(dict, pc),
),
) as pool:
yield cls(conn=pool, embedding=embedding)
@@ -3,20 +3,15 @@ import json
import logging
import threading
from collections import defaultdict
from collections.abc import Awaitable, Iterable, Iterator, Sequence
from contextlib import contextmanager
from dataclasses import dataclass
from datetime import datetime
from typing import (
Any,
Awaitable,
Callable,
Generic,
Iterable,
Iterator,
List,
Optional,
Sequence,
Type,
TypeVar,
Union,
cast,
@@ -28,6 +23,7 @@ from psycopg import Capabilities, Connection, Cursor, Pipeline
from psycopg.errors import UndefinedTable
from psycopg.rows import DictRow, dict_row
from psycopg.types.json import Jsonb
from psycopg_pool import ConnectionPool
from typing_extensions import TypedDict
from langgraph.checkpoint.postgres import _ainternal as _ainternal
@@ -177,6 +173,31 @@ CREATE INDEX IF NOT EXISTS store_vectors_embedding_idx ON store_vectors
C = TypeVar("C", bound=Union[_pg_internal.Conn, _ainternal.Conn])
class PoolConfig(TypedDict, total=False):
"""Connection pool settings for PostgreSQL connections.
Controls connection lifecycle and resource utilization:
- Small pools (1-5) suit low-concurrency workloads
- Larger pools handle concurrent requests but consume more resources
- Setting max_size prevents resource exhaustion under load
"""
min_size: int
"""Minimum number of connections maintained in the pool. Defaults to 1."""
max_size: Optional[int]
"""Maximum number of connections allowed in the pool. None means unlimited."""
kwargs: dict
"""Additional connection arguments passed to each connection in the pool.
Default kwargs set automatically:
- autocommit: True
- prepare_threshold: 0
- row_factory: dict_row
"""
class BasePostgresStore(Generic[C]):
MIGRATIONS = MIGRATIONS
conn: C
@@ -459,6 +480,7 @@ class PostgresStore(BaseStore, BasePostgresStore[_pg_internal.Conn]):
conn_string: str,
*,
pipeline: bool = False,
pool_config: Optional[PoolConfig] = None,
embedding: Optional[EmbeddingConfig] = None,
) -> Iterator["PostgresStore"]:
"""Create a new PostgresStore instance from a connection string.
@@ -466,19 +488,41 @@ class PostgresStore(BaseStore, BasePostgresStore[_pg_internal.Conn]):
Args:
conn_string (str): The Postgres connection info string.
pipeline (bool): whether to use Pipeline
pool_config (Optional[PoolArgs]): Configuration for the connection pool.
If provided, will create a connection pool and use it instead of a single connection.
This overrides the `pipeline` argument.
embedding (Optional[EmbeddingConfig]): The embedding config.
Returns:
PostgresStore: A new PostgresStore instance.
"""
with Connection.connect(
conn_string, autocommit=True, prepare_threshold=0, row_factory=dict_row
) as conn:
if pipeline:
with conn.pipeline() as pipe:
yield cls(conn, pipe=pipe, embedding=embedding)
else:
yield cls(conn, embedding=embedding)
if pool_config is not None:
pc = pool_config.copy()
with cast(
ConnectionPool[Connection[DictRow]],
ConnectionPool(
conn_string,
min_size=pc.pop("min_size", 1),
max_size=pc.pop("max_size", None),
kwargs={
"autocommit": True,
"prepare_threshold": 0,
"row_factory": dict_row,
**(pc.pop("kwargs", None) or {}),
},
**cast(dict, pc),
),
) as pool:
yield cls(conn=pool, embedding=embedding)
else:
with Connection.connect(
conn_string, autocommit=True, prepare_threshold=0, row_factory=dict_row
) as conn:
if pipeline:
with conn.pipeline() as pipe:
yield cls(conn, pipe=pipe, embedding=embedding)
else:
yield cls(conn, embedding=embedding)
@contextmanager
def _cursor(self, *, pipeline: bool = False) -> Iterator[Cursor[DictRow]]:
@@ -504,14 +548,18 @@ class PostgresStore(BaseStore, BasePostgresStore[_pg_internal.Conn]):
# a connection not in pipeline mode can only be used by one
# thread/coroutine at a time, so we acquire a lock
if self.supports_pipeline:
with self.lock, conn.pipeline(), conn.cursor(
binary=True, row_factory=dict_row
) as cur:
with (
self.lock,
conn.pipeline(),
conn.cursor(binary=True, row_factory=dict_row) as cur,
):
yield cur
else:
with self.lock, conn.transaction(), conn.cursor(
binary=True, row_factory=dict_row
) as cur:
with (
self.lock,
conn.transaction(),
conn.cursor(binary=True, row_factory=dict_row) as cur,
):
yield cur
else:
with conn.cursor(binary=True, row_factory=dict_row) as cur:
@@ -617,7 +665,7 @@ class PostgresStore(BaseStore, BasePostgresStore[_pg_internal.Conn]):
for (idx, _), (query, params) in zip(search_ops, queries):
cur.execute(query, params)
rows = cast(list[Row], cur.fetchall())
items = [
results[idx] = [
_row_to_item(
_decode_ns_bytes(row["prefix"]),
row,
@@ -626,7 +674,6 @@ class PostgresStore(BaseStore, BasePostgresStore[_pg_internal.Conn]):
)
for row in rows
]
results[idx] = items
def _batch_list_namespaces_ops(
self,
@@ -726,7 +773,7 @@ def _row_to_item(
row: Row,
*,
loader: Optional[Callable[[Union[bytes, orjson.Fragment]], dict[str, Any]]] = None,
cls: Union[Type[SearchItem], Type[Item]] = Item,
cls: Union[type[SearchItem], type[Item]] = Item,
) -> Union[Item, SearchItem]:
"""Convert a row from the database into an Item.
@@ -796,7 +843,7 @@ def _tokenize_path(path: str) -> list[str]:
return []
tokens = []
current: List[str] = []
current: list[str] = []
i = 0
while i < len(path):
char = path[i]
+1 -1
View File
@@ -1,4 +1,4 @@
from typing import AsyncIterator
from collections.abc import AsyncIterator
import pytest
from psycopg import AsyncConnection
@@ -1,7 +1,7 @@
# type: ignore
import sys
import uuid
from typing import AsyncIterator
from collections.abc import AsyncIterator
import pytest
from conftest import DEFAULT_URI # type: ignore
@@ -43,9 +43,8 @@ async def store(request) -> AsyncIterator[AsyncPostgresStore]:
yield store
elif request.param == "pool":
async with AsyncPostgresStore.from_conn_string(
conn_string, use_pool=True, max_size=10
conn_string, pool_config={"min_size": 1, "max_size": 10}
) as store:
await store.setup()
yield store
else: # default
async with AsyncPostgresStore.from_conn_string(conn_string) as store:
+4 -6
View File
@@ -7,7 +7,6 @@ import pytest
from conftest import DEFAULT_URI # type: ignore
from langchain_core.embeddings import Embeddings
from psycopg import Connection
from psycopg_pool import ConnectionPool
from utils import CharacterEmbeddings
from langgraph.store.base import (
@@ -45,10 +44,9 @@ def store(request) -> PostgresStore:
with PostgresStore.from_conn_string(conn_string, pipeline=True) as store:
yield store
elif request.param == "pool":
with ConnectionPool(
conn_string, max_size=10, kwargs={"autocommit": True}
) as pool:
store = PostgresStore(pool)
with PostgresStore.from_conn_string(
conn_string, pool_config={"min_size": 1, "max_size": 10}
) as store:
yield store
else: # default
with PostgresStore.from_conn_string(conn_string) as store:
@@ -566,7 +564,7 @@ def test_extract_text_by_path():
assert _extract_text_by_path(nested_data, "empty_dict") == ["{}"]
zeros = _extract_text_by_path(nested_data, "zeros[*]")
assert set(zeros) == {"0", "0.0", "0"}
assert set(zeros) == {"0", "0.0"}
assert _extract_text_by_path(nested_data, "items[].value") == []
assert _extract_text_by_path(nested_data, "items[abc].value") == []
+8
View File
@@ -1,3 +1,4 @@
import os
import pathlib
import shutil
import sys
@@ -598,6 +599,13 @@ def dev(
) from None
config_json = langgraph_cli.config.validate_config_file(config)
cwd = os.getcwd()
sys.path.append(cwd)
dependencies = config_json.get("dependencies", [])
for dep in dependencies:
dep_path = pathlib.Path(cwd) / dep
if dep_path.is_dir() and dep_path.exists():
sys.path.append(str(dep_path))
graphs = config_json.get("graphs", {})
run_server(
+1 -1
View File
@@ -1,6 +1,6 @@
[tool.poetry]
name = "langgraph-cli"
version = "0.1.58"
version = "0.1.59"
description = "CLI for interacting with LangGraph API"
authors = []
license = "MIT"
+4 -6
View File
@@ -309,8 +309,7 @@ async def _store_postgres_aio_pool():
try:
async with AsyncPostgresStore.from_conn_string(
DEFAULT_POSTGRES_URI + database,
use_pool=True,
max_size=10,
pool_config={"max_size": 10},
) as store:
await store.setup()
yield store
@@ -373,10 +372,9 @@ def store_postgres_pool():
conn.execute(f"CREATE DATABASE {database}")
try:
# yield store
with ConnectionPool(
DEFAULT_POSTGRES_URI + database, max_size=10, kwargs={"autocommit": True}
) as pool:
store = PostgresStore(pool)
with PostgresStore.from_conn_string(
DEFAULT_POSTGRES_URI + database, pool_config={"max_size": 10}
) as store:
store.setup()
yield store
finally: