WEBDIENSTE/langgraph
1
0
Fork 0
mirror of https://github.com/langchain-ai/langgraph.git synced 2026-09-21 17:18:09 +02:00
Code Issues Packages Projects Releases Wiki Activity

docs: node caching (#4749)

Browse Source
This commit is contained in:
Sydney Runkle
2025-05-19 15:49:31 -04:00
committed by GitHub
parent 46a9d3159d
commit 1e938a692f
7 changed files with 189 additions and 56 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/docs/concepts/low_level.md
+48
View File
@@ -247,6 +247,54 @@ from langgraph.graph import END
graph.add_edge("node_a", END)
```
### Node Caching
LangGraph supports caching of tasks/nodes based on the input to the node. To use caching:
* Specify a cache when compiling a graph (or specifying an entrypoint)
* Specify a cache policy for nodes. Each cache policy supports:
* `key_func` used to generate a cache key based on the input to a node, which defaults to a `hash` of the input with pickle.
* `ttl`, the time to live for the cache in seconds. If not specified, the cache will never expire.
For example:
```py
import time
from typing_extensions import TypedDict
from langgraph.graph import StateGraph
from langgraph.cache.memory import InMemoryCache
from langgraph.types import CachePolicy
class State(TypedDict):
x: int
result: int
builder = StateGraph(State)
def expensive_node(state: State) -> dict[str, int]:
# expensive computation
time.sleep(2)
return {"result": state["x"] * 2}
builder.add_node("expensive_node", expensive_node, cache_policy=CachePolicy(ttl=3))
builder.set_entry_point("expensive_node")
builder.set_finish_point("expensive_node")
graph = builder.compile(cache=InMemoryCache())
print(graph.invoke({"x": 5}, stream_mode='updates')) # (1)!
[{'expensive_node': {'result': 10}}]
print(graph.invoke({"x": 5}, stream_mode='updates')) # (2)!
[{'expensive_node': {'result': 10}, '__metadata__': {'cached': True}}]
```
1. First run takes the full second to run (due to mocked expensive computation).
2. Second run utilizes cache and returns quickly.
## Edges
Edges define how the logic is routed and how the graph decides to stop. This is a big part of how your agents work and how different nodes communicate with each other. There are a few key types of edges:
docs/docs/how-tos/graph-api.ipynb
+32 -1
View File
@@ -1288,12 +1288,43 @@
},
{
"cell_type": "markdown",
"id": "4eeb895c-adca-40ab-b289-93ee56e18661",
"id": "068f806a",
"metadata": {},
"source": [
"</details>"
]
},
{
"cell_type": "markdown",
"id": "6d99d63c",
"metadata": {},
"source": [
"## Add node caching\n",
"\n",
"Node caching is useful in cases where you want to avoid repeating operations, like when doing something expensive (either in terms of time or cost). LangGraph lets you add individualized caching policies to nodes in a graph.\n",
"\n",
"To configure a cache policy, pass the `cache_policy` parameter to the [add_node](https://langchain-ai.github.io/langgraph/reference/graphs/#langgraph.graph.state.StateGraph.add_node) function. In the following example, a [`CachePolicy`](https://langchain-ai.github.io/langgraph/reference/types/?h=cachepolicy#langgraph.types.CachePolicy) object is instantiated with a time to live of 120 seconds and the default `key_func` generator. Then it is associated with a node:\n",
"\n",
"```python\n",
"from langgraph.types import CachePolicy\n",
"\n",
"builder.add_node(\n",
" \"node_name\",\n",
" node_function,\n",
" cache_policy=CachePolicy(ttl=120),\n",
")\n",
"```\n",
"\n",
"Then, to enable node-level caching for a graph, set the `cache` argument when compiling the graph. The example below uses `InMemoryCache` to set up a graph with in-memory cache, but `SqliteCache` is also available.\n",
"\n",
"```python\n",
"from langgraph.cache.memory import InMemoryCache\n",
"\n",
"\n",
"graph = builder.compile(cache=InMemoryCache())\n",
"```"
]
},
{
"cell_type": "markdown",
"id": "e1a0213e-282f-4fad-b048-5f7465edfccb",
docs/docs/how-tos/use-functional-api.md
+32
View File
@@ -349,6 +349,38 @@ main.invoke({'any_input': 'foobar'}, config=config)
'OK'
```
## Caching Tasks
```python
import time
from langgraph.cache.memory import InMemoryCache
from langgraph.func import entrypoint, task
from langgraph.types import CachePolicy
@task(cache_policy=CachePolicy(ttl=120)) # (1)!
def slow_add(x: int) -> int:
time.sleep(1)
return x * 2
@entrypoint(cache=InMemoryCache())
def main(inputs: dict) -> dict[str, int]:
result1 = slow_add(inputs["x"]).result()
result2 = slow_add(inputs["x"]).result()
return {"result1": result1, "result2": result2}
for chunk in main.stream({"x": 5}, stream_mode="updates"):
print(chunk)
#> {'slow_add': 10}
#> {'slow_add': 10, '__metadata__': {'cached': True}}
#> {'main': {'result1': 10, 'result2': 10}}
```
1. `ttl` is specified in seconds. The cache will be invalidated after this time.
## Resuming after an error
```python
docs/docs/reference/cache.md
+5
View File
@@ -0,0 +1,5 @@
## Caching
::: langgraph.cache.base
::: langgraph.cache.memory
::: langgraph.cache.sqlite
docs/docs/reference/index.md
+1
View File
@@ -29,6 +29,7 @@ The core APIs for the LangGraph opens source library.
- [Pregel](pregel.md): Pregel-inspired computation model.
- [Checkpointing](checkpoints.md): Saving and restoring graph state.
- [Storage](store.md): Storage backends and options.
- [Caching](cache.md): Caching mechanisms for performance.
- [Types](types.md): Type definitions for graph components.
- [Config](config.md): Configuration options.
- [Errors](errors.md): Error types and handling.
docs/mkdocs.yml
+1
View File
@@ -255,6 +255,7 @@ nav:
- Pregel: reference/pregel.md
- Checkpointing: reference/checkpoints.md
- Storage: reference/store.md
- Caching: reference/cache.md
- Types: reference/types.md
- Config: reference/config.md
- Errors: reference/errors.md
docs/uv.lock
Generated
+70 -55
View File
@@ -291,16 +291,16 @@ name = "autogen"
version = "0.3.2"
source = { registry = "https://pypi.org/simple" }
dependencies = [
{ name = "diskcache" },
{ name = "docker" },
{ name = "flaml" },
{ name = "numpy" },
{ name = "openai" },
{ name = "packaging" },
{ name = "pydantic" },
{ name = "python-dotenv" },
{ name = "termcolor" },
{ name = "tiktoken" },
{ name = "diskcache", marker = "python_full_version < '3.13'" },
{ name = "docker", marker = "python_full_version < '3.13'" },
{ name = "flaml", marker = "python_full_version < '3.13'" },
{ name = "numpy", marker = "python_full_version < '3.13'" },
{ name = "openai", marker = "python_full_version < '3.13'" },
{ name = "packaging", marker = "python_full_version < '3.13'" },
{ name = "pydantic", marker = "python_full_version < '3.13'" },
{ name = "python-dotenv", marker = "python_full_version < '3.13'" },
{ name = "termcolor", marker = "python_full_version < '3.13'" },
{ name = "tiktoken", marker = "python_full_version < '3.13'" },
]
sdist = { url = "https://files.pythonhosted.org/packages/7b/e8/33b7fb072fbcf63b8a1b5bbba15570e4e8c86d6374da398889b92fc420c8/autogen-0.3.2.tar.gz", hash = "sha256:9f8a1170ac2e5a1fc9efc3cfa6e23261dd014db97b17c8c416f97ee14951bc7b", size = 306281 }
wheels = [
@@ -1008,9 +1008,9 @@ name = "docker"
version = "7.1.0"
source = { registry = "https://pypi.org/simple" }
dependencies = [
{ name = "pywin32", marker = "sys_platform == 'win32'" },
{ name = "requests" },
{ name = "urllib3" },
{ name = "pywin32", marker = "python_full_version < '3.13' and sys_platform == 'win32'" },
{ name = "requests", marker = "python_full_version < '3.13'" },
{ name = "urllib3", marker = "python_full_version < '3.13'" },
]
sdist = { url = "https://files.pythonhosted.org/packages/91/9b/4a2ea29aeba62471211598dac5d96825bb49348fa07e906ea930394a83ce/docker-7.1.0.tar.gz", hash = "sha256:ad8c70e6e3f8926cb8a92619b832b4ea5299e2831c14284663184e200546fa6c", size = 117834 }
wheels = [
@@ -1052,7 +1052,7 @@ name = "exceptiongroup"
version = "1.3.0"
source = { registry = "https://pypi.org/simple" }
dependencies = [
{ name = "typing-extensions", marker = "python_full_version < '3.12.4'" },
{ name = "typing-extensions", marker = "python_full_version < '3.11'" },
]
sdist = { url = "https://files.pythonhosted.org/packages/0b/9f/a65090624ecf468cdca03533906e7c69ed7588582240cfe7cc9e770b50eb/exceptiongroup-1.3.0.tar.gz", hash = "sha256:b241f5885f560bc56a59ee63ca4c6a8bfa46ae4ad651af316d4e81817bb9fd88", size = 29749 }
wheels = [
@@ -1153,7 +1153,7 @@ name = "flaml"
version = "2.3.4"
source = { registry = "https://pypi.org/simple" }
dependencies = [
{ name = "numpy" },
{ name = "numpy", marker = "python_full_version < '3.13'" },
]
sdist = { url = "https://files.pythonhosted.org/packages/20/a8/17322311b77f3012194f92c47c81455463f99c48d358c463fa45bd3c8541/flaml-2.3.4.tar.gz", hash = "sha256:308c3e769976d8a0272f2fd7d98258d7d4a4fd2e4525ba540d1ba149ae266c54", size = 284728 }
wheels = [
@@ -2590,7 +2590,7 @@ wheels = [
[[package]]
name = "langgraph"
version = "0.4.3"
version = "0.4.5"
source = { editable = "../libs/langgraph" }
dependencies = [
{ name = "langchain-core" },
@@ -2641,7 +2641,7 @@ dev = [
[[package]]
name = "langgraph-checkpoint"
version = "2.0.25"
version = "2.0.26"
source = { editable = "../libs/checkpoint" }
dependencies = [
{ name = "langchain-core" },
@@ -2715,17 +2715,19 @@ dev = [
[[package]]
name = "langgraph-checkpoint-sqlite"
version = "2.0.7"
version = "2.0.10"
source = { editable = "../libs/checkpoint-sqlite" }
dependencies = [
{ name = "aiosqlite" },
{ name = "langgraph-checkpoint" },
{ name = "sqlite-vec" },
]
[package.metadata]
requires-dist = [
{ name = "aiosqlite", specifier = ">=0.20" },
{ name = "langgraph-checkpoint", editable = "../libs/checkpoint" },
{ name = "sqlite-vec", specifier = ">=0.1.6" },
]
[package.metadata.requires-dev]
@@ -2736,6 +2738,7 @@ dev = [
{ name = "pytest" },
{ name = "pytest-asyncio" },
{ name = "pytest-mock" },
{ name = "pytest-retry", specifier = ">=1.7.0" },
{ name = "pytest-watcher" },
{ name = "ruff" },
]
@@ -2825,11 +2828,11 @@ requires-dist = [
[package.metadata.requires-dev]
docs = [
{ name = "click", specifier = ">=8.1.7,<9" },
{ name = "jupyter", specifier = ">=1.1.1,<2" },
{ name = "langchain-cohere", specifier = ">=0.4.2,<0.5" },
{ name = "click" },
{ name = "jupyter" },
{ name = "langchain-cohere" },
{ name = "langchain-mcp-adapters", git = "https://github.com/langchain-ai/langchain-mcp-adapters" },
{ name = "langchain-ollama", specifier = ">=0.2.3,<0.3" },
{ name = "langchain-ollama" },
{ name = "langgraph", editable = "../libs/langgraph" },
{ name = "langgraph-checkpoint", editable = "../libs/checkpoint" },
{ name = "langgraph-checkpoint-postgres", editable = "../libs/checkpoint-postgres" },
@@ -2849,41 +2852,41 @@ docs = [
{ name = "mkdocs-rss-plugin" },
{ name = "mkdocstrings" },
{ name = "mkdocstrings-python" },
{ name = "psycopg", extras = ["binary"], specifier = ">=3.2.0,<4" },
{ name = "psycopg-pool", specifier = ">=3.2.0,<4" },
{ name = "pygments-ansi-color", specifier = ">=0.3" },
{ name = "ruff", specifier = ">=0.6.8,<0.7" },
{ name = "vcrpy", specifier = ">=6.0.1,<7" },
{ name = "psycopg", extras = ["binary"] },
{ name = "psycopg-pool" },
{ name = "pygments-ansi-color" },
{ name = "ruff" },
{ name = "vcrpy" },
]
test = [
{ name = "autogen", marker = "python_full_version >= '3.8' and python_full_version < '3.13'", specifier = ">=0.3.0,<0.4" },
{ name = "chromadb", specifier = ">=0.5.5,<0.6" },
{ name = "gpt4all", specifier = ">=2.8.2,<3" },
{ name = "grandalf", specifier = ">=0.8,<0.9" },
{ name = "langchain", specifier = ">=0.3.8,<0.4" },
{ name = "langchain-anthropic", specifier = ">=0.3.8,<0.4" },
{ name = "langchain-community", specifier = ">=0.3.0,<0.4" },
{ name = "langchain-core", specifier = ">=0.3.54,<0.4" },
{ name = "langchain-experimental", specifier = ">=0.3.2,<0.4" },
{ name = "langchain-fireworks", specifier = ">=0.2.0,<0.3" },
{ name = "langchain-mistralai", specifier = ">=0.2.6,<0.3" },
{ name = "langchain-nomic", specifier = ">=0.1.3,<0.2" },
{ name = "langchain-openai", specifier = ">=0.3.7,<0.4" },
{ name = "langchain-tavily", specifier = ">=0.1.5,<0.2" },
{ name = "langgraph-checkpoint-mongodb", specifier = ">=0.1.0,<0.2" },
{ name = "langmem", specifier = ">=0.0.19,<0.0.20" },
{ name = "langsmith", specifier = ">=0.3.0,<0.4" },
{ name = "matplotlib", specifier = ">=3.9.2,<4" },
{ name = "motor", specifier = ">=3.5.1,<4" },
{ name = "networkx", specifier = "~=3.3" },
{ name = "numexpr", specifier = ">=2.10.1,<3" },
{ name = "numpy", specifier = ">=1.26.4,<2" },
{ name = "pymongo", specifier = ">=4.8.0,<5" },
{ name = "pyppeteer", specifier = ">=2.0.0,<3" },
{ name = "pytest", specifier = ">=8.3.5,<9" },
{ name = "pytest-check-links", specifier = ">=0.10.1,<0.11" },
{ name = "redis", specifier = ">=5.0.8,<6" },
{ name = "scikit-learn", specifier = ">=1.5.2,<2" },
{ name = "autogen", marker = "python_full_version >= '3.8' and python_full_version < '3.13'" },
{ name = "chromadb" },
{ name = "gpt4all" },
{ name = "grandalf" },
{ name = "langchain" },
{ name = "langchain-anthropic" },
{ name = "langchain-community" },
{ name = "langchain-core" },
{ name = "langchain-experimental" },
{ name = "langchain-fireworks" },
{ name = "langchain-mistralai" },
{ name = "langchain-nomic" },
{ name = "langchain-openai" },
{ name = "langchain-tavily" },
{ name = "langgraph-checkpoint-mongodb" },
{ name = "langmem" },
{ name = "langsmith" },
{ name = "matplotlib" },
{ name = "motor" },
{ name = "networkx" },
{ name = "numexpr" },
{ name = "numpy" },
{ name = "pymongo" },
{ name = "pyppeteer" },
{ name = "pytest" },
{ name = "pytest-check-links" },
{ name = "redis" },
{ name = "scikit-learn" },
]
[[package]]
@@ -5769,6 +5772,18 @@ wheels = [
{ url = "https://files.pythonhosted.org/packages/1c/fc/9ba22f01b5cdacc8f5ed0d22304718d2c758fce3fd49a5372b886a86f37c/sqlalchemy-2.0.41-py3-none-any.whl", hash = "sha256:57df5dc6fdb5ed1a88a1ed2195fd31927e705cad62dedd86b46972752a80f576", size = 1911224 },
]
[[package]]
name = "sqlite-vec"
version = "0.1.6"
source = { registry = "https://pypi.org/simple" }
wheels = [
{ url = "https://files.pythonhosted.org/packages/88/ed/aabc328f29ee6814033d008ec43e44f2c595447d9cccd5f2aabe60df2933/sqlite_vec-0.1.6-py3-none-macosx_10_6_x86_64.whl", hash = "sha256:77491bcaa6d496f2acb5cc0d0ff0b8964434f141523c121e313f9a7d8088dee3", size = 164075 },
{ url = "https://files.pythonhosted.org/packages/a7/57/05604e509a129b22e303758bfa062c19afb020557d5e19b008c64016704e/sqlite_vec-0.1.6-py3-none-macosx_11_0_arm64.whl", hash = "sha256:fdca35f7ee3243668a055255d4dee4dea7eed5a06da8cad409f89facf4595361", size = 165242 },
{ url = "https://files.pythonhosted.org/packages/f2/48/dbb2cc4e5bad88c89c7bb296e2d0a8df58aab9edc75853728c361eefc24f/sqlite_vec-0.1.6-py3-none-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:7b0519d9cd96164cd2e08e8eed225197f9cd2f0be82cb04567692a0a4be02da3", size = 103704 },
{ url = "https://files.pythonhosted.org/packages/80/76/97f33b1a2446f6ae55e59b33869bed4eafaf59b7f4c662c8d9491b6a714a/sqlite_vec-0.1.6-py3-none-manylinux_2_17_x86_64.manylinux2014_x86_64.manylinux1_x86_64.whl", hash = "sha256:823b0493add80d7fe82ab0fe25df7c0703f4752941aee1c7b2b02cec9656cb24", size = 151556 },
{ url = "https://files.pythonhosted.org/packages/6a/98/e8bc58b178266eae2fcf4c9c7a8303a8d41164d781b32d71097924a6bebe/sqlite_vec-0.1.6-py3-none-win_amd64.whl", hash = "sha256:c65bcfd90fa2f41f9000052bcb8bb75d38240b2dae49225389eca6c3136d3f0c", size = 281540 },
]
[[package]]
name = "sse-starlette"
version = "2.3.5"