diff --git a/docs/docs/concepts/low_level.md b/docs/docs/concepts/low_level.md index cc2cb7382..bd75f026e 100644 --- a/docs/docs/concepts/low_level.md +++ b/docs/docs/concepts/low_level.md @@ -247,6 +247,54 @@ from langgraph.graph import END graph.add_edge("node_a", END) ``` +### Node Caching + +LangGraph supports caching of tasks/nodes based on the input to the node. To use caching: + +* Specify a cache when compiling a graph (or specifying an entrypoint) +* Specify a cache policy for nodes. Each cache policy supports: + * `key_func` used to generate a cache key based on the input to a node, which defaults to a `hash` of the input with pickle. + * `ttl`, the time to live for the cache in seconds. If not specified, the cache will never expire. + +For example: + +```py +import time +from typing_extensions import TypedDict +from langgraph.graph import StateGraph +from langgraph.cache.memory import InMemoryCache +from langgraph.types import CachePolicy + + +class State(TypedDict): + x: int + result: int + + +builder = StateGraph(State) + + +def expensive_node(state: State) -> dict[str, int]: + # expensive computation + time.sleep(2) + return {"result": state["x"] * 2} + + +builder.add_node("expensive_node", expensive_node, cache_policy=CachePolicy(ttl=3)) +builder.set_entry_point("expensive_node") +builder.set_finish_point("expensive_node") + +graph = builder.compile(cache=InMemoryCache()) + +print(graph.invoke({"x": 5}, stream_mode='updates')) # (1)! +[{'expensive_node': {'result': 10}}] +print(graph.invoke({"x": 5}, stream_mode='updates')) # (2)! +[{'expensive_node': {'result': 10}, '__metadata__': {'cached': True}}] +``` + +1. First run takes the full second to run (due to mocked expensive computation). +2. Second run utilizes cache and returns quickly. + ## Edges Edges define how the logic is routed and how the graph decides to stop. This is a big part of how your agents work and how different nodes communicate with each other. There are a few key types of edges: diff --git a/docs/docs/how-tos/graph-api.ipynb b/docs/docs/how-tos/graph-api.ipynb index 17c3e3684..5c0ea0e43 100644 --- a/docs/docs/how-tos/graph-api.ipynb +++ b/docs/docs/how-tos/graph-api.ipynb @@ -1288,12 +1288,43 @@ }, { "cell_type": "markdown", - "id": "4eeb895c-adca-40ab-b289-93ee56e18661", + "id": "068f806a", "metadata": {}, "source": [ "" ] }, + { + "cell_type": "markdown", + "id": "6d99d63c", + "metadata": {}, + "source": [ + "## Add node caching\n", + "\n", + "Node caching is useful in cases where you want to avoid repeating operations, like when doing something expensive (either in terms of time or cost). LangGraph lets you add individualized caching policies to nodes in a graph.\n", + "\n", + "To configure a cache policy, pass the `cache_policy` parameter to the [add_node](https://langchain-ai.github.io/langgraph/reference/graphs/#langgraph.graph.state.StateGraph.add_node) function. In the following example, a [`CachePolicy`](https://langchain-ai.github.io/langgraph/reference/types/?h=cachepolicy#langgraph.types.CachePolicy) object is instantiated with a time to live of 120 seconds and the default `key_func` generator. Then it is associated with a node:\n", + "\n", + "```python\n", + "from langgraph.types import CachePolicy\n", + "\n", + "builder.add_node(\n", + " \"node_name\",\n", + " node_function,\n", + " cache_policy=CachePolicy(ttl=120),\n", + ")\n", + "```\n", + "\n", + "Then, to enable node-level caching for a graph, set the `cache` argument when compiling the graph. The example below uses `InMemoryCache` to set up a graph with in-memory cache, but `SqliteCache` is also available.\n", + "\n", + "```python\n", + "from langgraph.cache.memory import InMemoryCache\n", + "\n", + "\n", + "graph = builder.compile(cache=InMemoryCache())\n", + "```" + ] + }, { "cell_type": "markdown", "id": "e1a0213e-282f-4fad-b048-5f7465edfccb", diff --git a/docs/docs/how-tos/use-functional-api.md b/docs/docs/how-tos/use-functional-api.md index 597c293fa..4f9b0e864 100644 --- a/docs/docs/how-tos/use-functional-api.md +++ b/docs/docs/how-tos/use-functional-api.md @@ -349,6 +349,38 @@ main.invoke({'any_input': 'foobar'}, config=config) 'OK' ``` +## Caching Tasks + +```python +import time +from langgraph.cache.memory import InMemoryCache +from langgraph.func import entrypoint, task +from langgraph.types import CachePolicy + + +@task(cache_policy=CachePolicy(ttl=120)) # (1)! +def slow_add(x: int) -> int: + time.sleep(1) + return x * 2 + + +@entrypoint(cache=InMemoryCache()) +def main(inputs: dict) -> dict[str, int]: + result1 = slow_add(inputs["x"]).result() + result2 = slow_add(inputs["x"]).result() + return {"result1": result1, "result2": result2} + + +for chunk in main.stream({"x": 5}, stream_mode="updates"): + print(chunk) + +#> {'slow_add': 10} +#> {'slow_add': 10, '__metadata__': {'cached': True}} +#> {'main': {'result1': 10, 'result2': 10}} +``` + +1. `ttl` is specified in seconds. The cache will be invalidated after this time. + ## Resuming after an error ```python diff --git a/docs/docs/reference/cache.md b/docs/docs/reference/cache.md new file mode 100644 index 000000000..05d65af59 --- /dev/null +++ b/docs/docs/reference/cache.md @@ -0,0 +1,5 @@ +## Caching + +::: langgraph.cache.base +::: langgraph.cache.memory +::: langgraph.cache.sqlite \ No newline at end of file diff --git a/docs/docs/reference/index.md b/docs/docs/reference/index.md index e84688d8b..71d201902 100644 --- a/docs/docs/reference/index.md +++ b/docs/docs/reference/index.md @@ -29,6 +29,7 @@ The core APIs for the LangGraph opens source library. - [Pregel](pregel.md): Pregel-inspired computation model. - [Checkpointing](checkpoints.md): Saving and restoring graph state. - [Storage](store.md): Storage backends and options. +- [Caching](cache.md): Caching mechanisms for performance. - [Types](types.md): Type definitions for graph components. - [Config](config.md): Configuration options. - [Errors](errors.md): Error types and handling. diff --git a/docs/mkdocs.yml b/docs/mkdocs.yml index 2709c96c9..85a405068 100644 --- a/docs/mkdocs.yml +++ b/docs/mkdocs.yml @@ -255,6 +255,7 @@ nav: - Pregel: reference/pregel.md - Checkpointing: reference/checkpoints.md - Storage: reference/store.md + - Caching: reference/cache.md - Types: reference/types.md - Config: reference/config.md - Errors: reference/errors.md diff --git a/docs/uv.lock b/docs/uv.lock index 83c960dc0..3bdc91742 100644 --- a/docs/uv.lock +++ b/docs/uv.lock @@ -291,16 +291,16 @@ name = "autogen" version = "0.3.2" source = { registry = "https://pypi.org/simple" } dependencies = [ - { name = "diskcache" }, - { name = "docker" }, - { name = "flaml" }, - { name = "numpy" }, - { name = "openai" }, - { name = "packaging" }, - { name = "pydantic" }, - { name = "python-dotenv" }, - { name = "termcolor" }, - { name = "tiktoken" }, + { name = "diskcache", marker = "python_full_version < '3.13'" }, + { name = "docker", marker = "python_full_version < '3.13'" }, + { name = "flaml", marker = "python_full_version < '3.13'" }, + { name = "numpy", marker = "python_full_version < '3.13'" }, + { name = "openai", marker = "python_full_version < '3.13'" }, + { name = "packaging", marker = "python_full_version < '3.13'" }, + { name = "pydantic", marker = "python_full_version < '3.13'" }, + { name = "python-dotenv", marker = "python_full_version < '3.13'" }, + { name = "termcolor", marker = "python_full_version < '3.13'" }, + { name = "tiktoken", marker = "python_full_version < '3.13'" }, ] sdist = { url = "https://files.pythonhosted.org/packages/7b/e8/33b7fb072fbcf63b8a1b5bbba15570e4e8c86d6374da398889b92fc420c8/autogen-0.3.2.tar.gz", hash = "sha256:9f8a1170ac2e5a1fc9efc3cfa6e23261dd014db97b17c8c416f97ee14951bc7b", size = 306281 } wheels = [ @@ -1008,9 +1008,9 @@ name = "docker" version = "7.1.0" source = { registry = "https://pypi.org/simple" } dependencies = [ - { name = "pywin32", marker = "sys_platform == 'win32'" }, - { name = "requests" }, - { name = "urllib3" }, + { name = "pywin32", marker = "python_full_version < '3.13' and sys_platform == 'win32'" }, + { name = "requests", marker = "python_full_version < '3.13'" }, + { name = "urllib3", marker = "python_full_version < '3.13'" }, ] sdist = { url = "https://files.pythonhosted.org/packages/91/9b/4a2ea29aeba62471211598dac5d96825bb49348fa07e906ea930394a83ce/docker-7.1.0.tar.gz", hash = "sha256:ad8c70e6e3f8926cb8a92619b832b4ea5299e2831c14284663184e200546fa6c", size = 117834 } wheels = [ @@ -1052,7 +1052,7 @@ name = "exceptiongroup" version = "1.3.0" source = { registry = "https://pypi.org/simple" } dependencies = [ - { name = "typing-extensions", marker = "python_full_version < '3.12.4'" }, + { name = "typing-extensions", marker = "python_full_version < '3.11'" }, ] sdist = { url = "https://files.pythonhosted.org/packages/0b/9f/a65090624ecf468cdca03533906e7c69ed7588582240cfe7cc9e770b50eb/exceptiongroup-1.3.0.tar.gz", hash = "sha256:b241f5885f560bc56a59ee63ca4c6a8bfa46ae4ad651af316d4e81817bb9fd88", size = 29749 } wheels = [ @@ -1153,7 +1153,7 @@ name = "flaml" version = "2.3.4" source = { registry = "https://pypi.org/simple" } dependencies = [ - { name = "numpy" }, + { name = "numpy", marker = "python_full_version < '3.13'" }, ] sdist = { url = "https://files.pythonhosted.org/packages/20/a8/17322311b77f3012194f92c47c81455463f99c48d358c463fa45bd3c8541/flaml-2.3.4.tar.gz", hash = "sha256:308c3e769976d8a0272f2fd7d98258d7d4a4fd2e4525ba540d1ba149ae266c54", size = 284728 } wheels = [ @@ -2590,7 +2590,7 @@ wheels = [ [[package]] name = "langgraph" -version = "0.4.3" +version = "0.4.5" source = { editable = "../libs/langgraph" } dependencies = [ { name = "langchain-core" }, @@ -2641,7 +2641,7 @@ dev = [ [[package]] name = "langgraph-checkpoint" -version = "2.0.25" +version = "2.0.26" source = { editable = "../libs/checkpoint" } dependencies = [ { name = "langchain-core" }, @@ -2715,17 +2715,19 @@ dev = [ [[package]] name = "langgraph-checkpoint-sqlite" -version = "2.0.7" +version = "2.0.10" source = { editable = "../libs/checkpoint-sqlite" } dependencies = [ { name = "aiosqlite" }, { name = "langgraph-checkpoint" }, + { name = "sqlite-vec" }, ] [package.metadata] requires-dist = [ { name = "aiosqlite", specifier = ">=0.20" }, { name = "langgraph-checkpoint", editable = "../libs/checkpoint" }, + { name = "sqlite-vec", specifier = ">=0.1.6" }, ] [package.metadata.requires-dev] @@ -2736,6 +2738,7 @@ dev = [ { name = "pytest" }, { name = "pytest-asyncio" }, { name = "pytest-mock" }, + { name = "pytest-retry", specifier = ">=1.7.0" }, { name = "pytest-watcher" }, { name = "ruff" }, ] @@ -2825,11 +2828,11 @@ requires-dist = [ [package.metadata.requires-dev] docs = [ - { name = "click", specifier = ">=8.1.7,<9" }, - { name = "jupyter", specifier = ">=1.1.1,<2" }, - { name = "langchain-cohere", specifier = ">=0.4.2,<0.5" }, + { name = "click" }, + { name = "jupyter" }, + { name = "langchain-cohere" }, { name = "langchain-mcp-adapters", git = "https://github.com/langchain-ai/langchain-mcp-adapters" }, - { name = "langchain-ollama", specifier = ">=0.2.3,<0.3" }, + { name = "langchain-ollama" }, { name = "langgraph", editable = "../libs/langgraph" }, { name = "langgraph-checkpoint", editable = "../libs/checkpoint" }, { name = "langgraph-checkpoint-postgres", editable = "../libs/checkpoint-postgres" }, @@ -2849,41 +2852,41 @@ docs = [ { name = "mkdocs-rss-plugin" }, { name = "mkdocstrings" }, { name = "mkdocstrings-python" }, - { name = "psycopg", extras = ["binary"], specifier = ">=3.2.0,<4" }, - { name = "psycopg-pool", specifier = ">=3.2.0,<4" }, - { name = "pygments-ansi-color", specifier = ">=0.3" }, - { name = "ruff", specifier = ">=0.6.8,<0.7" }, - { name = "vcrpy", specifier = ">=6.0.1,<7" }, + { name = "psycopg", extras = ["binary"] }, + { name = "psycopg-pool" }, + { name = "pygments-ansi-color" }, + { name = "ruff" }, + { name = "vcrpy" }, ] test = [ - { name = "autogen", marker = "python_full_version >= '3.8' and python_full_version < '3.13'", specifier = ">=0.3.0,<0.4" }, - { name = "chromadb", specifier = ">=0.5.5,<0.6" }, - { name = "gpt4all", specifier = ">=2.8.2,<3" }, - { name = "grandalf", specifier = ">=0.8,<0.9" }, - { name = "langchain", specifier = ">=0.3.8,<0.4" }, - { name = "langchain-anthropic", specifier = ">=0.3.8,<0.4" }, - { name = "langchain-community", specifier = ">=0.3.0,<0.4" }, - { name = "langchain-core", specifier = ">=0.3.54,<0.4" }, - { name = "langchain-experimental", specifier = ">=0.3.2,<0.4" }, - { name = "langchain-fireworks", specifier = ">=0.2.0,<0.3" }, - { name = "langchain-mistralai", specifier = ">=0.2.6,<0.3" }, - { name = "langchain-nomic", specifier = ">=0.1.3,<0.2" }, - { name = "langchain-openai", specifier = ">=0.3.7,<0.4" }, - { name = "langchain-tavily", specifier = ">=0.1.5,<0.2" }, - { name = "langgraph-checkpoint-mongodb", specifier = ">=0.1.0,<0.2" }, - { name = "langmem", specifier = ">=0.0.19,<0.0.20" }, - { name = "langsmith", specifier = ">=0.3.0,<0.4" }, - { name = "matplotlib", specifier = ">=3.9.2,<4" }, - { name = "motor", specifier = ">=3.5.1,<4" }, - { name = "networkx", specifier = "~=3.3" }, - { name = "numexpr", specifier = ">=2.10.1,<3" }, - { name = "numpy", specifier = ">=1.26.4,<2" }, - { name = "pymongo", specifier = ">=4.8.0,<5" }, - { name = "pyppeteer", specifier = ">=2.0.0,<3" }, - { name = "pytest", specifier = ">=8.3.5,<9" }, - { name = "pytest-check-links", specifier = ">=0.10.1,<0.11" }, - { name = "redis", specifier = ">=5.0.8,<6" }, - { name = "scikit-learn", specifier = ">=1.5.2,<2" }, + { name = "autogen", marker = "python_full_version >= '3.8' and python_full_version < '3.13'" }, + { name = "chromadb" }, + { name = "gpt4all" }, + { name = "grandalf" }, + { name = "langchain" }, + { name = "langchain-anthropic" }, + { name = "langchain-community" }, + { name = "langchain-core" }, + { name = "langchain-experimental" }, + { name = "langchain-fireworks" }, + { name = "langchain-mistralai" }, + { name = "langchain-nomic" }, + { name = "langchain-openai" }, + { name = "langchain-tavily" }, + { name = "langgraph-checkpoint-mongodb" }, + { name = "langmem" }, + { name = "langsmith" }, + { name = "matplotlib" }, + { name = "motor" }, + { name = "networkx" }, + { name = "numexpr" }, + { name = "numpy" }, + { name = "pymongo" }, + { name = "pyppeteer" }, + { name = "pytest" }, + { name = "pytest-check-links" }, + { name = "redis" }, + { name = "scikit-learn" }, ] [[package]] @@ -5769,6 +5772,18 @@ wheels = [ { url = "https://files.pythonhosted.org/packages/1c/fc/9ba22f01b5cdacc8f5ed0d22304718d2c758fce3fd49a5372b886a86f37c/sqlalchemy-2.0.41-py3-none-any.whl", hash = "sha256:57df5dc6fdb5ed1a88a1ed2195fd31927e705cad62dedd86b46972752a80f576", size = 1911224 }, ] +[[package]] +name = "sqlite-vec" +version = "0.1.6" +source = { registry = "https://pypi.org/simple" } +wheels = [ + { url = "https://files.pythonhosted.org/packages/88/ed/aabc328f29ee6814033d008ec43e44f2c595447d9cccd5f2aabe60df2933/sqlite_vec-0.1.6-py3-none-macosx_10_6_x86_64.whl", hash = "sha256:77491bcaa6d496f2acb5cc0d0ff0b8964434f141523c121e313f9a7d8088dee3", size = 164075 }, + { url = "https://files.pythonhosted.org/packages/a7/57/05604e509a129b22e303758bfa062c19afb020557d5e19b008c64016704e/sqlite_vec-0.1.6-py3-none-macosx_11_0_arm64.whl", hash = "sha256:fdca35f7ee3243668a055255d4dee4dea7eed5a06da8cad409f89facf4595361", size = 165242 }, + { url = "https://files.pythonhosted.org/packages/f2/48/dbb2cc4e5bad88c89c7bb296e2d0a8df58aab9edc75853728c361eefc24f/sqlite_vec-0.1.6-py3-none-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:7b0519d9cd96164cd2e08e8eed225197f9cd2f0be82cb04567692a0a4be02da3", size = 103704 }, + { url = "https://files.pythonhosted.org/packages/80/76/97f33b1a2446f6ae55e59b33869bed4eafaf59b7f4c662c8d9491b6a714a/sqlite_vec-0.1.6-py3-none-manylinux_2_17_x86_64.manylinux2014_x86_64.manylinux1_x86_64.whl", hash = "sha256:823b0493add80d7fe82ab0fe25df7c0703f4752941aee1c7b2b02cec9656cb24", size = 151556 }, + { url = "https://files.pythonhosted.org/packages/6a/98/e8bc58b178266eae2fcf4c9c7a8303a8d41164d781b32d71097924a6bebe/sqlite_vec-0.1.6-py3-none-win_amd64.whl", hash = "sha256:c65bcfd90fa2f41f9000052bcb8bb75d38240b2dae49225389eca6c3136d3f0c", size = 281540 }, +] + [[package]] name = "sse-starlette" version = "2.3.5"