Compare commits

..
Author SHA1 Message Date
William Fu-Hinthorn 52910f3b04 InMemorySaver 2024-10-06 20:49:11 -07:00
60 changed files with 323 additions and 610 deletions
+3 -3
View File
@@ -60,7 +60,7 @@ from typing import Annotated, Literal, TypedDict
from langchain_core.messages import HumanMessage
from langchain_anthropic import ChatAnthropic
from langchain_core.tools import tool
from langgraph.checkpoint.memory import MemorySaver
from langgraph.checkpoint.memory import InMemorySaver
from langgraph.graph import END, START, StateGraph, MessagesState
from langgraph.prebuilt import ToolNode
@@ -125,7 +125,7 @@ workflow.add_conditional_edges(
workflow.add_edge("tools", 'agent')
# Initialize memory to persist state between graph runs
checkpointer = MemorySaver()
checkpointer = InMemorySaver()
# Finally, we compile it!
# This compiles it into a LangChain Runnable,
@@ -201,7 +201,7 @@ final_state["messages"][-1].content
<summary>Compile the graph.</summary>
- When we compile the graph, we turn it into a LangChain [Runnable](https://python.langchain.com/v0.2/docs/concepts/#runnable-interface), which automatically enables calling `.invoke()`, `.stream()` and `.batch()` with your inputs
- We can also optionally pass checkpointer object for persisting state between graph runs, and enabling memory, human-in-the-loop workflows, time travel and more. In our case we use `MemorySaver` - a simple in-memory checkpointer
- We can also optionally pass checkpointer object for persisting state between graph runs, and enabling memory, human-in-the-loop workflows, time travel and more. In our case we use `InMemorySaver` - a simple in-memory checkpointer
</details>
6. <details>
@@ -392,7 +392,7 @@
"metadata": {},
"outputs": [],
"source": [
"from langgraph.checkpoint.memory import MemorySaver"
"from langgraph.checkpoint.memory import InMemorySaver"
]
},
{
@@ -402,7 +402,7 @@
"metadata": {},
"outputs": [],
"source": [
"checkpointer = MemorySaver()\n",
"checkpointer = InMemorySaver()\n",
"graph_with_memory = create_react_agent(model, tools, checkpointer=checkpointer)"
]
},
Binary file not shown.

Before

Width:  |  Height:  |  Size: 20 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 59 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 40 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 28 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 44 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 43 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 46 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 177 KiB

+1 -10
View File
@@ -323,16 +323,7 @@ graph.add_conditional_edges("node_a", continue_to_jokes)
## Persistence
LangGraph provides built-in persistence for your agent's state using [checkpointers][langgraph.checkpoint.base.BaseCheckpointSaver]. Checkpointers save snapshots of the graph state at every superstep, allowing resumption at any time. This enables features like human-in-the-loop interactions, memory management, and fault-tolerance. You can even directly manipulate a graph's state after its execution using the
appropriate `get` and `update` methods. For more details, see the [persistence conceptual guide](./persistence.md).
## Threads
Threads in LangGraph represent individual sessions or conversations between your graph and a user. When using checkpointing, turns in a single conversation (and even steps within a single graph execution) are organized by a unique thread ID.
## Storage
LangGraph provides built-in document storage through the [BaseStore][langgraph.store.base.BaseStore] interface. Unlike checkpointers, which save state by thread ID, stores use custom namespaces for organizing data. This enables cross-thread persistence, allowing agents to maintain long-term memories, learn from past interactions, and accumulate knowledge over time. Common use cases include storing user profiles, building knowledge bases, and managing global preferences across all threads.
LangGraph has a built-in persistence layer, implemented through [checkpointers][langgraph.checkpoint.base.BaseCheckpointSaver]. When you use a checkpointer with a graph, you can interact with and manage the graph's state after the execution. The checkpointer saves a _checkpoint_ (a snapshot) of the graph state at every superstep, enabling several powerful capabilities, including human-in-the-loop, memory and fault-tolerance. See this [conceptual guide](./persistence.md) for more information.
## Graph Migrations
+120 -255
View File
@@ -2,145 +2,41 @@
## What is Memory?
Memory in AI applications refers to the ability to process, store, and effectively recall information from past interactions. With memory, your agents can learn from feedback and adapt to users' preferences. This guide is divided into two sections based on the scope of memory recall: short-term memory and long-term memory.
Memory in the context of LLMs and AI applications refers to the ability to process, retain, and utilize information from past interactions or data sources. Examples include:
**Short-term memory**, or [thread](persistence.md#threads)-scoped memory, can be recalled at any time **from within** a single conversational thread with a user. LangGraph manages short-term memory as a part of your agent's [state](low_level.md#state). State is persisted to a database using a [checkpointer](persistence.md#checkpoints) so the thread can be resumed at any time. Short-term memory updates when the graph is invoked or a step is completed, and the State is read at the start of each step.
- Managing what messages (e.g., from a long message history) are sent to a chat model to limit token usage
- Summarizing past conversations to give a chat model context from prior interactions
- Selecting few shot examples (e.g., from a dataset) to guide model responses
- Maintaining persistent data (e.g., user preferences) across multiple chat sessions
- Allowing an LLM to update its own prompt using past information (e.g., meta-prompting)
- Retrieving information relevant to a conversation or question from a long-term storage system
**Long-term memory** is shared **across** conversational threads. It can be recalled _at any time_ and **in any thread**. Memories are scoped to any custom namespace, not just within a single thread ID. LangGraph provides [stores](persistence.md#memory-store) ([reference doc](https://langchain-ai.github.io/langgraph/reference/store/#langgraph.store.base.BaseStore)) to let you save and recall long-term memories.
Below, we'll discuss each of these examples in some detail.
Both are important to understand and implement for your application.
![](img/memory/short-vs-long.png)
## Short-term memory
Short-term memory lets your application remember previous interactions within a single [thread](persistence.md#threads) or conversation. A [thread](persistence.md#threads) organizes multiple interactions in a session, similar to the way email groups messages in a single conversation.
LangGraph manages short-term memory as part of the agent's state, persisted via thread-scoped checkpoints. This state can normally include the conversation history along with other stateful data, such as uploaded files, retrieved documents, or generated artifacts. By storing these in the graph's state, the bot can access the full context for a given conversation while maintaining separation between different threads.
Since conversation history is the most common form of representing short-term memory, in the next section, we will cover techniques for managing conversation history when the list of messages becomes **long**. If you want to stick to the high-level concepts, continue on to the [long-term memory](#long-term-memory) section.
### Managing long conversation history
Long conversations pose a challenge to today's LLMs. The full history may not even fit inside an LLM's context window, resulting in an irrecoverable error. Even _if_ your LLM technically supports the full context length, most LLMs still perform poorly over long contexts. They get "distracted" by stale or off-topic content, all while suffering from slower response times and higher costs.
Managing short-term memory is an exercise of balancing [precision & recall](https://en.wikipedia.org/wiki/Precision_and_recall#:~:text=Precision%20can%20be%20seen%20as,irrelevant%20ones%20are%20also%20returned) with your application's other performance requirements (latency & cost). As always, it's important to think critically about how you represent information for your LLM and to look at your data. We cover a few common techniques for managing message lists below and hope to provide sufficient context for you to pick the best tradeoffs for your application:
- [Editing message lists](#editing-message-lists): How to think about trimming and filtering a list of messages before passing to language model.
- [Summarizing past conversations](#summarizing-past-conversations): A common technique to use when you don't just want to filter the list of messages.
## Managing Messages
### Editing message lists
Chat models accept context using [messages](https://python.langchain.com/docs/concepts/#messages), which include developer provided instructions (a system message) and user inputs (human messages). In chat applications, messages alternate between human inputs and model responses, resulting in a list of messages that grows longer over time. Because context windows are limited and token-rich message lists can be costly, many applications can benefit from using techniques to manually remove or forget stale information.
Chat models accept instructions through [messages](https://python.langchain.com/docs/concepts/#messages), which can serve as general instructions (e.g., a system message) or user-provided instructions (e.g., human messages). In chat applications, messages often alternate between human inputs and model responses, accumulating in a list over time. Because context windows are limited and token-rich message lists can be costly, many applications can benefit from approaches to actively manage messages.
![](img/memory/filter.png)
The most direct approach is to remove old messages from a list (similar to a [least-recently used cache](https://en.wikipedia.org/wiki/Page_replacement_algorithm#Least_recently_used)).
The typical technique for deleting content from a list in LangGraph is to return an update from a node telling the system to delete some portion of the list. You get to define what this update looks like, but a common approach would be to let you return an object or dictionary specifying which values to retain.
The most directed approach is to remove specific messages from a list. This can be done using [RemoveMessage](https://langchain-ai.github.io/langgraph/how-tos/memory/delete-messages/#manually-deleting-messages) based upon the message `id`, a unique identifier for each message. In the below example, we keep only the last two messages in the list using `RemoveMessage` to remove older messages based upon their `id`.
```python
def manage_list(existing: list, updates: Union[list, dict]):
if isinstance(updates, list):
# Normal case, add to the history
return existing + updates
elif isinstance(updates, dict) and updates["type"] == "keep":
# You get to decide what this looks like.
# For example, you could simplify and just accept a string "DELETE"
# and clear the entire list.
return existing[updates["from"]:updates["to"]]
# etc. We define how to interpret updates
from langchain_core.messages import RemoveMessage
class State(TypedDict):
my_list: Annotated[list, manage_list]
# Message list
messages = [AIMessage("Hi.", name="Bot", id="1")]
messages.append(HumanMessage("Hi.", name="Lance", id="2"))
messages.append(AIMessage("So you said you were researching ocean mammals?", name="Bot", id="3"))
messages.append(HumanMessage("Yes, I know about whales. But what others should I learn about?", name="Lance", id="4"))
def my_node(state: State):
return {
# We return an update for the field "my_list" saying to
# keep only values from index -5 to the end (deleting the rest)
"my_list": {"type": "keep", "from": -5, "to": None}
}
# Isolate messages to delete
delete_messages = [RemoveMessage(id=m.id) for m in messages[:-2]]
print(delete_messages)
[RemoveMessage(content='', id='1'), RemoveMessage(content='', id='2')]
```
LangGraph will call the `manage_list` "[reducer](low_level.md#reducers)" function any time an update is returned under the key "my_list". Within that function, we define what types of updates to accept. Typically, messages will be added to the existing list (the conversation will grow); however, we've also added support to accept a dictionary that lets you "keep" certain parts of the state. This lets you programmatically drop old message context.
Another common approach is to let you return a list of "remove" objects that specify the IDs of all messages to delete. If you're using the LangChain messages and the [`add_messages`](https://langchain-ai.github.io/langgraph/reference/graphs/#langgraph.graph.message.add_messages) reducer (or `MessagesState`, which uses the same underlying functionality) in LangGraph, you can do this using a `RemoveMessage`.
```python
from langchain_core.messages import RemoveMessage, AIMessage
from langgraph.graph import add_messages
# ... other imports
class State(TypedDict):
# add_messages will default to upserting messages by ID to the existing list
# if a RemoveMessage is returned, it will delete the message in the list by ID
messages: Annotated[list, add_messages]
def my_node_1(state: State):
# Add an AI message to the `messages` list in the state
return {"messages": [AIMessage(content="Hi")]}
def my_node_2(state: State):
# Delete all but the last 2 messages from the `messages` list in the state
delete_messages = [RemoveMessage(id=m.id) for m in state['messages'][:-2]]
return {"messages": delete_messages}
```
In the example above, the `add_messages` reducer allows us to [append](https://langchain-ai.github.io/langgraph/concepts/low_level/#serialization) new messages to the `messages` state key as shown in `my_node_1`. When it sees a `RemoveMessage`, it will delete the message with that ID from the list (and the RemoveMessage will then be discarded). For more information on LangChain-specific message handling, check out [this how-to on using `RemoveMessage` ](https://langchain-ai.github.io/langgraph/how-tos/memory/delete-messages/).
See this how-to [guide](https://langchain-ai.github.io/langgraph/how-tos/memory/manage-conversation-history/) and module 2 from our [LangChain Academy](https://github.com/langchain-ai/langchain-academy/tree/main/module-2) course for example usage.
### Summarizing past conversations
The problem with trimming or removing messages, as shown above, is that we may lose information from culling of the message queue. Because of this, some applications benefit from a more sophisticated approach of summarizing the message history using a chat model.
![](img/memory/summary.png)
Simple prompting and orchestration logic can be used to achieve this. As an example, in LangGraph we can extend the [MessagesState](https://langchain-ai.github.io/langgraph/concepts/low_level/#working-with-messages-in-graph-state) to include a `summary` key.
```python
from langgraph.graph import MessagesState
class State(MessagesState):
summary: str
```
Then, we can generate a summary of the chat history, using any existing summary as context for the next summary. This `summarize_conversation` node can be called after some number of messages have accumulated in the `messages` state key.
```python
def summarize_conversation(state: State):
# First, we get any existing summary
summary = state.get("summary", "")
# Create our summarization prompt
if summary:
# A summary already exists
summary_message = (
f"This is a summary of the conversation to date: {summary}\n\n"
"Extend the summary by taking into account the new messages above:"
)
else:
summary_message = "Create a summary of the conversation above:"
# Add prompt to our history
messages = state["messages"] + [HumanMessage(content=summary_message)]
response = model.invoke(messages)
# Delete all but the 2 most recent messages
delete_messages = [RemoveMessage(id=m.id) for m in state["messages"][:-2]]
return {"summary": response.content, "messages": delete_messages}
```
See this how-to [here](https://langchain-ai.github.io/langgraph/how-tos/memory/add-summary-conversation-history/) and module 2 from our [LangChain Academy](https://github.com/langchain-ai/langchain-academy/tree/main/module-2) course for example usage.
### Knowing **when** to remove messages
Most LLMs have a maximum supported context window (denominated in tokens). A simple way to decide when to truncate messages is to count the tokens in the message history and truncate whenever it approaches that limit. Naive truncation is straightforward to implement on your own, though there are a few "gotchas". Some model APIs further restrict the sequence of message types (must start with human message, cannot have consecutive messages of the same type, etc.). If you're using LangChain, you can use the [`trim_messages`](https://python.langchain.com/docs/how_to/trim_messages/#trimming-based-on-token-count) utility and specify the number of tokens to keep from the list, as well as the `strategy` (e.g., keep the last `max_tokens`) to use for handling the boundary.
Below is an example.
Because the context window for chat model is denominated in tokens, it can be useful to trim message lists based upon some number of tokens that we want to retain. To do this, we can use [`trim_messages`](https://python.langchain.com/docs/how_to/trim_messages/#trimming-based-on-token-count) and specify number of token to keep from the list, as well as the `strategy` (e.g., keep the last `max_tokens`).
```python
from langchain_core.messages import trim_messages
@@ -150,7 +46,10 @@ trim_messages(
strategy="last",
# Remember to adjust based on your model
# or else pass a custom token_encoder
token_counter=ChatOpenAI(model="gpt-4"),
token_counter=ChatOpenAI(model="gpt-4o"),
# Most chat models expect that chat history starts with either:
# (1) a HumanMessage or
# (2) a SystemMessage followed by a HumanMessage
# Remember to adjust based on the desired conversation
# length
max_tokens=45,
@@ -168,143 +67,109 @@ trim_messages(
include_system=True,
)
```
### Usage with LangGraph
## Long-term memory
Long-term memory in LangGraph allows systems to retain information across different conversations or sessions. Unlike short-term memory, which is thread-scoped, long-term memory is saved within custom "namespaces."
LangGraph stores long-term memories as JSON documents in a [store](persistence.md#memory-store) ([reference doc](https://langchain-ai.github.io/langgraph/reference/store/#langgraph.store.base.BaseStore)). Each memory is organized under a custom `namespace` (similar to a folder) and a distinct `key` (like a filename). Namespaces often include user or org IDs or other labels that makes it easier to organize information. This structure enables hierarchical organization of memories. Cross-namespace searching is then supported through content filters. See the example below for an example.
When building agents in LangGraph, we commonly want to manage a list of messages in the graph state. Because this is such a common use case, [MessagesState](https://langchain-ai.github.io/langgraph/concepts/low_level/#working-with-messages-in-graph-state) is a built-in LangGraph state schema that includes a `messages` key, which is a list of messages. `MessagesState` also includes an `add_messages` reducer for updating the messages list with new messages as the application runs. The `add_messages` reducer allows us to [append](https://langchain-ai.github.io/langgraph/concepts/low_level/#serialization) new messages to the `messages` state key as shown below. When we perform a state update with `{"messages": new_message}` returned from `my_node`, the `add_messages` reducer appends `new_message` to the existing list of messages.
```python
from langgraph.store.memory import InMemoryStore
# InMemoryStore saves data to an in-memory dictionary. Use a DB-backed store in production use.
store = InMemoryStore()
user_id = "my-user"
application_context = "chitchat"
namespace = (user_id, application_context)
store.put(namespace, key="a-memory", {"rules": ["User likes short, direct language", "User only speaks English & python"], "my-key": "my-value"})
# get the "memory" by ID
item = store.get(namespace)
# list "memories" within this namespace, filtering on content equivalence
items = store.search(namespace, filter={"my-key": "my-value"})
def my_node(state: State):
# Add a new message to the state
new_message = HumanMessage(content="message")
return {"messages": new_message}
```
When adding long-term memory to your agent, it's important to think about how to **write memories**, how to **store and manage memory updates**, and how to **recall & represent memories** for the LLM in your application. These questions are all interdependent: how you want to recall & format memories for the LLM dictates what you should store and how to manage it. Furthermore, each technique has tradeoffs. The right approach for you largely depends on your application's needs.
LangGraph aims to give you the low-level primitives to directly control the long-term memory of your application, based on memory [Store](persistence.md#memory-store)'s.
Long-term memory is far from a solved problem. While it is hard to provide generic advice, we have provided a few reliable patterns below for your consideration as you implement long-term memory.
**Do you want to write memories "on the hot path" or "in the background"**
Memory can be updated either as part of your primary application logic (e.g. "on the hot path" of the application) or as a background task (as a separate function that generates memories based on the primary application's state). We document some tradeoffs for each approach in [the writing memories section below](#writing-memories).
**Do you want to manage memories as a single profile or as a collection of documents?**
We provide two main approaches to managing long-term memory: a single, continuously updated document (referred to as a "profile" or "schema") or a collection of documents. Each method offers its own benefits, depending on the type of information you need to store and how you intend to access it.
Managing memories as a single, continuously updated "profile" or "schema" is useful when there is well-scoped, specific information you want to remember about a user, organization, or other entity (including the agent itself). You can define the schema of the profile ahead of time, and then use an LLM to update this based on interactions. Querying the "memory" is easy since it's a simple GET operation on a JSON document. We explain this in more detail in [remember a profile](#manage-individual-profiles). This technique can provide higher precision (on known information use cases) at the expense of lower recall (since you have to anticipate and model your domain, and updates to the doc tend to delete or rewrite away old information at a greater frequency).
Managing long-term memory as a collection of documents, on the other hand, lets you store an unbounded amount of information. This technique is useful when you want to repeatedly extract & remember items over a long time horizon but can be more complicated to query and manage over time.
Similar to the "profile" memory, you still define schema(s) for each memory. Rather than overwriting a single document, you instead will insert new ones (and potentially update or re-contextualize existing ones in the process). We explain this approach in more detail in ["managing a collection of memories"](#manage-a-collection-of-memories).
**Do you want to present memories to your agent as updated instructions or as few-shot examples?**
Memories are typically provided to the LLM as a part of the system prompt. Some common ways to "frame" memories for the LLM include providing raw information as "memories from previous interactions with user A", as system instructions or rules, or as few-shot examples.
Framing memories as "learning rules or instructions" typically means dedicating a portion of the system prompt to instructions the LLM can manage itself. After each conversation, you can prompt the LLM to evaluate its performance and update the instructions to better handle this type of task in the future. We explain this approach in more detail in [this section](#update-own-instructions).
Storing memories as few-shot examples lets you store and manage instructions as cause and effect. Each memory stores an input or context and expected response. Including a reasoning trajectory (a chain-of-thought) can also help provide sufficient context so that the memory is less likely to be mis-used in the future. We elaborate on this concept more in [this section](#few-shot-examples).
We will expand on techniques for writing, managing, and recalling & formatting memories in the following section.
### Writing memories
Humans form long-term memories when we sleep, but when and how should our agents create new memories? The two most common ways we see agents write memories are "on the hot path" and "in the background".
![](img/memory/hot_path_vs_background.png)
#### Writing memories in the hot path
This involves creating memories while the application is running. To provide a popular production example, ChatGPT manages memories using a "save_memories" tool to upsert memories as content strings. It decides whether (and how) to use this tool every time it receives a user message and multi-tasks memory management with the rest of the user instructions.
This has a few benefits. First of all, it happens "in real time". If the user starts a new thread right away that memory will be present. The user also transparently sees when memories are stored, since the bot has to explicitly decide to store information and can relate that to the user.
This also has several downsides. It complicates the decisions the agent must make (what to commit to memory). This complication can degrade its tool-calling performance and reduce task completion rates. It will slow down the final response since it needs to decide what to commit to memory. It also typically leads to fewer things being saved to memory (since the assistant is multi-tasking), which will cause **lower recall** in later conversations.
#### Writing memories in the background
This involves updating memory as a conceptually separate task, typically as a completely separate graph or function. Since it happens in the background, it incurs no latency. It also splits up the application logic from the memory logic, making it more modular and easy to manage. It also lets you separate the timing of memory creation, letting you avoid redundant work. Your agent can focus on accomplishing its immediate task without having to consciously think about what it needs to remember.
This approach is not without its downsides, however. You have to think about how often to write memories. If it doesn't run in realtime, the user's interactions on other threads won't benefit from the new context. You also have to think about when to trigger this job. We typically recommend scheduling memories after some point of time, cancelling and re-scheduling for the future if new events occur on a given thread. Other popular choices are to form memories on some cron schedule or to let the user or application logic manually trigger memory formation.
### Managing memories
Once you've sorted out memory scheduling, it's important to think about **how to update memory with new information**.
There are two main approaches: you can either continuously update a single document (memory profile) or insert new documents each time you receive new information.
We will outline some tradeoffs between these two approaches below, understanding that most people will find it most appropriate to combine approaches and to settle somewhere in the middle.
#### Manage individual profiles
A profile is generally just a JSON document with various key-value pairs you've selected to represent your domain. When remembering a profile, you will want to make sure that you are **updating** the profile each time. As a result, you will want to pass in the previous profile and ask the LLM to generate a new profile (or some JSON patch to apply to the old profile).
The larger the document, the more error-prone this can become. If your document becomes **too** large, you may want to consider splitting up the profiles into separate sections. You will likely need to use generation with retries and/or **strict** decoding when generating documents to ensure the memory schemas remains valid.
![](img/memory/update-profile.png)
#### Manage a collection of memories
Saving memories as a collection of documents simplifies some things. Each individual memory can be more narrowly scoped and easier to generate. It also means you're less likely to **lose** information over time, since it's easier for an LLM to generate _new_ objects for new information than it is for it to reconcile that new information with information in a dense profile. This tends to lead to higher recall downstream.
This approach shifts some complexity to how you prompt the LLM to apply memory updates. You now have to enable the LLM to _delete_ or _update_ existing items in the list. This can be tricky to prompt the LLM to do. Some LLMs may default to over-inserting; others may default to over-updating. Tuning the behavior here is best done through evals, something you can do with a tool like [LangSmith](https://docs.smith.langchain.com/tutorials/Developers/evaluation).
This also shifts complexity to memory **search** (recall). You have to think about what relevant items to use. Right now we support filtering by metadata. We will be adding semantic search shortly.
Finally, this shifts some complexity to how you represent the memories for the LLM (and by extension, the schemas you use to save each memories). It's very easy to write memories that can easily be mistaken out-of-context. It's important to prompt the LLM to include all necessary contextual information in the given memory so that when you use it in later conversations it doesn't mistakenly mis-apply that information.
![](img/memory/update-list.png)
### Representing memories
Once you have saved memories, the way you then retrieve and present the memory content for the LLM can play a large role in how well your LLM incorporates that information in its responses.
The following sections present a couple of common approaches. Note that these sections also will largely inform how you write and manage memories. Everything in memory is connected!
#### Update own instructions
While instructions are often static text written by the developer, many AI applications benefit from letting the users personalize the rules and instructions the agent should follow whenever it interacts with that user. This ideally can be inferred by its interactions with the user (so the user doesn't have to explicitly change settings in yoru app). In this sense, instructions are a form of long-form memory!
One way to apply this is using "reflection" or "Meta-prompting" steps. Prompt the LLM with the current instruction set (from the system prompt) and a conversation with the user, and instruct the LLM to refine its instructions. This approach allows the system to dynamically update and improve its own behavior, potentially leading to better performance on various tasks. This is particularly useful for tasks where the instructions are challenging to specify a priori.
Meta-prompting uses past information to refine prompts. For instance, a [Tweet generator](https://www.youtube.com/watch?v=Vn8A3BxfplE) employs meta-prompting to enhance its paper summarization prompt for Twitter. You could implement this using LangGraph's memory store to save updated instructions in a shared namespace. In this case, we will namespace the memories as "agent_instructions" and key the memory based on the agent.
The `add_messages` reducer built into `MessagesState` [also works with the `RemoveMessage` utility that we discussed above](https://langchain-ai.github.io/langgraph/how-tos/memory/delete-messages/). In this case, we can perform a state update with a list of `delete_messages` to remove specific messages from the `messages` list.
```python
# Node that *uses* the instructions
def call_model(state: State, store: BaseStore):
namespace = ("agent_instructions", )
instructions = store.get(namespace, key="agent_a")[0]
# Application logic
prompt = prompt_template.format(instructions=instructions.value["instructions"])
...
# Node that updates instructions
def update_instructions(state: State, store: BaseStore):
namespace = ("instructions",)
current_instructions = store.search(namespace)[0]
# Memory logic
prompt = prompt_template.format(instructions=instructions.value["instructions"], conversation=state["messages"])
output = llm.invoke(prompt)
new_instructions = output['new_instructions']
store.put(("agent_instructions",), "agent_a", {"instructions": new_instructions})
...
def my_node(state: State):
# Delete messages from state
delete_messages = [RemoveMessage(id=m.id) for m in state['messages'][:-2]]
return {"messages": delete_messages}
```
![](img/memory/update-instructions.png)
See this how-to [guide](https://langchain-ai.github.io/langgraph/how-tos/memory/manage-conversation-history/) and module 2 from our [LangChain Academy](https://github.com/langchain-ai/langchain-academy/tree/main/module-2) course for example usage.
#### Few-shot examples
## Summarizing Past Conversations
Sometimes it's easier to "show" than "tell." LLMs learn well from examples. Few-shot learning lets you ["program"](https://x.com/karpathy/status/1627366413840322562) your LLM by updating the prompt with input-output examples to illustrate the intended behavior. While various [best-practices](https://python.langchain.com/docs/concepts/#1-generating-examples) can be used to generate few-shot examples, often the challenge lies in selecting the most relevant examples based on user input.
The problem with trimming or removing messages, as shown above, is that we may loose information from culling of the message queue. Because of this, some applications benefit from a more sophisticated approach of summarizing the message history using a chat model.
Note that the memory store is just one way to store data as few-shot examples. If you want to have more developer involvement, or tie few-shots more closely to your evaluation harness, you can also use a [LangSmith Dataset](https://docs.smith.langchain.com/how_to_guides/datasets) to store your data. Then dynamic few-shot example selectors can be used out-of-the box to achieve this same goal. LangSmith will index the dataset for you and enable retrieval of few shot examples that are most relevant to the user input based upon keyword similarity ([using a BM25-like algorithm](https://docs.smith.langchain.com/how_to_guides/datasets/index_datasets_for_dynamic_few_shot_example_selection) for keyword based similarity). See this how-to [video](https://www.youtube.com/watch?v=37VaU7e7t5o) for example usage of dynamic few-shot example selection in LangSmith. Also, see this [blog post](https://blog.langchain.dev/few-shot-prompting-to-improve-tool-calling-performance/) showcasing few-shot prompting to improve tool calling performance and this [blog post](https://blog.langchain.dev/aligning-llm-as-a-judge-with-human-preferences/) using few-shot example to align an LLMs to human preferences.
Simple prompting and orchestration logic can be used to achieve this. As an example, in LangGraph we can extend the [MessagesState](https://langchain-ai.github.io/langgraph/concepts/low_level/#working-with-messages-in-graph-state) to include a `summary` key.
```python
from langgraph.graph import MessagesState
class State(MessagesState):
summary: str
```
Then, we can generate a summary of the chat history, using any existing summary as context for the next summary. This `summarize_conversation` node can be called after some number of messages have accumulated in the `messages` state key.
```python
def summarize_conversation(state: State):
# First, we get any existing summary
summary = state.get("summary", "")
# Create our summarization prompt
if summary:
# A summary already exists
summary_message = (
f"This is summary of the conversation to date: {summary}\n\n"
"Extend the summary by taking into account the new messages above:"
)
else:
summary_message = "Create a summary of the conversation above:"
# Add prompt to our history
messages = state["messages"] + [HumanMessage(content=summary_message)]
response = model.invoke(messages)
# Delete all but the 2 most recent messages
delete_messages = [RemoveMessage(id=m.id) for m in state["messages"][:-2]]
return {"summary": response.content, "messages": delete_messages}
```
See this how-to [here](https://langchain-ai.github.io/langgraph/how-tos/memory/add-summary-conversation-history/) and module 2 from our [LangChain Academy](https://github.com/langchain-ai/langchain-academy/tree/main/module-2) course for example usage.
## Few Shot Examples
Few-shot learning is a powerful technique where LLMs can be ["programmed"](https://x.com/karpathy/status/1627366413840322562) inside the prompt with input-output examples to perform diverse tasks. While various [best-practices](https://python.langchain.com/docs/concepts/#1-generating-examples) can be used to generate few-shot examples, often the challenge lies in selecting the most relevant examples based on user input.
LangChain [`ExampleSelectors`](https://python.langchain.com/docs/how_to/#example-selectors) can be used to customize few-shot example selection from a collection of examples using criteria such as length, semantic similarity, semantic ngram overlap, or maximal marginal relevance.
If few-shot examples are stored in a [LangSmith Dataset](https://docs.smith.langchain.com/how_to_guides/datasets), then dynamic few-shot example selectors can be used out-of-the box to achieve this same goal. LangSmith will index the dataset for you and enable retrieval of few shot examples that are most relevant to the user input based upon keyword similarity ([using a BM25-like algorithm](https://docs.smith.langchain.com/how_to_guides/datasets/index_datasets_for_dynamic_few_shot_example_selection) for keyword based similarity).
See this how-to [video](https://www.youtube.com/watch?v=37VaU7e7t5o) for example usage of dynamic few-shot example selection in LangSmith. Also, see this [blog post](https://blog.langchain.dev/few-shot-prompting-to-improve-tool-calling-performance/) showcasing few-shot prompting to improve tool calling performance and this [blog post](https://blog.langchain.dev/aligning-llm-as-a-judge-with-human-preferences/) using few-shot example to align an LLMs to human preferences.
## Maintaining Data Across Chat Sessions
LangGraph's [persistence layer](https://langchain-ai.github.io/langgraph/concepts/persistence/#persistence) has checkpointers that utilize various storage systems, including an in-memory key-value store or different databases. These checkpoints capture the graph state at each execution step and accumulate in a thread, which can be accessed at a later time using a thread ID to resume a previous graph execution. We add persistence to our graph by passing a checkpointer to the `compile` method, as shown here.
```python
# Compile the graph with a checkpointer
checkpointer = InMemorySaver()
graph = workflow.compile(checkpointer=checkpointer)
# Invoke the graph with a thread ID
config = {"configurable": {"thread_id": "1"}}
graph.invoke(input_state, config)
# get the latest state snapshot at a later time
config = {"configurable": {"thread_id": "1"}}
graph.get_state(config)
```
Persistence is critical sustaining a long-running chat sessions. For example, a chat between a user and an AI assistant may have interruptions. Persistence ensures that a user can continue that particular chat session at any later point in time. However, what happens if a user initiates a new chat session with an assistant? This spawns a new thread, and the information from the previous session (thread) is not retained. This motivates the need for a memory service that can maintain data across chat sessions (threads).
## Meta-prompting
Meta-prompting uses an LLM to generate or refine its own prompts or instructions. This approach allows the system to dynamically update and improve its own behavior, potentially leading to better performance on various tasks. This is particularly useful for tasks where the instructions are challenging to specify a priori.
Meta-prompting can use past information to update the prompt. As an example, this [Tweet generator](https://www.youtube.com/watch?v=Vn8A3BxfplE) uses meta-prompting to iteratively improve the summarization prompt used to generate high quality paper summaries for Twitter. In this case, we used a LangSmith dataset to house several papers that we wanted to summarize, generated summaries using a naive summarization prompt, manually reviewed the summaries, captured feedback from human review using the LangSmith Annotation Queue, and passed this feedback to a chat model to re-generate the summarization prompt. The process was repeated in a loop until the summaries met our criteria in human review.
## Retrieving relevant information from long-term storage
A central challenge that spans many different memory use-case can be summarized simply: how can we retrieve *relevant information* from a long-term storage system and pass it to a chat model? As an example, assume we have a system that stores a large number of specific details about a user, but the user asks a specific question related to restaurant recommendations. It would be costly to trivially extract *all* personal user information and pass it to a chat model. Instead, we want to extract only the information that is most relevant to the user's current chat interaction (e,g,. food preferences, location, etc.) and pass it to the chat model.
There is a large body of work on retrieval that aims to address this challenge. See our tutorials focused on [RAG, or Retrieval Augmented Generation](https://langchain-ai.github.io/langgraph/tutorials/rag/langgraph_adaptive_rag/), our conceptual docs on [retrieval](https://python.langchain.com/docs/concepts/#retrieval), and our [open source repository](https://github.com/langchain-ai/rag-from-scratch) along with [videos](https://www.youtube.com/playlist?list=PLfaIDFEXuae2LXbO1_PKyVJiQ23ZztA0x) on this topic.
+4 -146
View File
@@ -26,7 +26,7 @@ Let's see what checkpoints are saved when a simple graph is invoked as follows:
```python
from langgraph.graph import StateGraph, START, END
from langgraph.checkpoint.memory import MemorySaver
from langgraph.checkpoint.memory import InMemorySaver
from typing import Annotated
from typing_extensions import TypedDict
from operator import add
@@ -49,7 +49,7 @@ workflow.add_edge(START, "node_a")
workflow.add_edge("node_a", "node_b")
workflow.add_edge("node_b", END)
checkpointer = MemorySaver()
checkpointer = InMemorySaver()
graph = workflow.compile(checkpointer=checkpointer)
config = {"configurable": {"thread_id": "1"}}
@@ -216,153 +216,11 @@ The final thing you can optionally specify when calling `update_state` is `as_no
![Update](img/persistence/checkpoints_full_story.jpg)
## Memory Store
![Update](img/persistence/shared_state.png)
A [state schema](low_level.md#schema) specifies a set of keys that are populated as a graph is executed. As discussed above, state can be written by a checkpointer to a thread at each graph step, enabling state persistence.
But, what if we want to retrain some information *across threads*? Consider the case of a chatbot where we want to retain specific information about the user across *all* chat conversations (e.g., threads) with that user!
With checkpointers alone, we cannot share information across threads. This motivates the need for the `Store` interface. As an illustration, we can define an `InMemoryStore` to store information about a user across threads. We simply compile our graph with a checkpointer, as before, and will our new `in_memory_store`.
First, let's showcase this in isolation without using LangGraph.
```python
from langgraph.store.memory import InMemoryStore
in_memory_store = InMemoryStore()
```
Memories are namespaced by a `tuple`, which in this specific example will be `(<user_id>, "memories")`. The namespace can be any length and represent anything, does not have be user specific.
```python
user_id = "1"
namespace_for_memory = (user_id, "memories")
```
We use the `store.put` to save memories to our namespace in the store. When we do this, we specify the namespace, as defined above, and a key-value pair for the memory: the key is simply a unique identifier for the memory (`memory_id`) and the value (a dictionary) is the memory itself.
```python
memory_id = str(uuid.uuid4())
memory = {"food_preference" : "I like pizza"}
in_memory_store.put(namespace_for_memory, memory_id, memory)
```
We can read out memories in our namespace using `store.search`, which will return all memories for a given user as a list. The most recent memory is the last in the list.
```python
memories = in_memory_store.search(namespace_for_memory)
memories[-1].dict()
{'value': {'food_preference': 'I like pizza'},
'key': '07e0caf4-1631-47b7-b15f-65515d4c1843',
'namespace': ['1', 'memories'],
'created_at': '2024-10-02T17:22:31.590602+00:00',
'updated_at': '2024-10-02T17:22:31.590605+00:00'}
```
Each memory type is a Python class with certain attributes. We can access it as a dictionary by converting via `.dict` as above.
The attributes it has are:
- `value`: The value (itself a dictionary) of this memory
- `key`: The UUID for this memory in this namespace
- `namespace`: A list of strings, the namespace of this memory type
- `created_at`: Timestamp for when this memory was created
- `updated_at`: Timestamp for when this memory was updated
With this all in place, we use the `in_memory_store` in LangGraph. The `in_memory_store` works hand-in-hand with the checkpointer: the checkpointer saves state to threads, as discussed above, and the the `in_memory_store` allows us to store arbitrary information for access *across* threads. We compile the graph with both the checkpointer and the `in_memory_store` as follows.
```python
from langgraph.checkpoint.memory import MemorySaver
# We need this because we want to enable threads (conversations)
checkpointer = MemorySaver()
# ... Define the graph ...
# Compile the graph with the checkpointer and store
graph = graph.compile(checkpointer=checkpointer, store=in_memory_store)
```
We invoke the graph with a `thread_id`, as before, and also with a `user_id`, which we'll use to namespace our memories to this particular user as we showed above.
```python
# Invoke the graph
user_id = "1"
config = {"configurable": {"thread_id": "1", "user_id": user_id}}
# First let's just say hi to the AI
for update in graph.stream(
{"messages": [{"role": "user", "content": "hi"}]}, config, stream_mode="updates"
):
print(update)
```
We can access the `in_memory_store` and the `user_id` in *any node* by passing `store: BaseStore` and `config: RunnableConfig` as node arguments. Just as we saw above, simply use the `put` method to save memories to the store.
```python
def update_memory(state: MessagesState, config: RunnableConfig, *, store: BaseStore):
# Get the user id from the config
user_id = config["configurable"]["user_id"]
# Namespace the memory
namespace = (user_id, "memories")
# ... Analyze conversation and create a new memory
# Create a new memory ID
memory_id = str(uuid.uuid4())
# We create a new memory
store.put(namespace, memory_id, {"memory": memory})
```
As we showed above, we can also access the store in any node and use `search` to get memories. Recall the the memories are returned as a list of objects that can be converted to a dictionary.
```python
memories[-1].dict()
{'value': {'food_preference': 'I like pizza'},
'key': '07e0caf4-1631-47b7-b15f-65515d4c1843',
'namespace': ['1', 'memories'],
'created_at': '2024-10-02T17:22:31.590602+00:00',
'updated_at': '2024-10-02T17:22:31.590605+00:00'}
```
We can access the memories and use them in our model call.
```python
def call_model(state: MessagesState, config: RunnableConfig, *, store: BaseStore):
# Get the user id from the config
user_id = config["configurable"]["user_id"]
# Get the memories for the user from the store
memories = store.search(("memories", user_id))
info = "\n".join([d.value["memory"] for d in memories])
# ... Use memories in the model call
```
If we create a new thread, we can still access the same memories so long as the `user_id` is the same.
```python
# Invoke the graph
config = {"configurable": {"thread_id": "2", "user_id": "1"}}
# Let's say hi again
for update in graph.stream(
{"messages": [{"role": "user", "content": "hi, tell me about my memories"}]}, config, stream_mode="updates"
):
print(update)
```
When we use the LangGraph API, either locally (e.g., in LangGraph Studio) or with LangGraph Cloud, the memory store is available to use by default and does not need to be specified during graph compilation.
## Checkpointer libraries
Under the hood, checkpointing is powered by checkpointer objects that conform to [BaseCheckpointSaver][langgraph.checkpoint.base.BaseCheckpointSaver] interface. LangGraph provides several checkpointer implementations, all implemented via standalone, installable libraries:
* `langgraph-checkpoint`: The base interface for checkpointer savers ([BaseCheckpointSaver][langgraph.checkpoint.base.BaseCheckpointSaver]) and serialization/deserialization interface ([SerializerProtocol][langgraph.checkpoint.serde.base.SerializerProtocol]). Includes in-memory checkpointer implementation ([MemorySaver][langgraph.checkpoint.memory.MemorySaver]) for experimentation. LangGraph comes with `langgraph-checkpoint` included.
* `langgraph-checkpoint`: The base interface for checkpointer savers ([BaseCheckpointSaver][langgraph.checkpoint.base.BaseCheckpointSaver]) and serialization/deserialization interface ([SerializerProtocol][langgraph.checkpoint.serde.base.SerializerProtocol]). Includes in-memory checkpointer implementation ([InMemorySaver][langgraph.checkpoint.memory.InMemorySaver]) for experimentation. LangGraph comes with `langgraph-checkpoint` included.
* `langgraph-checkpoint-sqlite`: An implementation of LangGraph checkpointer that uses SQLite database ([SqliteSaver][langgraph.checkpoint.sqlite.SqliteSaver] / [AsyncSqliteSaver][langgraph.checkpoint.sqlite.aio.AsyncSqliteSaver]). Ideal for experimentation and local workflows. Needs to be installed separately.
* `langgraph-checkpoint-postgres`: An advanced checkpointer that uses Postgres database ([PostgresSaver][langgraph.checkpoint.postgres.PostgresSaver] / [AsyncPostgresSaver][langgraph.checkpoint.postgres.aio.AsyncPostgresSaver]), used in LangGraph Cloud. Ideal for using in production. Needs to be installed separately.
@@ -378,7 +236,7 @@ Each checkpointer conforms to [BaseCheckpointSaver][langgraph.checkpoint.base.Ba
If the checkpointer is used with asynchronous graph execution (i.e. executing the graph via `.ainvoke`, `.astream`, `.abatch`), asynchronous versions of the above methods will be used (`.aput`, `.aput_writes`, `.aget_tuple`, `.alist`).
!!! note Note
For running your graph asynchronously, you can use `MemorySaver`, or async versions of Sqlite/Postgres checkpointers -- `AsyncSqliteSaver` / `AsyncPostgresSaver` checkpointers.
For running your graph asynchronously, you can use `InMemorySaver`, or async versions of Sqlite/Postgres checkpointers -- `AsyncSqliteSaver` / `AsyncPostgresSaver` checkpointers.
### Serializer
@@ -135,9 +135,9 @@
"tools = [get_weather]\n",
"\n",
"# We need a checkpointer to enable human-in-the-loop patterns\n",
"from langgraph.checkpoint.memory import MemorySaver\n",
"from langgraph.checkpoint.memory import InMemorySaver\n",
"\n",
"memory = MemorySaver()\n",
"memory = InMemorySaver()\n",
"\n",
"# Define the graph\n",
"\n",
@@ -142,9 +142,9 @@
"\n",
"# We can add \"chat memory\" to the graph with LangGraph's checkpointer\n",
"# to retain the chat context between interactions\n",
"from langgraph.checkpoint.memory import MemorySaver\n",
"from langgraph.checkpoint.memory import InMemorySaver\n",
"\n",
"memory = MemorySaver()\n",
"memory = InMemorySaver()\n",
"\n",
"# Define the graph\n",
"\n",
@@ -154,7 +154,7 @@
"from langchain_anthropic import ChatAnthropic\n",
"from langchain_core.runnables import RunnableConfig\n",
"from langgraph.graph import StateGraph, MessagesState, START\n",
"from langgraph.checkpoint.memory import MemorySaver\n",
"from langgraph.checkpoint.memory import InMemorySaver\n",
"from langgraph.store.base import BaseStore\n",
"\n",
"\n",
@@ -187,7 +187,7 @@
"builder.add_edge(START, \"call_model\")\n",
"\n",
"# NOTE: we're passing the store object here when compiling the graph\n",
"graph = builder.compile(checkpointer=MemorySaver(), store=in_memory_store)\n",
"graph = builder.compile(checkpointer=InMemorySaver(), store=in_memory_store)\n",
"# If you're using LangGraph Cloud or LangGraph Studio, you don't need to pass the store or checkpointer when compiling the graph, since it's done automatically."
]
},
@@ -124,7 +124,7 @@
"source": [
"from typing_extensions import TypedDict\n",
"from langgraph.graph import StateGraph, START, END\n",
"from langgraph.checkpoint.memory import MemorySaver\n",
"from langgraph.checkpoint.memory import InMemorySaver\n",
"from IPython.display import Image, display\n",
"\n",
"\n",
@@ -157,7 +157,7 @@
"builder.add_edge(\"step_3\", END)\n",
"\n",
"# Set up memory\n",
"memory = MemorySaver()\n",
"memory = InMemorySaver()\n",
"\n",
"# Add\n",
"graph = builder.compile(checkpointer=memory, interrupt_before=[\"step_3\"])\n",
@@ -270,7 +270,7 @@
"from langgraph.graph import MessagesState, START\n",
"from langgraph.prebuilt import ToolNode\n",
"from langgraph.graph import END, StateGraph\n",
"from langgraph.checkpoint.memory import MemorySaver\n",
"from langgraph.checkpoint.memory import InMemorySaver\n",
"\n",
"\n",
"@tool\n",
@@ -352,7 +352,7 @@
"workflow.add_edge(\"action\", \"agent\")\n",
"\n",
"# Set up memory\n",
"memory = MemorySaver()\n",
"memory = InMemorySaver()\n",
"\n",
"# Finally, we compile it!\n",
"# This compiles it into a LangChain Runnable,\n",
@@ -78,7 +78,7 @@
"from IPython.display import Image, display\n",
"\n",
"from langgraph.graph import StateGraph, START, END\n",
"from langgraph.checkpoint.memory import MemorySaver\n",
"from langgraph.checkpoint.memory import InMemorySaver\n",
"from langgraph.errors import NodeInterrupt\n",
"\n",
"\n",
@@ -118,7 +118,7 @@
"builder.add_edge(\"step_3\", END)\n",
"\n",
"# Set up memory\n",
"memory = MemorySaver()\n",
"memory = InMemorySaver()\n",
"\n",
"# Compile the graph with memory\n",
"graph = builder.compile(checkpointer=memory)\n",
@@ -126,7 +126,7 @@
"source": [
"from typing_extensions import TypedDict\n",
"from langgraph.graph import StateGraph, START, END\n",
"from langgraph.checkpoint.memory import MemorySaver\n",
"from langgraph.checkpoint.memory import InMemorySaver\n",
"from IPython.display import Image, display\n",
"\n",
"\n",
@@ -159,7 +159,7 @@
"builder.add_edge(\"step_3\", END)\n",
"\n",
"# Set up memory\n",
"memory = MemorySaver()\n",
"memory = InMemorySaver()\n",
"\n",
"# Add\n",
"graph = builder.compile(checkpointer=memory, interrupt_before=[\"step_2\"])\n",
@@ -279,7 +279,7 @@
"from langchain_core.tools import tool\n",
"from langgraph.graph import MessagesState, START, END, StateGraph\n",
"from langgraph.prebuilt import ToolNode\n",
"from langgraph.checkpoint.memory import MemorySaver\n",
"from langgraph.checkpoint.memory import InMemorySaver\n",
"\n",
"\n",
"@tool\n",
@@ -361,7 +361,7 @@
"workflow.add_edge(\"action\", \"agent\")\n",
"\n",
"# Set up memory\n",
"memory = MemorySaver()\n",
"memory = InMemorySaver()\n",
"\n",
"# Finally, we compile it!\n",
"# This compiles it into a LangChain Runnable,\n",
@@ -120,7 +120,7 @@
"source": [
"from typing_extensions import TypedDict, Literal\n",
"from langgraph.graph import StateGraph, START, END, MessagesState\n",
"from langgraph.checkpoint.memory import MemorySaver\n",
"from langgraph.checkpoint.memory import InMemorySaver\n",
"from langchain_anthropic import ChatAnthropic\n",
"from langchain_core.tools import tool\n",
"from langchain_core.messages import AIMessage\n",
@@ -195,7 +195,7 @@
"builder.add_edge(\"run_tool\", \"call_llm\")\n",
"\n",
"# Set up memory\n",
"memory = MemorySaver()\n",
"memory = InMemorySaver()\n",
"\n",
"# Add\n",
"graph = builder.compile(checkpointer=memory, interrupt_before=[\"human_review_node\"])\n",
@@ -115,7 +115,7 @@
"from langgraph.graph import MessagesState, START\n",
"from langgraph.prebuilt import ToolNode\n",
"from langgraph.graph import END, StateGraph\n",
"from langgraph.checkpoint.memory import MemorySaver\n",
"from langgraph.checkpoint.memory import InMemorySaver\n",
"\n",
"\n",
"@tool\n",
@@ -201,7 +201,7 @@
"workflow.add_edge(\"action\", \"agent\")\n",
"\n",
"# Set up memory\n",
"memory = MemorySaver()\n",
"memory = InMemorySaver()\n",
"\n",
"# Finally, we compile it!\n",
"# This compiles it into a LangChain Runnable,\n",
@@ -120,7 +120,7 @@
"source": [
"from typing_extensions import TypedDict\n",
"from langgraph.graph import StateGraph, START, END\n",
"from langgraph.checkpoint.memory import MemorySaver\n",
"from langgraph.checkpoint.memory import InMemorySaver\n",
"from IPython.display import Image, display\n",
"\n",
"\n",
@@ -154,7 +154,7 @@
"builder.add_edge(\"step_3\", END)\n",
"\n",
"# Set up memory\n",
"memory = MemorySaver()\n",
"memory = InMemorySaver()\n",
"\n",
"# Add\n",
"graph = builder.compile(checkpointer=memory, interrupt_before=[\"human_feedback\"])\n",
@@ -475,9 +475,9 @@
"workflow.add_edge(\"ask_human\", \"agent\")\n",
"\n",
"# Set up memory\n",
"from langgraph.checkpoint.memory import MemorySaver\n",
"from langgraph.checkpoint.memory import InMemorySaver\n",
"\n",
"memory = MemorySaver()\n",
"memory = InMemorySaver()\n",
"\n",
"# Finally, we compile it!\n",
"# This compiles it into a LangChain Runnable,\n",
@@ -99,10 +99,10 @@
"\n",
"from langchain_anthropic import ChatAnthropic\n",
"from langchain_core.messages import SystemMessage, RemoveMessage\n",
"from langgraph.checkpoint.memory import MemorySaver\n",
"from langgraph.checkpoint.memory import InMemorySaver\n",
"from langgraph.graph import MessagesState, StateGraph, START, END\n",
"\n",
"memory = MemorySaver()\n",
"memory = InMemorySaver()\n",
"\n",
"\n",
"# We will add a `summary` attribute (in addition to `messages` key,\n",
@@ -106,11 +106,11 @@
"from langchain_anthropic import ChatAnthropic\n",
"from langchain_core.tools import tool\n",
"\n",
"from langgraph.checkpoint.memory import MemorySaver\n",
"from langgraph.checkpoint.memory import InMemorySaver\n",
"from langgraph.graph import MessagesState, StateGraph, START, END\n",
"from langgraph.prebuilt import ToolNode\n",
"\n",
"memory = MemorySaver()\n",
"memory = InMemorySaver()\n",
"\n",
"\n",
"@tool\n",
@@ -97,11 +97,11 @@
"from langchain_anthropic import ChatAnthropic\n",
"from langchain_core.tools import tool\n",
"\n",
"from langgraph.checkpoint.memory import MemorySaver\n",
"from langgraph.checkpoint.memory import InMemorySaver\n",
"from langgraph.graph import MessagesState, StateGraph, START, END\n",
"from langgraph.prebuilt import ToolNode\n",
"\n",
"memory = MemorySaver()\n",
"memory = InMemorySaver()\n",
"\n",
"\n",
"@tool\n",
@@ -228,11 +228,11 @@
"from langchain_anthropic import ChatAnthropic\n",
"from langchain_core.tools import tool\n",
"\n",
"from langgraph.checkpoint.memory import MemorySaver\n",
"from langgraph.checkpoint.memory import InMemorySaver\n",
"from langgraph.graph import MessagesState, StateGraph, START\n",
"from langgraph.prebuilt import ToolNode\n",
"\n",
"memory = MemorySaver()\n",
"memory = InMemorySaver()\n",
"\n",
"\n",
"@tool\n",
@@ -322,7 +322,7 @@
"source": [
"from langchain_openai import ChatOpenAI\n",
"from langgraph.prebuilt import ToolNode, create_react_agent\n",
"from langgraph.checkpoint.memory import MemorySaver\n",
"from langgraph.checkpoint.memory import InMemorySaver\n",
"\n",
"model = ChatOpenAI(model=\"gpt-4o\", temperature=0)\n",
"tools = [get_context]\n",
@@ -330,7 +330,7 @@
"# ToolNode will automatically take care of injecting state into tools\n",
"tool_node = ToolNode(tools)\n",
"\n",
"checkpointer = MemorySaver()\n",
"checkpointer = InMemorySaver()\n",
"graph = create_react_agent(model, tools, state_schema=State, checkpointer=checkpointer)"
]
},
@@ -524,7 +524,7 @@
"# ToolNode will automatically take care of injecting Store into tools\n",
"tool_node = ToolNode(tools)\n",
"\n",
"checkpointer = MemorySaver()\n",
"checkpointer = InMemorySaver()\n",
"# NOTE: we need to pass our store to `create_react_agent` to make sure our graph is aware of it\n",
"graph = create_react_agent(model, tools, checkpointer=checkpointer, store=doc_store)"
]
+3 -3
View File
@@ -88,7 +88,7 @@
"metadata": {},
"outputs": [
{
"name": "stdin",
"name": "stdout",
"output_type": "stream",
"text": [
"ANTHROPIC_API_KEY: ········\n"
@@ -237,9 +237,9 @@
"metadata": {},
"outputs": [],
"source": [
"from langgraph.checkpoint.memory import MemorySaver\n",
"from langgraph.checkpoint.memory import InMemorySaver\n",
"\n",
"memory = MemorySaver()\n",
"memory = InMemorySaver()\n",
"graph = builder.compile(checkpointer=memory)\n",
"# If you're using LangGraph Cloud or LangGraph Studio, you don't need to pass the checkpointer when compiling the graph, since it's done automatically."
]
@@ -250,6 +250,7 @@
],
"source": [
"from langgraph.graph import StateGraph, END\n",
"from langgraph.constants import END\n",
"\n",
"# Define a new graph\n",
"workflow = StateGraph(AgentState)\n",
+1 -1
View File
@@ -70,7 +70,7 @@
"source": [
"from typing import Optional, Annotated\n",
"from typing_extensions import TypedDict\n",
"from langgraph.checkpoint.memory import MemorySaver\n",
"from langgraph.checkpoint.memory import InMemorySaver\n",
"from langgraph.graph import StateGraph, START, END\n",
"\n",
"\n",
+1 -1
View File
@@ -104,7 +104,7 @@
"source": [
"from typing import Optional, Annotated\n",
"from typing_extensions import TypedDict\n",
"from langgraph.checkpoint.memory import MemorySaver\n",
"from langgraph.checkpoint.memory import InMemorySaver\n",
"from langgraph.graph import StateGraph, START, END\n",
"\n",
"\n",
@@ -134,10 +134,10 @@
"source": [
"from typing import Literal\n",
"from typing_extensions import TypedDict\n",
"from langgraph.checkpoint.memory import MemorySaver\n",
"from langgraph.checkpoint.memory import InMemorySaver\n",
"\n",
"\n",
"memory = MemorySaver()\n",
"memory = InMemorySaver()\n",
"\n",
"\n",
"class RouterState(MessagesState):\n",
@@ -739,10 +739,10 @@
"source": [
"from typing import Literal\n",
"from typing_extensions import TypedDict\n",
"from langgraph.checkpoint.memory import MemorySaver\n",
"from langgraph.checkpoint.memory import InMemorySaver\n",
"\n",
"\n",
"memory = MemorySaver()\n",
"memory = InMemorySaver()\n",
"\n",
"\n",
"class RouterState(MessagesState):\n",
@@ -794,9 +794,9 @@
"metadata": {},
"outputs": [],
"source": [
"from langgraph.checkpoint.memory import MemorySaver\n",
"from langgraph.checkpoint.memory import InMemorySaver\n",
"\n",
"memory = MemorySaver()\n",
"memory = InMemorySaver()\n",
"\n",
"\n",
"class GrandfatherState(MessagesState):\n",
@@ -823,7 +823,7 @@
" \"router_node\", route_after_prediction, [\"graph\", END]\n",
")\n",
"grandparent_graph.add_edge(\"graph\", END)\n",
"grandparent_graph = grandparent_graph.compile(checkpointer=MemorySaver())"
"grandparent_graph = grandparent_graph.compile(checkpointer=InMemorySaver())"
]
},
{
@@ -256,7 +256,7 @@
"metadata": {},
"outputs": [],
"source": [
"from langgraph.checkpoint.memory import MemorySaver\n",
"from langgraph.checkpoint.memory import InMemorySaver\n",
"from langgraph.graph import StateGraph, START\n",
"from langgraph.graph.message import add_messages\n",
"from typing import Annotated\n",
@@ -267,7 +267,7 @@
" messages: Annotated[list, add_messages]\n",
"\n",
"\n",
"memory = MemorySaver()\n",
"memory = InMemorySaver()\n",
"workflow = StateGraph(State)\n",
"workflow.add_node(\"info\", info_chain)\n",
"workflow.add_node(\"prompt\", prompt_gen_chain)\n",
@@ -1119,7 +1119,7 @@
"metadata": {},
"outputs": [],
"source": [
"from langgraph.checkpoint.memory import MemorySaver\n",
"from langgraph.checkpoint.memory import InMemorySaver\n",
"from langgraph.graph import END, StateGraph, START\n",
"from langgraph.prebuilt import tools_condition\n",
"\n",
@@ -1139,7 +1139,7 @@
"\n",
"# The checkpointer lets the graph persist its state\n",
"# this is a complete memory for the entire graph.\n",
"memory = MemorySaver()\n",
"memory = InMemorySaver()\n",
"part_1_graph = builder.compile(checkpointer=memory)"
]
},
@@ -1938,7 +1938,7 @@
"metadata": {},
"outputs": [],
"source": [
"from langgraph.checkpoint.memory import MemorySaver\n",
"from langgraph.checkpoint.memory import InMemorySaver\n",
"from langgraph.graph import StateGraph\n",
"from langgraph.prebuilt import tools_condition\n",
"\n",
@@ -1962,7 +1962,7 @@
")\n",
"builder.add_edge(\"tools\", \"assistant\")\n",
"\n",
"memory = MemorySaver()\n",
"memory = InMemorySaver()\n",
"part_2_graph = builder.compile(\n",
" checkpointer=memory,\n",
" # NEW: The graph will always halt before executing the \"tools\" node.\n",
@@ -2524,7 +2524,7 @@
"source": [
"from typing import Literal\n",
"\n",
"from langgraph.checkpoint.memory import MemorySaver\n",
"from langgraph.checkpoint.memory import InMemorySaver\n",
"from langgraph.graph import StateGraph\n",
"from langgraph.prebuilt import tools_condition\n",
"\n",
@@ -2568,7 +2568,7 @@
"builder.add_edge(\"safe_tools\", \"assistant\")\n",
"builder.add_edge(\"sensitive_tools\", \"assistant\")\n",
"\n",
"memory = MemorySaver()\n",
"memory = InMemorySaver()\n",
"part_3_graph = builder.compile(\n",
" checkpointer=memory,\n",
" # NEW: The graph will always halt before executing the \"tools\" node.\n",
@@ -3466,7 +3466,7 @@
"source": [
"from typing import Literal\n",
"\n",
"from langgraph.checkpoint.memory import MemorySaver\n",
"from langgraph.checkpoint.memory import InMemorySaver\n",
"from langgraph.graph import StateGraph\n",
"from langgraph.prebuilt import tools_condition\n",
"\n",
@@ -3830,7 +3830,7 @@
"builder.add_conditional_edges(\"fetch_user_info\", route_to_workflow)\n",
"\n",
"# Compile graph\n",
"memory = MemorySaver()\n",
"memory = InMemorySaver()\n",
"part_4_graph = builder.compile(\n",
" checkpointer=memory,\n",
" # Let the user approve or deny the use of sensitive tools\n",
+16 -16
View File
@@ -793,7 +793,7 @@
"\n",
"We will see later that **checkpointing** is _much_ more powerful than simple chat memory - it lets you save and resume complex state at any time for error recovery, human-in-the-loop workflows, time travel interactions, and more. But before we get too ahead of ourselves, let's add checkpointing to enable multi-turn conversations.\n",
"\n",
"To get started, create a `MemorySaver` checkpointer."
"To get started, create a `InMemorySaver` checkpointer."
]
},
{
@@ -803,9 +803,9 @@
"metadata": {},
"outputs": [],
"source": [
"from langgraph.checkpoint.memory import MemorySaver\n",
"from langgraph.checkpoint.memory import InMemorySaver\n",
"\n",
"memory = MemorySaver()"
"memory = InMemorySaver()"
]
},
{
@@ -1135,7 +1135,7 @@
"from langchain_core.messages import BaseMessage\n",
"from typing_extensions import TypedDict\n",
"\n",
"from langgraph.checkpoint.memory import MemorySaver\n",
"from langgraph.checkpoint.memory import InMemorySaver\n",
"from langgraph.graph import StateGraph\n",
"from langgraph.graph.message import add_messages\n",
"from langgraph.prebuilt import ToolNode\n",
@@ -1203,12 +1203,12 @@
"from langchain_community.tools.tavily_search import TavilySearchResults\n",
"from typing_extensions import TypedDict\n",
"\n",
"from langgraph.checkpoint.memory import MemorySaver\n",
"from langgraph.checkpoint.memory import InMemorySaver\n",
"from langgraph.graph import StateGraph, START\n",
"from langgraph.graph.message import add_messages\n",
"from langgraph.prebuilt import ToolNode, tools_condition\n",
"\n",
"memory = MemorySaver()\n",
"memory = InMemorySaver()\n",
"\n",
"\n",
"class State(TypedDict):\n",
@@ -1456,7 +1456,7 @@
"from langchain_core.messages import BaseMessage\n",
"from typing_extensions import TypedDict\n",
"\n",
"from langgraph.checkpoint.memory import MemorySaver\n",
"from langgraph.checkpoint.memory import InMemorySaver\n",
"from langgraph.graph import StateGraph\n",
"from langgraph.graph.message import add_messages\n",
"from langgraph.prebuilt import ToolNode, tools_condition\n",
@@ -1491,7 +1491,7 @@
"graph_builder.add_edge(\"tools\", \"chatbot\")\n",
"graph_builder.set_entry_point(\"chatbot\")\n",
"\n",
"memory = MemorySaver()\n",
"memory = InMemorySaver()\n",
"graph = graph_builder.compile(\n",
" checkpointer=memory,\n",
" # This is new!\n",
@@ -1531,7 +1531,7 @@
"from langchain_community.tools.tavily_search import TavilySearchResults\n",
"from typing_extensions import TypedDict\n",
"\n",
"from langgraph.checkpoint.memory import MemorySaver\n",
"from langgraph.checkpoint.memory import InMemorySaver\n",
"from langgraph.graph import StateGraph, START\n",
"from langgraph.graph.message import add_messages\n",
"from langgraph.prebuilt import ToolNode, tools_condition\n",
@@ -1565,7 +1565,7 @@
")\n",
"graph_builder.add_edge(\"tools\", \"chatbot\")\n",
"graph_builder.add_edge(START, \"chatbot\")\n",
"memory = MemorySaver()\n",
"memory = InMemorySaver()\n",
"graph = graph_builder.compile(\n",
" checkpointer=memory,\n",
" # This is new!\n",
@@ -2066,7 +2066,7 @@
"from langchain_community.tools.tavily_search import TavilySearchResults\n",
"from typing_extensions import TypedDict\n",
"\n",
"from langgraph.checkpoint.memory import MemorySaver\n",
"from langgraph.checkpoint.memory import InMemorySaver\n",
"from langgraph.graph import StateGraph, START\n",
"from langgraph.graph.message import add_messages\n",
"from langgraph.prebuilt import ToolNode, tools_condition\n",
@@ -2267,7 +2267,7 @@
"graph_builder.add_edge(\"tools\", \"chatbot\")\n",
"graph_builder.add_edge(\"human\", \"chatbot\")\n",
"graph_builder.add_edge(START, \"chatbot\")\n",
"memory = MemorySaver()\n",
"memory = InMemorySaver()\n",
"graph = graph_builder.compile(\n",
" checkpointer=memory,\n",
" # We interrupt before 'human' here instead.\n",
@@ -2542,7 +2542,7 @@
"from pydantic import BaseModel\n",
"from typing_extensions import TypedDict\n",
"\n",
"from langgraph.checkpoint.memory import MemorySaver\n",
"from langgraph.checkpoint.memory import InMemorySaver\n",
"from langgraph.graph import StateGraph\n",
"from langgraph.graph.message import add_messages\n",
"from langgraph.prebuilt import ToolNode, tools_condition\n",
@@ -2629,7 +2629,7 @@
"graph_builder.add_edge(\"tools\", \"chatbot\")\n",
"graph_builder.add_edge(\"human\", \"chatbot\")\n",
"graph_builder.set_entry_point(\"chatbot\")\n",
"memory = MemorySaver()\n",
"memory = InMemorySaver()\n",
"graph = graph_builder.compile(\n",
" checkpointer=memory,\n",
" interrupt_before=[\"human\"],\n",
@@ -2674,7 +2674,7 @@
"from pydantic import BaseModel\n",
"from typing_extensions import TypedDict\n",
"\n",
"from langgraph.checkpoint.memory import MemorySaver\n",
"from langgraph.checkpoint.memory import InMemorySaver\n",
"from langgraph.graph import StateGraph, START\n",
"from langgraph.graph.message import add_messages\n",
"from langgraph.prebuilt import ToolNode, tools_condition\n",
@@ -2761,7 +2761,7 @@
"graph_builder.add_edge(\"tools\", \"chatbot\")\n",
"graph_builder.add_edge(\"human\", \"chatbot\")\n",
"graph_builder.add_edge(START, \"chatbot\")\n",
"memory = MemorySaver()\n",
"memory = InMemorySaver()\n",
"graph = graph_builder.compile(\n",
" checkpointer=memory,\n",
" interrupt_before=[\"human\"],\n",
@@ -322,7 +322,7 @@
"from typing import Annotated, List, Sequence\n",
"from langgraph.graph import END, StateGraph, START\n",
"from langgraph.graph.message import add_messages\n",
"from langgraph.checkpoint.memory import MemorySaver\n",
"from langgraph.checkpoint.memory import InMemorySaver\n",
"from typing_extensions import TypedDict\n",
"\n",
"\n",
@@ -361,7 +361,7 @@
"\n",
"builder.add_conditional_edges(\"generate\", should_continue)\n",
"builder.add_edge(\"reflect\", \"generate\")\n",
"memory = MemorySaver()\n",
"memory = InMemorySaver()\n",
"graph = builder.compile(checkpointer=memory)"
]
},
+2 -2
View File
@@ -1514,7 +1514,7 @@
"metadata": {},
"outputs": [],
"source": [
"from langgraph.checkpoint.memory import MemorySaver\n",
"from langgraph.checkpoint.memory import InMemorySaver\n",
"\n",
"builder_of_storm = StateGraph(ResearchState)\n",
"\n",
@@ -1534,7 +1534,7 @@
"\n",
"builder_of_storm.add_edge(START, nodes[0][0])\n",
"builder_of_storm.add_edge(nodes[-1][0], END)\n",
"storm = builder_of_storm.compile(checkpointer=MemorySaver())"
"storm = builder_of_storm.compile(checkpointer=InMemorySaver())"
]
},
{
+4 -4
View File
@@ -1029,7 +1029,7 @@
"metadata": {},
"outputs": [],
"source": [
"from langgraph.checkpoint.memory import MemorySaver\n",
"from langgraph.checkpoint.memory import InMemorySaver\n",
"from langgraph.graph import END, StateGraph, START\n",
"\n",
"builder = StateGraph(State)\n",
@@ -1053,7 +1053,7 @@
"builder.add_conditional_edges(\"evaluate\", control_edge, {END: END, \"solve\": \"solve\"})\n",
"\n",
"\n",
"checkpointer = MemorySaver()\n",
"checkpointer = InMemorySaver()\n",
"graph = builder.compile(checkpointer=checkpointer)"
]
},
@@ -1327,7 +1327,7 @@
"outputs": [],
"source": [
"# This is all the same as before\n",
"from langgraph.checkpoint.memory import MemorySaver\n",
"from langgraph.checkpoint.memory import InMemorySaver\n",
"from langgraph.graph import END, StateGraph, START\n",
"\n",
"builder = StateGraph(State)\n",
@@ -1353,7 +1353,7 @@
"\n",
"\n",
"builder.add_conditional_edges(\"evaluate\", control_edge, {END: END, \"solve\": \"solve\"})\n",
"checkpointer = MemorySaver()"
"checkpointer = InMemorySaver()"
]
},
{
@@ -154,7 +154,7 @@
"id": "2dff2209-44c7-4e2c-b607-ba6675f9e45f",
"metadata": {},
"outputs": [],
"source": ["from langgraph.checkpoint.memory import MemorySaver\nfrom langgraph.graph import END, StateGraph, START\n\nbuilder = StateGraph(GraphState)\n\n# Define the nodes\nbuilder.add_node(\"generate\", generate) # generation solution\nbuilder.add_node(\"check_code\", code_check) # check code\n\n# Build graph\nbuilder.add_edge(START, \"generate\")\nbuilder.add_edge(\"generate\", \"check_code\")\nbuilder.add_conditional_edges(\n \"check_code\",\n decide_to_finish,\n {\n \"end\": END,\n \"generate\": \"generate\",\n },\n)\n\nmemory = MemorySaver()\ngraph = builder.compile(checkpointer=memory)"]
"source": ["from langgraph.checkpoint.memory import InMemorySaver\nfrom langgraph.graph import END, StateGraph, START\n\nbuilder = StateGraph(GraphState)\n\n# Define the nodes\nbuilder.add_node(\"generate\", generate) # generation solution\nbuilder.add_node(\"check_code\", code_check) # check code\n\n# Build graph\nbuilder.add_edge(START, \"generate\")\nbuilder.add_edge(\"generate\", \"check_code\")\nbuilder.add_conditional_edges(\n \"check_code\",\n decide_to_finish,\n {\n \"end\": END,\n \"generate\": \"generate\",\n },\n)\n\nmemory = InMemorySaver()\ngraph = builder.compile(checkpointer=memory)"]
},
{
"cell_type": "code",
@@ -18,7 +18,6 @@ from psycopg.errors import UndefinedTable
from psycopg.rows import dict_row
from langgraph.store.base import GetOp, ListNamespacesOp, Op, PutOp, Result, SearchOp
from langgraph.store.base.batch import AsyncBatchedBaseStore
from langgraph.store.postgres.base import (
BasePostgresStore,
Row,
@@ -30,9 +29,7 @@ from langgraph.store.postgres.base import (
logger = logging.getLogger(__name__)
class AsyncPostgresStore(AsyncBatchedBaseStore, BasePostgresStore[AsyncConnection]):
__slots__ = ("_deserializer",)
class AsyncPostgresStore(BasePostgresStore[AsyncConnection]):
def __init__(
self,
conn: AsyncConnection[Any],
@@ -41,8 +38,7 @@ class AsyncPostgresStore(AsyncBatchedBaseStore, BasePostgresStore[AsyncConnectio
Callable[[Union[bytes, orjson.Fragment]], dict[str, Any]]
] = None,
) -> None:
super().__init__()
self._deserializer = deserializer
super().__init__(deserializer=deserializer)
self.conn = conn
self.conn = conn
self.loop = asyncio.get_running_loop()
@@ -59,10 +59,20 @@ CREATE INDEX IF NOT EXISTS store_prefix_idx ON store USING btree (prefix text_pa
C = TypeVar("C", bound=BaseConnection)
class BasePostgresStore(Generic[C]):
class BasePostgresStore(BaseStore, Generic[C]):
MIGRATIONS = MIGRATIONS
conn: C
_deserializer: Optional[Callable[[Union[bytes, orjson.Fragment]], dict[str, Any]]]
__slots__ = ("_deserializer",)
def __init__(
self,
*,
deserializer: Optional[
Callable[[Union[bytes, orjson.Fragment]], dict[str, Any]]
] = None,
) -> None:
super().__init__()
self._deserializer = deserializer
def _get_batch_GET_ops_queries(
self,
@@ -156,9 +166,7 @@ class BasePostgresStore(Generic[C]):
params.extend([key, json.dumps(value)])
query += " AND " + " AND ".join(filter_conditions)
# Note: we will need to not do this if sim/keyword search
# is used
query += " ORDER BY updated_at DESC LIMIT %s OFFSET %s"
query += " LIMIT %s OFFSET %s"
params.extend([op.limit, op.offset])
queries.append((query, params))
@@ -219,9 +227,7 @@ class BasePostgresStore(Generic[C]):
return queries
class PostgresStore(BaseStore, BasePostgresStore[Connection]):
__slots__ = ("_deserializer",)
class PostgresStore(BasePostgresStore[Connection]):
def __init__(
self,
conn: Connection[Any],
@@ -230,8 +236,7 @@ class PostgresStore(BaseStore, BasePostgresStore[Connection]):
Callable[[Union[bytes, orjson.Fragment]], dict[str, Any]]
] = None,
) -> None:
super().__init__()
self._deserializer = deserializer
super().__init__(deserializer=deserializer)
self.conn = conn
def batch(self, ops: Iterable[Op]) -> list[Result]:
+1 -1
View File
@@ -1,6 +1,6 @@
[tool.poetry]
name = "langgraph-checkpoint-postgres"
version = "2.0.1"
version = "2.0.0"
description = "Library with a Postgres implementation of LangGraph checkpoint saver."
authors = []
license = "MIT"
+2 -2
View File
@@ -44,12 +44,12 @@ If the checkpointer will be used with asynchronous graph execution (i.e. executi
## Usage
```python
from langgraph.checkpoint.memory import MemorySaver
from langgraph.checkpoint.memory import InMemorySaver
write_config = {"configurable": {"thread_id": "1", "checkpoint_ns": ""}}
read_config = {"configurable": {"thread_id": "1"}}
checkpointer = MemorySaver()
checkpointer = InMemorySaver()
checkpoint = {
"v": 1,
"ts": "2024-07-31T20:14:19.804150+00:00",
@@ -178,7 +178,7 @@ CheckpointNS = ConfigurableFieldSpec(
annotation=str,
name="Checkpoint NS",
description='Checkpoint namespace. Denotes the path to the subgraph node the checkpoint originates from, separated by `|` character, e.g. `"child|grandchild"`. Defaults to "" (root graph).',
default="",
default=None,
is_shared=True,
)
@@ -21,7 +21,7 @@ from langgraph.checkpoint.base import (
from langgraph.checkpoint.serde.types import TASKS, ChannelProtocol
class MemorySaver(
class InMemorySaver(
BaseCheckpointSaver[str], AbstractContextManager, AbstractAsyncContextManager
):
"""An in-memory checkpoint saver.
@@ -29,7 +29,7 @@ class MemorySaver(
This checkpoint saver stores checkpoints in memory using a defaultdict.
Note:
Only use `MemorySaver` for debugging or testing purposes.
Only use `InMemorySaver` for debugging or testing purposes.
For production use cases we recommend installing [langgraph-checkpoint-postgres](https://pypi.org/project/langgraph-checkpoint-postgres/) and using `PostgresSaver` / `AsyncPostgresSaver`.
Args:
@@ -39,7 +39,7 @@ class MemorySaver(
import asyncio
from langgraph.checkpoint.memory import MemorySaver
from langgraph.checkpoint.memory import InMemorySaver
from langgraph.graph import StateGraph
builder = StateGraph(int)
@@ -47,7 +47,7 @@ class MemorySaver(
builder.set_entry_point("add_one")
builder.set_finish_point("add_one")
memory = MemorySaver()
memory = InMemorySaver()
graph = builder.compile(checkpointer=memory)
coro = graph.ainvoke(1, {"configurable": {"thread_id": "thread-1"}})
asyncio.run(coro) # Output: 2
@@ -73,7 +73,7 @@ class MemorySaver(
self.storage = defaultdict(lambda: defaultdict(dict))
self.writes = defaultdict(dict)
def __enter__(self) -> "MemorySaver":
def __enter__(self) -> "InMemorySaver":
return self
def __exit__(
@@ -84,7 +84,7 @@ class MemorySaver(
) -> Optional[bool]:
return
async def __aenter__(self) -> "MemorySaver":
async def __aenter__(self) -> "InMemorySaver":
return self
async def __aexit__(
@@ -135,15 +135,17 @@ class MemorySaver(
pending_writes=[
(id, c, self.serde.loads_typed(v)) for id, c, v in writes
],
parent_config={
"configurable": {
"thread_id": thread_id,
"checkpoint_ns": checkpoint_ns,
"checkpoint_id": parent_checkpoint_id,
parent_config=(
{
"configurable": {
"thread_id": thread_id,
"checkpoint_ns": checkpoint_ns,
"checkpoint_id": parent_checkpoint_id,
}
}
}
if parent_checkpoint_id
else None,
if parent_checkpoint_id
else None
),
)
else:
if checkpoints := self.storage[thread_id][checkpoint_ns]:
@@ -176,15 +178,17 @@ class MemorySaver(
pending_writes=[
(id, c, self.serde.loads_typed(v)) for id, c, v in writes
],
parent_config={
"configurable": {
"thread_id": thread_id,
"checkpoint_ns": checkpoint_ns,
"checkpoint_id": parent_checkpoint_id,
parent_config=(
{
"configurable": {
"thread_id": thread_id,
"checkpoint_ns": checkpoint_ns,
"checkpoint_id": parent_checkpoint_id,
}
}
}
if parent_checkpoint_id
else None,
if parent_checkpoint_id
else None
),
)
def list(
@@ -285,15 +289,17 @@ class MemorySaver(
"pending_sends": [self.serde.loads_typed(s) for s in sends],
},
metadata=metadata,
parent_config={
"configurable": {
"thread_id": thread_id,
"checkpoint_ns": checkpoint_ns,
"checkpoint_id": parent_checkpoint_id,
parent_config=(
{
"configurable": {
"thread_id": thread_id,
"checkpoint_ns": checkpoint_ns,
"checkpoint_id": parent_checkpoint_id,
}
}
}
if parent_checkpoint_id
else None,
if parent_checkpoint_id
else None
),
pending_writes=[
(id, c, self.serde.loads_typed(v)) for id, c, v in writes
],
@@ -474,3 +480,6 @@ class MemorySaver(
next_v = current_v + 1
next_h = random.random()
return f"{next_v:032}.{next_h:016}"
MemorySaver = InMemorySaver # Kept for backwards compatibility
@@ -182,11 +182,7 @@ def _validate_namespace(namespace: tuple[str, ...]) -> None:
class BaseStore(ABC):
"""Abstract base class for persistent key-value stores.
Stores enable persistence and memory that can be shared across threads,
scoped to user IDs, assistant IDs, or other arbitrary namespaces.
"""
"""Abstract base class for key-value stores."""
__slots__ = ("__weakref__",)
+2 -2
View File
@@ -9,13 +9,13 @@ from langgraph.checkpoint.base import (
create_checkpoint,
empty_checkpoint,
)
from langgraph.checkpoint.memory import MemorySaver
from langgraph.checkpoint.memory import InMemorySaver
class TestMemorySaver:
@pytest.fixture(autouse=True)
def setup(self) -> None:
self.memory_saver = MemorySaver()
self.memory_saver = InMemorySaver()
# objects for test setup
self.config_1: RunnableConfig = {
+3 -3
View File
@@ -60,7 +60,7 @@ from typing import Annotated, Literal, TypedDict
from langchain_core.messages import HumanMessage
from langchain_anthropic import ChatAnthropic
from langchain_core.tools import tool
from langgraph.checkpoint.memory import MemorySaver
from langgraph.checkpoint.memory import InMemorySaver
from langgraph.graph import END, START, StateGraph, MessagesState
from langgraph.prebuilt import ToolNode
@@ -125,7 +125,7 @@ workflow.add_conditional_edges(
workflow.add_edge("tools", 'agent')
# Initialize memory to persist state between graph runs
checkpointer = MemorySaver()
checkpointer = InMemorySaver()
# Finally, we compile it!
# This compiles it into a LangChain Runnable,
@@ -201,7 +201,7 @@ final_state["messages"][-1].content
<summary>Compile the graph.</summary>
- When we compile the graph, we turn it into a LangChain [Runnable](https://python.langchain.com/v0.2/docs/concepts/#runnable-interface), which automatically enables calling `.invoke()`, `.stream()` and `.batch()` with your inputs
- We can also optionally pass checkpointer object for persisting state between graph runs, and enabling memory, human-in-the-loop workflows, time travel and more. In our case we use `MemorySaver` - a simple in-memory checkpointer
- We can also optionally pass checkpointer object for persisting state between graph runs, and enabling memory, human-in-the-loop workflows, time travel and more. In our case we use `InMemorySaver` - a simple in-memory checkpointer
</details>
6. <details>
+15 -15
View File
@@ -8,7 +8,7 @@ from uvloop import new_event_loop
from bench.fanout_to_subgraph import fanout_to_subgraph, fanout_to_subgraph_sync
from bench.react_agent import react_agent
from bench.wide_state import wide_state
from langgraph.checkpoint.memory import MemorySaver
from langgraph.checkpoint.memory import InMemorySaver
from langgraph.pregel import Pregel
@@ -55,8 +55,8 @@ benchmarks = (
),
(
"fanout_to_subgraph_10x_checkpoint",
fanout_to_subgraph().compile(checkpointer=MemorySaver()),
fanout_to_subgraph_sync().compile(checkpointer=MemorySaver()),
fanout_to_subgraph().compile(checkpointer=InMemorySaver()),
fanout_to_subgraph_sync().compile(checkpointer=InMemorySaver()),
{
"subjects": [
random.choices("abcdefghijklmnopqrstuvwxyz", k=1000) for _ in range(10)
@@ -75,8 +75,8 @@ benchmarks = (
),
(
"fanout_to_subgraph_100x_checkpoint",
fanout_to_subgraph().compile(checkpointer=MemorySaver()),
fanout_to_subgraph_sync().compile(checkpointer=MemorySaver()),
fanout_to_subgraph().compile(checkpointer=InMemorySaver()),
fanout_to_subgraph_sync().compile(checkpointer=InMemorySaver()),
{
"subjects": [
random.choices("abcdefghijklmnopqrstuvwxyz", k=1000) for _ in range(100)
@@ -91,8 +91,8 @@ benchmarks = (
),
(
"react_agent_10x_checkpoint",
react_agent(10, checkpointer=MemorySaver()),
react_agent(10, checkpointer=MemorySaver()),
react_agent(10, checkpointer=InMemorySaver()),
react_agent(10, checkpointer=InMemorySaver()),
{"messages": [HumanMessage("hi?")]},
),
(
@@ -103,8 +103,8 @@ benchmarks = (
),
(
"react_agent_100x_checkpoint",
react_agent(100, checkpointer=MemorySaver()),
react_agent(100, checkpointer=MemorySaver()),
react_agent(100, checkpointer=InMemorySaver()),
react_agent(100, checkpointer=InMemorySaver()),
{"messages": [HumanMessage("hi?")]},
),
(
@@ -125,8 +125,8 @@ benchmarks = (
),
(
"wide_state_25x300_checkpoint",
wide_state(300).compile(checkpointer=MemorySaver()),
wide_state(300).compile(checkpointer=MemorySaver()),
wide_state(300).compile(checkpointer=InMemorySaver()),
wide_state(300).compile(checkpointer=InMemorySaver()),
{
"messages": [
{
@@ -157,8 +157,8 @@ benchmarks = (
),
(
"wide_state_15x600_checkpoint",
wide_state(600).compile(checkpointer=MemorySaver()),
wide_state(600).compile(checkpointer=MemorySaver()),
wide_state(600).compile(checkpointer=InMemorySaver()),
wide_state(600).compile(checkpointer=InMemorySaver()),
{
"messages": [
{
@@ -189,8 +189,8 @@ benchmarks = (
),
(
"wide_state_9x1200_checkpoint",
wide_state(1200).compile(checkpointer=MemorySaver()),
wide_state(1200).compile(checkpointer=MemorySaver()),
wide_state(1200).compile(checkpointer=InMemorySaver()),
wide_state(1200).compile(checkpointer=InMemorySaver()),
{
"messages": [
{
+2 -2
View File
@@ -107,9 +107,9 @@ if __name__ == "__main__":
import uvloop
from langgraph.checkpoint.memory import MemorySaver
from langgraph.checkpoint.memory import InMemorySaver
graph = fanout_to_subgraph().compile(checkpointer=MemorySaver())
graph = fanout_to_subgraph().compile(checkpointer=InMemorySaver())
input = {
"subjects": [
random.choices("abcdefghijklmnopqrstuvwxyz", k=1000) for _ in range(1000)
+2 -2
View File
@@ -68,9 +68,9 @@ if __name__ == "__main__":
import uvloop
from langgraph.checkpoint.memory import MemorySaver
from langgraph.checkpoint.memory import InMemorySaver
graph = react_agent(100, checkpointer=MemorySaver())
graph = react_agent(100, checkpointer=InMemorySaver())
input = {"messages": [HumanMessage("hi?")]}
config = {"configurable": {"thread_id": "1"}, "recursion_limit": 20000000000}
+2 -2
View File
@@ -116,9 +116,9 @@ if __name__ == "__main__":
import uvloop
from langgraph.checkpoint.memory import MemorySaver
from langgraph.checkpoint.memory import InMemorySaver
graph = wide_state(1000).compile(checkpointer=MemorySaver())
graph = wide_state(1000).compile(checkpointer=InMemorySaver())
input = {
"messages": [
{
+1 -1
View File
@@ -90,7 +90,7 @@ class StateGraph(Graph):
Examples:
>>> from langchain_core.runnables import RunnableConfig
>>> from typing_extensions import Annotated, TypedDict
>>> from langgraph.checkpoint.memory import MemorySaver
>>> from langgraph.checkpoint.memory import InMemorySaver
>>> from langgraph.graph import StateGraph
>>>
>>> def reducer(a: list, b: int | None) -> list:
@@ -372,8 +372,8 @@ def create_react_agent(
Add thread-level "chat memory" to the graph:
```pycon
>>> from langgraph.checkpoint.memory import MemorySaver
>>> graph = create_react_agent(model, tools, checkpointer=MemorySaver())
>>> from langgraph.checkpoint.memory import InMemorySaver
>>> graph = create_react_agent(model, tools, checkpointer=InMemorySaver())
>>> config = {"configurable": {"thread_id": "thread-1"}}
>>> def print_stream(graph, inputs, config):
... for s in graph.stream(inputs, config, stream_mode="values"):
@@ -408,7 +408,7 @@ def create_react_agent(
```pycon
>>> graph = create_react_agent(
... model, tools, interrupt_before=["tools"], checkpointer=MemorySaver()
... model, tools, interrupt_before=["tools"], checkpointer=InMemorySaver()
>>> )
>>> config = {"configurable": {"thread_id": "thread-1"}}
@@ -442,10 +442,10 @@ def create_react_agent(
... system_msg = f"User memories: {', '.join(memories)}"
... return [{"role": "system", "content": system_msg)] + state["messages"]
>>> from langgraph.checkpoint.memory import MemorySaver
>>> from langgraph.checkpoint.memory import InMemorySaver
>>> from langgraph.store.memory import InMemoryStore
>>> store = InMemoryStore()
>>> graph = create_react_agent(model, [save_memory], state_modifier=prepare_model_inputs, store=store, checkpointer=MemorySaver())
>>> graph = create_react_agent(model, [save_memory], state_modifier=prepare_model_inputs, store=store, checkpointer=InMemorySaver())
>>> config = {"configurable": {"thread_id": "thread-1", "user_id": "1"}}
>>> inputs = {"messages": [("user", "Hey I'm Will, how's it going?")]}
+4 -4
View File
@@ -12,7 +12,7 @@ from langgraph.checkpoint.base import (
SerializerProtocol,
copy_checkpoint,
)
from langgraph.checkpoint.memory import MemorySaver
from langgraph.checkpoint.memory import InMemorySaver
class NoopSerializer(SerializerProtocol):
@@ -23,7 +23,7 @@ class NoopSerializer(SerializerProtocol):
return "type", obj
class MemorySaverAssertImmutable(MemorySaver):
class MemorySaverAssertImmutable(InMemorySaver):
storage_for_copies: defaultdict[str, dict[str, dict[str, Checkpoint]]]
def __init__(
@@ -64,7 +64,7 @@ class MemorySaverAssertImmutable(MemorySaver):
return super().put(config, checkpoint, metadata, new_versions)
class MemorySaverAssertCheckpointMetadata(MemorySaver):
class MemorySaverAssertCheckpointMetadata(InMemorySaver):
"""This custom checkpointer is for verifying that a run's configurable
fields are merged with the previous checkpoint config for each step in
the run. This is the desired behavior. Because the checkpointer's (a)put()
@@ -119,7 +119,7 @@ class MemorySaverAssertCheckpointMetadata(MemorySaver):
)
class MemorySaverNoPending(MemorySaver):
class MemorySaverNoPending(InMemorySaver):
def get_tuple(self, config: RunnableConfig) -> Optional[CheckpointTuple]:
result = super().get_tuple(config)
if result:
+7 -7
View File
@@ -53,7 +53,7 @@ from langgraph.checkpoint.base import (
CheckpointMetadata,
CheckpointTuple,
)
from langgraph.checkpoint.memory import MemorySaver
from langgraph.checkpoint.memory import InMemorySaver
from langgraph.constants import ERROR, PULL, PUSH
from langgraph.errors import InvalidUpdateError, MultipleSubgraphsError, NodeInterrupt
from langgraph.graph import END, Graph
@@ -268,11 +268,11 @@ def test_graph_validation() -> None:
def test_checkpoint_errors() -> None:
class FaultyGetCheckpointer(MemorySaver):
class FaultyGetCheckpointer(InMemorySaver):
def get_tuple(self, config: RunnableConfig) -> Optional[CheckpointTuple]:
raise ValueError("Faulty get_tuple")
class FaultyPutCheckpointer(MemorySaver):
class FaultyPutCheckpointer(InMemorySaver):
def put(
self,
config: RunnableConfig,
@@ -282,13 +282,13 @@ def test_checkpoint_errors() -> None:
) -> RunnableConfig:
raise ValueError("Faulty put")
class FaultyPutWritesCheckpointer(MemorySaver):
class FaultyPutWritesCheckpointer(InMemorySaver):
def put_writes(
self, config: RunnableConfig, writes: List[Tuple[str, Any]], task_id: str
) -> RunnableConfig:
raise ValueError("Faulty put_writes")
class FaultyVersionCheckpointer(MemorySaver):
class FaultyVersionCheckpointer(InMemorySaver):
def get_next_version(self, current: Optional[int], channel: BaseChannel) -> int:
raise ValueError("Faulty get_next_version")
@@ -11250,7 +11250,7 @@ def test_xray_lance(snapshot: SnapshotAssertion):
interview_builder.add_conditional_edges("answer_question", route_messages)
# Set up memory
memory = MemorySaver()
memory = InMemorySaver()
# Interview
interview_graph = interview_builder.compile(checkpointer=memory).with_config(
@@ -11478,7 +11478,7 @@ def test_subgraph_retries():
parent.add_edge("parent_node", "child_graph")
parent.set_entry_point("parent_node")
checkpointer = MemorySaver()
checkpointer = InMemorySaver()
app = parent.compile(checkpointer=checkpointer)
with pytest.raises(RandomError):
app.invoke({"count": 0}, {"configurable": {"thread_id": "foo"}})
+5 -5
View File
@@ -49,7 +49,7 @@ from langgraph.checkpoint.base import (
CheckpointMetadata,
CheckpointTuple,
)
from langgraph.checkpoint.memory import MemorySaver
from langgraph.checkpoint.memory import InMemorySaver
from langgraph.constants import ERROR, PULL, PUSH
from langgraph.errors import InvalidUpdateError, MultipleSubgraphsError, NodeInterrupt
from langgraph.graph import END, Graph, StateGraph
@@ -89,11 +89,11 @@ pytestmark = pytest.mark.anyio
async def test_checkpoint_errors() -> None:
class FaultyGetCheckpointer(MemorySaver):
class FaultyGetCheckpointer(InMemorySaver):
async def aget_tuple(self, config: RunnableConfig) -> Optional[CheckpointTuple]:
raise ValueError("Faulty get_tuple")
class FaultyPutCheckpointer(MemorySaver):
class FaultyPutCheckpointer(InMemorySaver):
async def aput(
self,
config: RunnableConfig,
@@ -103,13 +103,13 @@ async def test_checkpoint_errors() -> None:
) -> RunnableConfig:
raise ValueError("Faulty put")
class FaultyPutWritesCheckpointer(MemorySaver):
class FaultyPutWritesCheckpointer(InMemorySaver):
async def aput_writes(
self, config: RunnableConfig, writes: List[Tuple[str, Any]], task_id: str
) -> RunnableConfig:
raise ValueError("Faulty put_writes")
class FaultyVersionCheckpointer(MemorySaver):
class FaultyVersionCheckpointer(InMemorySaver):
def get_next_version(self, current: Optional[int], channel: BaseChannel) -> int:
raise ValueError("Faulty get_next_version")
@@ -69,7 +69,6 @@ class AsyncKafkaExecutor(AbstractAsyncContextManager):
self.retry_policy = retry_policy
async def __aenter__(self) -> Self:
loop = asyncio.get_running_loop()
self.subgraphs = {
k: v async for k, v in self.graph.aget_subgraphs(recurse=True)
}
@@ -82,7 +81,6 @@ class AsyncKafkaExecutor(AbstractAsyncContextManager):
auto_offset_reset="earliest",
group_id="executor",
enable_auto_commit=False,
loop=loop,
**self.kwargs,
)
)
@@ -91,7 +89,6 @@ class AsyncKafkaExecutor(AbstractAsyncContextManager):
self.producer = await self.stack.enter_async_context(
DefaultAsyncProducer(
loop=loop,
**self.kwargs,
)
)
@@ -67,7 +67,6 @@ class AsyncKafkaOrchestrator(AbstractAsyncContextManager):
self.retry_policy = retry_policy
async def __aenter__(self) -> Self:
loop = asyncio.get_running_loop()
self.subgraphs = {
k: v async for k, v in self.graph.aget_subgraphs(recurse=True)
}
@@ -80,7 +79,6 @@ class AsyncKafkaOrchestrator(AbstractAsyncContextManager):
auto_offset_reset="earliest",
group_id="orchestrator",
enable_auto_commit=False,
loop=loop,
**self.kwargs,
)
)
@@ -89,7 +87,6 @@ class AsyncKafkaOrchestrator(AbstractAsyncContextManager):
self.producer = await self.stack.enter_async_context(
DefaultAsyncProducer(
loop=loop,
**self.kwargs,
)
)
+1 -1
View File
@@ -1,6 +1,6 @@
{
"name": "@langchain/langgraph-sdk",
"version": "0.0.16",
"version": "0.0.15",
"description": "Client library for interacting with the LangGraph API",
"type": "module",
"packageManager": "yarn@1.22.19",
-2
View File
@@ -611,7 +611,6 @@ export class ThreadsClient extends BaseClient {
options?: {
limit?: number;
before?: Config;
checkpoint?: Partial<Omit<Checkpoint, "thread_id">>;
metadata?: Metadata;
},
): Promise<ThreadState<ValuesType>[]> {
@@ -623,7 +622,6 @@ export class ThreadsClient extends BaseClient {
limit: options?.limit ?? 10,
before: options?.before,
metadata: options?.metadata,
checkpoint: options?.checkpoint,
},
},
);