diff --git a/docs/docs/agents/memory.md b/docs/docs/agents/memory.md index 3ea899f37..3c6268859 100644 --- a/docs/docs/agents/memory.md +++ b/docs/docs/agents/memory.md @@ -92,15 +92,26 @@ When the agent is invoked the second time with the same `thread_id`, the origina If you're using [LangGraph Platform](./deployment.md), during deployment your checkpointer will be automatically configured to use a production-ready database. -### Message history summarization +### Manage message history + +Long conversations can exceed the LLM's context window. Common solutions are: + +* [Summarization](#summarize-message-history): Maintain a running summary of the conversation +* [Trimming](#trim-message-history): Remove first or last N messages in the history + +This allows the agent to keep track of the conversation without exceeding the LLM's context window. + +To manage message history, specify `pre_model_hook` — a function ([node](../concepts/low_level.md#nodes)) that will always run before calling the language model. + +#### Summarize message history
![image](./assets/summary.png){: style="max-height:400px"} -
Message history can grow quickly and exceed the LLM's context window. A common solution is to maintain a running summary of the conversation. This allows the agent to keep track of the conversation without exceeding the LLM's context window. +
Long conversations can exceed the LLM's context window. A common solution is to maintain a running summary of the conversation. This allows the agent to keep track of the conversation without exceeding the LLM's context window.
-Long conversations can exceed the LLM's context window. To handle this, you can summarize older messages by specifying a [`pre_model_hook`][langgraph.prebuilt.chat_agent_executor.create_react_agent], such as the prebuilt [`SummarizationNode`](https://langchain-ai.github.io/langmem/reference/short_term/#langmem.short_term.SummarizationNode): +To summarize message history, you can use [`pre_model_hook`][langgraph.prebuilt.chat_agent_executor.create_react_agent] with a prebuilt [`SummarizationNode`](https://langchain-ai.github.io/langmem/reference/short_term/#langmem.short_term.SummarizationNode): ```python from langchain_anthropic import ChatAnthropic @@ -147,6 +158,44 @@ agent = create_react_agent( 4. The `pre_model_hook` is set to the `SummarizationNode`. This node will summarize the message history before sending it to the LLM. The summarization node will automatically handle the summarization process and update the agent's state with the new summary. You can replace this with a custom implementation if you prefer. Please see the [create_react_agent][langgraph.prebuilt.chat_agent_executor.create_react_agent] API reference for more details. 5. The `state_schema` is set to the `State` class, which is the custom state that contains an extra `context` key. +#### Trim message history + +To trim message history, you can use [`pre_model_hook`][langgraph.prebuilt.chat_agent_executor.create_react_agent] with [`trim_messages`](https://python.langchain.com/api_reference/core/messages/langchain_core.messages.utils.trim_messages.html) function: + +```python +# highlight-next-line +from langchain_core.messages.utils import ( + # highlight-next-line + trim_messages, + # highlight-next-line + count_tokens_approximately +# highlight-next-line +) +from langgraph.prebuilt import create_react_agent + +# This function will be called every time before the node that calls LLM +def pre_model_hook(state): + trimmed_messages = trim_messages( + state["messages"], + strategy="last", + token_counter=count_tokens_approximately, + max_tokens=384, + start_on="human", + end_on=("human", "tool"), + ) + # highlight-next-line + return {"llm_input_messages": trimmed_messages} + +checkpointer = InMemorySaver() +agent = create_react_agent( + model, + tools, + # highlight-next-line + pre_model_hook=pre_model_hook, + checkpointer=checkpointer, +) +``` + To learn more about using `pre_model_hook` for managing message history, see this [how-to guide](../how-tos/create-react-agent-manage-message-history.ipynb) ### Read in tools { #read-short-term } diff --git a/docs/docs/how-tos/create-react-agent-manage-message-history.ipynb b/docs/docs/how-tos/create-react-agent-manage-message-history.ipynb index 3cc1368dc..22fcdf7f9 100644 --- a/docs/docs/how-tos/create-react-agent-manage-message-history.ipynb +++ b/docs/docs/how-tos/create-react-agent-manage-message-history.ipynb @@ -17,8 +17,8 @@ "\n", "Message history can grow quickly and exceed LLM context window size, whether you're building chatbots with many conversation turns or agentic systems with numerous tool calls. There are several strategies for managing the message history:\n", "\n", - "* [message trimming](#keep-the-original-message-history-unmodified) - remove first or last N messages in the history\n", - "* [summarization](#summarizing-message-history) - summarize earlier messages in the history and replace them with a summary\n", + "* [message trimming](#keep-the-original-message-history-unmodified) — remove first or last N messages in the history\n", + "* [summarization](#summarizing-message-history) — summarize earlier messages in the history and replace them with a summary\n", "* custom strategies (e.g., message filtering, etc.)\n", "\n", "To manage message history in `create_react_agent`, you need to define a `pre_model_hook` function or [runnable](https://python.langchain.com/docs/concepts/runnables/) that takes graph state an returns a state update:\n",