langgraph/examples/configuration.ipynb

How to add runtime configuration to your graph

Sometimes you want to be able to configure your agent when calling it. Examples of this include configuring which LLM to use. Below we walk through an example of doing so.

Base

First, let's create a very simple graph

In [1]:

import operator
from typing import Annotated, Sequence, TypedDict

from langchain_anthropic import ChatAnthropic
from langchain_core.messages import BaseMessage, HumanMessage

from langgraph.graph import END, StateGraph, START

model = ChatAnthropic(model_name="claude-2.1")


class AgentState(TypedDict):
    messages: Annotated[Sequence[BaseMessage], operator.add]


def _call_model(state):
    response = model.invoke(state["messages"])
    return {"messages": [response]}


# Define a new graph
workflow = StateGraph(AgentState)
workflow.add_node("model", _call_model)
workflow.add_edge(START, "model")
workflow.add_edge("model", END)

app = workflow.compile()

In [2]:

app.invoke({"messages": [HumanMessage(content="hi")]})

Out [2]:

{'messages': [HumanMessage(content='hi'),
  AIMessage(content='Hello!', response_metadata={'id': 'msg_012SakNGNitBcKJgc9yZ1Asv', 'model': 'claude-2.1', 'stop_reason': 'end_turn', 'stop_sequence': None, 'usage': {'input_tokens': 10, 'output_tokens': 6}}, id='run-9e375cd7-ae84-4db2-981c-c7e18ecabddf-0', usage_metadata={'input_tokens': 10, 'output_tokens': 6, 'total_tokens': 16})]}

Configure the graph

Great! Now let's suppose that we want to extend this example so the user is able to choose from multiple llms. We can easily do that by passing in a config. Any configuration information needs to be passed inside configurable key as shown below. This config is meant to contain things are not part of the input (and therefore that we don't want to track as part of the state).

In [6]:

from langchain_openai import ChatOpenAI
from typing import Optional
from langchain_core.runnables.config import RunnableConfig

openai_model = ChatOpenAI()

models = {
    "anthropic": model,
    "openai": openai_model,
}

def _call_model(state: AgentState, config: RunnableConfig):
    # Access the config through the configurable key
    model_name = config["configurable"].get("model", "anthropic")
    model = models[model_name]
    response = model.invoke(state["messages"])
    return {"messages": [response]}


# Define a new graph
workflow = StateGraph(AgentState)
workflow.add_node("model", _call_model)
workflow.add_edge(START, "model")
workflow.add_edge("model", END)

app = workflow.compile()

If we call it with no configuration, it will use the default as we defined it (Anthropic).

In [7]:

app.invoke({"messages": [HumanMessage(content="hi")]})

Out [7]:

{'messages': [HumanMessage(content='hi'),
  AIMessage(content='Hello!', response_metadata={'id': 'msg_0133PAX5DyoUYL1gZiGR8NXs', 'model': 'claude-2.1', 'stop_reason': 'end_turn', 'stop_sequence': None, 'usage': {'input_tokens': 10, 'output_tokens': 6}}, id='run-03e8bd8b-fa09-4258-920d-8f53a7b91fcc-0', usage_metadata={'input_tokens': 10, 'output_tokens': 6, 'total_tokens': 16})]}

We can also call it with a config to get it to use a different model.

In [8]:

config = {"configurable": {"model": "openai"}}
app.invoke({"messages": [HumanMessage(content="hi")]}, config=config)

Out [8]:

{'messages': [HumanMessage(content='hi'),
  AIMessage(content='Hello! How can I assist you today?', additional_kwargs={'refusal': None}, response_metadata={'token_usage': {'completion_tokens': 9, 'prompt_tokens': 8, 'total_tokens': 17}, 'model_name': 'gpt-3.5-turbo-0125', 'system_fingerprint': None, 'finish_reason': 'stop', 'logprobs': None}, id='run-6d0c7c25-03de-49d6-b3be-ff0858d17122-0', usage_metadata={'input_tokens': 8, 'output_tokens': 9, 'total_tokens': 17})]}

We can also adapt our graph to take in more configuration! Like a system message for example.

In [9]:

from langchain_core.messages import SystemMessage

# We can define a config schema to specify the configuration options for the graph
# A config schema is useful for indicating which fields are available in the configurable dict inside the config
class ConfigSchema(TypedDict):
    model: Optional[str]
    system_message: Optional[str]

def _call_model(state: AgentState, config: RunnableConfig):
    # Access the config through the configurable key
    model_name = config["configurable"].get("model", "anthropic")
    model = models[model_name]
    messages = state["messages"]
    if "system_message" in config["configurable"]:
        messages = [
            SystemMessage(content=config["configurable"]["system_message"])
        ] + messages
    response = model.invoke(messages)
    return {"messages": [response]}


# Define a new graph - note that we pass in the configuration schema here, but it is not necessary
workflow = StateGraph(AgentState, ConfigSchema)
workflow.add_node("model", _call_model)
workflow.add_edge(START, "model")
workflow.add_edge("model", END)

app = workflow.compile()

In [10]:

app.invoke({"messages": [HumanMessage(content="hi")]})

Out [10]:

{'messages': [HumanMessage(content='hi'),
  AIMessage(content='Hello!', response_metadata={'id': 'msg_01TVJvxCXsCT9JVe7A4iUUi9', 'model': 'claude-2.1', 'stop_reason': 'end_turn', 'stop_sequence': None, 'usage': {'input_tokens': 10, 'output_tokens': 6}}, id='run-627eb685-c4d7-481d-9095-c0a1822e8c10-0', usage_metadata={'input_tokens': 10, 'output_tokens': 6, 'total_tokens': 16})]}

In [11]:

config = {"configurable": {"system_message": "respond in italian"}}
app.invoke({"messages": [HumanMessage(content="hi")]}, config=config)

Out [11]:

{'messages': [HumanMessage(content='hi'),
  AIMessage(content='Ciao!', response_metadata={'id': 'msg_01CpBD1cMCYvvPX2cogUawJj', 'model': 'claude-2.1', 'stop_reason': 'end_turn', 'stop_sequence': None, 'usage': {'input_tokens': 14, 'output_tokens': 7}}, id='run-6ef2fea6-9bfa-4266-bd05-263160a1db7b-0', usage_metadata={'input_tokens': 14, 'output_tokens': 7, 'total_tokens': 21})]}