langgraph/examples/chat_agent_executor_with_function_calling/anthropic.ipynb

Chat Agent Executor with Anthropic

In this example we will build a chat executor that uses tool calling and the prebuilt ToolNode with Anthropic.

Setup

First we need to install the packages required

In [1]:

%%capture --no-stderr
%pip install install --quiet -U langchain langchain_anthropic tavily-python

Next, we need to set API keys for OpenAI (the LLM we will use) and Tavily (the search tool we will use)

In [ ]:

import os
import getpass

os.environ["OPENAI_API_KEY"] = getpass.getpass("OpenAI API Key:")
os.environ["TAVILY_API_KEY"] = getpass.getpass("Tavily API Key:")

Optionally, we can set API key for LangSmith tracing, which will give us best-in-class observability.

In [ ]:

os.environ["LANGCHAIN_TRACING_V2"] = "true"
os.environ["LANGCHAIN_API_KEY"] = getpass.getpass("LangSmith API Key:")

Set up the tools

We will first define the tools we want to use. For this simple example, we will use a built-in search tool via Tavily. However, it is really easy to create your own tools - see documentation here on how to do that.

MODIFICATION

We don't need a ToolExecutor when using ToolNode.

In [1]:

from langchain_community.tools.tavily_search import TavilySearchResults

tools = [TavilySearchResults(max_results=1)]

Set up the model

Now we need to load the chat model we want to use. Importantly, this should satisfy two criteria:

1. It should work with messages. We will represent all agent state in the form of messages, so it needs to be able to work well with them.
2. It should work with tool calling. This means it should be a model that implements .bind_tools().

Note: these model requirements are not requirements for using LangGraph - they are just requirements for this one example.

In [2]:

from langchain_anthropic import ChatAnthropic

model = ChatAnthropic(temperature=0, model_name="claude-3-opus-20240229")

After we've done this, we should make sure the model knows that it has these tools available to call. We can do this by converting the LangChain tools into the format for OpenAI function calling, and then bind them to the model class.

In [3]:

model = model.bind_tools(tools)

/Users/nuno/dev/langgraph/.venv/lib/python3.11/site-packages/langchain_core/_api/beta_decorator.py:87: LangChainBetaWarning: The method `ChatAnthropic.bind_tools` is in beta. It is actively being worked on, so the API may change.
  warn_beta(

In [4]:

from typing import TypedDict, Annotated, Sequence
import operator
from langchain_core.messages import BaseMessage


class AgentState(TypedDict):
    messages: Annotated[Sequence[BaseMessage], operator.add]

Define the nodes

We now need to define a few different nodes in our graph. In langgraph, a node can be either a function or a runnable. There are two main nodes we need for this:

1. The agent: responsible for deciding what (if any) actions to take.
2. MODIFICATION The prebuilt ToolNode, given the list of tools. This will take tool calls from the most recent AIMessage, execute them, and return the result as ToolMessages.

We will also need to define some edges. Some of these edges may be conditional. The reason they are conditional is that based on the output of a node, one of several paths may be taken. The path that is taken is not known until that node is run (the LLM decides).

1. Conditional Edge: after the agent is called, we should either: a. If the agent said to take an action, then the function to invoke tools should be called b. If the agent said that it was finished, then it should finish
2. Normal Edge: after the tools are invoked, it should always go back to the agent to decide what to do next

Let's define the nodes, as well as a function to decide how what conditional edge to take.

In [5]:

from langgraph.prebuilt import ToolNode


# Define the function that determines whether to continue or not
def should_continue(state):
    messages = state["messages"]
    last_message = messages[-1]
    # If there are no tool calls, then we finish
    if not last_message.tool_calls:
        return "end"
    # Otherwise if there is, we continue
    else:
        return "continue"


# Define the function that calls the model
def call_model(state):
    messages = state["messages"]
    response = model.invoke(messages)
    # We return a list, because this will get added to the existing list
    return {"messages": [response]}


# Define the function to execute tools
tool_node = ToolNode(tools)

Define the graph

We can now put it all together and define the graph!

In [6]:

from langgraph.graph import StateGraph, END

# Define a new graph
workflow = StateGraph(AgentState)

# Define the two nodes we will cycle between
workflow.add_node("agent", call_model)
workflow.add_node("action", tool_node)

# Set the entrypoint as `agent`
# This means that this node is the first one called
workflow.set_entry_point("agent")

# We now add a conditional edge
workflow.add_conditional_edges(
    # First, we define the start node. We use `agent`.
    # This means these are the edges taken after the `agent` node is called.
    "agent",
    # Next, we pass in the function that will determine which node is called next.
    should_continue,
    # Finally we pass in a mapping.
    # The keys are strings, and the values are other nodes.
    # END is a special node marking that the graph should finish.
    # What will happen is we will call `should_continue`, and then the output of that
    # will be matched against the keys in this mapping.
    # Based on which one it matches, that node will then be called.
    {
        # If `tools`, then we call the tool node.
        "continue": "action",
        # Otherwise we finish.
        "end": END,
    },
)

# We now add a normal edge from `tools` to `agent`.
# This means that after `tools` is called, `agent` node is called next.
workflow.add_edge("action", "agent")

# Finally, we compile it!
# This compiles it into a LangChain Runnable,
# meaning you can use it as you would any other runnable
app = workflow.compile()

Use it!

We can now use it! This now exposes the same interface as all other LangChain runnables.

In [7]:

from langchain_core.messages import HumanMessage

inputs = {"messages": [HumanMessage(content="what is the weather in sf")]}
app.invoke(inputs)

Out [7]:

{'messages': [HumanMessage(content='what is the weather in sf'),
  AIMessage(content=[{'text': '<thinking>\nThe relevant tool to answer this question is tavily_search_results_json, which can provide comprehensive information about current events like weather.\n\nTo call this function, I need to provide a value for the required "query" parameter. The user\'s request directly specifies they want to know the weather in "sf", which I can reasonably infer refers to San Francisco.\n\nTherefore, I have enough information to populate the required parameter:\nquery = "weather in San Francisco"\n\n</thinking>', 'type': 'text'}, {'id': 'toolu_0183a3MorRJu43zykiCWKAyo', 'input': {'query': 'weather in San Francisco'}, 'name': 'tavily_search_results_json', 'type': 'tool_use'}], response_metadata={'id': 'msg_01Lg8ZNFNwbDXz9VfxZyRCSb', 'model': 'claude-3-opus-20240229', 'stop_reason': 'tool_use', 'stop_sequence': None, 'usage': {'input_tokens': 507, 'output_tokens': 166}}, id='run-587209cf-1406-47f1-9476-73f9c75f4650-0', tool_calls=[{'name': 'tavily_search_results_json', 'args': {'query': 'weather in San Francisco'}, 'id': 'toolu_0183a3MorRJu43zykiCWKAyo'}]),
  ToolMessage(content='[{"url": "https://www.weatherapi.com/", "content": "{\'location\': {\'name\': \'San Francisco\', \'region\': \'California\', \'country\': \'United States of America\', \'lat\': 37.78, \'lon\': -122.42, \'tz_id\': \'America/Los_Angeles\', \'localtime_epoch\': 1714170321, \'localtime\': \'2024-04-26 15:25\'}, \'current\': {\'last_updated_epoch\': 1714169700, \'last_updated\': \'2024-04-26 15:15\', \'temp_c\': 17.2, \'temp_f\': 63.0, \'is_day\': 1, \'condition\': {\'text\': \'Partly cloudy\', \'icon\': \'//cdn.weatherapi.com/weather/64x64/day/116.png\', \'code\': 1003}, \'wind_mph\': 34.9, \'wind_kph\': 56.2, \'wind_degree\': 280, \'wind_dir\': \'W\', \'pressure_mb\': 1017.0, \'pressure_in\': 30.02, \'precip_mm\': 0.0, \'precip_in\': 0.0, \'humidity\': 60, \'cloud\': 50, \'feelslike_c\': 17.2, \'feelslike_f\': 63.0, \'vis_km\': 16.0, \'vis_miles\': 9.0, \'uv\': 4.0, \'gust_mph\': 39.4, \'gust_kph\': 63.4}}"}]', name='tavily_search_results_json', tool_call_id='toolu_0183a3MorRJu43zykiCWKAyo'),
  AIMessage(content="<search_quality_reflection>\nThe search results provide a comprehensive and up-to-date weather report for San Francisco, including key details like the current temperature, weather conditions, wind, humidity, and more. This should be sufficient to fully answer the question of what the current weather is like in San Francisco.\n</search_quality_reflection>\n\n<search_quality_score>5</search_quality_score>\n\n<result>\nAccording to the current weather report, the weather in San Francisco right now is:\n\nTemperature: 63°F (17.2°C)\nConditions: Partly cloudy \nWind: 34.9 mph (56.2 km/h) winds from the west\nHumidity: 60%\n\nIt feels like 63°F (17.2°C). Visibility is good at 9 miles (16 km). The UV index is moderate at 4.0 out of 11. \n\nOverall, it's a mild spring day in San Francisco with some cloud cover and breezy conditions. A light jacket or sweater should suffice for being outdoors.\n</result>", response_metadata={'id': 'msg_01LS72RMeicMF1xT7enopKpJ', 'model': 'claude-3-opus-20240229', 'stop_reason': 'end_turn', 'stop_sequence': None, 'usage': {'input_tokens': 1097, 'output_tokens': 251}}, id='run-794deb88-bea5-4d0d-93db-bf5dc38445f0-0')]}

This may take a little bit - it's making a few calls behind the scenes. In order to start seeing some intermediate results as they happen, we can use streaming - see below for more information on that.

Streaming

LangGraph has support for several different types of streaming.

Streaming Node Output

One of the benefits of using LangGraph is that it is easy to stream output as it's produced by each node.

In [9]:

inputs = {"messages": [HumanMessage(content="what is the weather in sf")]}
for output in app.stream(inputs):
    # stream() yields dictionaries with output keyed by node name
    for key, value in output.items():
        print(f"Output from node '{key}':")
        print("---")
        print(value)
    print("\n---\n")

Output from node 'agent':
---
{'messages': [AIMessage(content=[{'text': '<thinking>\nThe relevant tool to answer this question is tavily_search_results_json, which can provide comprehensive results about current events like weather.\n\nTo call this function, I need to provide a value for the required "query" parameter. The user\'s request directly specifies the query to search for: "weather in sf". "sf" here likely refers to San Francisco.\n\nSince I have a value for the required parameter, I can proceed with the function call.\n</thinking>', 'type': 'text'}, {'id': 'toolu_01XgUtdMt17UaBS8BUN2ZRyn', 'input': {'query': 'weather in San Francisco'}, 'name': 'tavily_search_results_json', 'type': 'tool_use'}], response_metadata={'id': 'msg_01SyKFjD9dxUNxwTQ5FiT3Yr', 'model': 'claude-3-opus-20240229', 'stop_reason': 'tool_use', 'stop_sequence': None, 'usage': {'input_tokens': 507, 'output_tokens': 162}}, id='run-42b25509-f322-4c4b-9817-f9ae154b8293-0', tool_calls=[{'name': 'tavily_search_results_json', 'args': {'query': 'weather in San Francisco'}, 'id': 'toolu_01XgUtdMt17UaBS8BUN2ZRyn'}])]}

---

Output from node 'action':
---
{'messages': [ToolMessage(content='[{"url": "https://www.weatherapi.com/", "content": "{\'location\': {\'name\': \'San Francisco\', \'region\': \'California\', \'country\': \'United States of America\', \'lat\': 37.78, \'lon\': -122.42, \'tz_id\': \'America/Los_Angeles\', \'localtime_epoch\': 1712857380, \'localtime\': \'2024-04-11 10:43\'}, \'current\': {\'last_updated_epoch\': 1712856600, \'last_updated\': \'2024-04-11 10:30\', \'temp_c\': 15.6, \'temp_f\': 60.1, \'is_day\': 1, \'condition\': {\'text\': \'Partly cloudy\', \'icon\': \'//cdn.weatherapi.com/weather/64x64/day/116.png\', \'code\': 1003}, \'wind_mph\': 4.3, \'wind_kph\': 6.8, \'wind_degree\': 50, \'wind_dir\': \'NE\', \'pressure_mb\': 1015.0, \'pressure_in\': 29.96, \'precip_mm\': 0.0, \'precip_in\': 0.0, \'humidity\': 78, \'cloud\': 25, \'feelslike_c\': 15.6, \'feelslike_f\': 60.1, \'vis_km\': 16.0, \'vis_miles\': 9.0, \'uv\': 5.0, \'gust_mph\': 5.1, \'gust_kph\': 8.3}}"}]', name='tavily_search_results_json', tool_call_id='toolu_01XgUtdMt17UaBS8BUN2ZRyn')]}

---

Output from node 'agent':
---
{'messages': [AIMessage(content='<search_quality_reflection>\nThe search results provide a comprehensive and up-to-date weather report for San Francisco, including key details like temperature, conditions, wind, humidity, and more. This should be sufficient to fully answer the question of what the current weather is like in San Francisco.\n</search_quality_reflection>\n<search_quality_score>5</search_quality_score>\n\n<result>\nAccording to the latest weather report, the current weather in San Francisco is:\n\nTemperature: 60.1°F (15.6°C)\nConditions: Partly cloudy \nWind: 4.3 mph (6.8 km/h) from the NE\nHumidity: 78%\nPrecipitation: 0 inches\nVisibility: 9 miles\nUV Index: 5.0\n\nIt feels like 60.1°F (15.6°C). The report indicates it is a partly cloudy day with no rain expected. Winds are light out of the northeast.\n</result>', response_metadata={'id': 'msg_01X8S82ECeXU8px2TpMPfkce', 'model': 'claude-3-opus-20240229', 'stop_reason': 'end_turn', 'stop_sequence': None, 'usage': {'input_tokens': 1094, 'output_tokens': 232}}, id='run-772e7225-dc58-4b63-a0d7-6d7d39e3b059-0')]}

---