langgraph/examples/sdk/js_sdk.ipynb

This how-to takes you through the steps of using the JS/TS SDK for interacting with deployed Langgraph Cloud APIs.

Initialization

Initializing client

To get started we need to initialize our client. The process for initializing our client is almost identical for both the local deployment and cloud deployment using Langsmith.

In [ ]:

import { Client } from "@langchain/langgraph-sdk";



// If you deployed using Langsmith use this option
// Find this url on your Langsmith deployment page
const example_deployed_url = "https://ht-unhealthy-buffalo25-39d00f953458585aa9f7b5a4fa-g3ps4aazkq-uc.a.run.app";

// If you deployed locally using langgraph up -c langgraph.json use this option
// This is the default URL, and you can just call get_client() to use it
const example_local_url = "http://localhost:8123";

const client = new Client({apiUrl:"whatever-your-url-is"});

Selecting an Assistant

To select an assistant we can search the assistants that are hosted on our client, and then select the one we want:

In [ ]:

// List all assistants
const assistants = await client.assistants.search({
    metadata: null,
    offset: 0,
    limit: 10,
});

// We auto-create an assistant for each graph you register in config.
const agent = assistants[0];

In our example we are only hosting a single assistant, but you have the option to host many, in which case you will most likely want to do more filtering than just selecting the first one. Each assistant is a JSON object with the following format, allowing you to select based on a variety of parameters.

In [ ]:

console.log(agent);

In [ ]:

{
    assistant_id: 'fe096781-5601-53d2-b2f6-0d3403f7e9ca',
    graph_id: 'agent',
    created_at: '2024-06-11T20:12:45.862108+00:00',
    updated_at: '2024-06-11T20:12:45.862108+00:00',
    config: {},
    metadata: { created_by: 'system' }
}

Configuring an assistant

One important thing to know is that graph can be defined to be configurable, meaning that not every instance of the graph needs to be the same (read up on this guide to learn more about how to create your own configurable graphs). Let's briefly show how we can configure an assistant. The first step to do is find the assistant we want to configure. In our simple example we are only hosting a single graph, so we must choose it as the graph to configure.

In [ ]:

const base_assistant = assistants[0];

This assistant has one configurable argument called model, which can take on two values: openai or anthropic. In this case, I want to create a graph that originates from this assistant that uses the openai option, which we can do as follows:

In [ ]:

const config_graph = await client.assistants.create({graphId: base_assistant['graph_id'],config: {'configurable':{'model':'openai'}}});
console.log(config_graph);

In [ ]:

{
    assistant_id: '414a1ddd-4453-4500-ab0b-7bc4f5bb41b1',
    graph_id: 'agent',
    created_at: '2024-06-21T23:42:16.388341+00:00',
    updated_at: '2024-06-21T23:42:16.388341+00:00',
    config: { configurable: { model: 'openai' } },
    metadata: {}
}

Being able to create configurable assistants allows you to create different graphs all based on the same underlying structure. This can be very powerful for testing different configurations of a graph or allowing users to customize their graph.

Creating a thread

Threads are what we will actually use to run our graphs (assistants). Each thread will update the same state for the graph, meaning we can run the graph multiple times while the state will persist. We can also look back at our thread history, add meta data to different steps of our thread, and update the thread state manually if we wish. We will dive into all of those topics later in this article, but for now let’s just see how to start a thread:

In [ ]:

// Start a new thread
const thread = await client.threads.create();

We can examine the structure of our thread, which similar to the assistants object provides us with some information about the thread itself, including its id, timestamps, and metadata:

In [ ]:

console.log(thread);

In [ ]:

{
    thread_id: '12a112dc-d175-42ce-8b06-85697733cccc',
    created_at: '2024-06-20T22:01:18.715497+00:00',
    updated_at: '2024-06-20T22:01:18.715497+00:00',
    metadata: {}
}

Now we are ready to actually use our graph!

Invoking the graph

The graph used in this example is a simple example of a StateGraph, but it allows us to show most of the API functionality. The state of our graph is defined as follows (Langgraph Cloud is coming in the future for JS!):

In [ ]:

from typing import Annotated, TypedDict

from langchain_core.messages import AnyMessage

from langgraph.graph import add_messages


def update_user_info(old_info, new_info):
    if "name" not in new_info or new_info["age"] == -1:
        return old_info
    return new_info


class UserInformation(TypedDict):
    age: int
    name: str


class AgentState(TypedDict):
    messages: Annotated[list[AnyMessage], add_messages]
    user_info: Annotated[UserInformation, update_user_info]

It is important to note that our member variables can be updated by using the Annotated class. This is especially important when we make API calls that will update our variables. This is also a good example that your graph can hold much more information than just messages. In our example we use a very simple UserInformation class, but you can imagine holding much richer information in your state.

Our graph looks like follows:

The workflow is as follows: first the user inputs some message, our llm decides how to configure the call to our tool get_user_info , and after getting the results of the tool call we respond to our user using another LLM.

Simple Invocation

Ok, now that we have set up our client, assistant, and thread we can actually invoke the graph above. Let’s first define the function we will use to invoke the graph, since we don’t want to have to rewrite this code every single run.

In [ ]:

async function processStreamedMessages(client, thread, agent, messages, metadata = {}) {
    var answer = [];
    try {
        const streamResponse = client.runs.stream(
            thread["thread_id"],
            agent["assistant_id"],
            {
                input: { messages },
                config: {"configurable": metadata}
            }
        );

        for await (const chunk of streamResponse) {
            answer.concat(chunk);
            // Process each chunk of streamed data here
        }
        return answer;
    } catch (error) {
        console.error('Error processing streamed messages:', error);
    }
}

Let’s now see what happens to our graph when we run it with a simple sentence:

In [ ]:

var messages = [{ role: "human", content: "My name is Bagatur and I am 26 years old." }];

await processStreamedMessages(client, thread, agent, messages);

In order to see what happened to our graph, let's examine the state which was updated after the run we just sent through.

In [ ]:

var state = await client.threads.getState(thread['thread_id']);
console.log(state);

In [ ]:

{
    values: {
      messages: [ [Object], [Object], [Object] ],
      user_info: { age: 26, name: 'Bagatur' }
    },
    next: [],
    config: {
      configurable: {
        thread_id: '15292d98-6aca-4df8-b2a1-f4508e11abb1',
        thread_ts: '1ef2f516-ebd2-68f4-8003-2a1251f89da5'
      }
    },
    metadata: {
      step: 3,
      run_id: '1ef2f516-e211-6e60-8d19-6a1309008ece',
      source: 'loop',
      writes: { respond_to_user: [Object] },
      user_id: '',
      graph_id: 'agent',
      thread_id: '15292d98-6aca-4df8-b2a1-f4508e11abb1',
      created_by: 'system',
      assistant_id: 'fe096781-5601-53d2-b2f6-0d3403f7e9ca'
    },
    created_at: '2024-06-20T22:07:06.852149+00:00',
    parent_config: {
      configurable: {
        thread_id: '15292d98-6aca-4df8-b2a1-f4508e11abb1',
        thread_ts: '1ef2f516-e6c7-6698-8002-80a6bedfa731'
      }
    }
  }

Our state variable contains a variety of important information. Here is a quick summary of the keys and what they represent:

• values contains the actual state values, so in our case you could call state['values']['messages'] or state['values']['user_info'] and get the actual values of each of the state variables.
• next tells us what action in the graph is next at the current state. Since we just finished running our graph and reached the end node, it is currently empty because there is no next action to take. However, if you go through the state at each point of the run you will see that the next value goes from __start__ → llm → get_user_info →respond_to_user .
• config tells us what the configuration of the state is. This is important for when we want to run a query starting at a previous state instead of the one we are at. An example of this is shown in the Invoking from a previous checkpoint section
• metadata stores the metadata associated with our state. This is data that is outside of the agent state, but is important to keep track of across multiple runs. An example of this is shown in the next section.
• created_at is information on the date and time the state was created at.
• parent_config is config for the previous step of the graph. Note that the previous step is not the previous run, but rather the previous node that the graph was at.

Invoking with Metadata

Let’s create a new thread to reset our state and start fresh.

In [ ]:

const thread_2 = await client.threads.create();

Now let’s add some metadata to our request. In this example we are going to treat each run of our assistant as a separate “node”. For each run, we will pass in a “node_id” as well as a “parent_node” in the metadata. This way we can easily go “back in time” and rerun our graph from a previous checkpoint.

NOTE: The reason we add this metadata instead of using the parent_config attribute is because parent_config tracks every individual step of a run, not the entire run itself.

In [ ]:

var messages = [{ role: "human", content: "My name is Bagatur and I am 26 years old." }];

await processStreamedMessages(client, thread_2, agent, messages, {"node_id": 1, "parent_node": null});

Using our run_input function makes it easy to pass in metadata and you can inspect the function as well as the API docs to see exactly how metadata gets passed.

We can continue our thread by creating a second node as follows:

In [ ]:

messages = [{ role: "human", content: "What is my name?" }];

await processStreamedMessages(client, thread_2, agent, messages, {"node_id": 2, "parent_node": 1});

Let's check our state to make sure the graph remembered our name on the second run:

In [ ]:

var state = await client.threads.getState(thread_2['thread_id']);
var graph_messages = state.values['messages'].map((message) => message.content);
console.log(graph_messages);

In [ ]:

[
    'My name is Bagatur and I am 26 years old.',
    '',
    'Hello Bagatur! How can I assist you today?',
    'What is my name?',
    '',
    'Your name is Bagatur. How can I assist you today, Bagatur?'
]

Perfect! The state persisted across separate runs, and the LLM remembers the name of our user. In a future we will explore non-sequential runs, i.e. not having each run just follow the last one but choosing which checkpoint we start our run from.

Querying and Updating the thread

Getting checkpoints by metadata

Let’s say we want to start a new run from a previous state (not the current state). This state lives somewhere in our history, so we can utilize the get_history function to try and find it.

In [ ]:

var all_history = await client.threads.getHistory(thread_2["thread_id"])

This is helpful for inspecting the specifics of our current thread, but remember that the history contains all the intermediate steps a graph takes. In our case, where the graph has 5 nodes (remember that Start and End both count as nodes), our history array grows quickly. Luckily, there is a way to query by using metadata. For example if we wanted to start a run from Node 1(from the example from above) we need to find the state from the end of run with metadata node_id:1 , which we can do like so:

In [ ]:

var node_1_history = await client.threads.getHistory(thread_2["thread_id"],{metadata: {'node_id':1}})

var node_1_end_of_run = history.filter(node => node.next.length === 0)[0];

Now let’s explore how we could use this information to create a new branch in our thread.

Invoking from a previous checkpoint

The following diagram describes what we would like to happen:

Basically, we want to have 3 runs of our graph, but instead of having them sequentially - we want both the second and third run to originate from the same state. We can do this by utilizing the code we used above, and passing additional metadata to our run.

In [ ]:

messages = [{ role: "human", content: "What is my age?" }];

await processStreamedMessages(client, thread_2, agent, messages, {"node_id": 3, "parent_node": 1, "thread_ts":node_1_end_of_run.checkpoint_id});

To check that everything actually worked as planned, let’s check our current state and check that message history to ensure that the message we passed to Node 2 is nowhere to be found.

In [ ]:

var state = await client.threads.getState(thread_2['thread_id']);
var graph_messages = state.values['messages'].map((message) => message.content);
console.log(graph_messages);

In [ ]:

[
    'My name is Bagatur and I am 26 years old.',
    '',
    'Hello Bagatur! How can I assist you today?',
    'What is my age?',
    '',
    'You are 26 years old, Bagatur. How can I assist you today?'
]

Great! This has worked as expected. Being able to go back to previous states and execute new graph runs from those checkpoints is a great way to develop flexible applications that don’t require reloading or restarting everything when an error is detected or a user changes their mind.

Updating/Patching the thread state

Lastly, let’s discuss the ability to manually change both the thread state as well as the metadata for a given state. Let’s say we incorrectly inputted data to the LLM and we want to rectify it.

Continuing our previous example, let’s say the user mistyped their age and we want to let the graph know that without actually running it. In this case we can rectify this by using update_state

In [ ]:

await client.threads.updateState(thread_2['thread_id'], { values: { "user_info": { "name": "Bagatur", "age": 35 } } });

Let’s make sure that the state did in fact update by checking the current state:

In [ ]:

var state = await client.threads.getState(thread_2['thread_id']);
console.log(state.values['user_info']);

In [ ]:

{ name: 'Bagatur', age: 35 }

Voila! The LLM knows our users age updated without us having to prompt it at all.

The last thing we will talk about is patching the thread, which is used when we want to update the metadata of a state. For example, say we actually wanted to update our last state to have node_id:4 instead of node_id:3. To do this, we can call:

In [ ]:

await client.threads.patchState(thread_2['thread_id'],{"node_id": 4});

We can check that this worked by checking the metadata of our state

In [ ]:

var state = await client.threads.getState(thread_2['thread_id']);
console.log(state.metadata['node_id']);

In [ ]:

4

Perfect! The patch worked as expected.