diff --git a/docs/docs/cloud/how-tos/clone_traces_studio.md b/docs/docs/cloud/how-tos/clone_traces_studio.md index 19ec75eeb..31cbbf10d 100644 --- a/docs/docs/cloud/how-tos/clone_traces_studio.md +++ b/docs/docs/cloud/how-tos/clone_traces_studio.md @@ -1,31 +1,36 @@ -# Testing local agents with remote traces +# Debug LangSmith traces -## Overview +This guide explains how to open LangSmith traces in LangGraph Studio for interactive investigation and debugging. -A common workflow when debugging production-deployed agents is to test the same thread against a local version of the same agent, which may have modifications. +## Open deployed threads -To support this, LangGraph Studio, in combination with LangSmith, allows you to clone remote threads traced in LangSmith into your locally running agent. This cloned thread can then be used to re-run specific nodes within Studio. +1. Open the LangSmith trace, selecting the root run. +2. Click "Run in Studio". -## Requirements +This will open LangGraph Studio connected to the associated LangGraph Platform deployment with the trace's parent thread selected. -!!! info "Prerequisites" +## Testing local agents with remote traces + +This section explains how to test a local agent against remote traces from LangSmith. This enables you to use production traces as input for local testing, allowing you to debug and verify agent modifications in your development environment. + +### Requirements + +- A LangSmith traced thread +- A locally running agent. See [here](../how-tos/studio/quick_start.md#local-development-server) for setup + instructions. + +!!! info "Local agent requirements" - langgraph>=0.3.18 - langgraph-api>=0.0.32 + - Contains the same set of nodes present in the remote trace -- A thread traced in LangSmith. -- A locally running agent. See [here](../how-tos/studio/quick_start.md#local-development-server) for setup instructions. - - Note that your local agent must be using the above specified `langgraph` and `langgraph-api` versions. - - The nodes present in the remote trace must exist in at least one of the graphs in your local agent. +### Cloning Thread -## Cloning Thread +1. Open the LangSmith trace, selecting the root run. +2. Click the dropdown next to "Run in Studio". +3. Enter your local agent's URL. +4. Select "Clone thread locally". +5. If multiple graphs exist, select the target graph. -First navigate to the LangSmith trace. Here you should see a button to "Run in Studio". - -![Run in Studio](img/run_in_studio.png){width=1200} - -This will prompt you to enter the url that your locally running agent is accessible at. Once provided, select "Clone thread locally". If you have multiple graphs in your agent, you will also be prompted to select a graph to clone this thread under. - -Once selected, a will a new thread in your local agent will be created and the thread history will be reconstruced to reflect the original trace. - -Alternatively, if your trace originates from an agent deployed on LangGraph Platform, you can "View original thread" to open Studio with the actual deployed thread. +A new thread will be created in your local agent with the thread history inferred and copied from the remote thread, and you will be navigated to LangGraph Studio for your locally running application. diff --git a/docs/docs/cloud/how-tos/datasets_studio.md b/docs/docs/cloud/how-tos/datasets_studio.md index 89a5114d4..9b474224a 100644 --- a/docs/docs/cloud/how-tos/datasets_studio.md +++ b/docs/docs/cloud/how-tos/datasets_studio.md @@ -1,17 +1,12 @@ -# Adding nodes as dataset examples in Studio +# Add node to dataset -In LangGraph Studio you can create dataset examples from the thread history in the right-hand pane. This can be especially useful when you want to evaluate intermediate steps of the agent. +This guide shows how to add examples to [LangSmith datasets](https://docs.smith.langchain.com/evaluation/how_to_guides#dataset-management) from nodes in the thread log. This is useful to evaluate indivudal steps of the agent. -1. Click on the `Add to Dataset` button to enter the dataset mode. -1. Select nodes which you want to add to dataset. -1. Select the target dataset to create the example in. - -You can edit the example payload before sending it to the dataset, which is useful if you need to make changes to conform the example to the dataset schema. - -Finally, you can customise the target dataset by clicking on the `Settings` button. +1. Select a thread. +2. Click on the `Add to Dataset` button. +3. Select nodes whose input/output you want to add to a dataset. +4. For each selected node, select the target dataset to create the example in. By default a dataset for the specific assistant and node will be selected. If this dataset does not yet exist, it will be created. +5. Edit the example's input/output as needed before adding it to the dataset. +6. Select "Add to dataset" at the bottom of the page to add all selected nodes to their respective datasets. See [Evaluating intermediate steps](https://docs.smith.langchain.com/evaluation/how_to_guides/langgraph#evaluating-intermediate-steps) for more details on how to evaluate intermediate steps. - - diff --git a/docs/docs/cloud/how-tos/invoke_studio.md b/docs/docs/cloud/how-tos/invoke_studio.md index 7751d1020..2cac1a7e8 100644 --- a/docs/docs/cloud/how-tos/invoke_studio.md +++ b/docs/docs/cloud/how-tos/invoke_studio.md @@ -1,19 +1,42 @@ -# How to manage Assistants +# Run application -!!! info "Prerequisites" +!!!info "Prerequisites" + - [Running agents](../../agents/run_agents.md#running-agents) - - [Assistants Overview](../../concepts/assistants.md) - -LangGraph Studio lets you view, edit, and update your assistants, and allows you to run your graph using these assistant configurations. +This guide shows how to submit a [run](../concepts/runs.md) to your application. ## Graph mode -To view your assistants, click the "Manage Assistants" button in the bottom left corner. +### Specify input +First define the input to your graph with in the "Input" section on the left side of the page, below the graph interface. -This opens a modal for you to view all the assistants for the selected graph. Specify the assistant and its version you would like to mark as "Active", and this assistant will be used when submitting runs. +Studio will attempt to render a form for your input based on the graph's defined [state schema](../../concepts/low_level.md/#schema). To disable this, click the "View Raw" button, which will present you with a JSON editor. + +Click the up/down arrows at the top of the "Input" section to toggle through and use previously submitted inputs. + +### Run settings + +#### Assistant + +To specify the [assistant](../../concepts/assistants.md) that is used for the run click the settings button in the bottom left corner. If an assistant is currently selected the button will also list the assistant name. If no assistant is selected it will say "Manage Assistants". + +Select the assistant to run and click the "Active" toggle at the top of the modal to activate it. [See here](./studio/manage_assistants.md) for more information on managing assistants. + +#### Streaming +Click the dropdown next to "Submit" and click the toggle to enable/disable streaming. + +### Submit run + +To submit the run with the specified input and run settings, click the "Submit" button. This will add a [run](../concepts/runs.md) to the existing selected [thread](../concepts/threads.md). If no thread is currently selected, a new one will be created. + +To cancel the ongoing run, click the "Cancel" button. -By default, the "Default configuration" option will be active. This option reflects the default configuration defined in your graph. Edits made to this configuration will be used to update the run-time configuration, but will not update or create a new assistant unless you click "Create new assistant". ## Chat mode +Specify the input to your chat application in the bottom of the conversation panel. Click the "Send message" button to submit the input as a Human message and have the response streamed back. -Chat mode enables you to switch through the different assistants in your graph via the dropdown selector at the top of the page. To create, edit, or delete assistants, use Graph mode. \ No newline at end of file +To cancel the ongoing run, click the "Cancel" button. Click the "Show tool calls" toggle to hide/show tool calls in the conversation. + +## Learn more + +To run your application from a specific checkpoint in an existing thread, see [this guide](./threads_studio.md#edit-thread-history). \ No newline at end of file diff --git a/docs/docs/cloud/how-tos/iterate_graph_studio.md b/docs/docs/cloud/how-tos/iterate_graph_studio.md index 1ff3734b0..490dca24c 100644 --- a/docs/docs/cloud/how-tos/iterate_graph_studio.md +++ b/docs/docs/cloud/how-tos/iterate_graph_studio.md @@ -1,23 +1,28 @@ -# Prompt Engineering in LangGraph Studio +# Iterate on prompts ## Overview -A central aspect of agent development is prompt engineering. LangGraph Studio makes it easy to iterate on the prompts used within your graph directly within the UI. +LangGraph Studio supports two methods for modifying prompts in your graph: direct node editing and the LangSmith Playground interface. -## Setup +## Direct Node Editing -The first step is to define your [configuration](https://langchain-ai.github.io/langgraph/how-tos/configuration/) such that LangGraph Studio is aware of the prompts you want to iterate on and which nodes they are associated with. +Studio allows you to edit prompts used inside individual nodes, directly from the graph interface. -### Reference +!!! info "Prerequisites" -When defining your configuration, you can use special metadata keys to instruct LangGraph Studio how to handle different fields. Here's a reference for the available configuration options: + - [Assistants overview](../../concepts/assistants.md) -#### `langgraph_nodes` +### Graph Configuration -- **Description**: Specifies which graph nodes a configuration field is associated with. +Define your [configuration](https://langchain-ai.github.io/langgraph/how-tos/configuration/) to specify prompt fields and their associated nodes using `langgraph_nodes` and `langgraph_type` keys. + +#### Configuration Reference + +##### `langgraph_nodes` + +- **Description**: Specifies which nodes of the graph a configuration field is associated with. - **Value Type**: Array of strings, where each string is the name of a node in your graph. - **Usage Context**: Include in the `json_schema_extra` dictionary for Pydantic models or the `metadata["json_schema_extra"]` dictionary for dataclasses. -- **Required**: No, but necessary if you want a field to be editable for specific nodes in the UI. - **Example**: ```python system_prompt: str = Field( @@ -26,14 +31,13 @@ When defining your configuration, you can use special metadata keys to instruct ) ``` -#### `langgraph_type` +##### `langgraph_type` - **Description**: Specifies the type of configuration field, which determines how it's handled in the UI. - **Value Type**: String - **Supported Values**: - `"prompt"`: Indicates the field contains prompt text that should be treated specially in the UI. - **Usage Context**: Include in the `json_schema_extra` dictionary for Pydantic models or the `metadata["json_schema_extra"]` dictionary for dataclasses. -- **Required**: No, but helpful for prompt fields to enable special handling. - **Example**: ```python system_prompt: str = Field( @@ -45,9 +49,7 @@ When defining your configuration, you can use special metadata keys to instruct ) ``` -### Example - -For example, if you have a node called `call_model` whose system prompt you want to iterate on, you can define a configuration like the following. +#### Example Configuration ```python ## Using Pydantic @@ -111,30 +113,22 @@ class Configuration: ``` -## Iterating on prompts +### Editing prompts in UI -### Node Configuration +1. Locate the gear icon on nodes with associated configuration fields +2. Click to open the configuration modal +3. Edit the values +4. Save to update the current assistant version or create a new one -With this set up, running your graph and viewing in LangGraph Studio will result in the graph rendering like such. +## LangSmith Playground -**Note the configuration icon in the top right corner of the `call_model` node**: +The [LangSmith Playground](https:// +docs.smith.langchain.com/prompt_engineering/how_to_guides#playground) interface allows testing individual LLM calls without running the full graph: -![Graph in Studio](img/studio_graph_with_configuration.png){width=1200} +1. Select a thread +2. Click "View LLM Runs" on a node. This lists all the LLM calls (if any) made inside the node. +3. Select an LLM run to open in Playground +4. Modify prompts and test different model and tool settings +5. Copy updated prompts back to your graph -Clicking this icon will open a modal where you can edit the configuration for all of the fields associated with the `call_model` node. From here, you can save your changes and apply them to the graph. Note that these values reflect the currently active assistant, and saving will update the assistant with the new values. - -![Configuration modal](img/studio_node_configuration.png){width=1200} - -### Playground - -LangGraph Studio also supports prompt engineering through an integration with the LangSmith Playground. To do so: - -1. Open an existing thread or create a new one. -2. Within the thread log, any nodes that have made an LLM call will have a "View LLM Runs" button. Clicking this will open a popover with the LLM runs for that node. -3. Select the LLM run you want to edit. This will open the LangSmith Playground with the selected LLM run. - -![Playground in Studio](img/studio_playground.png){width=1200} - -From here you can edit the prompt, test different model configurations and re-run just this LLM call without having to re-run the entire graph. When you are happy with your changes, you can copy the updated prompt back into your graph. - -For more information on how to use the LangSmith Playground, see the [LangSmith Playground documentation](https://docs.smith.langchain.com/prompt_engineering/how_to_guides#playground). +For advanced Playground features, click the expand button in the top right corner. diff --git a/docs/docs/cloud/how-tos/studio/faqs.md b/docs/docs/cloud/how-tos/studio/faqs.md index 4daa22564..706b71daf 100644 --- a/docs/docs/cloud/how-tos/studio/faqs.md +++ b/docs/docs/cloud/how-tos/studio/faqs.md @@ -47,4 +47,7 @@ def routing_function(state: GraphState) -> Literal["node_b","node_c"]: ## Why is my graph taking so long to startup? -The LangGraph Studio interacts with a local LangGraph API server. To stay aligned with ongoing updates, the LangGraph API requires regular rebuilding. As a result, you may occasionally experience slight delays when starting up your project. \ No newline at end of file +The LangGraph Studio interacts with a local LangGraph API server. To stay aligned with ongoing updates, the LangGraph API requires regular rebuilding. As a result, you may occasionally experience slight delays when starting up your project. + +## Can I use Studio without using LangSmith? +By default, LangGraph Studio is accessed from the LangSmith UI, within the LangGraph Platform Deployments tab. For local development, if you do not wish to have data traced to Langsmith, simply set `LANGSMITH_TRACING=false` in your application's `.env` file. With tracing disabled, no data will leave your local server. \ No newline at end of file diff --git a/docs/docs/cloud/how-tos/studio/manage_assistants.md b/docs/docs/cloud/how-tos/studio/manage_assistants.md new file mode 100644 index 000000000..43b60d485 --- /dev/null +++ b/docs/docs/cloud/how-tos/studio/manage_assistants.md @@ -0,0 +1,19 @@ +# Manage assistants + +!!! info "Prerequisites" + + - [Assistants Overview](../../concepts/assistants.md) + +LangGraph Studio lets you view, edit, and update your assistants, and allows you to run your graph using these assistant configurations. + +## Graph mode + +To view your assistants, click the "Manage Assistants" button in the bottom left corner. + +This opens a modal for you to view all the assistants for the selected graph. Specify the assistant and its version you would like to mark as "Active", and this assistant will be used when submitting runs. + +By default, the "Default configuration" option will be active. This option reflects the default configuration defined in your graph. Edits made to this configuration will be used to update the run-time configuration, but will not update or create a new assistant unless you click "Create new assistant". + +## Chat mode + +Chat mode enables you to switch through the different assistants in your graph via the dropdown selector at the top of the page. To create, edit, or delete assistants, use Graph mode. \ No newline at end of file diff --git a/docs/docs/cloud/how-tos/studio/quick_start.md b/docs/docs/cloud/how-tos/studio/quick_start.md index ca797b4ef..a1c75316c 100644 --- a/docs/docs/cloud/how-tos/studio/quick_start.md +++ b/docs/docs/cloud/how-tos/studio/quick_start.md @@ -7,13 +7,13 @@ LangGraph Studio supports connecting to two types of graphs: - Graphs deployed on [LangGraph Platform](../../../cloud/quick_start.md) - Graphs running locally via the [LangGraph Server](../../../tutorials/langgraph-platform/local-server.md). -## Deployed Application +## Deployed application For applications that are deployed on LangGraph Platform, you can access Studio as part of that deployment. To do so, navigate to the deployment in LangGraph Platform within the LangSmith UI and click the "LangGraph Studio" button. This will load the Studio UI connected to your live deployment, allowing you to create, read, and update the [threads](../../concepts/threads.md), [assistants](../../../concepts/assistants.md), and [memory](../../../concepts//memory.md) in that deployment. -## Local Development Server +## Local development server To test your locally running application using LangGraph Studio, ensure your application is set up following [this guide](https://langchain-ai.github.io/langgraph/cloud/deployment/setup/). @@ -47,7 +47,7 @@ If successful, you will see the following logs: Once running, you will automatically be directed to LangGraph Studio. -If your server is already running, to access Studio, either: +For an already running server, access Studio by either: 1. Directly navigate to the following URL: `https://smith.langchain.com/studio/?baseUrl=http://127.0.0.1:2024`. 2. Within LangSmith, navigate to the LangGraph Platform Deployments tab, click the "LangGraph Studio" button, enter `http://127.0.0.1:2024` and click "Connect". @@ -93,10 +93,11 @@ Then attach your preferred debugger: ## Next steps -See the following how-tos for more information on how to use Studio: +See the following guides for more information on how to use Studio: -- [How to manage Assistants](../invoke_studio.md) -- [How to manage Threads](../threads_studio.md) -- [How to create datasets](../datasets_studio.md) -- [How to prompt engineer](../iterate_graph_studio.md) -- [How to locally debug remote traces](../clone_traces_studio.md) +- [Run application](../invoke_studio.md) +- [Manage assistants](./manage_assistants.md) +- [Manage threads](../threads_studio.md) +- [Iterate on prompts](../iterate_graph_studio.md) +- [Debug LangSmith traces](../clone_traces_studio.md) +- [Add node to dataset](../datasets_studio.md) diff --git a/docs/docs/cloud/how-tos/threads_studio.md b/docs/docs/cloud/how-tos/threads_studio.md index 62ef864b0..da599cbbc 100644 --- a/docs/docs/cloud/how-tos/threads_studio.md +++ b/docs/docs/cloud/how-tos/threads_studio.md @@ -1,4 +1,4 @@ -# How to manage Threads +# Manage threads !!! info "Prerequisites" @@ -6,7 +6,7 @@ Studio allows you to view threads from the server and edit their state. -## View Threads +## View threads ### Graph mode @@ -19,11 +19,15 @@ Studio allows you to view threads from the server and edit their state. 1. View all threads in the right-hand pane of the page. 2. Click the plus button to create a new thread. -## Edit Thread State +## Edit thread history ### Graph mode -To edit the state of the thread, select "edit node state" next to the desired node. This enables you to edit the node's output and create a new fork of the thread history. For more information about time travel, [see here](../../concepts/time-travel.md). +To edit the state of the thread, select "edit node state" next to the desired node. Edit the node's output as desired and click "fork" to confirm. This will create a new forked run from the checkpoint of the selected node. + +If you instead want to re-run the thread from a given checkpoint without editing the state, click the "Re-run from here". This will again create a new forked run from the selected checkpoint. This is useful for re-running with changes that are not specific to the state, such as the selected assistant. + + For more information about time travel, [see here](../../concepts/time-travel.md). ### Chat mode diff --git a/docs/docs/concepts/langgraph_studio.md b/docs/docs/concepts/langgraph_studio.md index c447f8a0a..88af96126 100644 --- a/docs/docs/concepts/langgraph_studio.md +++ b/docs/docs/concepts/langgraph_studio.md @@ -17,23 +17,29 @@ LangGraph Studio is a specialized agent IDE that enables visualization, interact ## Features -The key features of LangGraph Studio are: +Key features of LangGraph Studio: - Visualize your graph architecture -- Run and interact with your agent in a GUI -- Create and manage [assistants](assistants.md) -- View and manage [threads](../cloud/concepts/threads.md) +- [Run and interact with your agent](../cloud/how-tos/invoke_studio.md) +- [Manage assistants](../cloud/how-tos/studio/manage_assistants.md.md) +- [Manage threads](../cloud/how-tos/threads_studio.md) +- [Iterate on prompts](../cloud/how-tos/iterate_graph_studio.md) - View and manage [long term memory](memory.md) - Debug agent state via [time travel](time-travel.md) - LangGraph Studio works for graphs that are deployed on [LangGraph Platform](../cloud/quick_start.md) or for graphs that are running locally via the [LangGraph Server](../tutorials/langgraph-platform/local-server.md). -LangGraph Studio supports two modes: +Studio supports two modes: -1. Graph -2. Chat +### Graph mode Graph mode exposes the full feature-set of Studio and is useful when you would like as many details about the execution of your agent, including the nodes traversed, intermediate states, and LangSmith integrations (such as adding to datasets an playground). -Chat mode is a simpler UI for iterating on and testing chat-specific agents. It is useful for business users and those who want to test overall agent behavior. \ No newline at end of file +### Chat mode + +Chat mode is a simpler UI for iterating on and testing chat-specific agents. It is useful for business users and those who want to test overall agent behavior. Chat mode is only supported for graph's whose state includes or extends [`MessagesState`](https://langchain-ai.github.io/langgraph/how-tos/graph-api/#messagesstate). + +## Learn more + +- See this guide on how to [get started](../cloud/how-tos/studio/quick_start.md) with LangGraph Studio. +- See [here](../cloud//how-tos/studio/faqs.md) for frequently asked questions. diff --git a/docs/mkdocs.yml b/docs/mkdocs.yml index 2abe4ca6e..56b416c80 100644 --- a/docs/mkdocs.yml +++ b/docs/mkdocs.yml @@ -178,10 +178,11 @@ nav: - Overview: concepts/langgraph_studio.md - Quickstart: cloud/how-tos/studio/quick_start.md - cloud/how-tos/invoke_studio.md + - cloud/how-tos/studio/manage_assistants.md - cloud/how-tos/threads_studio.md - - cloud/how-tos/datasets_studio.md - cloud/how-tos/iterate_graph_studio.md - cloud/how-tos/clone_traces_studio.md + - cloud/how-tos/datasets_studio.md - cloud/how-tos/studio/faqs.md - LangGraph SDK: concepts/sdk.md - Data management: