Merge branch 'main' into get-started

docs/docs/cloud/deployment/cloud.md

@@ -62,6 +62,15 @@ Starting from the `LangGraph Platform` view...
 1. In the panel, select the `Server` tab to view server logs for the revision. Server logs are only available after a revision has been deployed.
 1. Within the `Server` tab, adjust the date/time range picker as needed. By default, the date/time range picker is set to the `Last 7 days`.
 
+## View Deployment Metrics
+
+Starting from the <a href="https://smith.langchain.com/" target="_blank">LangSmith UI</a>...
+
+1. In the left-hand navigation panel, select `LangGraph Platform`. The `LangGraph Platform` view contains a list of existing LangGraph Platform deployments.
+1. Select an existing deployment to monitor.
+1. Select the `Monitoring` tab to view the deployment metrics. See a list of [all available metrics](../../concepts/langgraph_control_plane.md#monitoring).
+1. Within the `Monitoring` tab, use the date/time range picker as needed. By default, the date/time range picker is set to the `Last 15 minutes`.
+
 ## Interrupt Revision
 
 Interrupting a revision will stop deployment of the revision.

docs/docs/cloud/how-tos/studio/run_evals.md

@@ -0,0 +1,57 @@
+# Run experiments over a dataset
+
+LangGraph Studio supports evaluations by allowing you to run your assistant over a pre-defined LangSmith dataset. This enables you to understand how your application performs over a variety of inputs, compare the results to reference outputs, and score the results using [evaluators](../../../agents/evals.md).
+
+This guide shows you how to run an experiment end-to-end from Studio.
+
+---
+
+## Prerequisites
+
+Before running an experiment, ensure you have the following:
+
+1.  **A LangSmith dataset**: Your dataset should contain the inputs you want to test and optionally, reference outputs for comparison.
+
+    - The schema for the inputs must match the required input schema for the assistant. For more information on schemas, see [here](../../../concepts/low_level.md#schema).
+    - For more on creating datasets, see [How to Manage Datasets](https://docs.smith.langchain.com/evaluation/how_to_guides/manage_datasets_in_application#set-up-your-dataset).
+
+2.  **(Optional) Evaluators**: You can attach evaluators (e.g., LLM-as-a-Judge, heuristics, or custom functions) to your dataset in LangSmith. These will run automatically after the graph has processed all inputs.
+
+    - To learn more, read about [Evaluation Concepts](https://docs.smith.langchain.com/evaluation/concepts#evaluators).
+
+3.  **A running application**: The experiment can be run against:
+    - An application deployed on [LangGraph Platform](../../quick_start.md).
+    - A locally running application started via the [langgraph-cli](../../../tutorials/langgraph-platform/local-server.md).
+
+---
+
+## Step-by-step guide
+
+### 1. Launch the experiment
+
+Click the **Run experiment** button in the top right corner of the Studio page.
+
+### 2. Select your dataset
+
+In the modal that appears, select the dataset (or a specific dataset split) to use for the experiment and click **Start**.
+
+### 3. Monitor the progress
+
+All of the inputs in the dataset will now be run against the active assistant. Monitor the experiment's progress via the badge in the top right corner.
+
+You can continue to work in Studio while the experiment runs in the background. Click the arrow icon button at any time to navigate to LangSmith and view the detailed experiment results.
+
+---
+
+## Troubleshooting
+
+### "Run experiment" button is disabled
+
+If the "Run experiment" button is disabled, check the following:
+
+- **Deployed application**: If your application is deployed on LangGraph Platform, you may need to create a new revision to enable this feature.
+- **Local development server**: If you are running your application locally, make sure you have upgraded to the latest version of the `langgraph-cli` (`pip install -U langgraph-cli`). Additionally, ensure you have tracing enabled by setting the `LANGSMITH_API_KEY` in your project's `.env` file.
+
+### Evaluator results are missing
+
+When you run an experiment, any attached evaluators are scheduled for execution in a queue. If you don't see results immediately, it likely means they are still pending.

docs/docs/concepts/langgraph_control_plane.md

@@ -19,6 +19,7 @@ From the control plane UI, you can:
 - Update a deployment.
 - Update environment variables for a deployment.
 - View build and server logs of a deployment.
+- View deployment metrics like CPU and memory usage.
 - Delete a deployment.
 
 The Control Plane UI is embedded in [LangSmith](https://docs.smith.langchain.com/langgraph_cloud).
@@ -88,6 +89,15 @@ Infrastructure for deployments and revisions are provisioned and deployed asynch
 
 The control plane and [LangGraph Data Plane](./langgraph_data_plane.md) "listener" application coordinate to achieve asynchronous deployments.
 
+### Monitoring
+
+After a deployment is ready, the control plane monitors the deployment and records various metrics, such as:
+
+- CPU and memory usage of the deployment.
+- Number of container restarts.
+
+These metrics are displayed as charts in the Control Plane UI.
+
 ### LangSmith Integration
 
 A [LangSmith](https://docs.smith.langchain.com/) tracing project is automatically created for each deployment. The tracing project has the same name as the deployment. When creating a deployment, the `LANGCHAIN_TRACING` and `LANGSMITH_API_KEY`/`LANGCHAIN_API_KEY` environment variables do not need to be specified; they are set automatically by the control plane.

docs/docs/concepts/langgraph_studio.md

@@ -24,6 +24,7 @@ Key features of LangGraph Studio:
 - [Manage assistants](../cloud/how-tos/studio/manage_assistants.md)
 - [Manage threads](../cloud/how-tos/threads_studio.md)
 - [Iterate on prompts](../cloud/how-tos/iterate_graph_studio.md)
+- [Run experiments over a dataset](../cloud/how-tos/studio/run_evals.md)
 - Manage [long term memory](memory.md)
 - Debug agent state via [time travel](time-travel.md)
 
@@ -41,4 +42,4 @@ Chat mode is a simpler UI for iterating on and testing chat-specific agents. It
 
 ## Learn more
 
-- See this guide on how to [get started](../cloud/how-tos/studio/quick_start.md) with LangGraph Studio.
+- See this guide on how to [get started](../cloud/how-tos/studio/quick_start.md) with LangGraph Studio.

docs/mkdocs.yml

@@ -195,13 +195,14 @@ nav:
          - Standalone Container: cloud/deployment/standalone_container.md
     - Evaluation:
       - Basic implementation: agents/evals.md
-    - Commercial-only capabilities:
+    - Platform capabilities:
       - LangGraph Studio:
         - Quickstart: cloud/how-tos/studio/quick_start.md
         - cloud/how-tos/invoke_studio.md
         - cloud/how-tos/studio/manage_assistants.md
         - cloud/how-tos/threads_studio.md
         - cloud/how-tos/iterate_graph_studio.md
+        - cloud/how-tos/studio/run_evals.md
         - cloud/how-tos/clone_traces_studio.md
         - cloud/how-tos/datasets_studio.md
       - Authentication & access control: