Merge branch 'main' into eugene/document_interrupt

This commit is contained in:
Eugene Yurtsev
2024-12-11 13:14:01 -05:00
committed by GitHub
17 changed files with 424 additions and 89 deletions
+12
View File
@@ -21,6 +21,18 @@ See the [how-to guide](../cloud/deployment/cloud.md#create-new-deployment) for c
| Development | 1 CPU | 1 GB | Up to 1 container |
| Production | 1 CPU | 2 GB | Up to 10 containers |
## Autoscaling
`Production` type deployments automatically scale up to 10 containers. Scaling is based on the current request load for a single container. Specifically, the autoscaling implementation scales the deployment so that each container is processing about 10 concurrent requests. For example...
- If the deployment is processing 20 concurrent requests, the deployment will scale up from 1 container to 2 containers (20 requests / 2 containers = 10 requests per container).
- If a deployment of 2 containers is processing 10 requests, the deployment will scale down from 2 containers to 1 container (10 requests / 1 container = 10 requests per container).
10 concurrent requests per container is the target threshold. However, 10 concurrent requests per container is not a hard limit. The number of concurrent requests can exceed 10 if there is a sudden burst of requests.
Scale down actions are delayed for 30 minutes before any action is taken. In other words, if the autoscaling implementation decides to scale down a deployment, it will first wait for 30 minutes before scaling down. After 30 minutes, the concurrency metric is recomputed and the deployment will scale down if the concurrency metric has met the target threshold. Otherwise, the deployment remains scaled up. This "cool down" period ensures that deployments do not scale up and down too frequently.
In the future, the autoscaling implementation may evolve to accommodate other metrics such as background run queue size.
## Revision
A revision is an iteration of a [deployment](#deployment). When a new deployment is created, an initial revision is automatically created. To deploy new code changes or update environment variable configurations for a deployment, a new revision must be created. When a revision is created, a new container image is built automatically.
+6
View File
@@ -353,6 +353,12 @@ def my_node(state: State) -> Command[Literal["my_other_node"]]:
Check out this [how-to guide](../how-tos/command.ipynb) for an end-to-end example of how to use `Command`.
### When should I use Command instead of conditional edges?
Use `Command` when you need to **both** update the graph state **and** route to a different node. For example, when implementing [multi-agent handoffs](./multi_agent.md#handoffs) where it's important to route to a different agent and pass some information to that agent.
Use [conditional edges](#conditional-edges) to route between nodes conditionally without updating the state.
### Using inside tools
A common use case is updating graph state from inside a tool. For example, in a customer support application you might want to look up customer information based on their account number or ID in the beginning of the conversation. To update the graph state from the tool, you can return `Command(update={"my_custom_key": "foo", "messages": [...]})` from the tool:
+1 -1
View File
@@ -153,7 +153,7 @@ network = builder.compile()
### Supervisor
In this architecture, we define agents as nodes and add a supervisor node (LLM) that decides which agent nodes should be called next. We use [conditional edges](./low_level.md#conditional-edges) to route execution to the appropriate agent node based on supervisor's decision. This architecture also lends itself well to running multiple agents in parallel or using [map-reduce](../how-tos/map-reduce.ipynb) pattern.
In this architecture, we define agents as nodes and add a supervisor node (LLM) that decides which agent nodes should be called next. We use [`Command`](./low_level.md#command) to route execution to the appropriate agent node based on supervisor's decision. This architecture also lends itself well to running multiple agents in parallel or using [map-reduce](../how-tos/map-reduce.ipynb) pattern.
```python
from typing import Literal
+7 -4
View File
@@ -254,7 +254,7 @@
"name": "stdout",
"output_type": "stream",
"text": [
"{'travel_advisor': {'messages': {'role': 'ai', 'content': 'The Caribbean is a fantastic choice for warm, sunny weather and beautiful beaches. Here are a few destinations you might consider:\\n\\n1. **Jamaica**: Known for its vibrant culture, reggae music, and stunning beaches like Negril and Montego Bay.\\n\\n2. **Bahamas**: With over 700 islands, the Bahamas offers clear turquoise waters and beautiful sandy beaches, perfect for relaxation and water sports.\\n\\n3. **Dominican Republic**: Known for its resorts, beaches, and golfing. Punta Cana and Puerto Plata are popular destinations.\\n\\n4. **Barbados**: Offers beautiful beaches and a rich history, with plenty of activities and festivals.\\n\\n5. **Puerto Rico**: A mix of Spanish heritage and modern resorts, with opportunities for hiking in El Yunque National Forest and enjoying the vibrant nightlife in San Juan.\\n\\n6. **Aruba**: Known for its dry climate and sunny days, with beautiful beaches and activities like snorkeling and diving.\\n\\nEach of these destinations has its own unique charm and appeal. If you need specific sightseeing or hotel recommendations, let me know!', 'name': 'travel_advisor'}}}\n",
"{'travel_advisor': {'messages': {'role': 'ai', 'content': 'The Caribbean offers many warm destinations perfect for a relaxing getaway. Consider visiting Jamaica for its beautiful beaches and vibrant culture, the Bahamas for its stunning islands and clear waters, or the Dominican Republic for its all-inclusive resorts and rich history. Let me know if you need more information on sightseeing or hotel recommendations!', 'name': 'travel_advisor'}}}\n",
"\n",
"\n"
]
@@ -286,10 +286,13 @@
"name": "stdout",
"output_type": "stream",
"text": [
"{'travel_advisor': {'messages': {'role': 'ai', 'content': \"I recommend visiting Barbados for a warm Caribbean getaway. It's known for its beautiful beaches, vibrant culture, and friendly locals. Let me gather some sightseeing and hotel recommendations for you.\", 'name': 'travel_advisor'}}}\n",
"{'travel_advisor': {'messages': {'role': 'ai', 'content': 'I recommend visiting Jamaica, a beautiful Caribbean island known for its warm climate, stunning beaches, and vibrant culture.', 'name': 'travel_advisor'}}}\n",
"\n",
"\n",
"{'sightseeing_advisor': {'messages': {'role': 'ai', 'content': \"Barbados is a fantastic destination to experience the warmth of the Caribbean. Here are some things to do and places to stay:\\n\\n### Sightseeing Recommendations:\\n1. **Harrison's Cave**: Explore this stunning limestone cave with its impressive stalactites and stalagmites. \\n2. **Bathsheba Beach**: Known for its unique rock formations and surf-friendly waves, it's perfect for a day of relaxation and exploration.\\n3. **St. Nicholas Abbey**: Visit this historic plantation house, distillery, and museum for a glimpse into the island's colonial past.\\n4. **Animal Flower Cave**: Located at the northern tip of Barbados, this sea cave offers incredible views and natural rock pools.\\n5. **Oistins Fish Fry**: Experience local culture and cuisine with fresh seafood, music, and dancing every Friday night.\\n\\n### Hotel Recommendations:\\n1. **Sandy Lane**: A luxurious resort offering world-class amenities, golf courses, and a private beach.\\n2. **The Crane Resort**: Known for its historic charm and stunning ocean views, it provides a unique blend of luxury and culture.\\n3. **Sea Breeze Beach House**: Offers an all-inclusive experience with multiple dining options and beachfront access.\\n4. **The House by Elegant Hotels**: A boutique, adults-only hotel perfect for a romantic getaway with personalized service and beachfront location.\\n\\nEnjoy your trip to Barbados!\", 'name': 'sightseeing_advisor'}}}\n",
"{'sightseeing_advisor': {'messages': {'role': 'ai', 'content': \"Jamaica is a fantastic choice for a warm Caribbean getaway. Here are some top things to do while you're there:\\n\\n1. **Dunn's River Falls**: Located near Ocho Rios, this is one of Jamaica's most famous waterfalls. You can climb the falls, swim in the refreshing pools, or simply enjoy the beautiful surroundings.\\n\\n2. **Seven Mile Beach**: Located in Negril, this is one of the most beautiful beaches in the Caribbean. It's perfect for sunbathing, swimming, and enjoying water sports.\\n\\n3. **Bob Marley Museum**: Situated in Kingston, this museum is dedicated to the life and legacy of the reggae legend Bob Marley and is a must-visit for music lovers.\\n\\n4. **Blue Mountains**: Go hiking or take a tour to explore the Blue Mountains, where you can enjoy breathtaking views and taste some of the world's best coffee.\\n\\n5. **Luminous Lagoon**: Experience the natural wonder of the Luminous Lagoon in Falmouth, where the water glows at night due to bioluminescent microorganisms.\\n\\nFor hotel recommendations, I suggest checking with a hotel advisor for the best options that suit your budget and preferences.\", 'name': 'sightseeing_advisor'}}}\n",
"\n",
"\n",
"{'hotel_advisor': {'messages': {'role': 'ai', 'content': 'For hotel recommendations in Jamaica, here are a few options across different areas: \\n\\n1. **Sandals Montego Bay** (Montego Bay): A luxurious all-inclusive resort ideal for couples, offering beautiful beachfront views and a variety of dining options.\\n\\n2. **Half Moon Resort** (Montego Bay): A family-friendly resort with a private beach, golf course, and various activities for all ages.\\n\\n3. **Jamaica Inn** (Ocho Rios): A charming boutique hotel known for its excellent service and tranquil atmosphere.\\n\\n4. **The Caves** (Negril): A unique and romantic cliff-side resort offering stunning ocean views and intimate dining experiences.\\n\\n5. **Trident Hotel** (Port Antonio): A luxurious and contemporary hotel offering privacy, elegance, and beautiful views of the Caribbean Sea.\\n\\nThese options cater to different tastes and budgets, ensuring a comfortable and enjoyable stay in Jamaica.', 'name': 'hotel_advisor'}}}\n",
"\n",
"\n"
]
@@ -558,7 +561,7 @@
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.12.3"
"version": "3.11.9"
}
},
"nbformat": 4,
@@ -31,7 +31,7 @@
" # update the state keys\n",
" \"user_info\": user_info,\n",
" # update the message history\n",
" \"messages\": [ToolMessage(\"Successfully looked up user information\", tool_call_id=\"\")]\n",
" \"messages\": [ToolMessage(\"Successfully looked up user information\", tool_call_id=tool_call_id)]\n",
" }\n",
" )\n",
"```\n",