diff --git a/examples/llm-compiler/img/llm-compiler.png b/examples/llm-compiler/img/llm-compiler.png new file mode 100644 index 000000000..3c9ec7a57 Binary files /dev/null and b/examples/llm-compiler/img/llm-compiler.png differ diff --git a/examples/plan-and-execute/img/plan-and-execute.png b/examples/plan-and-execute/img/plan-and-execute.png new file mode 100644 index 000000000..967f2cf56 Binary files /dev/null and b/examples/plan-and-execute/img/plan-and-execute.png differ diff --git a/examples/rewoo/img/rewoo-paper-workflow.png b/examples/rewoo/img/rewoo-paper-workflow.png new file mode 100644 index 000000000..cdfc7261f Binary files /dev/null and b/examples/rewoo/img/rewoo-paper-workflow.png differ diff --git a/examples/rewoo/img/rewoo.png b/examples/rewoo/img/rewoo.png new file mode 100644 index 000000000..48af25d57 Binary files /dev/null and b/examples/rewoo/img/rewoo.png differ diff --git a/examples/rewoo/rewoo.ipynb b/examples/rewoo/rewoo.ipynb index 092207771..06e361074 100644 --- a/examples/rewoo/rewoo.ipynb +++ b/examples/rewoo/rewoo.ipynb @@ -1,30 +1,136 @@ { "cells": [ { - "cell_type": "code", - "execution_count": 23, - "id": "b3db60ea-d38b-4a55-a048-6c0222af6566", + "cell_type": "markdown", + "id": "1c161710-fc66-426f-8c96-28440b9c9626", "metadata": {}, - "outputs": [], "source": [ - "from langchain_openai import ChatOpenAI\n", - "from langchain_community.tools.tavily_search import TavilySearchResults\n", - "search = TavilySearchResults()" + "# Reasoning Without Observation\n", + "\n", + "In [ReWOO](https://arxiv.org/abs/2305.18323), Xu, et. al, propose an agent that combines a multi-step planner and variable substitution for effective tool use. It was designed to improve on the ReACT-style agent architecture in the following ways:\n", + "\n", + "1. Reduce token consumption and execution time by generating the full chain of tools used in a single pass. (_ReACT-style agent architecture requires many LLM calls with redundant prefixes (since the system prompt and previous steps are provided to the LLM for each reasoning step_)\n", + "2. Simplify the fine-tuning process. Since the planning data doesn't depend on the outputs of the tool, models can be fine-tuned without actually invoking the tools (in theory).\n", + "\n", + "\n", + "![ReWoo Diagram](./img/rewoo.png)\n", + "\n", + "ReWOO is made of 3 modules:\n", + "\n", + "1. 🧠**Planner**: Generate the plan in the following format:\n", + "```text\n", + "Plan: \n", + "#E1 = Tool[argument for tool]\n", + "Plan: \n", + "#E2 = Tool[argument for tool with #E1 variable substitution]\n", + "...\n", + "```\n", + "3. **Worker**: executes the tool with the provided arguments.\n", + "4. 🧠**Solver**: generates the answer for the initial task based on the tool observations.\n", + "\n", + "The modules with a 🧠 emoji depend on an LLM call. Notice that we avoid redundant calls to the planner LLM by using variable substitution.\n", + "\n", + "In this example, each module is represented by a LangGraph node. Let's get started!\n", + "\n", + "## 0. Prerequisites\n", + "\n", + "For this example, we will provide the agent with a Tavily search engine tool. You can get an API key [here](https://app.tavily.com/sign-in) or replace with a free tool option (e.g., [duck duck go search](https://python.langchain.com/docs/integrations/tools/ddg)).\n", + "\n", + "To see the full langsmith trace, you can s" ] }, { "cell_type": "code", - "execution_count": 10, + "execution_count": 1, + "id": "7f52bded-9d23-4826-8bfc-20b0d3a51182", + "metadata": {}, + "outputs": [], + "source": [ + "# %pip install -U langgraph langchain_community langchain_openai tavily-python" + ] + }, + { + "cell_type": "code", + "execution_count": 2, + "id": "4215f9fb-71ff-4d88-8484-f73174db5592", + "metadata": {}, + "outputs": [], + "source": [ + "import os\n", + "import getpass\n", + "\n", + "def _set_if_undefined(var: str):\n", + " if not os.environ.get(var):\n", + " os.environ[var] = getpass.getpass(f\"{var}=\")\n", + "\n", + "os.environ[\"LANGCHAIN_TRACING_V2\"] = \"true\"\n", + "os.environ[\"LANGCHAIN_PROJECT\"] = \"ReWOO\"\n", + "_set_if_undefined(\"TAVILY_API_KEY\")\n", + "_set_if_undefined(\"LANGCHAIN_API_KEY\")\n", + "os.environ['LANGCHAIN_API_KEY'] = \"ls__2d5a3fe2b3af4f4db79501c906e1c072\"\n", + "_set_if_undefined(\"OPENAI_API_KEY\")" + ] + }, + { + "cell_type": "markdown", + "id": "55239a14-a14d-4117-adb5-07199e1e5e16", + "metadata": {}, + "source": [ + "**Graph State**: In LangGraph, every node updates a shared graph state. The state is the input to any node whenever it is invoked.\n", + "\n", + "Below, we will define a state dict to contain the task, plan, steps, and other variables." + ] + }, + { + "cell_type": "code", + "execution_count": 3, + "id": "9a92c875-c20b-4b7e-9d88-61c62382f8e2", + "metadata": {}, + "outputs": [], + "source": [ + "from typing import TypedDict, List\n", + "class ReWOO(TypedDict):\n", + " task: str\n", + " plan_string: str\n", + " steps: List\n", + " results: dict\n", + " result: str" + ] + }, + { + "cell_type": "markdown", + "id": "997f9181-41c0-4c44-937d-94bd3946a929", + "metadata": {}, + "source": [ + "## 1. Planner\n", + "\n", + "The planner prompts an LLM to generate a plan in the form of a task list. The arguments to each task are strings that may contain special variables (`#E{0-9}+`) that are used for variable subtitution from other task results.\n", + "\n", + "\n", + "![ReWOO workflow](./img/rewoo-paper-workflow.png)\n", + "\n", + "Our example agent will have two tools: \n", + "1. Google - a search engine (in this case Tavily)\n", + "2. LLM - an LLM call to reason about previous outputs.\n", + "\n", + "The LLM tool receives less of the prompt context and so can be more token-efficient than the ReACT paradigm." + ] + }, + { + "cell_type": "code", + "execution_count": 4, "id": "c8836921-c89e-42b6-8c71-27aeaeac5368", "metadata": {}, "outputs": [], "source": [ + "from langchain_openai import ChatOpenAI\n", + "\n", "model = ChatOpenAI(temperature=0)" ] }, { "cell_type": "code", - "execution_count": 11, + "execution_count": 5, "id": "7e7faa92-30a1-4942-b3c7-acd3a7bfccbc", "metadata": {}, "outputs": [], @@ -57,7 +163,7 @@ }, { "cell_type": "code", - "execution_count": 12, + "execution_count": 6, "id": "72b4ab0f-7215-4f4b-9407-0ebad8b13b92", "metadata": {}, "outputs": [], @@ -67,7 +173,7 @@ }, { "cell_type": "code", - "execution_count": 14, + "execution_count": 7, "id": "56ecb45b-ea76-4303-a4f3-51406fe8312a", "metadata": {}, "outputs": [], @@ -77,7 +183,7 @@ }, { "cell_type": "code", - "execution_count": 18, + "execution_count": 8, "id": "8a733caa-d75b-422c-93aa-6ad913c995f3", "metadata": {}, "outputs": [ @@ -104,56 +210,78 @@ ] }, { - "cell_type": "code", - "execution_count": 27, - "id": "8b14c889-0f96-40d3-9200-7dc9afc190ce", + "cell_type": "markdown", + "id": "37166985-5bec-4615-bd40-54d16fd7b4ea", "metadata": {}, - "outputs": [], "source": [ + "#### Planner Node\n", "\n", - "from typing import TypedDict, Annotated, List\n", - "import operator" + "To connect the planner to our graph, we will create a `get_plan` node that accepts the `ReWOO` state and returns with a state update for the\n", + "`steps` and `plan_string` fields." ] }, { "cell_type": "code", - "execution_count": 92, - "id": "a3a0a670-686b-4767-92b7-b16fca748730", + "execution_count": 11, + "id": "f9f042b6-90d8-430f-abf3-04ad2bb047c7", "metadata": {}, "outputs": [], "source": [ - "class ReWOO(TypedDict):\n", - " task: str\n", - " plan_string: str\n", - " steps: List\n", - " results: dict\n", - " result: str" + "import re\n", + "from langchain_core.prompts import ChatPromptTemplate\n", + "# Regex to match expressions of the form E#... = ...[...]\n", + "regex_pattern = r\"Plan:\\s*(.+)\\s*(#E\\d+)\\s*=\\s*(\\w+)\\s*\\[([^\\]]+)\\]\"\n", + "prompt_template = ChatPromptTemplate.from_messages([(\"user\", prompt)]) \n", + "planner = prompt_template| model\n", + "\n", + "def get_plan(state: ReWOO):\n", + " task = state[\"task\"]\n", + " result = planner.invoke({\"task\": task})\n", + " # Find all matches in the sample text\n", + " matches = re.findall(regex_pattern, result.content)\n", + " return {\"steps\": matches, \"plan_string\": result.content}" + ] + }, + { + "cell_type": "markdown", + "id": "0d97942f-27d3-4761-b6cc-6614dbb90c77", + "metadata": {}, + "source": [ + "## 2. Executor\n", + "\n", + "The executor receives the plan and executes the tools in sequence.\n", + "\n", + "Below, instantiate the search engine and define the toole execution node." ] }, { "cell_type": "code", - "execution_count": 93, + "execution_count": 12, "id": "3412cfc4-6796-4295-aea4-7eeb304e10bd", "metadata": {}, "outputs": [], "source": [ - "def _get_current_task(state):\n", + "from langchain_community.tools.tavily_search import TavilySearchResults\n", + "search = TavilySearchResults()" + ] + }, + { + "cell_type": "code", + "execution_count": 13, + "id": "aa96fbac-28bc-4afe-ae35-ddb3383d1147", + "metadata": {}, + "outputs": [], + "source": [ + "def _get_current_task(state: ReWOO):\n", " if state[\"results\"] is None:\n", " return 1\n", " if len(state[\"results\"]) == len(state[\"steps\"]):\n", " return None\n", " else:\n", - " return len(state[\"results\"]) + 1" - ] - }, - { - "cell_type": "code", - "execution_count": 110, - "id": "aa96fbac-28bc-4afe-ae35-ddb3383d1147", - "metadata": {}, - "outputs": [], - "source": [ - "def tool_execution(state):\n", + " return len(state[\"results\"]) + 1\n", + " \n", + "def tool_execution(state: ReWOO):\n", + " \"\"\"Worker node that executes the tools of a given plan.\"\"\"\n", " _step =_get_current_task(state)\n", " _, step_name, tool, tool_input = state[\"steps\"][_step - 1]\n", " _results = state[\"results\"] or {}\n", @@ -170,36 +298,19 @@ ] }, { - "cell_type": "code", - "execution_count": 111, - "id": "c0eba982-6b8c-4ad9-8971-10c18d7c9b75", + "cell_type": "markdown", + "id": "28e20b31-d721-470d-94d2-db0c177fae75", "metadata": {}, - "outputs": [], "source": [ - "import re\n", - "# Regex to match expressions of the form E#... = ...[...]\n", - "regex_pattern = r\"Plan:\\s*(.+)\\s*(#E\\d+)\\s*=\\s*(\\w+)\\s*\\[([^\\]]+)\\]\"" + "## 3. Solver\n", + "\n", + "The solver receives the full plan and generates the final response based on the responses of the tool calls from the worker." ] }, { "cell_type": "code", - "execution_count": 112, - "id": "2791b020-c182-4b54-8912-f9e3396196b8", - "metadata": {}, - "outputs": [], - "source": [ - "def get_plan(state):\n", - " task = state[\"task\"]\n", - " result = model.invoke(prompt.format(task=task))\n", - " # Find all matches in the sample text\n", - " matches = re.findall(regex_pattern, result.content)\n", - " return {\"steps\": matches, \"plan_string\": result.content}" - ] - }, - { - "cell_type": "code", - "execution_count": 113, - "id": "9f60d611-9e15-4621-8c03-aabace7d2a8b", + "execution_count": 14, + "id": "0a4d9851-8590-42be-8c53-9969ebff85f4", "metadata": {}, "outputs": [], "source": [ @@ -214,7 +325,8 @@ "\n", "Task: {task}\n", "Response:\"\"\"\n", - "def solve(state):\n", + "\n", + "def solve(state: ReWOO):\n", " plan = \"\"\n", " for _plan, step_name, tool, tool_input in state[\"steps\"]:\n", " _results = state[\"results\"] or {}\n", @@ -226,9 +338,19 @@ " return {\"result\": result.content}" ] }, + { + "cell_type": "markdown", + "id": "8ce26c3f-6ced-4a91-a9f2-d0bc235e4010", + "metadata": {}, + "source": [ + "## 4. Define Graph\n", + "\n", + "Our graph defines the workflow. Each of the planner, tool executor, and solver modules are added as nodes." + ] + }, { "cell_type": "code", - "execution_count": 114, + "execution_count": 15, "id": "73b235d7-fa83-4e84-9f2e-2908f16deb26", "metadata": {}, "outputs": [], @@ -236,14 +358,16 @@ "def _route(state):\n", " _step = _get_current_task(state)\n", " if _step is None:\n", + " # We have executed all tasks\n", " return \"solve\"\n", " else:\n", + " # We are still executing tasks, loop back to the \"tool\" node\n", " return \"tool\"" ] }, { "cell_type": "code", - "execution_count": 115, + "execution_count": 16, "id": "cf173aa1-ce31-4dca-8111-30c91e209652", "metadata": {}, "outputs": [], @@ -258,12 +382,13 @@ "graph.add_edge(\"solve\", END)\n", "graph.add_conditional_edges(\"tool\", _route)\n", "graph.set_entry_point(\"plan\")\n", + "\n", "app = graph.compile()" ] }, { "cell_type": "code", - "execution_count": 116, + "execution_count": 18, "id": "badaca52-5d55-433f-8770-1bd50c10bf7f", "metadata": {}, "outputs": [ @@ -272,27 +397,59 @@ "output_type": "stream", "text": [ "{'plan': {'steps': [('Use Google to search for the 2024 Australian Open winner.', '#E1', 'Google', '2024 Australian Open winner'), ('Retrieve the name of the 2024 Australian Open winner from the search results.', '#E2', 'LLM', 'What is the name of the 2024 Australian Open winner, given #E1'), ('Use Google to search for the hometown of the 2024 Australian Open winner.', '#E3', 'Google', 'hometown of 2024 Australian Open winner, given #E2'), ('Retrieve the hometown of the 2024 Australian Open winner from the search results.', '#E4', 'LLM', 'What is the hometown of the 2024 Australian Open winner, given #E3')], 'plan_string': 'Plan: Use Google to search for the 2024 Australian Open winner.\\n#E1 = Google[2024 Australian Open winner]\\n\\nPlan: Retrieve the name of the 2024 Australian Open winner from the search results.\\n#E2 = LLM[What is the name of the 2024 Australian Open winner, given #E1]\\n\\nPlan: Use Google to search for the hometown of the 2024 Australian Open winner.\\n#E3 = Google[hometown of 2024 Australian Open winner, given #E2]\\n\\nPlan: Retrieve the hometown of the 2024 Australian Open winner from the search results.\\n#E4 = LLM[What is the hometown of the 2024 Australian Open winner, given #E3]'}}\n", + "---\n", "{'tool': {'results': {'#E1': '[{\\'url\\': \\'https://www.cbssports.com/tennis/news/australian-open-2024-jannik-sinner-aryna-sabalenka-crowned-as-grand-slam-singles-champions-at-melbourne-park/\\', \\'content\\': \\'2024 Australian Open odds, Sinner vs. Medvedev picks Sabalenka defeats Zheng to win 2024 Australian Open Australian Open 2024: Jannik Sinner, Aryna Sabalenka crowned as Grand Slam singles champions at Melbourne Park 2024 Australian Open odds, Sabalenka vs. Zheng picks 2024 Australian Open odds, Medvedev vs. Zverev picks Sinner, Sabalenka win Australian Open singles titles Sinner makes epic comeback to win Australian OpenJan 28, 2024 — Jan 28, 2024Australian Open 2024: Jannik Sinner, Aryna Sabalenka crowned as Grand Slam singles champions at Melbourne Park ... Watch Now: Jannik Sinner came\\\\xa0...\\'}, {\\'url\\': \\'https://en.wikipedia.org/wiki/2024_Australian_Open\\', \\'content\\': \"Contents 2024 Australian Open The 2024 Australian Open was a Grand Slam level tennis tournament held at Melbourne Park, from 14–28 January 2024.[1] The Australian Open total prize money for 2024 increased by 13.07% year on year to a tournament record A$86,500,000. In the tournament\\'s 119-year history, this was the first Australian Open Tennis Championships to be held on an openingNovak Djokovic was the defending men\\'s singles champion. ... He was defeated in the semifinals by Jannik Sinner, who went on to beat Daniil Medvedev in a five-set\\\\xa0...\"}, {\\'url\\': \\'https://en.wikipedia.org/wiki/2024_Australian_Open_%E2%80%93_Men%27s_singles\\', \\'content\\': \"Contents 2024 Australian Open – Men\\'s singles The entry list was released by Tennis Australia based on the ATP rankings for the week of 4 December 2023.[15] matches, tying the Open Era record set at the 1983 US Open.[14] feature any of the Big Three members.[4] It was the second time Medvedev lost the Australian Open final after winningJannik Sinner defeated Daniil Medvedev in the final, 3–6, 3–6, 6–4, 6–4, 6–3, to win the men\\'s singles tennis title at the 2024 Australian Open.\"}]'}}}\n", + "---\n", "{'tool': {'results': {'#E1': '[{\\'url\\': \\'https://www.cbssports.com/tennis/news/australian-open-2024-jannik-sinner-aryna-sabalenka-crowned-as-grand-slam-singles-champions-at-melbourne-park/\\', \\'content\\': \\'2024 Australian Open odds, Sinner vs. Medvedev picks Sabalenka defeats Zheng to win 2024 Australian Open Australian Open 2024: Jannik Sinner, Aryna Sabalenka crowned as Grand Slam singles champions at Melbourne Park 2024 Australian Open odds, Sabalenka vs. Zheng picks 2024 Australian Open odds, Medvedev vs. Zverev picks Sinner, Sabalenka win Australian Open singles titles Sinner makes epic comeback to win Australian OpenJan 28, 2024 — Jan 28, 2024Australian Open 2024: Jannik Sinner, Aryna Sabalenka crowned as Grand Slam singles champions at Melbourne Park ... Watch Now: Jannik Sinner came\\\\xa0...\\'}, {\\'url\\': \\'https://en.wikipedia.org/wiki/2024_Australian_Open\\', \\'content\\': \"Contents 2024 Australian Open The 2024 Australian Open was a Grand Slam level tennis tournament held at Melbourne Park, from 14–28 January 2024.[1] The Australian Open total prize money for 2024 increased by 13.07% year on year to a tournament record A$86,500,000. In the tournament\\'s 119-year history, this was the first Australian Open Tennis Championships to be held on an openingNovak Djokovic was the defending men\\'s singles champion. ... He was defeated in the semifinals by Jannik Sinner, who went on to beat Daniil Medvedev in a five-set\\\\xa0...\"}, {\\'url\\': \\'https://en.wikipedia.org/wiki/2024_Australian_Open_%E2%80%93_Men%27s_singles\\', \\'content\\': \"Contents 2024 Australian Open – Men\\'s singles The entry list was released by Tennis Australia based on the ATP rankings for the week of 4 December 2023.[15] matches, tying the Open Era record set at the 1983 US Open.[14] feature any of the Big Three members.[4] It was the second time Medvedev lost the Australian Open final after winningJannik Sinner defeated Daniil Medvedev in the final, 3–6, 3–6, 6–4, 6–4, 6–3, to win the men\\'s singles tennis title at the 2024 Australian Open.\"}]', '#E2': \"content='The name of the 2024 Australian Open winner is Jannik Sinner.'\"}}}\n", - "{'tool': {'results': {'#E1': '[{\\'url\\': \\'https://www.cbssports.com/tennis/news/australian-open-2024-jannik-sinner-aryna-sabalenka-crowned-as-grand-slam-singles-champions-at-melbourne-park/\\', \\'content\\': \\'2024 Australian Open odds, Sinner vs. Medvedev picks Sabalenka defeats Zheng to win 2024 Australian Open Australian Open 2024: Jannik Sinner, Aryna Sabalenka crowned as Grand Slam singles champions at Melbourne Park 2024 Australian Open odds, Sabalenka vs. Zheng picks 2024 Australian Open odds, Medvedev vs. Zverev picks Sinner, Sabalenka win Australian Open singles titles Sinner makes epic comeback to win Australian OpenJan 28, 2024 — Jan 28, 2024Australian Open 2024: Jannik Sinner, Aryna Sabalenka crowned as Grand Slam singles champions at Melbourne Park ... Watch Now: Jannik Sinner came\\\\xa0...\\'}, {\\'url\\': \\'https://en.wikipedia.org/wiki/2024_Australian_Open\\', \\'content\\': \"Contents 2024 Australian Open The 2024 Australian Open was a Grand Slam level tennis tournament held at Melbourne Park, from 14–28 January 2024.[1] The Australian Open total prize money for 2024 increased by 13.07% year on year to a tournament record A$86,500,000. In the tournament\\'s 119-year history, this was the first Australian Open Tennis Championships to be held on an openingNovak Djokovic was the defending men\\'s singles champion. ... He was defeated in the semifinals by Jannik Sinner, who went on to beat Daniil Medvedev in a five-set\\\\xa0...\"}, {\\'url\\': \\'https://en.wikipedia.org/wiki/2024_Australian_Open_%E2%80%93_Men%27s_singles\\', \\'content\\': \"Contents 2024 Australian Open – Men\\'s singles The entry list was released by Tennis Australia based on the ATP rankings for the week of 4 December 2023.[15] matches, tying the Open Era record set at the 1983 US Open.[14] feature any of the Big Three members.[4] It was the second time Medvedev lost the Australian Open final after winningJannik Sinner defeated Daniil Medvedev in the final, 3–6, 3–6, 6–4, 6–4, 6–3, to win the men\\'s singles tennis title at the 2024 Australian Open.\"}]', '#E2': \"content='The name of the 2024 Australian Open winner is Jannik Sinner.'\", '#E3': '[{\\'url\\': \\'https://www.tennis.com/news/articles/soccer-mad-italy-is-now-obsessed-with-tennis-player-jannik-sinner-after-his-australian-open-title\\', \\'content\\': \"Soccer-mad Italy is now obsessed with tennis player Jannik Sinner after his Australian Open title Play & Win Advertising Soccer-mad Italy is now obsessed with tennis player Jannik Sinner after his Australian Open title \\'Grandissimo\\': Italian Premier Giorgia Meloni welcomes home Australian Open champion Jannik Sinner First of many? Jannik Sinner\\'s five-set comeback sinks Daniil Medvedev in Australian Open finalJan 28, 2024 — Jan 28, 2024In Sinner\\'s tiny hometown of Sesto (population 1,860) near the Austrian border, about 70 people gathered inside the two-court indoor tennis\\\\xa0...\"}, {\\'url\\': \\'https://apnews.com/article/jannik-sinner-italy-australian-open-03573689c4c58c2851d1006e26546ac9\\', \\'content\\': \"Soccer-mad Italy is now obsessed with tennis player Jannik Sinner after his Australian Open title Jannik Sinner, left, of Italy gestures as he holds the Norman Brookes Challenge Cup after defeating Daniil Medvedev, Jannik Sinner, left, of Italy gestures as he holds the Norman Brookes Challenge Cup after defeating Daniil Medvedev, Jannik Sinner, left, of Italy gestures as he holds the Norman Brookes Challenge Cup after defeating Daniil Medvedev,Jan 28, 2024 — Jan 28, 2024Soccer-mad Italy has a new obsession. Jannik Sinner\\'s Australian Open performance on the tennis court has captured the country\\'s attention.\"}, {\\'url\\': \\'https://en.wikipedia.org/wiki/Jannik_Sinner\\', \\'content\\': \\'At the 2024 Australian Open, Sinner defeated world No. 1 Novak Djokovic in the semifinals to reach his first major Sinner is a major champion, having won the 2024 Australian Open.[3] He has won a further ten ATP Tour singles titles, Early in the year Sinner made the second round of the 2020 Australian Open, recording his first Grand Slam main draw a match since Janko Tipsarević in London in 2011.[59][60] Sinner played Daniil Medvedev next in the round robin stage,Since making his professional debut in 2018, Sinner has won 11 ATP Tour singles titles, including a Grand Slam at the 2024 Australian Open and a Masters 1000 at\\\\xa0...\\'}, {\\'url\\': \\'https://ausopen.com/players/italy/jannik-sinner\\', \\'content\\': \"Jannik Sinner weathered an early onslaught to reel in Daniil Medvedev, growing in potency to win the Australian Open the Australian Open 2024 final – his first Grand Slam singles title. Jannik Sinner will contest his first Grand Slam final after scuttling Novak Djokovic’s bid for a record-extending 11th Jannick Sinner has form and fitness on his side ahead of his meeting with Andrey Rublev.Jannik Sinner Press Conference | Australian Open 2024 Final. 15:02 · Player & Career Overview. Career Wins 73% · Men\\'s Singles. Final • Rod Laver Arena · Comeback\\\\xa0...\"}]'}}}\n", - "{'tool': {'results': {'#E1': '[{\\'url\\': \\'https://www.cbssports.com/tennis/news/australian-open-2024-jannik-sinner-aryna-sabalenka-crowned-as-grand-slam-singles-champions-at-melbourne-park/\\', \\'content\\': \\'2024 Australian Open odds, Sinner vs. Medvedev picks Sabalenka defeats Zheng to win 2024 Australian Open Australian Open 2024: Jannik Sinner, Aryna Sabalenka crowned as Grand Slam singles champions at Melbourne Park 2024 Australian Open odds, Sabalenka vs. Zheng picks 2024 Australian Open odds, Medvedev vs. Zverev picks Sinner, Sabalenka win Australian Open singles titles Sinner makes epic comeback to win Australian OpenJan 28, 2024 — Jan 28, 2024Australian Open 2024: Jannik Sinner, Aryna Sabalenka crowned as Grand Slam singles champions at Melbourne Park ... Watch Now: Jannik Sinner came\\\\xa0...\\'}, {\\'url\\': \\'https://en.wikipedia.org/wiki/2024_Australian_Open\\', \\'content\\': \"Contents 2024 Australian Open The 2024 Australian Open was a Grand Slam level tennis tournament held at Melbourne Park, from 14–28 January 2024.[1] The Australian Open total prize money for 2024 increased by 13.07% year on year to a tournament record A$86,500,000. In the tournament\\'s 119-year history, this was the first Australian Open Tennis Championships to be held on an openingNovak Djokovic was the defending men\\'s singles champion. ... He was defeated in the semifinals by Jannik Sinner, who went on to beat Daniil Medvedev in a five-set\\\\xa0...\"}, {\\'url\\': \\'https://en.wikipedia.org/wiki/2024_Australian_Open_%E2%80%93_Men%27s_singles\\', \\'content\\': \"Contents 2024 Australian Open – Men\\'s singles The entry list was released by Tennis Australia based on the ATP rankings for the week of 4 December 2023.[15] matches, tying the Open Era record set at the 1983 US Open.[14] feature any of the Big Three members.[4] It was the second time Medvedev lost the Australian Open final after winningJannik Sinner defeated Daniil Medvedev in the final, 3–6, 3–6, 6–4, 6–4, 6–3, to win the men\\'s singles tennis title at the 2024 Australian Open.\"}]', '#E2': \"content='The name of the 2024 Australian Open winner is Jannik Sinner.'\", '#E3': '[{\\'url\\': \\'https://www.tennis.com/news/articles/soccer-mad-italy-is-now-obsessed-with-tennis-player-jannik-sinner-after-his-australian-open-title\\', \\'content\\': \"Soccer-mad Italy is now obsessed with tennis player Jannik Sinner after his Australian Open title Play & Win Advertising Soccer-mad Italy is now obsessed with tennis player Jannik Sinner after his Australian Open title \\'Grandissimo\\': Italian Premier Giorgia Meloni welcomes home Australian Open champion Jannik Sinner First of many? Jannik Sinner\\'s five-set comeback sinks Daniil Medvedev in Australian Open finalJan 28, 2024 — Jan 28, 2024In Sinner\\'s tiny hometown of Sesto (population 1,860) near the Austrian border, about 70 people gathered inside the two-court indoor tennis\\\\xa0...\"}, {\\'url\\': \\'https://apnews.com/article/jannik-sinner-italy-australian-open-03573689c4c58c2851d1006e26546ac9\\', \\'content\\': \"Soccer-mad Italy is now obsessed with tennis player Jannik Sinner after his Australian Open title Jannik Sinner, left, of Italy gestures as he holds the Norman Brookes Challenge Cup after defeating Daniil Medvedev, Jannik Sinner, left, of Italy gestures as he holds the Norman Brookes Challenge Cup after defeating Daniil Medvedev, Jannik Sinner, left, of Italy gestures as he holds the Norman Brookes Challenge Cup after defeating Daniil Medvedev,Jan 28, 2024 — Jan 28, 2024Soccer-mad Italy has a new obsession. Jannik Sinner\\'s Australian Open performance on the tennis court has captured the country\\'s attention.\"}, {\\'url\\': \\'https://en.wikipedia.org/wiki/Jannik_Sinner\\', \\'content\\': \\'At the 2024 Australian Open, Sinner defeated world No. 1 Novak Djokovic in the semifinals to reach his first major Sinner is a major champion, having won the 2024 Australian Open.[3] He has won a further ten ATP Tour singles titles, Early in the year Sinner made the second round of the 2020 Australian Open, recording his first Grand Slam main draw a match since Janko Tipsarević in London in 2011.[59][60] Sinner played Daniil Medvedev next in the round robin stage,Since making his professional debut in 2018, Sinner has won 11 ATP Tour singles titles, including a Grand Slam at the 2024 Australian Open and a Masters 1000 at\\\\xa0...\\'}, {\\'url\\': \\'https://ausopen.com/players/italy/jannik-sinner\\', \\'content\\': \"Jannik Sinner weathered an early onslaught to reel in Daniil Medvedev, growing in potency to win the Australian Open the Australian Open 2024 final – his first Grand Slam singles title. Jannik Sinner will contest his first Grand Slam final after scuttling Novak Djokovic’s bid for a record-extending 11th Jannick Sinner has form and fitness on his side ahead of his meeting with Andrey Rublev.Jannik Sinner Press Conference | Australian Open 2024 Final. 15:02 · Player & Career Overview. Career Wins 73% · Men\\'s Singles. Final • Rod Laver Arena · Comeback\\\\xa0...\"}]', '#E4': \"content='The hometown of the 2024 Australian Open winner, Jannik Sinner, is Sesto, a small town near the Austrian border in Italy.'\"}}}\n", + "---\n", + "{'tool': {'results': {'#E1': '[{\\'url\\': \\'https://www.cbssports.com/tennis/news/australian-open-2024-jannik-sinner-aryna-sabalenka-crowned-as-grand-slam-singles-champions-at-melbourne-park/\\', \\'content\\': \\'2024 Australian Open odds, Sinner vs. Medvedev picks Sabalenka defeats Zheng to win 2024 Australian Open Australian Open 2024: Jannik Sinner, Aryna Sabalenka crowned as Grand Slam singles champions at Melbourne Park 2024 Australian Open odds, Sabalenka vs. Zheng picks 2024 Australian Open odds, Medvedev vs. Zverev picks Sinner, Sabalenka win Australian Open singles titles Sinner makes epic comeback to win Australian OpenJan 28, 2024 — Jan 28, 2024Australian Open 2024: Jannik Sinner, Aryna Sabalenka crowned as Grand Slam singles champions at Melbourne Park ... Watch Now: Jannik Sinner came\\\\xa0...\\'}, {\\'url\\': \\'https://en.wikipedia.org/wiki/2024_Australian_Open\\', \\'content\\': \"Contents 2024 Australian Open The 2024 Australian Open was a Grand Slam level tennis tournament held at Melbourne Park, from 14–28 January 2024.[1] The Australian Open total prize money for 2024 increased by 13.07% year on year to a tournament record A$86,500,000. In the tournament\\'s 119-year history, this was the first Australian Open Tennis Championships to be held on an openingNovak Djokovic was the defending men\\'s singles champion. ... He was defeated in the semifinals by Jannik Sinner, who went on to beat Daniil Medvedev in a five-set\\\\xa0...\"}, {\\'url\\': \\'https://en.wikipedia.org/wiki/2024_Australian_Open_%E2%80%93_Men%27s_singles\\', \\'content\\': \"Contents 2024 Australian Open – Men\\'s singles The entry list was released by Tennis Australia based on the ATP rankings for the week of 4 December 2023.[15] matches, tying the Open Era record set at the 1983 US Open.[14] feature any of the Big Three members.[4] It was the second time Medvedev lost the Australian Open final after winningJannik Sinner defeated Daniil Medvedev in the final, 3–6, 3–6, 6–4, 6–4, 6–3, to win the men\\'s singles tennis title at the 2024 Australian Open.\"}]', '#E2': \"content='The name of the 2024 Australian Open winner is Jannik Sinner.'\", '#E3': '[{\\'url\\': \\'https://www.tennis.com/news/articles/soccer-mad-italy-is-now-obsessed-with-tennis-player-jannik-sinner-after-his-australian-open-title\\', \\'content\\': \"Soccer-mad Italy is now obsessed with tennis player Jannik Sinner after his Australian Open title Play & Win Advertising Soccer-mad Italy is now obsessed with tennis player Jannik Sinner after his Australian Open title \\'Grandissimo\\': Italian Premier Giorgia Meloni welcomes home Australian Open champion Jannik Sinner First of many? Jannik Sinner\\'s five-set comeback sinks Daniil Medvedev in Australian Open finalJan 28, 2024 — Jan 28, 2024In Sinner\\'s tiny hometown of Sesto (population 1,860) near the Austrian border, about 70 people gathered inside the two-court indoor tennis\\\\xa0...\"}, {\\'url\\': \\'https://apnews.com/article/jannik-sinner-italy-australian-open-03573689c4c58c2851d1006e26546ac9\\', \\'content\\': \"Soccer-mad Italy is now obsessed with tennis player Jannik Sinner after his Australian Open title Jannik Sinner, left, of Italy gestures as he holds the Norman Brookes Challenge Cup after defeating Daniil Medvedev, Jannik Sinner, left, of Italy gestures as he holds the Norman Brookes Challenge Cup after defeating Daniil Medvedev, Jannik Sinner, left, of Italy gestures as he holds the Norman Brookes Challenge Cup after defeating Daniil Medvedev,Jan 28, 2024 — Jan 28, 2024Soccer-mad Italy has a new obsession. Jannik Sinner\\'s Australian Open performance on the tennis court has captured the country\\'s attention.\"}, {\\'url\\': \\'https://en.wikipedia.org/wiki/Jannik_Sinner\\', \\'content\\': \\'Sinner is a major champion, having won the 2024 Australian Open.[3] He has won a further ten ATP Tour singles titles, At the 2024 Australian Open, Sinner defeated world No. 1 Novak Djokovic in the semifinals to reach his first major Early in the year Sinner made the second round of the 2020 Australian Open, recording his first Grand Slam main draw a match since Janko Tipsarević in London in 2011.[59][60] Sinner played Daniil Medvedev next in the round robin stage,Since making his professional debut in 2018, Sinner has won 11 ATP Tour singles titles, including a Grand Slam at the 2024 Australian Open and a Masters 1000 at\\\\xa0...\\'}, {\\'url\\': \\'https://ausopen.com/players/italy/jannik-sinner\\', \\'content\\': \"Jannik Sinner weathered an early onslaught to reel in Daniil Medvedev, growing in potency to win the Australian Open the Australian Open 2024 final – his first Grand Slam singles title. Jannik Sinner will contest his first Grand Slam final after scuttling Novak Djokovic’s bid for a record-extending 11th Jannick Sinner has form and fitness on his side ahead of his meeting with Andrey Rublev.Jannik Sinner Press Conference | Australian Open 2024 Final. 15:02 · Player & Career Overview. Career Wins 73% · Men\\'s Singles. Final • Rod Laver Arena · Comeback\\\\xa0...\"}]'}}}\n", + "---\n", + "{'tool': {'results': {'#E1': '[{\\'url\\': \\'https://www.cbssports.com/tennis/news/australian-open-2024-jannik-sinner-aryna-sabalenka-crowned-as-grand-slam-singles-champions-at-melbourne-park/\\', \\'content\\': \\'2024 Australian Open odds, Sinner vs. Medvedev picks Sabalenka defeats Zheng to win 2024 Australian Open Australian Open 2024: Jannik Sinner, Aryna Sabalenka crowned as Grand Slam singles champions at Melbourne Park 2024 Australian Open odds, Sabalenka vs. Zheng picks 2024 Australian Open odds, Medvedev vs. Zverev picks Sinner, Sabalenka win Australian Open singles titles Sinner makes epic comeback to win Australian OpenJan 28, 2024 — Jan 28, 2024Australian Open 2024: Jannik Sinner, Aryna Sabalenka crowned as Grand Slam singles champions at Melbourne Park ... Watch Now: Jannik Sinner came\\\\xa0...\\'}, {\\'url\\': \\'https://en.wikipedia.org/wiki/2024_Australian_Open\\', \\'content\\': \"Contents 2024 Australian Open The 2024 Australian Open was a Grand Slam level tennis tournament held at Melbourne Park, from 14–28 January 2024.[1] The Australian Open total prize money for 2024 increased by 13.07% year on year to a tournament record A$86,500,000. In the tournament\\'s 119-year history, this was the first Australian Open Tennis Championships to be held on an openingNovak Djokovic was the defending men\\'s singles champion. ... He was defeated in the semifinals by Jannik Sinner, who went on to beat Daniil Medvedev in a five-set\\\\xa0...\"}, {\\'url\\': \\'https://en.wikipedia.org/wiki/2024_Australian_Open_%E2%80%93_Men%27s_singles\\', \\'content\\': \"Contents 2024 Australian Open – Men\\'s singles The entry list was released by Tennis Australia based on the ATP rankings for the week of 4 December 2023.[15] matches, tying the Open Era record set at the 1983 US Open.[14] feature any of the Big Three members.[4] It was the second time Medvedev lost the Australian Open final after winningJannik Sinner defeated Daniil Medvedev in the final, 3–6, 3–6, 6–4, 6–4, 6–3, to win the men\\'s singles tennis title at the 2024 Australian Open.\"}]', '#E2': \"content='The name of the 2024 Australian Open winner is Jannik Sinner.'\", '#E3': '[{\\'url\\': \\'https://www.tennis.com/news/articles/soccer-mad-italy-is-now-obsessed-with-tennis-player-jannik-sinner-after-his-australian-open-title\\', \\'content\\': \"Soccer-mad Italy is now obsessed with tennis player Jannik Sinner after his Australian Open title Play & Win Advertising Soccer-mad Italy is now obsessed with tennis player Jannik Sinner after his Australian Open title \\'Grandissimo\\': Italian Premier Giorgia Meloni welcomes home Australian Open champion Jannik Sinner First of many? Jannik Sinner\\'s five-set comeback sinks Daniil Medvedev in Australian Open finalJan 28, 2024 — Jan 28, 2024In Sinner\\'s tiny hometown of Sesto (population 1,860) near the Austrian border, about 70 people gathered inside the two-court indoor tennis\\\\xa0...\"}, {\\'url\\': \\'https://apnews.com/article/jannik-sinner-italy-australian-open-03573689c4c58c2851d1006e26546ac9\\', \\'content\\': \"Soccer-mad Italy is now obsessed with tennis player Jannik Sinner after his Australian Open title Jannik Sinner, left, of Italy gestures as he holds the Norman Brookes Challenge Cup after defeating Daniil Medvedev, Jannik Sinner, left, of Italy gestures as he holds the Norman Brookes Challenge Cup after defeating Daniil Medvedev, Jannik Sinner, left, of Italy gestures as he holds the Norman Brookes Challenge Cup after defeating Daniil Medvedev,Jan 28, 2024 — Jan 28, 2024Soccer-mad Italy has a new obsession. Jannik Sinner\\'s Australian Open performance on the tennis court has captured the country\\'s attention.\"}, {\\'url\\': \\'https://en.wikipedia.org/wiki/Jannik_Sinner\\', \\'content\\': \\'Sinner is a major champion, having won the 2024 Australian Open.[3] He has won a further ten ATP Tour singles titles, At the 2024 Australian Open, Sinner defeated world No. 1 Novak Djokovic in the semifinals to reach his first major Early in the year Sinner made the second round of the 2020 Australian Open, recording his first Grand Slam main draw a match since Janko Tipsarević in London in 2011.[59][60] Sinner played Daniil Medvedev next in the round robin stage,Since making his professional debut in 2018, Sinner has won 11 ATP Tour singles titles, including a Grand Slam at the 2024 Australian Open and a Masters 1000 at\\\\xa0...\\'}, {\\'url\\': \\'https://ausopen.com/players/italy/jannik-sinner\\', \\'content\\': \"Jannik Sinner weathered an early onslaught to reel in Daniil Medvedev, growing in potency to win the Australian Open the Australian Open 2024 final – his first Grand Slam singles title. Jannik Sinner will contest his first Grand Slam final after scuttling Novak Djokovic’s bid for a record-extending 11th Jannick Sinner has form and fitness on his side ahead of his meeting with Andrey Rublev.Jannik Sinner Press Conference | Australian Open 2024 Final. 15:02 · Player & Career Overview. Career Wins 73% · Men\\'s Singles. Final • Rod Laver Arena · Comeback\\\\xa0...\"}]', '#E4': \"content='The hometown of the 2024 Australian Open winner, Jannik Sinner, is Sesto, a small town near the Austrian border in Italy.'\"}}}\n", + "---\n", "{'solve': {'result': 'The hometown of the 2024 Australian Open winner, Jannik Sinner, is Sesto, Italy.'}}\n", - "{'__end__': {'task': 'what is the hometown of the 2024 australian open winner', 'plan_string': 'Plan: Use Google to search for the 2024 Australian Open winner.\\n#E1 = Google[2024 Australian Open winner]\\n\\nPlan: Retrieve the name of the 2024 Australian Open winner from the search results.\\n#E2 = LLM[What is the name of the 2024 Australian Open winner, given #E1]\\n\\nPlan: Use Google to search for the hometown of the 2024 Australian Open winner.\\n#E3 = Google[hometown of 2024 Australian Open winner, given #E2]\\n\\nPlan: Retrieve the hometown of the 2024 Australian Open winner from the search results.\\n#E4 = LLM[What is the hometown of the 2024 Australian Open winner, given #E3]', 'steps': [('Use Google to search for the 2024 Australian Open winner.', '#E1', 'Google', '2024 Australian Open winner'), ('Retrieve the name of the 2024 Australian Open winner from the search results.', '#E2', 'LLM', 'What is the name of the 2024 Australian Open winner, given #E1'), ('Use Google to search for the hometown of the 2024 Australian Open winner.', '#E3', 'Google', 'hometown of 2024 Australian Open winner, given #E2'), ('Retrieve the hometown of the 2024 Australian Open winner from the search results.', '#E4', 'LLM', 'What is the hometown of the 2024 Australian Open winner, given #E3')], 'results': {'#E1': '[{\\'url\\': \\'https://www.cbssports.com/tennis/news/australian-open-2024-jannik-sinner-aryna-sabalenka-crowned-as-grand-slam-singles-champions-at-melbourne-park/\\', \\'content\\': \\'2024 Australian Open odds, Sinner vs. Medvedev picks Sabalenka defeats Zheng to win 2024 Australian Open Australian Open 2024: Jannik Sinner, Aryna Sabalenka crowned as Grand Slam singles champions at Melbourne Park 2024 Australian Open odds, Sabalenka vs. Zheng picks 2024 Australian Open odds, Medvedev vs. Zverev picks Sinner, Sabalenka win Australian Open singles titles Sinner makes epic comeback to win Australian OpenJan 28, 2024 — Jan 28, 2024Australian Open 2024: Jannik Sinner, Aryna Sabalenka crowned as Grand Slam singles champions at Melbourne Park ... Watch Now: Jannik Sinner came\\\\xa0...\\'}, {\\'url\\': \\'https://en.wikipedia.org/wiki/2024_Australian_Open\\', \\'content\\': \"Contents 2024 Australian Open The 2024 Australian Open was a Grand Slam level tennis tournament held at Melbourne Park, from 14–28 January 2024.[1] The Australian Open total prize money for 2024 increased by 13.07% year on year to a tournament record A$86,500,000. In the tournament\\'s 119-year history, this was the first Australian Open Tennis Championships to be held on an openingNovak Djokovic was the defending men\\'s singles champion. ... He was defeated in the semifinals by Jannik Sinner, who went on to beat Daniil Medvedev in a five-set\\\\xa0...\"}, {\\'url\\': \\'https://en.wikipedia.org/wiki/2024_Australian_Open_%E2%80%93_Men%27s_singles\\', \\'content\\': \"Contents 2024 Australian Open – Men\\'s singles The entry list was released by Tennis Australia based on the ATP rankings for the week of 4 December 2023.[15] matches, tying the Open Era record set at the 1983 US Open.[14] feature any of the Big Three members.[4] It was the second time Medvedev lost the Australian Open final after winningJannik Sinner defeated Daniil Medvedev in the final, 3–6, 3–6, 6–4, 6–4, 6–3, to win the men\\'s singles tennis title at the 2024 Australian Open.\"}]', '#E2': \"content='The name of the 2024 Australian Open winner is Jannik Sinner.'\", '#E3': '[{\\'url\\': \\'https://www.tennis.com/news/articles/soccer-mad-italy-is-now-obsessed-with-tennis-player-jannik-sinner-after-his-australian-open-title\\', \\'content\\': \"Soccer-mad Italy is now obsessed with tennis player Jannik Sinner after his Australian Open title Play & Win Advertising Soccer-mad Italy is now obsessed with tennis player Jannik Sinner after his Australian Open title \\'Grandissimo\\': Italian Premier Giorgia Meloni welcomes home Australian Open champion Jannik Sinner First of many? Jannik Sinner\\'s five-set comeback sinks Daniil Medvedev in Australian Open finalJan 28, 2024 — Jan 28, 2024In Sinner\\'s tiny hometown of Sesto (population 1,860) near the Austrian border, about 70 people gathered inside the two-court indoor tennis\\\\xa0...\"}, {\\'url\\': \\'https://apnews.com/article/jannik-sinner-italy-australian-open-03573689c4c58c2851d1006e26546ac9\\', \\'content\\': \"Soccer-mad Italy is now obsessed with tennis player Jannik Sinner after his Australian Open title Jannik Sinner, left, of Italy gestures as he holds the Norman Brookes Challenge Cup after defeating Daniil Medvedev, Jannik Sinner, left, of Italy gestures as he holds the Norman Brookes Challenge Cup after defeating Daniil Medvedev, Jannik Sinner, left, of Italy gestures as he holds the Norman Brookes Challenge Cup after defeating Daniil Medvedev,Jan 28, 2024 — Jan 28, 2024Soccer-mad Italy has a new obsession. Jannik Sinner\\'s Australian Open performance on the tennis court has captured the country\\'s attention.\"}, {\\'url\\': \\'https://en.wikipedia.org/wiki/Jannik_Sinner\\', \\'content\\': \\'At the 2024 Australian Open, Sinner defeated world No. 1 Novak Djokovic in the semifinals to reach his first major Sinner is a major champion, having won the 2024 Australian Open.[3] He has won a further ten ATP Tour singles titles, Early in the year Sinner made the second round of the 2020 Australian Open, recording his first Grand Slam main draw a match since Janko Tipsarević in London in 2011.[59][60] Sinner played Daniil Medvedev next in the round robin stage,Since making his professional debut in 2018, Sinner has won 11 ATP Tour singles titles, including a Grand Slam at the 2024 Australian Open and a Masters 1000 at\\\\xa0...\\'}, {\\'url\\': \\'https://ausopen.com/players/italy/jannik-sinner\\', \\'content\\': \"Jannik Sinner weathered an early onslaught to reel in Daniil Medvedev, growing in potency to win the Australian Open the Australian Open 2024 final – his first Grand Slam singles title. Jannik Sinner will contest his first Grand Slam final after scuttling Novak Djokovic’s bid for a record-extending 11th Jannick Sinner has form and fitness on his side ahead of his meeting with Andrey Rublev.Jannik Sinner Press Conference | Australian Open 2024 Final. 15:02 · Player & Career Overview. Career Wins 73% · Men\\'s Singles. Final • Rod Laver Arena · Comeback\\\\xa0...\"}]', '#E4': \"content='The hometown of the 2024 Australian Open winner, Jannik Sinner, is Sesto, a small town near the Austrian border in Italy.'\"}, 'result': 'The hometown of the 2024 Australian Open winner, Jannik Sinner, is Sesto, Italy.'}}\n" + "---\n", + "{'__end__': {'task': 'what is the hometown of the 2024 australian open winner', 'plan_string': 'Plan: Use Google to search for the 2024 Australian Open winner.\\n#E1 = Google[2024 Australian Open winner]\\n\\nPlan: Retrieve the name of the 2024 Australian Open winner from the search results.\\n#E2 = LLM[What is the name of the 2024 Australian Open winner, given #E1]\\n\\nPlan: Use Google to search for the hometown of the 2024 Australian Open winner.\\n#E3 = Google[hometown of 2024 Australian Open winner, given #E2]\\n\\nPlan: Retrieve the hometown of the 2024 Australian Open winner from the search results.\\n#E4 = LLM[What is the hometown of the 2024 Australian Open winner, given #E3]', 'steps': [('Use Google to search for the 2024 Australian Open winner.', '#E1', 'Google', '2024 Australian Open winner'), ('Retrieve the name of the 2024 Australian Open winner from the search results.', '#E2', 'LLM', 'What is the name of the 2024 Australian Open winner, given #E1'), ('Use Google to search for the hometown of the 2024 Australian Open winner.', '#E3', 'Google', 'hometown of 2024 Australian Open winner, given #E2'), ('Retrieve the hometown of the 2024 Australian Open winner from the search results.', '#E4', 'LLM', 'What is the hometown of the 2024 Australian Open winner, given #E3')], 'results': {'#E1': '[{\\'url\\': \\'https://www.cbssports.com/tennis/news/australian-open-2024-jannik-sinner-aryna-sabalenka-crowned-as-grand-slam-singles-champions-at-melbourne-park/\\', \\'content\\': \\'2024 Australian Open odds, Sinner vs. Medvedev picks Sabalenka defeats Zheng to win 2024 Australian Open Australian Open 2024: Jannik Sinner, Aryna Sabalenka crowned as Grand Slam singles champions at Melbourne Park 2024 Australian Open odds, Sabalenka vs. Zheng picks 2024 Australian Open odds, Medvedev vs. Zverev picks Sinner, Sabalenka win Australian Open singles titles Sinner makes epic comeback to win Australian OpenJan 28, 2024 — Jan 28, 2024Australian Open 2024: Jannik Sinner, Aryna Sabalenka crowned as Grand Slam singles champions at Melbourne Park ... Watch Now: Jannik Sinner came\\\\xa0...\\'}, {\\'url\\': \\'https://en.wikipedia.org/wiki/2024_Australian_Open\\', \\'content\\': \"Contents 2024 Australian Open The 2024 Australian Open was a Grand Slam level tennis tournament held at Melbourne Park, from 14–28 January 2024.[1] The Australian Open total prize money for 2024 increased by 13.07% year on year to a tournament record A$86,500,000. In the tournament\\'s 119-year history, this was the first Australian Open Tennis Championships to be held on an openingNovak Djokovic was the defending men\\'s singles champion. ... He was defeated in the semifinals by Jannik Sinner, who went on to beat Daniil Medvedev in a five-set\\\\xa0...\"}, {\\'url\\': \\'https://en.wikipedia.org/wiki/2024_Australian_Open_%E2%80%93_Men%27s_singles\\', \\'content\\': \"Contents 2024 Australian Open – Men\\'s singles The entry list was released by Tennis Australia based on the ATP rankings for the week of 4 December 2023.[15] matches, tying the Open Era record set at the 1983 US Open.[14] feature any of the Big Three members.[4] It was the second time Medvedev lost the Australian Open final after winningJannik Sinner defeated Daniil Medvedev in the final, 3–6, 3–6, 6–4, 6–4, 6–3, to win the men\\'s singles tennis title at the 2024 Australian Open.\"}]', '#E2': \"content='The name of the 2024 Australian Open winner is Jannik Sinner.'\", '#E3': '[{\\'url\\': \\'https://www.tennis.com/news/articles/soccer-mad-italy-is-now-obsessed-with-tennis-player-jannik-sinner-after-his-australian-open-title\\', \\'content\\': \"Soccer-mad Italy is now obsessed with tennis player Jannik Sinner after his Australian Open title Play & Win Advertising Soccer-mad Italy is now obsessed with tennis player Jannik Sinner after his Australian Open title \\'Grandissimo\\': Italian Premier Giorgia Meloni welcomes home Australian Open champion Jannik Sinner First of many? Jannik Sinner\\'s five-set comeback sinks Daniil Medvedev in Australian Open finalJan 28, 2024 — Jan 28, 2024In Sinner\\'s tiny hometown of Sesto (population 1,860) near the Austrian border, about 70 people gathered inside the two-court indoor tennis\\\\xa0...\"}, {\\'url\\': \\'https://apnews.com/article/jannik-sinner-italy-australian-open-03573689c4c58c2851d1006e26546ac9\\', \\'content\\': \"Soccer-mad Italy is now obsessed with tennis player Jannik Sinner after his Australian Open title Jannik Sinner, left, of Italy gestures as he holds the Norman Brookes Challenge Cup after defeating Daniil Medvedev, Jannik Sinner, left, of Italy gestures as he holds the Norman Brookes Challenge Cup after defeating Daniil Medvedev, Jannik Sinner, left, of Italy gestures as he holds the Norman Brookes Challenge Cup after defeating Daniil Medvedev,Jan 28, 2024 — Jan 28, 2024Soccer-mad Italy has a new obsession. Jannik Sinner\\'s Australian Open performance on the tennis court has captured the country\\'s attention.\"}, {\\'url\\': \\'https://en.wikipedia.org/wiki/Jannik_Sinner\\', \\'content\\': \\'Sinner is a major champion, having won the 2024 Australian Open.[3] He has won a further ten ATP Tour singles titles, At the 2024 Australian Open, Sinner defeated world No. 1 Novak Djokovic in the semifinals to reach his first major Early in the year Sinner made the second round of the 2020 Australian Open, recording his first Grand Slam main draw a match since Janko Tipsarević in London in 2011.[59][60] Sinner played Daniil Medvedev next in the round robin stage,Since making his professional debut in 2018, Sinner has won 11 ATP Tour singles titles, including a Grand Slam at the 2024 Australian Open and a Masters 1000 at\\\\xa0...\\'}, {\\'url\\': \\'https://ausopen.com/players/italy/jannik-sinner\\', \\'content\\': \"Jannik Sinner weathered an early onslaught to reel in Daniil Medvedev, growing in potency to win the Australian Open the Australian Open 2024 final – his first Grand Slam singles title. Jannik Sinner will contest his first Grand Slam final after scuttling Novak Djokovic’s bid for a record-extending 11th Jannick Sinner has form and fitness on his side ahead of his meeting with Andrey Rublev.Jannik Sinner Press Conference | Australian Open 2024 Final. 15:02 · Player & Career Overview. Career Wins 73% · Men\\'s Singles. Final • Rod Laver Arena · Comeback\\\\xa0...\"}]', '#E4': \"content='The hometown of the 2024 Australian Open winner, Jannik Sinner, is Sesto, a small town near the Austrian border in Italy.'\"}, 'result': 'The hometown of the 2024 Australian Open winner, Jannik Sinner, is Sesto, Italy.'}}\n", + "---\n" ] } ], "source": [ "for s in app.stream({\"task\":task}):\n", - " print(s)\n" + " print(s)\n", + " print('---')" ] }, { "cell_type": "code", - "execution_count": null, - "id": "ba4a3cc6-ae26-47e5-aabe-5a30637c94b8", + "execution_count": 20, + "id": "70e5aa0f-4d8b-4f65-817a-1c4ebf07d07a", "metadata": {}, - "outputs": [], - "source": [] + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "The hometown of the 2024 Australian Open winner, Jannik Sinner, is Sesto, Italy.\n" + ] + } + ], + "source": [ + "# Print out the final result\n", + "print(s[END][\"result\"])" + ] + }, + { + "cell_type": "markdown", + "id": "842954d7-0de0-4876-be63-46b4e14157b0", + "metadata": {}, + "source": [ + "## Conclusion\n", + "\n", + "Congratulations on implementing ReWOO! Before you leave, I'll leave you with a couple limitations of the current implementation from the paper:\n", + "\n", + "1. If little context of the environment is available, the planner will be ineffective in its tool use. This can typically be ameliorated through few-shot prompting and/or fine-tuning.\n", + "2. The tasks are still executed in sequence, meaning the total execution time is impacted by _every_ tool call, not just he longest-running in a given step." + ] } ], "metadata": { @@ -311,7 +468,7 @@ "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", - "version": "3.11.1" + "version": "3.11.2" } }, "nbformat": 4,