{
"cells": [
{
"cell_type": "markdown",
"id": "f1a7d688-561c-4175-acfc-a6537f6dd042",
"metadata": {},
"source": [
"# Web Voyager\n",
"\n",
"[WebVoyager](https://arxiv.org/abs/2401.13919) by He, et. al., is a vision-enabled web-browsing agent capable of controlling the mouse and keyboard.\n",
"\n",
"It works by viewing annotated browser screenshots for each turn, then choosing the next step to take. The agent architecture is a basic reasoning and action (ReAct) loop. \n",
"The unique aspects of this agent are:\n",
"- It's usage of [Set-of-Marks](https://som-gpt4v.github.io/)-like image annotations to serve as UI affordances for the agent\n",
"- It's application in the browser by using tools to control both the mouse and keyboard\n",
"\n",
"The overall design looks like the following:\n",
"\n",
"\n",
"
\n",
"\n",
"\n",
"## Setup\n",
"\n",
"First, let's install our required packages:"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "af83b042",
"metadata": {},
"outputs": [],
"source": [
"%%capture --no-stderr\n",
"%pip install -U --quiet langgraph langsmith langchain_openai"
]
},
{
"cell_type": "code",
"execution_count": 2,
"id": "02f6e962-c6a5-43fa-b069-616c78c86c26",
"metadata": {},
"outputs": [],
"source": [
"import os\n",
"from getpass import getpass\n",
"\n",
"\n",
"def _getpass(env_var: str):\n",
" if not os.environ.get(env_var):\n",
" os.environ[env_var] = getpass(f\"{env_var}=\")\n",
"\n",
"\n",
"_getpass(\"OPENAI_API_KEY\")"
]
},
{
"cell_type": "markdown",
"id": "8251cc1a",
"metadata": {},
"source": [
"
Set up LangSmith for LangGraph development
\n", "\n", " Sign up for LangSmith to quickly spot issues and improve the performance of your LangGraph projects. LangSmith lets you use trace data to debug, test, and monitor your LLM apps built with LangGraph — read more about how to get started here. \n", "
\n", "