WEBDIENSTE/langgraph
1
0
Fork 0
mirror of https://github.com/langchain-ai/langgraph.git synced 2026-08-31 12:19:58 +02:00
Code Issues Packages Projects Releases Wiki Activity

Add img

Browse Source
This commit is contained in:
William Fu-Hinthorn
2024-03-01 00:34:51 -08:00
parent 2bd5e3606e
commit afbe9360bb
2 changed files with 44 additions and 11 deletions
Download Patch File Download Diff File Expand all files Collapse all files
examples/storm/img/storm.png
BIN
View File
Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 1.8 MiB

examples/storm/storm.ipynb
+44 -11
View File
@@ -6,25 +6,35 @@
"source": [
"# STORM\n",
"\n",
"[STORM](https://arxiv.org/abs/2402.14207) is a research assistant by Shao, et. al that extends the idea of \"outline-driven RAG\" for richer article generation.\n",
"[STORM](https://arxiv.org/abs/2402.14207) is a research assistant designed by Shao, et. al that extends the idea of \"outline-driven RAG\" for richer article generation.\n",
"\n",
"It is tasked with generating Wikipedia-like ariticles on a user-provided topic. It has a few main stages:\n",
"STORM is designed to generate Wikipedia-style ariticles on a user-provided topic. It applies two main insights to produce more organized and comprehensive articles:\n",
"\n",
"1. Creating an outline (planning) by querying similar topics helps improve coverage.\n",
"2. Multi-perspective, grounded (in search) conversation simulation helps increase the reference count and information density. \n",
"\n",
"The control flow looks like the diagram below.\n",
"\n",
"![STORM diagram](./img/storm.png)\n",
"\n",
"STORM has a few main stages:\n",
"\n",
"1. Generate initial outline + Survey related subjects\n",
"2. Identify distinct perspectives\n",
"3. \"Interview subject matter experts\" (role-playing LLMs)\n",
"4. Refine outline\n",
"5. Write article\n",
"4. Refine outline (using references)\n",
"5. Write sections, then write article\n",
"\n",
"\n",
"The expert interviews stage ocurrs between the article writer and each role-playing agent and itself is a loop, where the \"expert\" is able to query external knowledge and respond to pointed questions.\n",
"The expert interviews stage ocurrs between the role-playing article writer and a research expert. The \"expert\" is able to query external knowledge and respond to pointed questions, saving cited sources to a vectorstore so that the later refinement stages can synthesize the full article.\n",
"\n",
"Couple hyperparameters to restrict the infinite research breadth:\n",
"There are a couple hyperparameters you can set to restrict the (potentially) infinite research breadth:\n",
"\n",
"N: Number of perspectives to survey / use (2->3)\n",
"M: Max number of conversation turns in step (3)\n",
"N: Number of perspectives to survey / use (Steps 2->3)\n",
"M: Max number of conversation turns in step (Step 3)\n",
"\n",
"The paper uses DSPY and few-shot examples to adapt but we'll just use functioncalling here."
"\n",
"## Prerequisites"
]
},
{
@@ -33,7 +43,9 @@
"metadata": {},
"outputs": [],
"source": [
"# %pip install langchain_community langchain_openai langchain_fireworks langgraph wikipedia tavily-python scikit-learn duckduckgo"
"# %pip install -U langchain_community langchain_openai langgraph wikipedia scikit-learn langchain_fireworks\n",
"# We use one or the other search engine below\n",
"# %pip install -U duckduckgo tavily-python"
]
},
{
@@ -48,13 +60,34 @@
"# !CFLAGS=\"-I $(brew --prefix graphviz)/include\" LDFLAGS=\"-L $(brew --prefix graphviz)/lib\" pip install -U pygraphviz"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"import os\n",
"import getpass\n",
"\n",
"def _set_env(var: str):\n",
" if os.environ.get(var):\n",
" return\n",
" os.environ[var] = getpass.getpass(var + \":\")\n",
"\n",
"# Set for tracing\n",
"os.environ[\"LANGCHAIN_TRACING_V2\"[ = \"true\"\n",
"os.environ[\"LANGCHAIN_PROJECT\"] = \"STORM\"\n",
"_set_env(\"LANGCHAIN_API_KEY\") \n",
"_set_env(\"OPENAI_API_KEY\")"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"#### Select LLMs\n",
"\n",
"We will have a faster LLM do most of the work, but a slower, long-context model distill the conversations and write the final report."
"We will have a faster LLM do most of the work, but a slower, long-context model to distill the conversations and write the final report."
]
},
{