WEBDIENSTE/langgraph
1
0
Fork 0
mirror of https://github.com/langchain-ai/langgraph.git synced 2026-08-26 17:42:24 +02:00
Code Issues Packages Projects Releases Wiki Activity

Add img

Browse Source
This commit is contained in:
William Fu-Hinthorn
2024-03-31 18:25:12 -07:00
parent 62ac195d39
commit ab0b6d3c0d
2 changed files with 29 additions and 38 deletions
Download Patch File Download Diff File Expand all files Collapse all files
examples/tnt-llm/img/tnt_llm.png
BIN
View File
Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 501 KiB

examples/tnt-llm/tnt-llm.ipynb
+29 -38
View File
@@ -18,7 +18,9 @@
"3. Finetune classifier + deploy\n",
"\n",
"\n",
"In this notebook, we will focus on the first phase: taxonomy generation.\n",
"In this notebook, we will focus on the first phase: taxonomy generation (blue in the diagram below).\n",
"\n",
"![TNT LLM Diagram](./img/tnt_llm.png)\n",
"\n",
"To generate the taxonomy, TNT-LLM proposes 5 steps:\n",
"\n",
@@ -612,24 +614,10 @@
},
{
"cell_type": "code",
"execution_count": 38,
"execution_count": null,
"id": "c2340177-f40c-407a-8e3e-cb06c2ef09ce",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"summarize {'documents': [{'id' ...\n",
"get_minibatches {'minibatches': [[39 ...\n",
"generate_taxonomy {'clusters': [[{'id' ...\n",
"update_taxonomy {'clusters': [[{'id' ...\n",
"update_taxonomy {'clusters': [[{'id' ...\n",
"review_taxonomy {'clusters': [[{'id' ...\n",
"__end__ {'documents': [{'id' ...\n"
]
}
],
"outputs": [],
"source": [
"# We will randomly sample down to 1K docs to speed things up\n",
"docs = [run_to_doc(run) for run in runs if run.inputs]\n",
@@ -640,28 +628,31 @@
" \" that would benefit the user.\"\n",
")\n",
"\n",
"stream = app.stream(\n",
" {\"documents\": docs},\n",
" {\n",
" \"configurable\": {\n",
" \"use_case\": use_case,\n",
" # Optional:\n",
" \"batch_size\": 400,\n",
" \"suggestion_length\": 30,\n",
" \"cluster_name_length\": 10,\n",
" \"cluster_description_length\": 30,\n",
" \"explanation_length\": 20,\n",
" \"max_num_clusters\": 25,\n",
" },\n",
" # We batch summarize the docs. To avoid getting errors, we will limit the\n",
" # degree of parallelism to permit.\n",
" \"max_concurrency\": 2,\n",
" },\n",
")\n",
"from langchain_core.tracers.context import tracing_v2_enabled\n",
"\n",
"for step in stream:\n",
" node, state = next(iter(step.items()))\n",
" print(node, str(state)[:20] + \" ...\")"
"with tracing_v2_enabled(client=Client(api_key=\"ls__eaec6db115fe4ad2af4fdf26fa553645\")):\n",
" stream = app.stream(\n",
" {\"documents\": docs},\n",
" {\n",
" \"configurable\": {\n",
" \"use_case\": use_case,\n",
" # Optional:\n",
" \"batch_size\": 400,\n",
" \"suggestion_length\": 30,\n",
" \"cluster_name_length\": 10,\n",
" \"cluster_description_length\": 30,\n",
" \"explanation_length\": 20,\n",
" \"max_num_clusters\": 25,\n",
" },\n",
" # We batch summarize the docs. To avoid getting errors, we will limit the\n",
" # degree of parallelism to permit.\n",
" \"max_concurrency\": 2,\n",
" },\n",
" )\n",
"\n",
" for step in stream:\n",
" node, state = next(iter(step.items()))\n",
" print(node, str(state)[:20] + \" ...\")"
]
},
{