commentary

examples/tnt-llm/tnt-llm.ipynb

@@ -18,7 +18,7 @@
     "3. Finetune classifier + deploy\n",
     "\n",
     "\n",
-    "In this notebook, we will focus on the first phase: taxonomy generation (blue in the diagram below).\n",
+    "When applying LangGraph in this notebook, we will focus on the first phase: taxonomy generation (blue in the diagram below). We then show how to label and fit the classifier in subsequent steps below.\n",
     "\n",
     "![TNT LLM Diagram](./img/tnt_llm.png)\n",
     "\n",
@@ -42,7 +42,9 @@
    "outputs": [],
    "source": [
     "%%capture --no-stderr\n",
-    "%pip install -U langgraph langchain_anthropic langsmith"
+    "%pip install -U langgraph langchain_anthropic langsmith\n",
+    "# For the embedding-based classifier use in phase 2\n",
+    "%pip install -U sklearn langchain_openai"
    ]
   },
   {
@@ -825,6 +827,17 @@
     "updated_docs = [{**doc, **category} for doc, category in zip(docs, results)]"
    ]
   },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "0ef9be82-278e-4501-8af9-70409ce15cc2",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "if \"OPENAI_API_KEY\" not in os.environ:\n",
+    "    os.environ[\"OPENAI_API_KEY\"] = getpass(\"Enter your OPENAI_API_KEY: \")"
+   ]
+  },
   {
    "cell_type": "code",
    "execution_count": 149,
@@ -834,6 +847,7 @@
    "source": [
     "from langchain_openai import OpenAIEmbeddings\n",
     "\n",
+    "# Consider using other embedding models here too!\n",
     "encoder = OpenAIEmbeddings(model=\"text-embedding-3-large\")\n",
     "vectors = encoder.embed_documents([doc[\"content\"] for doc in docs])\n",
     "embedded_docs = [{**doc, \"embedding\": v} for doc, v in zip(updated_docs, vectors)]"
@@ -849,16 +863,6 @@
     "Now that we've extracted the features from the text, we can generate the classifier on them."
    ]
   },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "id": "d8b8d386-d1eb-4894-abe7-9346ca363f50",
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "%pip install sklearn"
-   ]
-  },
   {
    "cell_type": "code",
    "execution_count": 196,
@@ -928,11 +932,9 @@
    "id": "32e7e994-a3c3-47f3-a860-32b847dfefee",
    "metadata": {},
    "source": [
-    "## Deploy\n",
+    "## Phase 3: Deploy\n",
     "\n",
-    "Now that you have your classifier, you can easily deploy it and apply to future runs! All you need is to embed the input and apply your LogisticRegression classifier. Let's try it.\n",
-    "\n",
-    "Assuming you've had some more data come in, you can fetch it and apply it below"
+    "Now that you have your classifier, you can easily deploy it and apply to future runs! All you need is to embed the input and apply your LogisticRegression classifier. Let's try it. We will use python's [joblib](https://joblib.readthedocs.io/en/stable/) library to serialize our sklearn classifier. Below is an example:"
    ]
   },
   {
@@ -956,7 +958,9 @@
    "id": "443f5f92-7f3c-4ce8-9104-b2b32ace0c42",
    "metadata": {},
    "source": [
-    "#### To deploy\n"
+    "#### To deploy\n",
+    "\n",
+    "When deploying, you can load the classifier and initialize your embeddings encoder. They fit together easily using LCEL:"
    ]
   },
   {
@@ -984,6 +988,16 @@
     ")"
    ]
   },
+  {
+   "cell_type": "markdown",
+   "id": "3114657e-b558-4a72-91af-68ae3fccacd7",
+   "metadata": {},
+   "source": [
+    "#### Example:\n",
+    "\n",
+    "Assuming you've had some more data come in, you can fetch it and apply it below"
+   ]
+  },
   {
    "cell_type": "code",
    "execution_count": 194,