From 2f22bb9f10f19dd5f2ebe0ff1012982731c43206 Mon Sep 17 00:00:00 2001
From: William Fu-Hinthorn <13333726+hinthornw@users.noreply.github.com>
Date: Mon, 1 Apr 2024 00:32:55 -0700
Subject: [PATCH] commentary

---
 examples/tnt-llm/tnt-llm.ipynb | 48 ++++++++++++++++++++++------------
 1 file changed, 31 insertions(+), 17 deletions(-)

diff --git a/examples/tnt-llm/tnt-llm.ipynb b/examples/tnt-llm/tnt-llm.ipynb
index eb51a1064..c98bf5598 100644
--- a/examples/tnt-llm/tnt-llm.ipynb
+++ b/examples/tnt-llm/tnt-llm.ipynb
@@ -18,7 +18,7 @@
     "3. Finetune classifier + deploy\n",
     "\n",
     "\n",
-    "In this notebook, we will focus on the first phase: taxonomy generation (blue in the diagram below).\n",
+    "When applying LangGraph in this notebook, we will focus on the first phase: taxonomy generation (blue in the diagram below). We then show how to label and fit the classifier in subsequent steps below.\n",
     "\n",
     "![TNT LLM Diagram](./img/tnt_llm.png)\n",
     "\n",
@@ -42,7 +42,9 @@
    "outputs": [],
    "source": [
     "%%capture --no-stderr\n",
-    "%pip install -U langgraph langchain_anthropic langsmith"
+    "%pip install -U langgraph langchain_anthropic langsmith\n",
+    "# For the embedding-based classifier use in phase 2\n",
+    "%pip install -U sklearn langchain_openai"
    ]
   },
   {
@@ -825,6 +827,17 @@
     "updated_docs = [{**doc, **category} for doc, category in zip(docs, results)]"
    ]
   },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "0ef9be82-278e-4501-8af9-70409ce15cc2",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "if \"OPENAI_API_KEY\" not in os.environ:\n",
+    "    os.environ[\"OPENAI_API_KEY\"] = getpass(\"Enter your OPENAI_API_KEY: \")"
+   ]
+  },
   {
    "cell_type": "code",
    "execution_count": 149,
@@ -834,6 +847,7 @@
    "source": [
     "from langchain_openai import OpenAIEmbeddings\n",
     "\n",
+    "# Consider using other embedding models here too!\n",
     "encoder = OpenAIEmbeddings(model=\"text-embedding-3-large\")\n",
     "vectors = encoder.embed_documents([doc[\"content\"] for doc in docs])\n",
     "embedded_docs = [{**doc, \"embedding\": v} for doc, v in zip(updated_docs, vectors)]"
@@ -849,16 +863,6 @@
     "Now that we've extracted the features from the text, we can generate the classifier on them."
    ]
   },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "id": "d8b8d386-d1eb-4894-abe7-9346ca363f50",
-   "metadata": {},
-   "outputs": [],
-   "source": [
-    "%pip install sklearn"
-   ]
-  },
   {
    "cell_type": "code",
    "execution_count": 196,
@@ -928,11 +932,9 @@
    "id": "32e7e994-a3c3-47f3-a860-32b847dfefee",
    "metadata": {},
    "source": [
-    "## Deploy\n",
+    "## Phase 3: Deploy\n",
     "\n",
-    "Now that you have your classifier, you can easily deploy it and apply to future runs! All you need is to embed the input and apply your LogisticRegression classifier. Let's try it.\n",
-    "\n",
-    "Assuming you've had some more data come in, you can fetch it and apply it below"
+    "Now that you have your classifier, you can easily deploy it and apply to future runs! All you need is to embed the input and apply your LogisticRegression classifier. Let's try it. We will use python's [joblib](https://joblib.readthedocs.io/en/stable/) library to serialize our sklearn classifier. Below is an example:"
    ]
   },
   {
@@ -956,7 +958,9 @@
    "id": "443f5f92-7f3c-4ce8-9104-b2b32ace0c42",
    "metadata": {},
    "source": [
-    "#### To deploy\n"
+    "#### To deploy\n",
+    "\n",
+    "When deploying, you can load the classifier and initialize your embeddings encoder. They fit together easily using LCEL:"
    ]
   },
   {
@@ -984,6 +988,16 @@
     ")"
    ]
   },
+  {
+   "cell_type": "markdown",
+   "id": "3114657e-b558-4a72-91af-68ae3fccacd7",
+   "metadata": {},
+   "source": [
+    "#### Example:\n",
+    "\n",
+    "Assuming you've had some more data come in, you can fetch it and apply it below"
+   ]
+  },
   {
    "cell_type": "code",
    "execution_count": 194,