WEBDIENSTE/langgraph
1
0
Fork 0
mirror of https://github.com/langchain-ai/langgraph.git synced 2026-08-17 21:25:46 +02:00
Code Issues Packages Projects Releases Wiki Activity

Clean, update all ntbks

Browse Source
This commit is contained in:
Lance Martin
2024-04-01 14:48:42 -07:00
parent 3a9387e53f
commit 736b307647
4 changed files with 1034 additions and 210 deletions
Download Patch File Download Diff File Expand all files Collapse all files
examples/rag/langgraph_adaptive_rag.ipynb
+3 -1
View File
@@ -47,7 +47,7 @@
"metadata": {},
"outputs": [],
"source": [
"! pip install langchain_community tiktoken langchain-openai langchain-cohere langchainhub chromadb langchain langgraph"
"! pip install langchain_community tiktoken langchain-openai langchain-cohere langchainhub chromadb langchain langgraph tavily-python"
]
},
{
@@ -762,6 +762,8 @@
}
],
"source": [
"from pprint import pprint\n",
"\n",
"# Run \n",
"inputs = {\"question\": \"What player at the Bears expected to draft first in the 2024 NFL draft?\"}\n",
"for output in app.stream(inputs):\n",
examples/rag/langgraph_adaptive_rag_local.ipynb
+813
View File
File diff suppressed because one or more lines are too long
examples/rag/langgraph_crag_mistral.ipynb → examples/rag/langgraph_crag_local.ipynb
+112 -112
View File
@@ -87,7 +87,7 @@
"\n",
"(2) Download a `Mistral` model from various Mistral versions [here](https://ollama.ai/library/mistral) and Mixtral versions [here](https://ollama.ai/library/mixtral) available.\n",
"```\n",
"ollama pull mistral:instruct\n",
"ollama pull mistral\n",
"```"
]
},
@@ -157,13 +157,13 @@
},
{
"cell_type": "code",
"execution_count": 7,
"execution_count": 2,
"id": "2f4db331-c4d0-4c7c-a9a5-0bebc8a89c6c",
"metadata": {},
"outputs": [],
"source": [
"run_local = 'Yes'\n",
"local_llm = \"mistral:instruct\""
"local_llm = \"mistral:latest\""
]
},
{
@@ -178,7 +178,7 @@
},
{
"cell_type": "code",
"execution_count": null,
"execution_count": 3,
"id": "bb8b789b-475b-4e1b-9c66-03504c837830",
"metadata": {},
"outputs": [],
@@ -226,10 +226,18 @@
},
{
"cell_type": "code",
"execution_count": null,
"execution_count": 4,
"id": "0e75c029-6c10-47c7-871c-1f4932b25309",
"metadata": {},
"outputs": [],
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"{'score': 'yes'}\n"
]
}
],
"source": [
"### Retrieval Grader \n",
"\n",
@@ -266,10 +274,18 @@
},
{
"cell_type": "code",
"execution_count": null,
"execution_count": 6,
"id": "dad03302-bd93-43fc-949e-af51a3298cfa",
"metadata": {},
"outputs": [],
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
" The given text discusses the concept of building autonomous agents using a large language model (LLM) as its core controller. The text highlights several key components of an LLM-powered agent system, including observation, retrieval, reflection, planning & reacting, and relationships between agents. It also mentions some challenges such as finite context length, long-term planning and task decomposition, and reliability of natural language interface. The text provides examples of proof-of-concept demos like AutoGPT and discusses their limitations. The architecture of the generative agent is also described, which results in emergent social behavior.\n"
]
}
],
"source": [
"### Generate\n",
"\n",
@@ -285,7 +301,7 @@
"else:\n",
" llm = ChatMistralAI(\n",
" model=\"mistral-medium\", temperature=0, mistral_api_key=mistral_api_key\n",
" )ß\n",
" )\n",
"\n",
"# Post-processing\n",
"def format_docs(docs):\n",
@@ -301,10 +317,21 @@
},
{
"cell_type": "code",
"execution_count": null,
"execution_count": 9,
"id": "f4b61211-70b5-4471-a714-feb9cc91e860",
"metadata": {},
"outputs": [],
"outputs": [
{
"data": {
"text/plain": [
"' What is agent memory and how can it be effectively utilized in vector database retrieval?'"
]
},
"execution_count": 9,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"### Question Re-writer\n",
"\n",
@@ -317,13 +344,11 @@
" )\n",
"\n",
"# Prompt \n",
"system = \"\"\"You a question re-writer that converts an input question to a better version that is optimized \\n \n",
" for web search. Look at the input and try to reason about the underlying sematic intent / meaning.\"\"\"\n",
"re_write_prompt = ChatPromptTemplate.from_messages(\n",
" [\n",
" (\"system\", system),\n",
" (\"human\", \"Here is the initial question: \\n\\n {question} \\n Formulate an improved question.\"),\n",
" ]\n",
"re_write_prompt = PromptTemplate(\n",
" template=\"\"\"You a question re-writer that converts an input question to a better version that is optimized \\n \n",
" for vectorstore retrieval. Look at the initial and formulate an improved question. \\n\n",
" Here is the initial question: \\n\\n {question}. Improved question with no preamble: \\n \"\"\",\n",
" input_variables=[\"generation\", \"question\"],\n",
")\n",
"\n",
"question_rewriter = re_write_prompt | llm | StrOutputParser()\n",
@@ -340,7 +365,7 @@
},
{
"cell_type": "code",
"execution_count": null,
"execution_count": 10,
"id": "b36a2f36-bc5f-408d-a5e8-3fa203c233f6",
"metadata": {},
"outputs": [],
@@ -365,7 +390,7 @@
},
{
"cell_type": "code",
"execution_count": 4,
"execution_count": 11,
"id": "10028794-2fbc-43f9-aa4c-7fe3abd69c1e",
"metadata": {},
"outputs": [],
@@ -391,7 +416,7 @@
},
{
"cell_type": "code",
"execution_count": 5,
"execution_count": 15,
"id": "447d1333-082d-479a-a6fa-0ac0df78bb9d",
"metadata": {},
"outputs": [],
@@ -453,7 +478,7 @@
" web_search = \"No\"\n",
" for d in documents:\n",
" score = retrieval_grader.invoke({\"question\": question, \"document\": d.page_content})\n",
" grade = score.binary_score\n",
" grade = score['score']\n",
" if grade == \"yes\":\n",
" print(\"---GRADE: DOCUMENT RELEVANT---\")\n",
" filtered_docs.append(d)\n",
@@ -546,7 +571,7 @@
},
{
"cell_type": "code",
"execution_count": 6,
"execution_count": 16,
"id": "0a63776c-f9cd-46ce-b8cf-95c066dc5b06",
"metadata": {},
"outputs": [],
@@ -583,7 +608,7 @@
},
{
"cell_type": "code",
"execution_count": 8,
"execution_count": 17,
"id": "3ab1d8df-a74e-4b48-a30b-e39bbfd5925a",
"metadata": {},
"outputs": [
@@ -594,46 +619,50 @@
"---RETRIEVE---\n",
"\"Node 'retrieve':\"\n",
"'\\n---\\n'\n",
"---CHECK RELEVANCE---\n",
"---CHECK DOCUMENT RELEVANCE TO QUESTION---\n",
"---GRADE: DOCUMENT RELEVANT---\n",
"---GRADE: DOCUMENT RELEVANT---\n",
"---GRADE: DOCUMENT RELEVANT---\n",
"---GRADE: DOCUMENT RELEVANT---\n",
"\"Node 'grade_documents':\"\n",
"'\\n---\\n'\n",
"---DECIDE TO GENERATE---\n",
"---ASSESS GRADED DOCUMENTS---\n",
"---DECISION: GENERATE---\n",
"---GENERATE---\n",
"\"Node 'generate':\"\n",
"'\\n---\\n'\n",
"\"Node '__end__':\"\n",
"'\\n---\\n'\n",
"(' In an LLM (large language model)-powered autonomous agent system, LLM '\n",
" 'functions as the agent’s brain, complemented by several key components: '\n",
" 'planning and memory.\\n'\n",
"(' The given text discusses the concept of building autonomous agents using '\n",
" 'large language models (LLMs) as their core controllers. LLMs have the '\n",
" 'potential to be powerful general problem solvers, extending beyond '\n",
" 'generating well-written copies, stories, essays, and programs. In an '\n",
" \"LLM-powered agent system, the model functions as the agent's brain, \"\n",
" 'complemented by several key components: planning, memory, and tool use.\\n'\n",
" '\\n'\n",
" 'Planning involves breaking down large tasks into smaller subgoals for '\n",
" 'efficient handling of complex tasks and self-criticism and refinement to '\n",
" 'improve results.\\n'\n",
" '1. Planning: The agent breaks down large tasks into smaller subgoals for '\n",
" 'efficient handling of complex tasks and can do self-criticism and '\n",
" 'self-reflection to improve results.\\n'\n",
" '2. Memory: Short-term memory is utilized for in-context learning, while '\n",
" 'long-term memory provides the capability to retain and recall information '\n",
" 'over extended periods by leveraging an external vector store and fast '\n",
" 'retrieval.\\n'\n",
" '3. Tool use: The agent learns to call external APIs for missing information, '\n",
" 'including current information, code execution capability, access to '\n",
" 'proprietary information sources, and more.\\n'\n",
" '\\n'\n",
" 'Memory includes short-term memory, which utilizes in-context learning, and '\n",
" 'long-term memory, providing the agent with the capability to retain and '\n",
" 'recall information over extended periods using an external vector store and '\n",
" 'fast retrieval. The agent also learns to call external APIs for missing '\n",
" 'information.\\n'\n",
" 'The text also discusses the types of memory in human brains, including '\n",
" 'sensory memory, short-term memory (STM), and long-term memory (LTM). Sensory '\n",
" 'memory provides the ability to retain impressions of sensory information for '\n",
" 'a few seconds, while STM stores information needed for complex cognitive '\n",
" 'tasks and lasts for 20-30 seconds. LTM can store information for remarkably '\n",
" 'long periods with an essentially unlimited storage capacity and has two '\n",
" 'subtypes: explicit/declarative memory (memory of facts and events) and '\n",
" 'implicit/procedural memory (skills and routines).\\n'\n",
" '\\n'\n",
" 'Types of Memory:\\n'\n",
" '1. Sensory Memory: retains impressions of sensory information for a few '\n",
" 'seconds.\\n'\n",
" '2. Short-Term Memory (STM) or Working Memory: stores information needed for '\n",
" 'complex cognitive tasks and lasts for 20-30 seconds.\\n'\n",
" '3. Long-Term Memory (LTM): stores information for a remarkably long time, '\n",
" 'with two subtypes: explicit/declarative memory and implicit/procedural '\n",
" 'memory.\\n'\n",
" '\\n'\n",
" 'The agent uses LLM as its core controller, which can be extended beyond '\n",
" 'generating well-written copies, stories, essays, and programs to a powerful '\n",
" 'general problem solver.')\n"
" 'The text also includes a figure comparing different methods, including AD, '\n",
" 'ED, source policies, and RL^2, on environments that require memory and '\n",
" 'exploration.')\n"
]
}
],
@@ -659,15 +688,15 @@
"id": "03ee2be9-2368-46ea-9edd-dc064a7c7c96",
"metadata": {},
"source": [
"`Local (Ollama) -` \n",
"Trace: \n",
"\n",
"Trace for blow run: https://smith.langchain.com/public/3b23a1d4-720a-4b26-8f34-70d2f20f8832/r"
"https://smith.langchain.com/public/731df833-57de-4612-8fe8-07cb424bc9a6/r"
]
},
{
"cell_type": "code",
"execution_count": 11,
"id": "16ea2032-59c7-433d-aca4-2828a1239074",
"execution_count": 18,
"id": "deb28175-27a1-4afc-9747-2983e87fc881",
"metadata": {},
"outputs": [
{
@@ -677,75 +706,36 @@
"---RETRIEVE---\n",
"\"Node 'retrieve':\"\n",
"'\\n---\\n'\n",
"---CHECK RELEVANCE---\n",
"---GRADE: DOCUMENT RELEVANT---\n",
"---GRADE: DOCUMENT RELEVANT---\n",
"---GRADE: DOCUMENT RELEVANT---\n",
"---GRADE: DOCUMENT RELEVANT---\n",
"---CHECK DOCUMENT RELEVANCE TO QUESTION---\n",
"---GRADE: DOCUMENT NOT RELEVANT---\n",
"---GRADE: DOCUMENT NOT RELEVANT---\n",
"---GRADE: DOCUMENT NOT RELEVANT---\n",
"---GRADE: DOCUMENT NOT RELEVANT---\n",
"\"Node 'grade_documents':\"\n",
"'\\n---\\n'\n",
"---DECIDE TO GENERATE---\n",
"---DECISION: GENERATE---\n",
"---ASSESS GRADED DOCUMENTS---\n",
"---DECISION: ALL DOCUMENTS ARE NOT RELEVANT TO QUESTION, TRANSFORM QUERY---\n",
"---TRANSFORM QUERY---\n",
"\"Node 'transform_query':\"\n",
"'\\n---\\n'\n",
"---WEB SEARCH---\n",
"\"Node 'web_search_node':\"\n",
"'\\n---\\n'\n",
"---GENERATE---\n",
"\"Node 'generate':\"\n",
"'\\n---\\n'\n",
"\"Node '__end__':\"\n",
"'\\n---\\n'\n",
"(' In an LLM (large language model)-powered autonomous agent system, LLM '\n",
" 'functions as the agent’s brain, complemented by several key components: '\n",
" 'planning and memory.\\n'\n",
" '\\n'\n",
" 'Planning involves breaking down large tasks into smaller subgoals for '\n",
" 'efficient handling of complex tasks and self-criticism and refinement to '\n",
" 'improve results.\\n'\n",
" '\\n'\n",
" 'Memory includes short-term memory, which utilizes in-context learning, and '\n",
" 'long-term memory, providing the agent with the capability to retain and '\n",
" 'recall information over extended periods using an external vector store and '\n",
" 'fast retrieval. The agent also learns to call external APIs for missing '\n",
" 'information.\\n'\n",
" '\\n'\n",
" 'Types of Memory:\\n'\n",
" '1. Sensory Memory: retains impressions of sensory information for a few '\n",
" 'seconds.\\n'\n",
" '2. Short-Term Memory (STM) or Working Memory: stores information needed for '\n",
" 'complex cognitive tasks and lasts for 20-30 seconds.\\n'\n",
" '3. Long-Term Memory (LTM): stores information for a remarkably long time, '\n",
" 'with two subtypes: explicit/declarative memory and implicit/procedural '\n",
" 'memory.\\n'\n",
" '\\n'\n",
" 'The agent uses LLM as its core controller, which can be extended beyond '\n",
" 'generating well-written copies, stories, essays, and programs to a powerful '\n",
" 'general problem solver.')\n"
"(' AlphaCodium is a new approach to code generation by LLMs, proposed in a '\n",
" 'paper titled \"Code Generation with AlphaCodium: From Prompt Engineering to '\n",
" 'Flow Engineering.\" It\\'s described as a test-based, multi-stage flow that '\n",
" 'improves the performance of LLMs on code problems without requiring '\n",
" 'fine-tuning. The iterative process involves repeatedly running and fixing '\n",
" 'generated code against input-output tests, with two key elements being '\n",
" 'generating additional data for the process and enrichment.')\n"
]
}
],
"source": [
"# Run\n",
"inputs = {\n",
" \"keys\": {\n",
" \"question\": \"Explain how the different types of agent memory work?\",\n",
" \"local\": run_local,\n",
" }\n",
"}\n",
"for output in app.stream(inputs):\n",
" for key, value in output.items():\n",
" # Node\n",
" pprint.pprint(f\"Node '{key}':\")\n",
" # Optional: print full state at each node\n",
" # pprint.pprint(value[\"keys\"], indent=2, width=80, depth=None)\n",
" pprint.pprint(\"\\n---\\n\")\n",
"\n",
"# Final generation\n",
"pprint.pprint(value[\"keys\"][\"generation\"])"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "deb28175-27a1-4afc-9747-2983e87fc881",
"metadata": {},
"outputs": [],
"source": [
"from pprint import pprint\n",
"\n",
@@ -762,6 +752,16 @@
"# Final generation\n",
"pprint(value[\"generation\"])"
]
},
{
"cell_type": "markdown",
"id": "598fe73e-f4e6-479f-8d9d-fc81680fff21",
"metadata": {},
"source": [
"Trace: \n",
"\n",
"https://smith.langchain.com/public/c8b75f1b-38b7-48f2-a399-7ebb969d34f6/r"
]
}
],
"metadata": {
examples/rag/langgraph_self_rag_mistral_nomic.ipynb → examples/rag/langgraph_self_rag_local.ipynb
+106 -97
View File
@@ -60,7 +60,7 @@
"metadata": {},
"outputs": [],
"source": [
"! pip install -U llama-cpp-python langchain-nomic langchain_community tiktoken langchainhub chromadb langchain langgraph"
"! pip install -U langchain-nomic langchain_community tiktoken langchainhub chromadb langchain langgraph"
]
},
{
@@ -70,12 +70,6 @@
"source": [
"### LLMs\n",
"\n",
"You can run this in two ways:\n",
"\n",
"(1) Use [Mistral API](https://auth.mistral.ai/ui/login?flow=cc5d3fa5-122b-4c87-bcd8-81e8151c6753).\n",
"\n",
"(2) Run locally, as shown below.\n",
"\n",
"#### Local Embeddings\n",
"\n",
"You can use `GPT4AllEmbeddings()` from Nomic, which can access use Nomic's recently released [v1](https://blog.nomic.ai/posts/nomic-embed-text-v1) and [v1.5](https://blog.nomic.ai/posts/nomic-embed-matryoshka) embeddings.\n",
@@ -89,19 +83,19 @@
"\n",
"(2) Download a `Mistral` model from various Mistral versions [here](https://ollama.ai/library/mistral) and Mixtral versions [here](https://ollama.ai/library/mixtral) available.\n",
"```\n",
"ollama pull mistral:instruct\n",
"ollama pull mistral\n",
"```"
]
},
{
"cell_type": "code",
"execution_count": null,
"execution_count": 2,
"id": "bedffc73-6b10-42c8-8768-2085c8ed3398",
"metadata": {},
"outputs": [],
"source": [
"# Ollama model name\n",
"local_llm = \"mistral:instruct\""
"local_llm = \"mistral\""
]
},
{
@@ -138,7 +132,7 @@
},
{
"cell_type": "code",
"execution_count": 1,
"execution_count": 4,
"id": "c3bb9060-ad74-4470-9991-2ba167b6b8d8",
"metadata": {},
"outputs": [],
@@ -151,7 +145,7 @@
"urls = [\n",
" \"https://lilianweng.github.io/posts/2023-06-23-agent/\",\n",
" \"https://lilianweng.github.io/posts/2023-03-15-prompt-engineering/\",\n",
" \"https://lilianweng.github.io/posts/2023-10-25-adv-attack-llm/\",ß\n",
" \"https://lilianweng.github.io/posts/2023-10-25-adv-attack-llm/\",\n",
"]\n",
"\n",
"docs = [WebBaseLoader(url).load() for url in urls]\n",
@@ -181,22 +175,29 @@
},
{
"cell_type": "code",
"execution_count": null,
"execution_count": 6,
"id": "3aad0c60-3208-48fb-af82-0024630b4da1",
"metadata": {},
"outputs": [],
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"{'score': 'yes'}\n"
]
}
],
"source": [
"### Retrieval Grader \n",
"\n",
"from langchain.prompts import PromptTemplate\n",
"from langchain_community.chat_models import ChatOllama\n",
"from langchain_mistralai.chat_models import ChatMistralAI\n",
"from langchain_core.output_parsers import JsonOutputParser\n",
"\n",
"# LLM\n",
"llm = ChatOllama(model=local_llm, format=\"json\", temperature=0)\n",
"\n",
"prompt = PromptTemplate(ß\n",
"prompt = PromptTemplate(\n",
" template=\"\"\"You are a grader assessing relevance of a retrieved document to a user question. \\n \n",
" Here is the retrieved document: \\n\\n {document} \\n\\n\n",
" Here is the user question: {question} \\n\n",
@@ -216,10 +217,18 @@
},
{
"cell_type": "code",
"execution_count": null,
"execution_count": 7,
"id": "e5e45953-248d-492f-af28-d5e80c664c95",
"metadata": {},
"outputs": [],
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
" In an LLM-powered autonomous agent system, the Large Language Model (LLM) functions as the agent's brain. The agent has key components including memory, planning, and reflection mechanisms. The memory component is a long-term memory module that records a comprehensive list of agents’ experience in natural language. It includes a memory stream, which is an external database for storing past experiences. The reflection mechanism synthesizes memories into higher-level inferences over time and guides the agent's future behavior.\n"
]
}
],
"source": [
"### Generate\n",
"\n",
@@ -246,10 +255,21 @@
},
{
"cell_type": "code",
"execution_count": null,
"execution_count": 8,
"id": "56297862-df87-42a7-ba9d-310926dfb328",
"metadata": {},
"outputs": [],
"outputs": [
{
"data": {
"text/plain": [
"{'score': 'yes'}"
]
},
"execution_count": 8,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"### Hallucination Grader \n",
"\n",
@@ -275,10 +295,21 @@
},
{
"cell_type": "code",
"execution_count": null,
"execution_count": 10,
"id": "e1dd9174-2df6-45b1-8e69-13381f579c39",
"metadata": {},
"outputs": [],
"outputs": [
{
"data": {
"text/plain": [
"{'score': 'yes'}"
]
},
"execution_count": 10,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"### Answer Grader \n",
"\n",
@@ -299,15 +330,26 @@
")\n",
"\n",
"answer_grader = prompt | llm | JsonOutputParser()\n",
"answer_grader.invoke({\"question\": question,\"generation\": generation}"
"answer_grader.invoke({\"question\": question,\"generation\": generation})"
]
},
{
"cell_type": "code",
"execution_count": null,
"execution_count": 11,
"id": "5216d92b-1ca1-4bcf-a34f-c99ec2766b54",
"metadata": {},
"outputs": [],
"outputs": [
{
"data": {
"text/plain": [
"' What is agent memory and how can it be effectively utilized in vector database retrieval?'"
]
},
"execution_count": 11,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"### Question Re-writer\n",
"\n",
@@ -315,13 +357,11 @@
"llm = ChatOllama(model=local_llm, temperature=0)\n",
"\n",
"# Prompt \n",
"system = \"\"\"You a question re-writer that converts an input question to a better version that is optimized \\n \n",
" for vectorstore retrieval. Look at the input and try to reason about the underlying sematic intent / meaning.\"\"\"\n",
"re_write_prompt = ChatPromptTemplate.from_messages(\n",
" [\n",
" (\"system\", system),\n",
" (\"human\", \"Here is the initial question: \\n\\n {question} \\n Formulate an improved question.\"),\n",
" ]\n",
"re_write_prompt = PromptTemplate(\n",
" template=\"\"\"You a question re-writer that converts an input question to a better version that is optimized \\n \n",
" for vectorstore retrieval. Look at the initial and formulate an improved question. \\n\n",
" Here is the initial question: \\n\\n {question}. Improved question with no preamble: \\n \"\"\",\n",
" input_variables=[\"generation\", \"question\"],\n",
")\n",
"\n",
"question_rewriter = re_write_prompt | llm | StrOutputParser()\n",
@@ -342,7 +382,7 @@
},
{
"cell_type": "code",
"execution_count": 44,
"execution_count": 13,
"id": "90fb1dc6-c482-483a-8441-39965c401beb",
"metadata": {},
"outputs": [],
@@ -366,7 +406,7 @@
},
{
"cell_type": "code",
"execution_count": 45,
"execution_count": 14,
"id": "5324ea49-5745-47b5-a0a5-bf58c8babe46",
"metadata": {},
"outputs": [],
@@ -429,7 +469,7 @@
" filtered_docs = []\n",
" for d in documents:\n",
" score = retrieval_grader.invoke({\"question\": question, \"document\": d.page_content})\n",
" grade = score.binary_score\n",
" grade = score['score']\n",
" if grade == \"yes\":\n",
" print(\"---GRADE: DOCUMENT RELEVANT---\")\n",
" filtered_docs.append(d)\n",
@@ -501,7 +541,7 @@
" generation = state[\"generation\"]\n",
"\n",
" score = hallucination_grader.invoke({\"documents\": documents, \"generation\": generation})\n",
" grade = score.binary_score\n",
" grade = score['score']\n",
"\n",
" # Check hallucination\n",
" if grade == \"yes\":\n",
@@ -509,7 +549,7 @@
" # Check question-answering\n",
" print(\"---GRADE GENERATION vs QUESTION---\")\n",
" score = answer_grader.invoke({\"question\": question,\"generation\": generation})\n",
" grade = score.binary_score\n",
" grade = score['score']\n",
" if grade == \"yes\":\n",
" print(\"---DECISION: GENERATION ADDRESSES QUESTION---\")\n",
" return \"useful\"\n",
@@ -533,7 +573,7 @@
},
{
"cell_type": "code",
"execution_count": 46,
"execution_count": 15,
"id": "5605dee4-b2df-46ae-a640-cc2ed90c21a6",
"metadata": {},
"outputs": [],
@@ -579,29 +619,15 @@
"id": "105ae1b5-6963-4186-bb83-6d6cb96d095f",
"metadata": {},
"source": [
"## Run\n",
"\n",
"Trace for below run: https://smith.langchain.com/public/928651fd-85b3-49ff-b481-bd28417645e5/r"
"## Run\n"
]
},
{
"cell_type": "code",
"execution_count": 47,
"execution_count": 16,
"id": "26a64f7d-0c14-4e31-a67f-63021dee626e",
"metadata": {},
"outputs": [
{
"name": "stderr",
"output_type": "stream",
"text": [
"\n",
"llama_print_timings: load time = 149.49 ms\n",
"llama_print_timings: sample time = 0.00 ms / 1 runs ( 0.00 ms per token, inf tokens per second)\n",
"llama_print_timings: prompt eval time = 17.39 ms / 12 tokens ( 1.45 ms per token, 690.01 tokens per second)\n",
"llama_print_timings: eval time = 0.00 ms / 1 runs ( 0.00 ms per token, inf tokens per second)\n",
"llama_print_timings: total time = 17.39 ms / 13 tokens\n"
]
},
{
"name": "stdout",
"output_type": "stream",
@@ -609,61 +635,34 @@
"---RETRIEVE---\n",
"\"Node 'retrieve':\"\n",
"'\\n---\\n'\n",
"---CHECK RELEVANCE---\n",
"---CHECK DOCUMENT RELEVANCE TO QUESTION---\n",
"---GRADE: DOCUMENT RELEVANT---\n",
"---GRADE: DOCUMENT RELEVANT---\n",
"---GRADE: DOCUMENT RELEVANT---\n",
"---GRADE: DOCUMENT RELEVANT---\n",
"\"Node 'grade_documents':\"\n",
"'\\n---\\n'\n",
"---DECIDE TO GENERATE---\n",
"---ASSESS GRADED DOCUMENTS---\n",
"---DECISION: GENERATE---\n",
"---GENERATE---\n",
"\"Node 'generate':\"\n",
"'\\n---\\n'\n",
"---GRADE GENERATION vs DOCUMENTS---\n",
"---DECISION: SUPPORTED, MOVE TO FINAL GRADE---\n",
"---FINAL GRADE---\n",
"\"Node 'prepare_for_final_grade':\"\n",
"'\\n---\\n'\n",
"---CHECK HALLUCINATIONS---\n",
"---DECISION: GENERATION IS GROUNDED IN DOCUMENTS---\n",
"---GRADE GENERATION vs QUESTION---\n",
"---DECISION: USEFUL---\n",
"---DECISION: GENERATION ADDRESSES QUESTION---\n",
"\"Node '__end__':\"\n",
"'\\n---\\n'\n",
"(' In a LLM (large language model)-powered autonomous agent system, LLM '\n",
" 'functions as the agent’s brain, complemented by several key components. One '\n",
" 'of these components is memory. Memory can be defined as the processes used '\n",
" 'to acquire, store, retain, and later retrieve information. There are several '\n",
" 'types of memory in human brains:\\n'\n",
" '\\n'\n",
" '1. Sensory Memory: This is the earliest stage of memory, providing the '\n",
" 'ability to retain impressions of sensory information (visual, auditory, etc) '\n",
" 'after the original stimuli have ended. Sensory memory typically only lasts '\n",
" 'for up to a few seconds. Subcategories include iconic memory (visual), '\n",
" 'echoic memory (auditory), and haptic memory (touch).\\n'\n",
" '2. Short-Term Memory (STM) or Working Memory: It stores information that we '\n",
" 'are currently aware of and needed to carry out complex cognitive tasks such '\n",
" 'as learning and reasoning. Short-term memory is believed to have the '\n",
" 'capacity of about 7 items (Miller 1956) and lasts for 20-30 seconds.\\n'\n",
" '3. Long-Term Memory (LTM): Long-term memory can store information for a '\n",
" 'remarkably long time, ranging from a few days to decades, with an '\n",
" 'essentially unlimited storage capacity. There are two subtypes of LTM:\\n'\n",
" ' * Explicit / declarative memory: This is memory of facts and events, and '\n",
" 'refers to those memories that can be consciously recalled, including '\n",
" 'episodic memory (events and experiences) and semantic memory (facts and '\n",
" 'concepts).\\n'\n",
" ' * Implicit / procedural memory: This type of memory is unconscious and '\n",
" 'involves skills and routines that are performed automatically, like riding a '\n",
" 'bike or typing on a keyboard.\\n'\n",
" '\\n'\n",
" 'We can roughly consider the following mappings in an LLM-powered agent '\n",
" 'system:\\n'\n",
" '\\n'\n",
" '* Sensory Memory: Input data from sensors\\n'\n",
" '* Short-Term Memory: Active processing of information, temporary storage for '\n",
" 'complex tasks\\n'\n",
" '* Long-Term Memory: Stored knowledge and experiences that can be accessed '\n",
" 'and used to learn new tasks or make decisions.')\n"
"(' In a LLM-powered autonomous agent system, memory is a key component that '\n",
" 'enables agents to store and retrieve information. There are different types '\n",
" 'of memory in human brains, such as sensory memory which retains impressions '\n",
" 'of sensory information for a few seconds, and long-term memory which records '\n",
" \"experiences for extended periods (Lil'Log, 2023). In the context of LLM \"\n",
" 'agents, memory is often implemented as an external database or memory stream '\n",
" \"(Lil'Log, 2023). The agent can consult this memory to inform its behavior \"\n",
" 'based on relevance, recency, and importance. Additionally, reflection '\n",
" 'mechanisms synthesize memories into higher-level inferences over time and '\n",
" \"guide the agent's future behavior (Lil'Log, 2023).\")\n"
]
}
],
@@ -684,10 +683,20 @@
"pprint(value[\"generation\"])"
]
},
{
"cell_type": "markdown",
"id": "f9a27907-2611-4791-910b-0d66c59f5cf5",
"metadata": {},
"source": [
"Trace: \n",
"\n",
"https://smith.langchain.com/public/4163a342-5260-4852-8602-bda3f95177e7/r"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "4d40a430-be4a-4d9c-98f9-46c5eb3066e8",
"id": "953143c2-2f2a-4361-a36b-87db7cf21d63",
"metadata": {},
"outputs": [],
"source": []