diff --git a/examples/rag/langgraph_crag_mistral.ipynb b/examples/rag/langgraph_crag_mistral.ipynb index 807e6273c..502498cfa 100644 --- a/examples/rag/langgraph_crag_mistral.ipynb +++ b/examples/rag/langgraph_crag_mistral.ipynb @@ -7,7 +7,7 @@ "metadata": {}, "outputs": [], "source": [ - "! pip install --quiet langchain_community tiktoken langchain-openai langchainhub chromadb langchain langgraph tavily-python langchain-mistralai gpt4all" + "! pip install --quiet langchain_community tiktoken langchain-openai langchainhub chromadb langchain langgraph tavily-python langchain-mistralai gpt4all llama-cpp-python" ] }, { @@ -45,15 +45,42 @@ "\n", "### Running Locally \n", "\n", - "If you want to run this locally (e.g., on your laptop), use [Ollama](https://ollama.ai/library/mistral/tags):\n", + "#### Embeddings\n", + "\n", + "There are several options for local embeddings.\n", + "\n", + "(1) You can use `GPT4AllEmbeddings()` from Nomic.\n", + "\n", + "(2) You can also use Nomic's recently released [v1](https://blog.nomic.ai/posts/nomic-embed-text-v1) and [v1.5](https://blog.nomic.ai/posts/nomic-embed-matryoshka) embeddings.\n", + "\n", + "For these, simply:\n", + "\n", + "Clone [`llama.cpp`](https://github.com/ggerganov/llama.cpp):\n", "\n", - "* Download [Ollama app](https://ollama.ai/).\n", - "* Download a `Mistral` model e.g., `ollama pull mistral:instruct`, from various Mistral versions [here](https://ollama.ai/library/mistral) and Mixtral versions [here](https://ollama.ai/library/mixtral) available.\n", - "* Set flags indicating we will run locally and the Mistral model downloaded:\n", - " \n", "```\n", - "run_local = \"Yes\"\n", - "local_llm = \"mistral:instruct\"\n", + "git clone https://github.com/ggerganov/llama.cpp\n", + "```\n", + "\n", + "Download GGUF weights for Nomic's embedding model(s), allowing them to be run locally: \n", + "\n", + "* https://huggingface.co/nomic-ai/nomic-embed-text-v1-GGUF\n", + "* https://huggingface.co/nomic-ai/nomic-embed-text-v1.5-GGUF\n", + "\n", + "Add to `llama.cpp/model` directory.\n", + "\n", + "Build llama.cpp:\n", + "```\n", + "cd llama.cpp\n", + "make\n", + "```\n", + "\n", + "### LLM\n", + "\n", + "(1) Download [Ollama app](https://ollama.ai/).\n", + "\n", + "(2) Download a `Mistral` model from various Mistral versions [here](https://ollama.ai/library/mistral) and Mixtral versions [here](https://ollama.ai/library/mixtral) available.\n", + "```\n", + "ollama pull mistral:instruct\n", "```\n", "\n", "### Tracing \n", @@ -136,6 +163,7 @@ "from langchain_community.vectorstores import Chroma\n", "from langchain_mistralai import MistralAIEmbeddings\n", "from langchain_community.embeddings import GPT4AllEmbeddings\n", + "from langchain_community.embeddings import LlamaCppEmbeddings\n", "\n", "# Load\n", "url = \"https://lilianweng.github.io/posts/2023-06-23-agent/\"\n", @@ -150,7 +178,11 @@ "\n", "# Embed and index\n", "if run_local == \"Yes\":\n", + " # GPT4All\n", " embedding = GPT4AllEmbeddings()\n", + " # Nomic v1 or v1.5\n", + " # embd_model_path = \"/Users/rlm/Desktop/Code/llama.cpp/models/nomic-embd/nomic-embed-text-v1.Q4_K_S.gguf\"\n", + " # embedding = LlamaCppEmbeddings(model_path=embd_model_path, n_batch=512)\n", "else:\n", " embedding = MistralAIEmbeddings(mistral_api_key=mistral_api_key)\n", "\n", diff --git a/examples/rag/langgraph_self_rag_mistral_nomic.ipynb b/examples/rag/langgraph_self_rag_mistral_nomic.ipynb index f26ef00fb..d23c73d9d 100644 --- a/examples/rag/langgraph_self_rag_mistral_nomic.ipynb +++ b/examples/rag/langgraph_self_rag_mistral_nomic.ipynb @@ -68,7 +68,7 @@ "ollama pull mistral:instruct\n", "```\n", "\n", - "(3) Set flags indicating we will run locally and the Mistral model downloaded." + "(3) Set `local_llm` to the model downloaded." ] }, {