langgraph/examples/web-research.ipynb

In [1]:

from operator import itemgetter

from langchain.chat_models import ChatOpenAI, ChatAnthropic
from langchain.prompts import SystemMessagePromptTemplate, ChatPromptTemplate
from langchain.schema.output_parser import StrOutputParser
from langchain.runnables.openai_functions import OpenAIFunctionsRouter

from permchain.connection_inmemory import InMemoryPubSubConnection
from permchain.pubsub import PubSub
from permchain.topic import Topic

Content Fetcher

First, we are going to define our content fetcher. This is responsible for taking a search query an getting relevant web pages.

In [2]:

from langchain.utilities import GoogleSearchAPIWrapper
from langchain.document_loaders import AsyncHtmlLoader
from langchain.document_transformers import Html2TextTransformer

In [3]:

from duckduckgo_search import DDGS

ddgs = DDGS()

In [4]:

def retrieve_documents(query):
    query = query.strip().strip('"')
    search_results = ddgs.text(query)
    urls_to_look = []
    for res in search_results:
        if res.get("href", None):
            urls_to_look.append(res["href"])
        if len(urls_to_look) >= 4:
            break

    # Relevant urls
    # Load, split, and add new urls to vectorstore
    if urls_to_look:
        loader = AsyncHtmlLoader(urls_to_look)
        html2text = Html2TextTransformer()
        docs = loader.load()
        docs = list(html2text.transform_documents(docs))
    else:
        docs = []
    return docs

In [5]:

import nest_asyncio

nest_asyncio.apply()

In [6]:

# docs = retrieve_documents("langchain")

Summarizer

We will now come up with an actor to summarize the results given a query and some search results

In [7]:

prompt = ChatPromptTemplate.from_template(
    "Answer the user's question given the search results\n\n<question>{question}</question><search_results>{search_results}</search_results>"
)

In [8]:

summarizer_chain = (
    prompt
    | ChatOpenAI(max_retries=0).with_fallbacks(
        [ChatOpenAI(model="gpt-3.5-turbo-16k"), ChatAnthropic(model="claude-2")]
    )
    | StrOutputParser()
)

All together now!

In [9]:

summarizer_inbox = Topic("summarizer")

In [10]:

search_actor = (
    Topic.IN.subscribe()
    | {
        "search_results": retrieve_documents,
        "question": Topic.IN.current(),
    }
    | summarizer_inbox.publish()
)

In [11]:

summ_actor = (
    summarizer_inbox.subscribe() | {"answer": summarizer_chain} | Topic.OUT.publish()
)

In [12]:

web_researcher = PubSub(
    processes=(search_actor, summ_actor),
    connection=InMemoryPubSubConnection(),
).with_config(run_name="WebResearcher")

In [13]:

web_researcher.invoke("What is langsmith?")

Out [13]:

Fetching pages: 100%|###################################################| 4/4 [00:01<00:00,  2.46it/s]

[{'answer': 'LangSmith is a platform that helps developers build production-grade language model applications and allows for efficient development lifecycles, maintenance, and improvement of AI models. It is built by the developers who created LangChain and integrates seamlessly with that library. LangSmith provides features such as tracing runs associated with an active instance and testing and evaluating prompts or answers generated by the language model applications. It aims to address the challenges of building reliable and maintainable language model applications for production. For more information, you can refer to the LangSmith documentation.'}]

In [14]:

web_researcher.batch(["what is langsmith", "what is llama"])

Out [14]:

Fetching pages:   0%|                                                           | 0/4 [00:00<?, ?it/s]
Fetching pages: 100%|###################################################| 4/4 [00:00<00:00,  8.78it/s]
Fetching pages: 100%|###################################################| 4/4 [00:01<00:00,  2.76it/s]

[[{'answer': 'Based on the search results, LangSmith is a platform that helps developers build and evaluate language model applications. It is designed to assist in moving from prototyping to production and aims to address the challenges of building and maintaining reliable and consistent language model applications. LangSmith is built by the creators of LangChain, a popular language model software tool. It provides features for tracing, testing, evaluating, and monitoring language model applications. LangSmith integrates seamlessly with LangChain and requires a sign-up and API key to access its capabilities.'}],
 [{'answer': 'According to the search results, a llama is a domesticated livestock species that is a descendant of the guanaco and belongs to the camel family. Llamas are primarily used as pack animals and a source of food, wool, hides, tallow for candles, and dried dung for fuel. They are found primarily in South American countries such as Bolivia, Peru, Colombia, Ecuador, Chile, and Argentina. Llamas are known for their long necks, long legs, small heads, large pointed ears, and ability to graze on grass and other plants. They are gregarious animals and can interbreed with other lamoids, such as guanacos, vicuñas, and alpacas. Llama fleece is sheared every two years and consists of coarse guard hairs and short crimped fibers. The fleece is used for knitwear, woven fabrics, rugs, rope, and fabric.'}]]

Trying to use it as a sub component

In [15]:

from langchain.output_parsers.openai_functions import JsonKeyOutputFunctionsParser

In [16]:

template = """Write between 2 and 5 sub questions that serve as google search queries to search online that form an objective opinion from the following: {question}"""
functions = [
    {
        "name": "sub_questions",
        "description": "List of sub questions",
        "parameters": {
            "type": "object",
            "properties": {
                "questions": {
                    "type": "array",
                    "description": "List of sub questions to ask.",
                    "items": {"type": "string"},
                },
            },
        },
    },
]
prompt = ChatPromptTemplate.from_template(template)
question_chain = (
    prompt
    | ChatOpenAI(temperature=0).bind(
        functions=functions, function_call={"name": "sub_questions"}
    )
    | JsonKeyOutputFunctionsParser(key_name="questions")
)

In [17]:

question_chain.invoke({"question": "what is langsmith?"})

Out [17]:

['What is the purpose of Langsmith?',
 'Who developed Langsmith?',
 'What are the key features of Langsmith?',
 'Are there any alternatives to Langsmith?',
 'What are the reviews or feedback on Langsmith?']

In [18]:

template = """You are tasked with writing a research report to answer the following question:

<question>
{question}
</question>

In order to do that, you first came up with several sub questions and researched those. please find those below:

<research>
{research}
</research>

Now, write your final report answering the original question!"""
prompt = ChatPromptTemplate.from_template(template)
report_chain = prompt | ChatOpenAI() | StrOutputParser()

In [19]:

research_inbox = Topic("research")
writer_inbox = Topic("writer_inbox")

In [20]:

subquestion_actor = (
    # Listed in inputs
    Topic.IN.subscribe()
    | question_chain
    # The draft always goes to the editors inbox
    | research_inbox.publish()
)
research_actor = (
    research_inbox.subscribe()
    | {
        "research": web_researcher.map(),
        # "research": lambda x: [web_researcher.invoke({"question": i}) for i in x],
        "question": Topic.IN.current() | itemgetter("question"),
    }
    | writer_inbox.publish()
)
write_actor = writer_inbox.subscribe() | report_chain | Topic.OUT.publish()

In [21]:

longer_researcher = PubSub(
    processes=(subquestion_actor, research_actor, write_actor),
    connection=InMemoryPubSubConnection(),
).with_config(run_name="LongResearcher")

In [22]:

longer_researcher.invoke({"question": "what is langsmith?"})

Out [22]:

Fetching pages:   0%|                                                           | 0/4 [00:00<?, ?it/s]
Fetching pages:   0%|                                                           | 0/4 [00:00<?, ?it/s]


Fetching pages:   0%|                                                           | 0/4 [00:00<?, ?it/s]



Fetching pages:   0%|                                                           | 0/4 [00:00<?, ?it/s]

Fetching pages:   0%|                                                           | 0/4 [00:00<?, ?it/s]


Fetching pages: 100%|###################################################| 4/4 [00:01<00:00,  3.33it/s]




Fetching pages: 100%|###################################################| 4/4 [00:01<00:00,  3.02it/s]

Fetching pages: 100%|###################################################| 4/4 [00:01<00:00,  2.86it/s]


Fetching pages: 100%|###################################################| 4/4 [00:01<00:00,  2.33it/s]
Fetching pages: 100%|###################################################| 4/4 [00:02<00:00,  1.94it/s]

["Research Report: Understanding LangSmith - A Unified Platform for Building LLM Applications\n\nIntroduction:\nThe purpose of this research report is to provide a comprehensive understanding of LangSmith, a unified platform designed to assist developers in building production-grade Language Model applications (LLMs). By exploring various sub-questions, we have gathered information about LangSmith's features, development team, alternatives, and user feedback.\n\n1. What is LangSmith?\nLangSmith is a unified platform developed by LangChain, the creators of LangChain itself. It aims to streamline the development lifecycle, maintenance, and improvement of LLM applications. By providing tools for debugging, testing, evaluating, and monitoring LLM applications, LangSmith enables developers to transition from prototype to production seamlessly. It addresses the challenges associated with building reliable and maintainable LLM applications in production environments.\n\n2. Features and Integration:\nLangSmith offers a range of features to facilitate the development and management of LLM applications. These include tracing runs, testing and evaluating LLM-generated prompts or answers, and visualizing and replaying traces. The platform integrates seamlessly with LangChain, providing native integration for debugging and monitoring LLM applications.\n\n3. Getting Started with LangSmith:\nTo utilize LangSmith, users need to sign up for an account and create an API key. Additionally, having a GitHub repository and an OpenAI API key is necessary for integration. Although LangSmith is currently in beta, periodic access to new sign-ups is available.\n\n4. Alternatives to LangSmith:\nBased on our research, we have identified several alternatives to LangSmith for developers working with LLM applications. These alternatives offer similar functionalities and may serve as viable options depending on specific project requirements. Some notable alternatives include:\n\n- LangChain: An open source framework for building conversational AI agents, which integrates with LangSmith for debugging and monitoring.\n- Autoblocks: A tool that monitors and improves AI models powered by large language models, providing an SDK for easy integration.\n- BenchLLM: An open source tool specifically designed for evaluating large language models, supporting models from various providers including OpenAI and LangChain.\n- GradientJ: A comprehensive platform for building, evaluating, and managing large language models, offering features such as prompt chaining and data integration.\n- Portkey: An LMOps platform catering to the deployment of production-ready LLM applications, offering model management, monitoring, and versioning tools.\n- Vellum AI: A platform equipped with tools for developing LLM applications, including prompt engineering, version control, testing, and monitoring. Compatible with multiple LLM providers.\n- Openlayer: A collaborative platform that facilitates aligning expectations around LLM quality and performance, providing diagnostic tools for issue resolution and iteration.\n- Metal: A tool that focuses on providing LLM developers with a seamless workflow, although specific details about its features are not available.\n\n5. User Feedback:\nLangSmith has received positive reviews and feedback from the community. Although the reviews were based on the tool's description and potential rather than personal usage, LangSmith has an overall rating of 5/5. Users have praised the platform for simplifying the transition of projects from the prototyping phase to full-fledged production. LangSmith's ability to manage the complexity of LLM applications and enhance product development and iteration capabilities has been highlighted.\n\nConclusion:\nIn conclusion, LangSmith is a unified platform developed by LangChain to assist developers in building production-grade Language Model applications. It offers a range of features for debugging, testing, evaluating, and monitoring LLM applications, aiming to simplify the development lifecycle, maintenance, and improvement of these applications. Although alternatives exist, LangSmith has received positive feedback and has the potential to enhance productivity and workflow for developers working with LLMs."]

In [ ]: