langgraph/examples/web-research.ipynb

In [1]:

from operator import itemgetter

from langchain.chat_models import ChatOpenAI, ChatAnthropic
from langchain.prompts import SystemMessagePromptTemplate, ChatPromptTemplate
from langchain.schema.output_parser import StrOutputParser
from langchain.runnables.openai_functions import OpenAIFunctionsRouter

from permchain.connection_inmemory import InMemoryPubSubConnection
from permchain.pubsub import PubSub
from permchain.topic import Topic

Content Fetcher

First, we are going to define our content fetcher. This is responsible for taking a search query an getting relevant web pages.

In [2]:

from langchain.utilities import GoogleSearchAPIWrapper
from langchain.document_loaders import AsyncHtmlLoader
from langchain.document_transformers import Html2TextTransformer

In [3]:

from duckduckgo_search import DDGS

ddgs = DDGS()

In [4]:

def retrieve_documents(query):
    query = query.strip().strip('"')
    search_results = ddgs.text(query)
    urls_to_look = []
    for res in search_results:
        if res.get("href", None):
            urls_to_look.append(res["href"])
        if len(urls_to_look) >= 4:
            break

    # Relevant urls
    # Load, split, and add new urls to vectorstore
    if urls_to_look:
        loader = AsyncHtmlLoader(urls_to_look)
        html2text = Html2TextTransformer()
        docs = loader.load()
        docs = list(html2text.transform_documents(docs))
    else:
        docs = []
    return docs

In [5]:

import nest_asyncio

nest_asyncio.apply()

In [6]:

# docs = retrieve_documents("langchain")

Summarizer

We will now come up with an actor to summarize the results given a query and some search results

In [24]:

prompt = ChatPromptTemplate.from_template(
    "Answer the user's question given the search results\n\n<question>{question}</question><search_results>{search_results}</search_results>"
)

In [25]:

summarizer_chain = (
    prompt
    | ChatOpenAI(max_retries=0).with_fallbacks(
        [ChatOpenAI(model="gpt-3.5-turbo-16k"), ChatAnthropic(model="claude-2")]
    )
    | StrOutputParser()
)

All together now!

In [26]:

summarizer_inbox = Topic("summarizer")

In [27]:

search_actor = (
    Topic.IN.subscribe()
    | {
        "search_results": retrieve_documents,
        "question": Topic.IN.current(),
    }
    | summarizer_inbox.publish()
)

In [43]:

summ_actor = (
    summarizer_inbox.subscribe() | {"answer": summarizer_chain} | Topic.OUT.publish()
)

In [44]:

web_researcher = PubSub(
    processes=(search_actor, summ_actor),
    connection=InMemoryPubSubConnection(),
)

In [45]:

web_researcher.invoke("What is langsmith?")

Out [45]:

Fetching pages: 100%|#############################################################################################################################################################################################| 4/4 [00:00<00:00,  6.44it/s]

[{'answer': 'LangSmith is a platform for building production-grade language model applications. It helps trace and evaluate language model applications and intelligent agents, making it easier to move from prototype to production. LangSmith is developed by LangChain, the company behind the open source LangChain framework. More information can be found in the LangSmith documentation.'}]

In [47]:

web_researcher.batch(["what is langsmith", "what is llama"])

Fetching pages:   0%|                                                                                                                                                                                                     | 0/4 [00:00<?, ?it/s]
Fetching pages: 100%|#############################################################################################################################################################################################| 4/4 [00:00<00:00,  6.93it/s]
Fetching pages: 100%|#############################################################################################################################################################################################| 4/4 [00:00<00:00,  4.79it/s]

---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
Cell In[47], line 1
----> 1 web_researcher.batch(["what is langsmith", "what is llama"])

File ~/workplace/langchain/libs/langchain/langchain/schema/runnable/base.py:102, in Runnable.batch(self, inputs, config, max_concurrency)
     99     return [self.invoke(inputs[0], configs[0])]
    101 with ThreadPoolExecutor(max_workers=max_concurrency) as executor:
--> 102     return list(executor.map(self.invoke, inputs, configs))

File ~/.pyenv/versions/3.10.1/lib/python3.10/concurrent/futures/_base.py:608, in Executor.map.<locals>.result_iterator()
    605 while fs:
    606     # Careful not to keep a reference to the popped future
    607     if timeout is None:
--> 608         yield fs.pop().result()
    609     else:
    610         yield fs.pop().result(end_time - time.monotonic())

File ~/.pyenv/versions/3.10.1/lib/python3.10/concurrent/futures/_base.py:445, in Future.result(self, timeout)
    443     raise CancelledError()
    444 elif self._state == FINISHED:
--> 445     return self.__get_result()
    446 else:
    447     raise TimeoutError()

File ~/.pyenv/versions/3.10.1/lib/python3.10/concurrent/futures/_base.py:390, in Future.__get_result(self)
    388 if self._exception:
    389     try:
--> 390         raise self._exception
    391     finally:
    392         # Break a reference cycle with the exception in self._exception
    393         self = None

File ~/.pyenv/versions/3.10.1/lib/python3.10/concurrent/futures/thread.py:58, in _WorkItem.run(self)
     55     return
     57 try:
---> 58     result = self.fn(*self.args, **self.kwargs)
     59 except BaseException as exc:
     60     self.future.set_exception(exc)

File ~/workplace/permchain/permchain/pubsub.py:67, in PubSub.invoke(self, input, config)
     65 def invoke(self, input: Any, config: Optional[RunnableConfig] = None) -> Any:
     66     collected = []
---> 67     for chunk in self.stream(input, config):
     68         collected.append(chunk)
     69     return collected

File ~/workplace/permchain/permchain/pubsub.py:176, in PubSub.stream(self, input, config, max_concurrency)
    174             final_output = chunk
    175         else:
--> 176             final_output += chunk
    177 finally:
    178     # Cleanup
    179     for fut in inflight:

TypeError: unsupported operand type(s) for +=: 'dict' and 'dict'

Trying to use it as a sub component

In [31]:

from langchain.output_parsers.openai_functions import JsonKeyOutputFunctionsParser

In [32]:

template = """Write between 2 and 5 sub questions that serve as google search queries to search online that form an objective opinion from the following: {question}"""
functions = [
    {
        "name": "sub_questions",
        "description": "List of sub questions",
        "parameters": {
            "type": "object",
            "properties": {
                "questions": {
                    "type": "array",
                    "description": "List of sub questions to ask.",
                    "items": {"type": "string"},
                },
            },
        },
    },
]
prompt = ChatPromptTemplate.from_template(template)
question_chain = (
    prompt
    | ChatOpenAI(temperature=0).bind(
        functions=functions, function_call={"name": "sub_questions"}
    )
    | JsonKeyOutputFunctionsParser(key_name="questions")
)

In [33]:

question_chain.invoke({"question": "what is langsmith?"})

Out [33]:

['What is the purpose of Langsmith?',
 'Who developed Langsmith?',
 'What are the features of Langsmith?',
 'How does Langsmith work?',
 'Are there any alternatives to Langsmith?']

In [34]:

template = """You are tasked with writing a research report to answer the following question:

<question>
{question}
</question>

In order to do that, you first came up with several sub questions and researched those. please find those below:

<research>
{research}
</research>

Now, write your final report answering the original question!"""
prompt = ChatPromptTemplate.from_template(template)
report_chain = prompt | ChatOpenAI() | StrOutputParser()

In [35]:

research_inbox = Topic("research")
writer_inbox = Topic("writer_inbox")

In [36]:

subquestion_actor = (
    # Listed in inputs
    Topic.IN.subscribe()
    | question_chain
    # The draft always goes to the editors inbox
    | research_inbox.publish()
)
research_actor = (
    research_inbox.subscribe()
    | {
        "research": lambda x: web_researcher.batch(x),
        # "research": lambda x: [web_researcher.invoke({"question": i}) for i in x],
        "question": Topic.IN.current() | itemgetter("question"),
    }
    | writer_inbox.publish()
)
write_actor = writer_inbox.subscribe() | report_chain | Topic.OUT.publish()

In [37]:

longer_researcher = PubSub(
    processes=(subquestion_actor, research_actor, write_actor),
    connection=InMemoryPubSubConnection(),
)

In [38]:

longer_researcher.invoke({"question": "what is langsmith?"})

Out [38]:

Fetching pages:   0%|                                                                                                                                                                                               | 0/4 [00:00<?, ?it/s]
Fetching pages:   0%|                                                                                                                                                                                               | 0/4 [00:00<?, ?it/s]

Fetching pages:   0%|                                                                                                                                                                                               | 0/4 [00:00<?, ?it/s]


Fetching pages:   0%|                                                                                                                                                                                               | 0/4 [00:00<?, ?it/s]



Fetching pages:   0%|                                                                                                                                                                                               | 0/4 [00:00<?, ?it/s]
Fetching pages: 100%|#######################################################################################################################################################################################| 4/4 [00:01<00:00,  3.84it/s]




Fetching pages: 100%|#######################################################################################################################################################################################| 4/4 [00:01<00:00,  3.64it/s]
Fetching pages: 100%|#######################################################################################################################################################################################| 4/4 [00:01<00:00,  2.88it/s]



Fetching pages: 100%|#######################################################################################################################################################################################| 4/4 [00:01<00:00,  2.85it/s]


Fetching pages: 100%|#######################################################################################################################################################################################| 4/4 [00:01<00:00,  2.69it/s]
Retrying langchain.chat_models.openai.ChatOpenAI.completion_with_retry.<locals>._completion_with_retry in 4.0 seconds as it raised RateLimitError: Rate limit reached for default-gpt-3.5-turbo-16k in organization org-i0zjYONU3PemzJ222esBaAzZ on tokens per min. Limit: 180000 / min. Current: 173743 / min. Contact us through our help center at help.openai.com if you continue to have issues..
Retrying langchain.chat_models.openai.ChatOpenAI.completion_with_retry.<locals>._completion_with_retry in 4.0 seconds as it raised RateLimitError: Rate limit reached for default-gpt-3.5-turbo-16k in organization org-i0zjYONU3PemzJ222esBaAzZ on tokens per min. Limit: 180000 / min. Current: 161254 / min. Contact us through our help center at help.openai.com if you continue to have issues..

['Research Report: Understanding LangSmith\n\nIntroduction:\nThe purpose of this research report is to provide a comprehensive understanding of LangSmith, a developer platform designed to facilitate the development and management of Language Model applications (LLMs). Through an analysis of the gathered research, this report aims to answer the question: "What is LangSmith?"\n\nResearch Findings:\n\n1. LangSmith Overview:\nLangSmith is a unified platform that helps developers trace, evaluate, and monitor LLM applications and intelligent agents. It aims to simplify the process of moving from prototype to production by providing features such as tracing runs, testing prompts or answers, and exporting datasets and runs for further analysis. LangSmith offers comprehensive visibility into the chain sequence of calls, real-time insights, and observability features to monitor LLM applications. It emphasizes best practices and offers useful tools and resources for developers working with LLMs.\n\n2. Key Features of LangSmith:\nLangSmith offers several key features to assist developers in building and managing LLM applications. These include:\n- Tracing and evaluating the behavior of LLM applications.\n- Debugging and experimentation capabilities.\n- Sharing work with others.\n- Creating datasets for testing and evaluation.\n- Evaluating models based on created datasets.\n- Monitoring the behavior and performance of LLM applications.\n- Comprehensive visibility into the entire chain sequence of calls.\n\n3. LangSmith\'s Integration with LangChain:\nLangSmith seamlessly integrates with LangChain, a library for prototyping LLM applications. This integration allows developers to leverage the composability of LangChain and build applications with large language models effectively. LangChain supports features such as memory, custom datasets, and more.\n\n4. Potential Alternatives to LangSmith:\nBased on the research findings, several potential alternatives to LangSmith have been identified. These include:\n- LangChain: An open-source framework for building applications with large language models through composability.\n- GradientJ: A platform to build, orchestrate, and manage complex LLM applications at scale.\n- LLMOps.Space: A community and resource hub focused on deploying LLMs into production.\n- Vellum: A development platform aimed at production LLM applications, providing tools for monitoring, version control, and testing datasets.\n- Llama2: An open-source large language model from Meta that can be fine-tuned and deployed.\n- Openlayer: A platform focused on ML model testing, monitoring, and improvement.\n- Backengine: A platform that allows creating and deploying backend APIs using natural language descriptions.\n- QueryVary: A platform for systematically designing and refining prompts for LLMs.\n\nConclusion:\nIn conclusion, LangSmith is a developer platform designed to simplify the development and management of Language Model applications (LLMs). It provides developers with tools for tracing, testing, evaluating, and monitoring LLM applications, along with comprehensive visibility and real-time insights. LangSmith aims to empower developers and handle the complexity of LLM applications effectively. By integrating seamlessly with LangChain, it offers enhanced capabilities for building applications with large language models. While LangSmith is a prominent platform, developers may also consider other alternatives such as LangChain, GradientJ, LLMOps.Space, Vellum, and more, depending on their specific requirements.']

In [ ]: