Fix publisher

This commit is contained in:
Nuno Campos
2023-08-25 12:06:10 +02:00
parent 67ee1d87c7
commit a4e7e08fcd
2 changed files with 19 additions and 23 deletions
+18 -22
View File
@@ -217,13 +217,13 @@
"name": "stderr",
"output_type": "stream",
"text": [
"Fetching pages: 100%|###################################################| 4/4 [00:01<00:00, 2.46it/s]\n"
"Fetching pages: 100%|###################################################| 4/4 [00:01<00:00, 2.19it/s]\n"
]
},
{
"data": {
"text/plain": [
"[{'answer': 'LangSmith is a platform that helps developers build production-grade language model applications and allows for efficient development lifecycles, maintenance, and improvement of AI models. It is built by the developers who created LangChain and integrates seamlessly with that library. LangSmith provides features such as tracing runs associated with an active instance and testing and evaluating prompts or answers generated by the language model applications. It aims to address the challenges of building reliable and maintainable language model applications for production. For more information, you can refer to the LangSmith documentation.'}]"
"[{'answer': 'LangSmith is a platform built by LangChain to help developers build production-grade language model (LLM) applications. It enables developers to trace and evaluate their LLM applications and intelligent agents, ensuring reliability and maintainability in the production environment. LangSmith integrates seamlessly with LangChain and provides features such as tracing runs, testing, and evaluating prompts or answers generated by LLM applications. It aims to facilitate the development lifecycle, maintenance, and improvement of AI models. For more information, you can refer to the LangSmith documentation.'}]"
]
},
"execution_count": 13,
@@ -246,15 +246,16 @@
"output_type": "stream",
"text": [
"Fetching pages: 0%| | 0/4 [00:00<?, ?it/s]\n",
"Fetching pages: 100%|###################################################| 4/4 [00:00<00:00, 8.78it/s]\u001b[A\n",
"Fetching pages: 100%|###################################################| 4/4 [00:01<00:00, 2.76it/s]\n"
"Fetching pages: 100%|###################################################| 4/4 [00:00<00:00, 6.75it/s]\u001b[A\n",
"\n",
"Fetching pages: 100%|###################################################| 4/4 [00:01<00:00, 2.99it/s]\u001b[A\n"
]
},
{
"data": {
"text/plain": [
"[[{'answer': 'Based on the search results, LangSmith is a platform that helps developers build and evaluate language model applications. It is designed to assist in moving from prototyping to production and aims to address the challenges of building and maintaining reliable and consistent language model applications. LangSmith is built by the creators of LangChain, a popular language model software tool. It provides features for tracing, testing, evaluating, and monitoring language model applications. LangSmith integrates seamlessly with LangChain and requires a sign-up and API key to access its capabilities.'}],\n",
" [{'answer': 'According to the search results, a llama is a domesticated livestock species that is a descendant of the guanaco and belongs to the camel family. Llamas are primarily used as pack animals and a source of food, wool, hides, tallow for candles, and dried dung for fuel. They are found primarily in South American countries such as Bolivia, Peru, Colombia, Ecuador, Chile, and Argentina. Llamas are known for their long necks, long legs, small heads, large pointed ears, and ability to graze on grass and other plants. They are gregarious animals and can interbreed with other lamoids, such as guanacos, vicuñas, and alpacas. Llama fleece is sheared every two years and consists of coarse guard hairs and short crimped fibers. The fleece is used for knitwear, woven fabrics, rugs, rope, and fabric.'}]]"
"[[{'answer': 'LangSmith is a platform that helps developers build production-grade language model applications and provides tools for testing, evaluating, and monitoring these applications. It is built by the developers of LangChain and integrates seamlessly with that library. LangSmith aims to address the challenges of moving LLM applications from prototypes to production, ensuring reliability and maintainability. It offers features such as tracing, testing, and evaluating prompts and answers generated by language models. For more information, you can refer to the LangSmith documentation.'}],\n",
" [{'answer': 'According to the search results, a llama is a domesticated livestock species that is a descendant of the guanaco and belongs to the camel family. Llamas are primarily used as pack animals and a source of food, wool, hides, tallow, and dried dung. They are found in South American countries such as Bolivia, Peru, Colombia, Ecuador, Chile, and Argentina. Llamas are known for their long necks, long legs, small heads, and large pointed ears. They are gregarious animals that graze on grass and other plants. Llamas can interbreed with other lamoid species and produce fertile offspring. On the other hand, alpacas are smaller than llamas, have different face shapes and hair textures, and are primarily used for fleece production.'}]]"
]
},
"execution_count": 14,
@@ -329,9 +330,9 @@
"text/plain": [
"['What is the purpose of Langsmith?',\n",
" 'Who developed Langsmith?',\n",
" 'What are the key features of Langsmith?',\n",
" 'Are there any alternatives to Langsmith?',\n",
" 'What are the reviews or feedback on Langsmith?']"
" 'What are the features of Langsmith?',\n",
" 'How does Langsmith work?',\n",
" 'Are there any alternatives to Langsmith?']"
]
},
"execution_count": 17,
@@ -430,34 +431,29 @@
"Fetching pages: 0%| | 0/4 [00:00<?, ?it/s]\n",
"Fetching pages: 0%| | 0/4 [00:00<?, ?it/s]\u001b[A\n",
"\n",
"Fetching pages: 0%| | 0/4 [00:00<?, ?it/s]\u001b[A\u001b[A\n",
"\n",
"\n",
"Fetching pages: 0%| | 0/4 [00:00<?, ?it/s]\u001b[A\u001b[A\u001b[A\n",
"\n",
"\n",
"\n",
"Fetching pages: 0%| | 0/4 [00:00<?, ?it/s]\u001b[A\u001b[A\u001b[A\u001b[A\n",
"\n",
"Fetching pages: 0%| | 0/4 [00:00<?, ?it/s]\u001b[A\u001b[A\n",
"Fetching pages: 100%|###################################################| 4/4 [00:00<00:00, 5.71it/s]\u001b[A\u001b[A\u001b[A\u001b[A\n",
"Fetching pages: 100%|###################################################| 4/4 [00:00<00:00, 4.80it/s]\n",
"Fetching pages: 100%|###################################################| 4/4 [00:01<00:00, 2.77it/s]\n",
"\n",
"\n",
"Fetching pages: 100%|###################################################| 4/4 [00:01<00:00, 3.33it/s]\u001b[A\u001b[A\u001b[A\n",
"Fetching pages: 100%|###################################################| 4/4 [00:01<00:00, 2.70it/s]\u001b[A\u001b[A\n",
"\n",
"\n",
"\n",
"\n",
"Fetching pages: 100%|###################################################| 4/4 [00:01<00:00, 3.02it/s]\u001b[A\u001b[A\u001b[A\u001b[A\n",
"\n",
"Fetching pages: 100%|###################################################| 4/4 [00:01<00:00, 2.86it/s]\u001b[A\n",
"\n",
"\n",
"Fetching pages: 100%|###################################################| 4/4 [00:01<00:00, 2.33it/s]\u001b[A\u001b[A\n",
"Fetching pages: 100%|###################################################| 4/4 [00:02<00:00, 1.94it/s]\n"
"Fetching pages: 100%|###################################################| 4/4 [00:03<00:00, 1.22it/s]\u001b[A\u001b[A\u001b[A\n"
]
},
{
"data": {
"text/plain": [
"[\"Research Report: Understanding LangSmith - A Unified Platform for Building LLM Applications\\n\\nIntroduction:\\nThe purpose of this research report is to provide a comprehensive understanding of LangSmith, a unified platform designed to assist developers in building production-grade Language Model applications (LLMs). By exploring various sub-questions, we have gathered information about LangSmith's features, development team, alternatives, and user feedback.\\n\\n1. What is LangSmith?\\nLangSmith is a unified platform developed by LangChain, the creators of LangChain itself. It aims to streamline the development lifecycle, maintenance, and improvement of LLM applications. By providing tools for debugging, testing, evaluating, and monitoring LLM applications, LangSmith enables developers to transition from prototype to production seamlessly. It addresses the challenges associated with building reliable and maintainable LLM applications in production environments.\\n\\n2. Features and Integration:\\nLangSmith offers a range of features to facilitate the development and management of LLM applications. These include tracing runs, testing and evaluating LLM-generated prompts or answers, and visualizing and replaying traces. The platform integrates seamlessly with LangChain, providing native integration for debugging and monitoring LLM applications.\\n\\n3. Getting Started with LangSmith:\\nTo utilize LangSmith, users need to sign up for an account and create an API key. Additionally, having a GitHub repository and an OpenAI API key is necessary for integration. Although LangSmith is currently in beta, periodic access to new sign-ups is available.\\n\\n4. Alternatives to LangSmith:\\nBased on our research, we have identified several alternatives to LangSmith for developers working with LLM applications. These alternatives offer similar functionalities and may serve as viable options depending on specific project requirements. Some notable alternatives include:\\n\\n- LangChain: An open source framework for building conversational AI agents, which integrates with LangSmith for debugging and monitoring.\\n- Autoblocks: A tool that monitors and improves AI models powered by large language models, providing an SDK for easy integration.\\n- BenchLLM: An open source tool specifically designed for evaluating large language models, supporting models from various providers including OpenAI and LangChain.\\n- GradientJ: A comprehensive platform for building, evaluating, and managing large language models, offering features such as prompt chaining and data integration.\\n- Portkey: An LMOps platform catering to the deployment of production-ready LLM applications, offering model management, monitoring, and versioning tools.\\n- Vellum AI: A platform equipped with tools for developing LLM applications, including prompt engineering, version control, testing, and monitoring. Compatible with multiple LLM providers.\\n- Openlayer: A collaborative platform that facilitates aligning expectations around LLM quality and performance, providing diagnostic tools for issue resolution and iteration.\\n- Metal: A tool that focuses on providing LLM developers with a seamless workflow, although specific details about its features are not available.\\n\\n5. User Feedback:\\nLangSmith has received positive reviews and feedback from the community. Although the reviews were based on the tool's description and potential rather than personal usage, LangSmith has an overall rating of 5/5. Users have praised the platform for simplifying the transition of projects from the prototyping phase to full-fledged production. LangSmith's ability to manage the complexity of LLM applications and enhance product development and iteration capabilities has been highlighted.\\n\\nConclusion:\\nIn conclusion, LangSmith is a unified platform developed by LangChain to assist developers in building production-grade Language Model applications. It offers a range of features for debugging, testing, evaluating, and monitoring LLM applications, aiming to simplify the development lifecycle, maintenance, and improvement of these applications. Although alternatives exist, LangSmith has received positive feedback and has the potential to enhance productivity and workflow for developers working with LLMs.\"]"
"['Research Report: Understanding LangSmith\\n\\nIntroduction:\\nThe purpose of this research report is to explore and provide insights into the topic of LangSmith. LangSmith is a unified platform that aims to address the challenges developers face when building and deploying language model applications in production environments. By examining various sources, we have gathered information to answer the question, \"What is LangSmith?\"\\n\\nFindings:\\n\\n1. LangSmith\\'s Purpose and Features:\\nLangSmith is designed to assist developers in transitioning from prototype to production with their language model applications. It offers a range of features to trace, test, evaluate, and monitor LLM (Large Language Model) calls for production. The platform is part of the LangChain ecosystem and provides reliable and maintainable solutions for language model applications [1].\\n\\n2. Development and Integration:\\nLangSmith was developed by the same team that created LangChain, the popular language model software tool. With a focus on reliability and maintainability, LangSmith seamlessly integrates with LangChain, enabling developers to efficiently build production-grade LLM applications [2].\\n\\n3. Functionalities and Benefits:\\nLangSmith serves as a unified platform for debugging, testing, and monitoring language model applications. It aids developers in prototyping LLM applications and Agents, facilitating customization and iteration on prompts, chains, and other components. Additionally, LangSmith allows for quick debugging of new chains and agents, visualizes component relationships, evaluates prompts and LLMs, and captures usage traces for generating insights. It also provides benchmarking features to evaluate LLM applications [3].\\n\\n4. Availability and Documentation:\\nLangSmith is currently in beta and periodically allows access to new sign-ups. The platform offers documentation and walkthroughs to guide users through its features, making it easier for developers to utilize LangSmith effectively [3].\\n\\n5. Alternatives to LangSmith:\\nBased on our research, we identified several potential alternatives to LangSmith that offer similar functionalities. These alternatives include LangChain, GradientJ, Vellum, Llama 2, Openlayer, Backengine, Query Vary, and BenchLLM. Each alternative provides different tools and features to support the development, testing, and monitoring of language model applications [4].\\n\\nConclusion:\\nLangSmith is a unified platform developed by the creators of LangChain to address the challenges of building and deploying language model applications in production. It offers tracing, testing, evaluating, and monitoring features for LLM applications. By providing documentation and a walkthrough, LangSmith helps developers transition from prototyping to production. However, it is essential to consider alternative platforms based on specific requirements and needs.\\n\\nReferences:\\n1. [1] LangSmith: A unified platform for language model applications. Retrieved from [source 1].\\n2. [2] LangSmith: Tackling challenges in LLM application development. Retrieved from [source 2].\\n3. [3] Exploring the functionalities of LangSmith. Retrieved from [source 3].\\n4. [4] Alternatives to LangSmith. Retrieved from [source 4].\\n\\nPlease note that the sources mentioned above have not been provided and should be replaced with the actual sources used for research.']"
]
},
"execution_count": 22,
+1 -1
View File
@@ -94,7 +94,7 @@ class RunnableSubscriber(RunnableBinding[T, Any]):
raise NotImplementedError()
class RunnablePublisher(Runnable[T, T]):
class RunnablePublisher(Serializable, Runnable[T, T]):
topic: Topic[T]
def invoke(self, input: T, config: Optional[RunnableConfigForPubSub] = None) -> T: