Commit Graph
5 Commits
Author SHA1 Message Date
Nuno Campos 30b631aaff Rewrite imports 2024-09-21 17:41:05 -07:00
Nuno Campos 4accc19734 kafka: Implement sync orchestrator and executor classes 2024-09-11 16:06:50 -07:00
Nuno Campos 217b07ae65 kafka: Make consumer and producer classes configurable
- Define protocol for sync and async producer and consumer
- Accept consumer/producer as init args in Orchestrator/Executor
- If not passed in, create default consumer/producer as before
2024-09-11 14:27:08 -07:00
Nuno Campos 943cc4d83a Add readme, lint 2024-09-10 16:17:22 -07:00
Nuno Campos 841aabf5b3 Implement LangGraph Scheduler for Kafka
- Orchestrator and Executor classes to run LangGraph in a distributed fashion using Kafka as a message bus for communication
- Orchestrator and Executor run on-demand when a new message is published to the topic they listen to
- Orchestrator is responsible for running the Pregel algorithm (deciding next tasks to run) and sending messages to the executor topic
- Executor is responsible for executing each task (node), and sending messages to the orchestrator topic when done
2024-09-10 16:17:22 -07:00