From 97d0006461ee080546411ccba46f39fb517baf89 Mon Sep 17 00:00:00 2001 From: haikdc Date: Mon, 30 Mar 2026 19:11:19 -0700 Subject: [PATCH] [Haik]: ckpt, deleted all instructions for the assistant-ui refactoring --- ASSISTANT_UI_MIGRATION/MIGRATION_AGENT_1.md | 268 -------------- ASSISTANT_UI_MIGRATION/MIGRATION_AGENT_2.md | 184 ---------- ASSISTANT_UI_MIGRATION/MIGRATION_AGENT_3.md | 206 ----------- ASSISTANT_UI_MIGRATION/MIGRATION_AGENT_4.md | 262 -------------- ASSISTANT_UI_MIGRATION/MIGRATION_AGENT_5.md | 229 ------------ ASSISTANT_UI_MIGRATION/MIGRATION_AGENT_6.md | 369 -------------------- ASSISTANT_UI_MIGRATION/MIGRATION_AGENT_7.md | 257 -------------- ASSISTANT_UI_MIGRATION/OVERVIEW.md | 165 --------- 8 files changed, 1940 deletions(-) delete mode 100644 ASSISTANT_UI_MIGRATION/MIGRATION_AGENT_1.md delete mode 100644 ASSISTANT_UI_MIGRATION/MIGRATION_AGENT_2.md delete mode 100644 ASSISTANT_UI_MIGRATION/MIGRATION_AGENT_3.md delete mode 100644 ASSISTANT_UI_MIGRATION/MIGRATION_AGENT_4.md delete mode 100644 ASSISTANT_UI_MIGRATION/MIGRATION_AGENT_5.md delete mode 100644 ASSISTANT_UI_MIGRATION/MIGRATION_AGENT_6.md delete mode 100644 ASSISTANT_UI_MIGRATION/MIGRATION_AGENT_7.md delete mode 100644 ASSISTANT_UI_MIGRATION/OVERVIEW.md diff --git a/ASSISTANT_UI_MIGRATION/MIGRATION_AGENT_1.md b/ASSISTANT_UI_MIGRATION/MIGRATION_AGENT_1.md deleted file mode 100644 index 0f553919..00000000 --- a/ASSISTANT_UI_MIGRATION/MIGRATION_AGENT_1.md +++ /dev/null @@ -1,268 +0,0 @@ -# Migration Agent 1: Foundation & Packages - -## Objective - -Install all required packages, configure Tailwind CSS alongside MUI, set up shadcn/ui, create the ExternalStoreRuntime adapter, create the toolkit skeleton, and scaffold the new directory structure. This agent's output is the foundation that all Phase 2 agents build on. - -**This agent must complete before any other agent starts.** - -## Constraints - -- **No custom styling**: Do not theme assistant-ui or Tool UI components to match MUI. Use their default appearance. -- **Tailwind + MUI coexist**: Tailwind should not break existing MUI styles. Use Tailwind's `prefix` option (e.g., `tw-`) or scope it carefully. -- **Webpack 5**: This project uses Webpack 5 (not Next.js). Tailwind must be configured for Webpack with PostCSS. -- **React 18**: The project uses React 18.2. assistant-ui supports React 18 — check their compatibility docs at `/docs/react-compatibility`. -- **The assistant-ui MCP docs server is available** in `.cursor/mcp.json`. Use `assistantUIDocs` and `assistantUIExamples` tools to look up current API docs. - -## Step-by-Step - -### 1. Install Tailwind CSS for Webpack 5 - -```bash -cd frontend -npm install -D tailwindcss @tailwindcss/postcss postcss postcss-loader -``` - -Create `frontend/postcss.config.js`: -```js -module.exports = { - plugins: { - '@tailwindcss/postcss': {}, - }, -}; -``` - -Create `frontend/src/styles/tailwind.css`: -```css -@import "tailwindcss"; -``` - -Update `webpack.config.js` to add a CSS rule with PostCSS for `.css` files (not `.module.scss` — those stay as-is): -```js -{ - test: /\.css$/, - use: ['style-loader', 'css-loader', 'postcss-loader'] -} -``` - -Import the tailwind CSS in `frontend/src/index.tsx`: -```typescript -import './styles/tailwind.css'; -``` - -Verify: `npm run dev` should start without errors, existing MUI pages should look unchanged. - -### 2. Install shadcn/ui - -shadcn/ui typically assumes Next.js, but works with Webpack. Run: - -```bash -npx shadcn@latest init -``` - -When prompted: -- Style: Default -- Base color: Neutral (or Slate — doesn't matter since we're not custom styling) -- CSS variables: Yes -- Path alias: `@/` (matches existing tsconfig paths) -- Components directory: `src/components/ui` - -This creates `components.json` and a `lib/utils.ts` file. Ensure `cn()` utility works. - -### 3. Install assistant-ui - -```bash -npm install @assistant-ui/react @assistant-ui/react-markdown -``` - -Optionally, for the Lexical rich editor (used by Composer Mention): -```bash -npm install @assistant-ui/react-lexical lexical @lexical/react -``` - -Use `assistantUIDocs` MCP tool to check the installation page (`/docs/installation`) for the latest install instructions and any peer dependencies. - -Add assistant-ui's default styles. Check their docs for the exact import — it may be: -```typescript -import "@assistant-ui/react/styles/index.css"; -``` - -Or they may use Tailwind-based styling via shadcn components. Follow whatever the current docs say. - -### 4. Install ALL Tool UI components upfront - -Install every Tool UI component the migration needs. This prevents parallel agents from running conflicting `npx shadcn` installs. - -```bash -npx shadcn@latest add @tool-ui/terminal -npx shadcn@latest add @tool-ui/code-block -npx shadcn@latest add @tool-ui/code-diff -npx shadcn@latest add @tool-ui/approval-card -npx shadcn@latest add @tool-ui/question-flow -npx shadcn@latest add @tool-ui/option-list -npx shadcn@latest add @tool-ui/message-draft -npx shadcn@latest add @tool-ui/data-table -npx shadcn@latest add @tool-ui/item-carousel -npx shadcn@latest add @tool-ui/progress-tracker -``` - -Each installs to `src/components/tool-ui//` with a component file, schema, and types. - -### 5. Create the ExternalStoreRuntime adapter - -Create `frontend/src/app/pages/AgentChat/runtime/useOpenSwarmRuntime.ts`. - -This hook bridges the Redux store + WebSocket to assistant-ui's runtime. Use `assistantUIDocs` to look up the ExternalStoreRuntime API at path `runtimes/custom/external-store`. - -Key responsibilities: -- Read `session.messages` from Redux and convert to assistant-ui's message format -- Read `session.streamingMessage` and surface it as the in-progress message -- Map `session.status` to `isRunning` -- `onNew` → dispatch `sendMessage` thunk (existing in `agentsThunks.ts`) -- `onEdit` → dispatch `editMessage` thunk -- `onCancel` → dispatch `stopAgent` thunk -- Handle `session.branches` for branching support - -**Message format conversion** — the critical mapping: - -Redux `AgentMessage` types: -```typescript -interface AgentMessage { - id: string; - role: 'user' | 'assistant' | 'tool_call' | 'tool_result' | 'system'; - content: any; - timestamp: string; - branch_id: string; - parent_id: string | null; - // ... other fields -} -``` - -assistant-ui expects messages like: -```typescript -// User message -{ role: 'user', content: [{ type: 'text', text: '...' }] } - -// Assistant message -{ role: 'assistant', content: [{ type: 'text', text: '...' }] } - -// Tool call (part of assistant message) -{ role: 'assistant', content: [{ type: 'tool-call', toolCallId: '...', toolName: '...', args: {} }] } - -// Tool result -{ role: 'tool', content: [{ type: 'tool-result', toolCallId: '...', result: {} }] } -``` - -Write a `convertMessages(messages: AgentMessage[]): ThreadMessage[]` function that handles this. Note that consecutive tool_call + tool_result messages may need to be merged or paired. - -Look up the ExternalStoreRuntime docs carefully — the exact shape of `ThreadMessage` and how streaming is handled (via `status` field on messages) matters. - -### 6. Create the toolkit skeleton - -Create the directory `frontend/src/app/pages/AgentChat/toolkit/`. - -Create `frontend/src/app/pages/AgentChat/toolkit/index.ts`: -```typescript -import { type Toolkit } from '@assistant-ui/react'; -import { nativeToolkit } from './native-tools'; -import { approvalToolkit } from './approval-tools'; -import { mcpToolkit } from './mcp-tools'; -import { customToolkit } from './custom-tools'; - -export const toolkit: Toolkit = { - ...nativeToolkit, - ...approvalToolkit, - ...mcpToolkit, - ...customToolkit, -}; -``` - -Create placeholder files for each toolkit module: -- `toolkit/native-tools.tsx` — exports `nativeToolkit: Partial = {}` -- `toolkit/approval-tools.tsx` — exports `approvalToolkit: Partial = {}` -- `toolkit/mcp-tools.tsx` — exports `mcpToolkit: Partial = {}` -- `toolkit/custom-tools.tsx` — exports `customToolkit: Partial = {}` - -These are empty stubs. Phase 2 agents fill them in. - -### 7. Create Thread and Composer placeholder directories - -Create: -- `frontend/src/app/pages/AgentChat/thread/` — Agent 2 fills this -- `frontend/src/app/pages/AgentChat/composer/` — Agent 3 fills this - -Create placeholder files: -- `thread/OpenSwarmThread.tsx` — exports a simple `
Thread placeholder
` -- `composer/OpenSwarmComposer.tsx` — exports a simple `
Composer placeholder
` - -### 8. Scaffold AgentChat.tsx with provider wrapper - -Modify `AgentChat.tsx` to wrap the chat area with `AssistantRuntimeProvider`. Keep the existing structure but prepare for Phase 2 agents to swap in their components. - -The modified AgentChat should: -1. Import and use `useOpenSwarmRuntime(sessionId)` to get the runtime -2. Wrap the chat area with `` -3. Register the toolkit via `useAui` or `Tools` (check assistant-ui docs for the current API) -4. Keep existing imports for `ChatHeader`, `MessageQueue`, `ModelModeSelector` (these stay) -5. Keep the `useAgentChat` hook (it manages WS lifecycle, send handlers, etc.) -6. For now, keep the existing message rendering loop AND the placeholders — the app should still work with the old rendering. Agent 7 does the final swap. - -**Important**: The app must still work after this agent completes. Do not remove any existing components yet — just add the runtime wrapper and placeholders alongside them. - -### 9. Verify everything works - -- `npm run dev` starts without errors -- The existing chat page renders and works as before -- Tailwind utility classes work (test by adding a `tw-text-red-500` class somewhere temporarily) -- Tool UI component files exist in `src/components/tool-ui/` -- No TypeScript errors from the new imports - -## Files Created - -| File | Description | -|------|-------------| -| `frontend/postcss.config.js` | PostCSS config for Tailwind | -| `frontend/src/styles/tailwind.css` | Tailwind entry CSS | -| `frontend/components.json` | shadcn/ui config | -| `frontend/src/lib/utils.ts` | shadcn `cn()` utility | -| `frontend/src/app/pages/AgentChat/runtime/useOpenSwarmRuntime.ts` | ExternalStoreRuntime adapter | -| `frontend/src/app/pages/AgentChat/toolkit/index.ts` | Merged toolkit registry | -| `frontend/src/app/pages/AgentChat/toolkit/native-tools.tsx` | Stub for Agent 4 | -| `frontend/src/app/pages/AgentChat/toolkit/approval-tools.tsx` | Stub for Agent 5 | -| `frontend/src/app/pages/AgentChat/toolkit/mcp-tools.tsx` | Stub for Agent 6 | -| `frontend/src/app/pages/AgentChat/toolkit/custom-tools.tsx` | Stub for Agent 6 | -| `frontend/src/app/pages/AgentChat/thread/OpenSwarmThread.tsx` | Placeholder for Agent 2 | -| `frontend/src/app/pages/AgentChat/composer/OpenSwarmComposer.tsx` | Placeholder for Agent 3 | -| `frontend/src/components/tool-ui/*` | All Tool UI components (installed via shadcn) | - -## Files Modified - -| File | Change | -|------|--------| -| `frontend/package.json` | New dependencies added | -| `frontend/webpack.config.js` | CSS rule with postcss-loader added | -| `frontend/tsconfig.json` | May need path adjustments for shadcn | -| `frontend/src/index.tsx` | Import tailwind CSS | -| `frontend/src/app/pages/AgentChat/AgentChat.tsx` | Wrapped with `AssistantRuntimeProvider` | - -## Files Deleted - -None. This agent only adds — it never removes existing functionality. - -## Verification Checklist - -- [ ] `npm run dev` starts without errors -- [ ] Existing chat page works exactly as before -- [ ] Tailwind classes render correctly -- [ ] `src/components/tool-ui/terminal/` exists with component + schema -- [ ] `src/components/tool-ui/code-block/` exists with component + schema -- [ ] `src/components/tool-ui/code-diff/` exists with component + schema -- [ ] `src/components/tool-ui/approval-card/` exists with component + schema -- [ ] `src/components/tool-ui/question-flow/` exists with component + schema -- [ ] `src/components/tool-ui/option-list/` exists with component + schema -- [ ] `src/components/tool-ui/message-draft/` exists with component + schema -- [ ] `src/components/tool-ui/data-table/` exists with component + schema -- [ ] `src/components/tool-ui/progress-tracker/` exists with component + schema -- [ ] `useOpenSwarmRuntime` hook compiles without errors -- [ ] Toolkit index imports all stubs without errors -- [ ] `AgentChat.tsx` has `AssistantRuntimeProvider` wrapper diff --git a/ASSISTANT_UI_MIGRATION/MIGRATION_AGENT_2.md b/ASSISTANT_UI_MIGRATION/MIGRATION_AGENT_2.md deleted file mode 100644 index 11e5d7ff..00000000 --- a/ASSISTANT_UI_MIGRATION/MIGRATION_AGENT_2.md +++ /dev/null @@ -1,184 +0,0 @@ -# Migration Agent 2: Thread & Message Rendering - -## Objective - -Replace the custom message list, scroll viewport, message bubbles, action bar, branch picker, thinking indicator, and image rendering with assistant-ui's `Thread`, `Message`, `ActionBar`, `BranchPicker`, and `Reasoning` primitives. - -## Prerequisites - -- **Agent 1 must be complete.** The ExternalStoreRuntime adapter, toolkit skeleton, and `AssistantRuntimeProvider` wrapper must exist. - -## Constraints - -- **No custom styling**: Use default assistant-ui appearance. Do not add CSS to match MUI theme. -- **Only modify files in the `thread/` directory** and the files listed under "Files Deleted." Do NOT modify `AgentChat.tsx`, `toolkit/`, `composer/`, or any file owned by another agent. -- **The assistant-ui MCP docs server is available** in `.cursor/mcp.json`. Use `assistantUIDocs` and `assistantUIExamples` tools to look up current API docs. - -## Key Files to Read First - -Understand what you're replacing: -- `frontend/src/app/pages/AgentChat/useMessageRendering.ts` — branch resolution, render items -- `frontend/src/app/pages/AgentChat/MessageBubble.tsx` — routes to User/Assistant bubbles -- `frontend/src/app/pages/AgentChat/UserBubbleContent.tsx` — user message with context pills, images, editing -- `frontend/src/app/pages/AgentChat/AssistantBubbleContent.tsx` — markdown rendering -- `frontend/src/app/pages/AgentChat/MessageActionBar.tsx` — copy/edit/regenerate/branch actions -- `frontend/src/app/pages/AgentChat/BranchNavigator.tsx` — branch picker arrows -- `frontend/src/app/pages/AgentChat/ThinkingBubble.tsx` — thinking/reasoning display -- `frontend/src/app/pages/AgentChat/MessageImageThumbnails.tsx` — image display in messages -- `frontend/src/app/pages/AgentChat/AttachedContextSection.tsx` — context pills in user messages - -## Step-by-Step - -### 1. Look up assistant-ui docs - -Use the `assistantUIDocs` MCP tool to read these pages: -- `primitives/thread` — Thread primitives (viewport, messages, scroll) -- `primitives/message` — Message primitives (root, content, parts) -- `primitives/action-bar` — ActionBar primitives (copy, edit, reload, speak) -- `primitives/branch-picker` — BranchPicker primitives (prev/next/count) -- `ui/thread` — Pre-built Thread component -- `ui/markdown` — Markdown component -- `ui/reasoning` — Reasoning/thinking UI -- `guides/branching` — How branching works - -### 2. Build OpenSwarmThread - -Create `frontend/src/app/pages/AgentChat/thread/OpenSwarmThread.tsx`. - -This is the main thread component that replaces the scroll container + message render loop in `AgentChat.tsx`. It should: - -1. Use `ThreadPrimitive.Root` and `ThreadPrimitive.Viewport` for the scrollable container with auto-scroll -2. Use `ThreadPrimitive.Messages` to render the message list -3. Provide custom `UserMessage` and `AssistantMessage` components - -```tsx -import { ThreadPrimitive } from '@assistant-ui/react'; -import { UserMessage } from './UserMessage'; -import { AssistantMessage } from './AssistantMessage'; - -export const OpenSwarmThread = () => { - return ( - - - - - - - ); -}; -``` - -### 3. Build UserMessage component - -Create `thread/UserMessage.tsx`. - -This replaces `UserBubbleContent.tsx`. It should: -- Use `MessagePrimitive.Root` and `MessagePrimitive.Content` -- Render user text content -- Render attached context paths (files, directories) as chips/badges -- Render attached images using `MessagePrimitive.Attachments` or custom rendering -- Support inline editing via `MessagePrimitive.EditComposer` (or however assistant-ui handles edit) -- Show the `ActionBar` (copy, edit) on hover - -For context pills and attached skills, read the existing `AttachedContextSection.tsx` and `UserBubbleContent.tsx` to understand the data shape. These are in `message.context_paths` and `message.attached_skills`. - -Note: The ExternalStoreRuntime adapter (from Agent 1) should include these in the converted message format. If they're not accessible via assistant-ui's message API, render them by reading from Redux directly using the message ID. - -### 4. Build AssistantMessage component - -Create `thread/AssistantMessage.tsx`. - -This replaces `AssistantBubbleContent.tsx`. It should: -- Use `MessagePrimitive.Root` and `MessagePrimitive.Content` -- Render markdown content using assistant-ui's `MarkdownText` or `@assistant-ui/react-markdown` -- Render tool calls inline (assistant-ui handles this automatically when tools are registered in the toolkit) -- Show the `ActionBar` (copy, regenerate, branch) on hover -- Show the `BranchPicker` when the message has sibling branches - -Use assistant-ui's `makeMarkdownText` or the `Markdown` component from `@assistant-ui/react-markdown`. Look up the current API in the docs. - -### 5. Build ActionBar - -Create `thread/MessageActions.tsx` (or inline in UserMessage/AssistantMessage). - -This replaces `MessageActionBar.tsx`. Use assistant-ui's `ActionBarPrimitive`: -- `ActionBarPrimitive.Copy` — copy message text -- `ActionBarPrimitive.Edit` — edit user message -- `ActionBarPrimitive.Reload` — regenerate assistant message (maps to your "Regenerate" button) - -For the "Branch chat" action, this is custom. Use a custom button inside the ActionBar that calls the existing `duplicateSession` thunk. You can use `useMessage` hook to get the current message context. - -### 6. Build BranchPicker - -Create `thread/BranchPicker.tsx` (or inline in AssistantMessage). - -This replaces `BranchNavigator.tsx`. Use assistant-ui's `BranchPickerPrimitive`: -- `BranchPickerPrimitive.Previous` -- `BranchPickerPrimitive.Number` / `BranchPickerPrimitive.Count` -- `BranchPickerPrimitive.Next` - -The ExternalStoreRuntime adapter (Agent 1) should expose branch data. If not, this may need coordination with Agent 1's runtime adapter to ensure branches are surfaced correctly. - -### 7. Handle thinking/reasoning display - -Create `thread/ThinkingIndicator.tsx` (or use assistant-ui's built-in). - -This replaces `ThinkingBubble.tsx`. Check if assistant-ui's `Thread` automatically shows a loading indicator when `isRunning` is true. If so, the default behavior may be sufficient. If a custom thinking bubble is needed, use assistant-ui's reasoning or chain-of-thought UI. - -### 8. Handle streaming messages - -The ExternalStoreRuntime adapter should handle streaming state. When `session.streamingMessage` exists in Redux, the runtime should surface it as a message with `status: 'in_progress'`. assistant-ui will automatically animate streaming text. - -Verify that the streaming cursor / token animation works correctly with the runtime adapter from Agent 1. - -## Files Created - -| File | Description | -|------|-------------| -| `thread/OpenSwarmThread.tsx` | Main thread component (replaces scroll container + message loop) | -| `thread/UserMessage.tsx` | User message rendering | -| `thread/AssistantMessage.tsx` | Assistant message with markdown, tool calls | -| `thread/MessageActions.tsx` | Action bar (copy, edit, regenerate, branch) | -| `thread/BranchPicker.tsx` | Branch navigation | -| `thread/ThinkingIndicator.tsx` | Thinking/loading state (if needed beyond defaults) | - -All files go in `frontend/src/app/pages/AgentChat/thread/`. - -## Files Deleted (by this agent) - -These files are replaced by the new thread components. Delete them: - -| File | Lines | Replaced By | -|------|-------|------------| -| `MessageBubble.tsx` | 99 | `UserMessage.tsx` + `AssistantMessage.tsx` | -| `UserBubbleContent.tsx` | 152 | `UserMessage.tsx` | -| `AssistantBubbleContent.tsx` | 128 | `AssistantMessage.tsx` | -| `MessageActionBar.tsx` | 153 | `MessageActions.tsx` | -| `BranchNavigator.tsx` | 60 | `BranchPicker.tsx` | -| `ThinkingBubble.tsx` | 55 | `ThinkingIndicator.tsx` or built-in | -| `MessageImageThumbnails.tsx` | 108 | Handled in `UserMessage.tsx` via Attachment primitives | -| `messageBubbleUtils.ts` | 54 | No longer needed | -| `useMessageRendering.ts` | 196 | Thread handles message list rendering | - -**Total deleted: ~1,005 lines** - -## Files NOT Modified - -- `AgentChat.tsx` — Agent 7 wires in `OpenSwarmThread` -- Anything in `composer/`, `toolkit/`, or shared state - -## Verification Checklist - -- [ ] `OpenSwarmThread` renders a scrollable message list -- [ ] User messages display text content -- [ ] Assistant messages render markdown correctly -- [ ] Tool call messages are rendered (even if just as fallback text — toolkit fills in later) -- [ ] ActionBar shows copy/edit/regenerate on hover -- [ ] BranchPicker shows navigation when branches exist -- [ ] Streaming messages animate token-by-token -- [ ] Auto-scroll to bottom works on new messages -- [ ] Scroll-to-bottom button appears when scrolled up -- [ ] No TypeScript errors -- [ ] All deleted files are removed diff --git a/ASSISTANT_UI_MIGRATION/MIGRATION_AGENT_3.md b/ASSISTANT_UI_MIGRATION/MIGRATION_AGENT_3.md deleted file mode 100644 index 27136d08..00000000 --- a/ASSISTANT_UI_MIGRATION/MIGRATION_AGENT_3.md +++ /dev/null @@ -1,206 +0,0 @@ -# Migration Agent 3: Composer & Mention System - -## Objective - -Replace the custom `ChatInput` (contentEditable + CommandPicker) with assistant-ui's `Composer` primitives and `ComposerMentionPopover` for `@`/`/` trigger-based command picking. - -## Prerequisites - -- **Agent 1 must be complete.** The `AssistantRuntimeProvider`, runtime adapter, and assistant-ui packages must be installed. - -## Constraints - -- **No custom styling**: Use default assistant-ui Composer and Mention appearance. -- **Only modify files in the `composer/` directory** and the files listed under "Files Deleted." Do NOT modify `AgentChat.tsx`, `toolkit/`, `thread/`, or any file owned by another agent. -- **Keep `ModelModeSelector.tsx`** as-is — it's MUI and stays. The new Composer should render it in the footer area alongside the send button, similar to how it's currently positioned. -- **Keep `MessageQueue.tsx`** as-is — it wraps the composer. -- **Keep `TemplateInvokeModal.tsx`** as-is — it's triggered when a template is selected. -- **The assistant-ui MCP docs server is available** in `.cursor/mcp.json`. Use `assistantUIDocs` and `assistantUIExamples` tools to look up current API docs. - -## Key Files to Read First - -Understand what you're replacing: -- `frontend/src/app/pages/AgentChat/ChatInput.tsx` — main composer component -- `frontend/src/app/pages/AgentChat/hooks/useChatSubmit.ts` — send/paste/drop/pick logic -- `frontend/src/app/pages/AgentChat/AttachmentChips.tsx` — context path + tool chips -- `frontend/src/app/pages/AgentChat/ImageAttachments.tsx` — image thumbnails + lightbox -- `frontend/src/app/components/CommandPicker.tsx` — unified `@`/`/` command picker -- `frontend/src/app/components/commandPickerTypes.tsx` — picker types -- `frontend/src/app/components/useCommandPickerItems.tsx` — items for the picker (templates, skills, modes, tools, files) -- `frontend/src/app/components/CommandPickerIcons.tsx` — icons for picker categories -- `frontend/src/app/components/SlashCommandPicker.tsx` — legacy picker -- `frontend/src/app/components/richEditorUtils.ts` — skill pill serialize/deserialize -- `frontend/src/app/components/RichPromptEditor.tsx` — shared rich editor used in mode/template editors - -Also read the assistant-ui docs: -- `ui/mention` — ComposerMentionPopover, MentionAdapter, DirectiveText -- `primitives/composer` — Composer primitives (Root, Input, Send, Attachments) -- `primitives/attachment` — Attachment primitives -- `guides/attachments` — Attachment handling - -## Step-by-Step - -### 1. Look up assistant-ui Mention docs - -Use `assistantUIDocs` MCP tool to read: -- `ui/mention` — full Mention API -- `primitives/composer` — Composer primitives - -The Mention system provides: -- `ComposerMentionPopover.Root` — wraps the composer, detects `@` trigger -- `ComposerMentionPopover` — the popover UI with categories and keyboard nav -- `LexicalComposerInput` — rich editor with inline mention chips -- Custom `MentionAdapter` — you supply the list of mentionable items - -### 2. Create the MentionAdapter - -Create `frontend/src/app/pages/AgentChat/composer/OpenSwarmMentionAdapter.ts`. - -This replaces `useCommandPickerItems.tsx`. The adapter provides the list of mentionable items for both `@` and `/` triggers. - -Read the existing `useCommandPickerItems.tsx` to understand what categories exist: -- **Templates** (triggered by `/`) — from `state.templates.items` -- **Skills** (triggered by `/`) — from `state.skills.items` -- **Modes** (triggered by `/`) — from `state.modes.items` -- **File attach** (triggered by `@`) — opens file browser -- **Web search** (triggered by `@`) — web search context -- **MCP tool groups** (triggered by `@`) — from `state.mcpRegistry` -- **View/app outputs** (triggered by `@`) — from dashboard outputs - -The Mention adapter should return these as categories with items. The adapter's `search` function filters items based on the typed query. - -Note: The default trigger for assistant-ui Mention is `@`. To support both `@` and `/`, you may need two `ComposerMentionPopover.Root` instances with different `trigger` props, or a single adapter that handles both (check docs for multi-trigger support). - -### 3. Build OpenSwarmComposer - -Create `frontend/src/app/pages/AgentChat/composer/OpenSwarmComposer.tsx`. - -This replaces `ChatInput.tsx`. Structure: - -```tsx -import { ComposerPrimitive } from '@assistant-ui/react'; -import { ComposerMentionPopover } from '@/components/assistant-ui/composer-mention'; -import { LexicalComposerInput } from '@assistant-ui/react-lexical'; -import ModelModeSelector from '../ModelModeSelector'; - -export const OpenSwarmComposer = ({ mode, onModeChange, model, onModelChange, ...props }) => { - return ( - - - {/* Attachment display area (images, context paths) */} - - - {/* Rich text input with inline mention chips */} - - - {/* Mention popover (appears on @ or / trigger) */} - - - {/* Footer: mode/model selector + send button */} - - - {/* Stop button when running */} - - - ); -}; -``` - -Key behaviors to implement: -- **Send**: `ComposerPrimitive.Send` calls the runtime's `onNew` which dispatches `sendMessage` -- **Stop**: `ComposerPrimitive.Cancel` calls the runtime's `onCancel` which dispatches `stopAgent` -- **Image paste/drop**: Use `ComposerPrimitive.Attachments` for drag-drop and paste handling. Check assistant-ui's attachment guide. -- **File upload**: When `@file` is selected from the mention popover, trigger the existing file upload flow (POST to `/api/settings/upload-files`) -- **Template selection**: When a template with variables is selected from `/`, open `TemplateInvokeModal`. This requires a callback on mention selection. - -### 4. Handle the ChatInputHandle API - -The existing `ChatInput` exposes a `ChatInputHandle` ref with `getConfig()` and `setContent()` methods. These are used by: -- `SkillBuilderChat.tsx` — to programmatically set content -- `useAgentChat.ts` — to get config before sending - -With assistant-ui, programmatic control goes through the `ComposerRuntime`. Check docs at `api-reference/runtimes/composer-runtime` for `setText()`, `send()`, etc. - -Create a wrapper or hook that provides equivalent functionality: -- `getConfig()` → read from composer runtime state -- `setContent()` → use `composerRuntime.setText()` - -### 5. Handle attachment rendering in the composer - -The current `ChatInput` renders: -- **Image thumbnails** with lightbox and remove buttons (`ImageAttachments.tsx`) -- **Context path chips** with copy-to-clipboard and remove (`AttachmentChips.tsx`) -- **Forced tool chips** (`AttachmentChips.tsx`) -- **UI element selection chips** (`AttachmentChips.tsx`) - -With assistant-ui, use `ComposerPrimitive.Attachments` to render attachments. For custom chip types (context paths, forced tools, UI elements), you may need custom rendering inside the Composer area. - -### 6. Handle the RichPromptEditor - -`RichPromptEditor.tsx` is a shared component used outside the chat (in mode editors, template editors). It has similar `@`/`/` trigger detection and skill pill insertion. - -Two options: -- **Option A**: Keep `RichPromptEditor.tsx` as-is for now (it's not in the critical chat path) -- **Option B**: Refactor it to also use `ComposerMentionPopover` - -Go with **Option A** — keep it as-is. It's used in Settings/Modes pages, which are out of scope. Just make sure deleting `richEditorUtils.ts` doesn't break it. If it imports from there, extract the needed utilities. - -**Important**: Before deleting `richEditorUtils.ts`, check if `RichPromptEditor.tsx` imports from it. If so, either: -- Move the needed functions into `RichPromptEditor.tsx` itself -- Keep `richEditorUtils.ts` but only with the functions `RichPromptEditor` needs - -## Files Created - -| File | Description | -|------|-------------| -| `composer/OpenSwarmComposer.tsx` | Main composer component | -| `composer/OpenSwarmMentionAdapter.ts` | Mention adapter for templates/skills/modes/tools/files | -| `composer/useComposerHandle.ts` | Hook providing `getConfig`/`setContent` via ComposerRuntime | - -All files go in `frontend/src/app/pages/AgentChat/composer/`. - -Also install the composer-mention shadcn component if not already present: -```bash -npx shadcn@latest add composer-mention -``` - -## Files Deleted (by this agent) - -| File | Lines | Replaced By | -|------|-------|------------| -| `ChatInput.tsx` | 209 | `OpenSwarmComposer.tsx` | -| `hooks/useChatSubmit.ts` | 248 | Runtime `onNew`/`onEdit`/`onCancel` | -| `AttachmentChips.tsx` | 118 | Mention chips + Composer Attachments | -| `ImageAttachments.tsx` | 85 | Composer Attachment primitives | -| `CommandPicker.tsx` | 230 | `ComposerMentionPopover` | -| `commandPickerTypes.tsx` | 47 | `OpenSwarmMentionAdapter.ts` types | -| `useCommandPickerItems.tsx` | 225 | `OpenSwarmMentionAdapter.ts` | -| `CommandPickerIcons.tsx` | 40 | Category icons in adapter | -| `SlashCommandPicker.tsx` | 165 | `ComposerMentionPopover` | - -**About `richEditorUtils.ts` (196 lines)**: Check if `RichPromptEditor.tsx` imports from it. If yes, extract only the functions `RichPromptEditor` needs into that file, then delete `richEditorUtils.ts`. If no imports, delete it entirely. - -**Total deleted: ~1,367+ lines** - -## Files NOT Modified - -- `AgentChat.tsx` — Agent 7 wires in `OpenSwarmComposer` -- `ModelModeSelector.tsx` — kept as-is, rendered inside the new Composer -- `MessageQueue.tsx` — kept as-is, wraps the new Composer -- `TemplateInvokeModal.tsx` — kept as-is, triggered by mention selection -- Anything in `thread/`, `toolkit/`, or shared state - -## Verification Checklist - -- [ ] `OpenSwarmComposer` renders a text input area -- [ ] Typing `@` opens the mention popover with context categories (files, tools, etc.) -- [ ] Typing `/` opens the mention popover with command categories (templates, skills, modes) -- [ ] Keyboard navigation works (arrows, Enter, Escape) -- [ ] Selected mentions appear as inline chips in the editor -- [ ] Images can be pasted or dragged into the composer -- [ ] Send button calls the runtime's `onNew` -- [ ] Stop button appears when the agent is running and calls `onCancel` -- [ ] `ModelModeSelector` renders in the composer footer -- [ ] No TypeScript errors -- [ ] All deleted files are removed -- [ ] `RichPromptEditor` still works in Modes/Templates pages (if it existed before) diff --git a/ASSISTANT_UI_MIGRATION/MIGRATION_AGENT_4.md b/ASSISTANT_UI_MIGRATION/MIGRATION_AGENT_4.md deleted file mode 100644 index f7862f57..00000000 --- a/ASSISTANT_UI_MIGRATION/MIGRATION_AGENT_4.md +++ /dev/null @@ -1,262 +0,0 @@ -# Migration Agent 4: Tool Toolkit — Terminal, Code, Diffs - -## Objective - -Register Tool UI components (`Terminal`, `CodeBlock`, `CodeDiff`) in the toolkit for rendering native tool calls (bash, file read/write, edit, grep, glob, search). Replace `ToolCallBubble`, `toolCallColors`, `toolCallUtils`, `ElapsedTimer`, and `ToolGroupBubble`. - -## Prerequisites - -- **Agent 1 must be complete.** Tool UI components must be installed in `src/components/tool-ui/`, and the toolkit skeleton must exist at `toolkit/native-tools.tsx`. - -## Constraints - -- **No custom styling**: Use default Tool UI Terminal/CodeBlock/CodeDiff appearance. -- **Only modify `toolkit/native-tools.tsx`**. Do NOT modify `AgentChat.tsx`, `thread/`, `composer/`, or other toolkit files. -- **The assistant-ui MCP docs server is available** in `.cursor/mcp.json`. Use it for assistant-ui API lookups. For Tool UI docs, refer to the website or read the installed component schemas. - -## Key Files to Read First - -Understand what you're replacing: -- `frontend/src/app/pages/AgentChat/ToolCallBubble.tsx` — main tool call rendering (173 lines) -- `frontend/src/app/pages/AgentChat/toolCallUtils.ts` — `parseMcpToolName`, `getToolData`, `parseToolResult`, `formatInputDisplay`, etc. -- `frontend/src/app/pages/AgentChat/toolCallColors.tsx` — `colorizeInput`, `colorizeOutput`, terminal color constants -- `frontend/src/app/pages/AgentChat/ElapsedTimer.tsx` — duration timer component -- `frontend/src/app/pages/AgentChat/ToolGroupBubble.tsx` — grouped tool calls accordion - -Also read the installed Tool UI component schemas: -- `src/components/tool-ui/terminal/schema.ts` — Terminal props schema -- `src/components/tool-ui/code-block/schema.ts` — CodeBlock props schema -- `src/components/tool-ui/code-diff/schema.ts` — CodeDiff props schema - -## Background: How Tool Calls Work in This App - -Messages with `role: 'tool_call'` have this content shape: -```typescript -{ - tool: string; // tool name (e.g. "Bash", "Read", "Edit", "Grep", "mcp__google-gmail__search") - input: any; // tool input args (e.g. { command: "ls -la" }) -} -``` - -Messages with `role: 'tool_result'` have: -```typescript -{ - text: string; // raw result text - elapsed_ms?: number; // optional duration -} -// OR just a string -``` - -The `toolCallUtils.ts` file has `parseToolResult()` which parses the result into structured types: -- `{ type: 'bash', stdout, stderr, exitCode }` for Bash tool -- `{ type: 'text', content, isError }` for Read/Grep/etc. -- `{ type: 'mcp', service, action, data }` for MCP tools (handled by Agent 6) - -## Step-by-Step - -### 1. Read Tool UI component APIs - -Read the schema files for each installed component: -- `src/components/tool-ui/terminal/schema.ts` -- `src/components/tool-ui/code-block/schema.ts` -- `src/components/tool-ui/code-diff/schema.ts` - -Understand the props each component accepts. - -### 2. Understand the toolkit registration pattern - -In assistant-ui, tool calls are rendered by registering renderers in a `Toolkit` object. Each key is a tool name, and the value describes how to render it. - -Use `assistantUIDocs` to look up: -- `guides/tool-ui` — Generative UI / tool rendering -- `copilots/make-assistant-tool-ui` — `makeAssistantToolUI` API - -The toolkit pattern from Tool UI's quick-start: -```tsx -const toolkit: Toolkit = { - toolName: { - type: "backend", - render: ({ result, args }) => { - // Parse result, return Tool UI component - return ; - }, - }, -}; -``` - -### 3. Implement native-tools.tsx - -Fill in `frontend/src/app/pages/AgentChat/toolkit/native-tools.tsx`. - -Register renderers for each native tool type: - -#### Bash → Terminal -```tsx -import { Terminal } from '@/components/tool-ui/terminal'; - -Bash: { - type: "backend", - render: ({ result, args }) => { - const command = args?.command || ''; - // Parse result: { stdout, stderr, exitCode, elapsed_ms } - return ( - - ); - }, -} -``` - -#### Read → CodeBlock -```tsx -import { CodeBlock } from '@/components/tool-ui/code-block'; - -Read: { - type: "backend", - render: ({ result, args }) => { - const filePath = args?.file_path || args?.path || ''; - const language = guessLanguage(filePath); // from extension - return ( - - ); - }, -} -``` - -#### Edit / Write → CodeDiff -```tsx -import { CodeDiff } from '@/components/tool-ui/code-diff'; - -Edit: { - type: "backend", - render: ({ result, args }) => { - const filePath = args?.file_path || args?.path || ''; - return ( - - ); - }, -} -``` - -#### Grep / Glob / Search → Terminal -These are search commands; display results in Terminal: -```tsx -Grep: { - type: "backend", - render: ({ result, args }) => { - return ( - - ); - }, -} -``` - -### 4. Handle the "input" display - -Currently `ToolCallBubble` shows the tool input (command, file path, etc.) in a terminal-style header. With Tool UI's `Terminal`, the `command` prop handles this. For `CodeBlock` and `CodeDiff`, the `filename` prop shows the context. - -For tools that don't fit Terminal/CodeBlock/CodeDiff, create a simple fallback renderer that shows the tool name and JSON-formatted args. - -### 5. Handle tool groups (ToolGroupBubble replacement) - -`ToolGroupBubble` wraps multiple tool calls in a collapsible accordion with a generated SVG icon and completed/pending counters. - -Check if assistant-ui has a `ChainOfThought` or `ToolGroup` primitive: -- Look up `primitives/chain-of-thought` in assistant-ui docs -- Look up `ui/reasoning` for grouping - -If assistant-ui provides grouping, use it. If not, create a simple wrapper component that groups consecutive tool calls into a collapsible section. Put this in `toolkit/native-tools.tsx` as a helper. - -### 6. Write a `guessLanguage` utility - -Create a small utility function that maps file extensions to language names for CodeBlock: -```typescript -function guessLanguage(filePath: string): string { - const ext = filePath.split('.').pop()?.toLowerCase(); - const map: Record = { - ts: 'typescript', tsx: 'typescript', js: 'javascript', jsx: 'javascript', - py: 'python', rs: 'rust', go: 'go', rb: 'ruby', java: 'java', - json: 'json', yaml: 'yaml', yml: 'yaml', md: 'markdown', - html: 'html', css: 'css', scss: 'scss', sh: 'bash', bash: 'bash', - sql: 'sql', xml: 'xml', toml: 'toml', // ... etc - }; - return map[ext || ''] || 'text'; -} -``` - -### 7. Handle pending/streaming states - -Currently `ToolCallBubble` shows: -- A pulsing cursor when pending -- A blinking cursor when streaming -- An elapsed timer when pending - -Tool UI's `Terminal` has built-in support for these states through its props. For pending state without result, you can render `Terminal` without stdout/stderr and it shows as in-progress. Check the exact behavior. - -## Files Created / Modified - -| File | Action | Description | -|------|--------|-------------| -| `toolkit/native-tools.tsx` | **Fill in** (was stub from Agent 1) | All native tool registrations | - -## Files Deleted (by this agent) - -| File | Lines | Replaced By | -|------|-------|------------| -| `ToolCallBubble.tsx` | 172 | Toolkit registrations in `native-tools.tsx` | -| `toolCallColors.tsx` | 211 | Tool UI Terminal handles ANSI colors natively | -| `toolCallUtils.ts` | 249 | Simplified parsing in toolkit renderers | -| `ElapsedTimer.tsx` | 41 | Terminal's `durationMs` prop | -| `ToolGroupBubble.tsx` | 200 | ChainOfThought or custom grouping | - -**Important**: Before deleting `toolCallUtils.ts`, check if other files import from it: -- `McpServiceCards.tsx` imports `ParsedMcpResult`, `formatTimestamp` — these are owned by Agent 6 -- `GmailCard.tsx` imports `getGmailHeader`, `formatTimestamp`, `stripHtml` -- `approvalUtils.tsx` imports `parseMcpToolName` - -If other files import from `toolCallUtils.ts`, **do not delete it yet**. Instead, move the functions that only native-tools need into `native-tools.tsx`, and leave `toolCallUtils.ts` for other agents to consume. Agent 7 (Cleanup) will delete it once all consumers are gone. - -Similarly for `ToolGroupBubble.tsx` — check if `AgentChat.tsx` imports `isToolGroup`/`isToolPair` from it. If so, coordinate with Agent 7. - -**Total deleted: ~873 lines** (depending on shared imports) - -## Files NOT Modified - -- `AgentChat.tsx` — Agent 7 handles integration -- `thread/`, `composer/` — owned by Agents 2 and 3 -- `toolkit/approval-tools.tsx`, `toolkit/mcp-tools.tsx` — owned by Agents 5 and 6 - -## Verification Checklist - -- [ ] `toolkit/native-tools.tsx` exports a `nativeToolkit` object -- [ ] Bash tool calls render as `Terminal` with command, stdout, stderr, exitCode, duration -- [ ] Read tool calls render as `CodeBlock` with syntax highlighting and filename -- [ ] Edit tool calls render as `CodeDiff` with old/new code and filename -- [ ] Grep/Glob/Search tool calls render as `Terminal` -- [ ] Unknown native tools render a sensible fallback -- [ ] No TypeScript errors -- [ ] Deleted files don't break other imports (check before deleting) diff --git a/ASSISTANT_UI_MIGRATION/MIGRATION_AGENT_5.md b/ASSISTANT_UI_MIGRATION/MIGRATION_AGENT_5.md deleted file mode 100644 index 852ce589..00000000 --- a/ASSISTANT_UI_MIGRATION/MIGRATION_AGENT_5.md +++ /dev/null @@ -1,229 +0,0 @@ -# Migration Agent 5: Tool Toolkit — Approvals & Questions - -## Objective - -Register Tool UI components (`ApprovalCard`, `QuestionFlow`, `OptionList`) in the toolkit for rendering HITL approval requests and user questions. Replace `ApprovalBar`, `BatchApprovalBar`, `QuestionForm`, `ToolPreview`, and `approvalUtils`. - -## Prerequisites - -- **Agent 1 must be complete.** Tool UI components must be installed in `src/components/tool-ui/`, and the toolkit skeleton must exist at `toolkit/approval-tools.tsx`. - -## Constraints - -- **No custom styling**: Use default Tool UI ApprovalCard/QuestionFlow/OptionList appearance. -- **Only modify `toolkit/approval-tools.tsx`** and create any helpers needed within it. Do NOT modify `AgentChat.tsx`, `thread/`, `composer/`, or other toolkit files. -- **The assistant-ui MCP docs server is available** in `.cursor/mcp.json`. Use it for assistant-ui API lookups. - -## Key Files to Read First - -Understand what you're replacing: -- `frontend/src/app/pages/AgentChat/ApprovalBar.tsx` — single tool approval (195 lines) -- `frontend/src/app/pages/AgentChat/BatchApprovalBar.tsx` — mass approve/deny (179 lines) -- `frontend/src/app/pages/AgentChat/QuestionForm.tsx` — AskUserQuestion with options/multi-select/free-text (238 lines) -- `frontend/src/app/pages/AgentChat/ToolPreview.tsx` — code preview of bash/read/write/edit/grep args (140 lines) -- `frontend/src/app/pages/AgentChat/approvalUtils.tsx` — MCP tool metadata, integration icons (139 lines) - -Also read the installed Tool UI schemas: -- `src/components/tool-ui/approval-card/schema.ts` -- `src/components/tool-ui/question-flow/schema.ts` -- `src/components/tool-ui/option-list/schema.ts` - -## Background: How Approvals Work in This App - -The backend sends `approval_request` events over WebSocket. These are stored in `session.pending_approvals` in Redux: - -```typescript -interface ApprovalRequest { - id: string; - session_id: string; - tool_name: string; // e.g. "Bash", "mcp__google-gmail__sendEmail" - tool_input: Record; // the tool args needing approval - created_at: string; -} -``` - -The user approves or denies via `handleApprove(requestId, updatedInput?)` or `handleDeny(requestId, message?)` which dispatches to the backend. - -Special case: `tool_name === 'AskUserQuestion'` renders a `QuestionForm` instead of an approval bar. The `tool_input` contains: -```typescript -{ - question: string; - options?: Array<{ id, label, description? }>; - allow_multiple?: boolean; - allow_free_text?: boolean; -} -``` - -The user's response is sent back via `onApprove(requestId, { answer: selectedOptions })`. - -## Step-by-Step - -### 1. Read Tool UI component schemas - -Read the installed schema files to understand each component's props: -- `src/components/tool-ui/approval-card/schema.ts` — `ApprovalCard` props -- `src/components/tool-ui/question-flow/schema.ts` — `QuestionFlow` props -- `src/components/tool-ui/option-list/schema.ts` — `OptionList` props - -### 2. Understand the rendering context - -Approvals are NOT standard tool call results — they're rendered in a separate area below the message list in `AgentChat.tsx` (lines 199-205). They come from `session.pending_approvals`, not from the message stream. - -This means approvals might not fit neatly into the toolkit registration pattern (which is for rendering tool call results inside the message thread). Instead, they may need to be rendered as standalone components in the approval area. - -**Two approaches**: - -**Approach A — Keep approvals as standalone components**: Create wrapper components in `toolkit/approval-tools.tsx` that use Tool UI's `ApprovalCard`, `QuestionFlow`, and `OptionList` internally. These are rendered in the approval area of `AgentChat.tsx` (Agent 7 wires this in). - -**Approach B — Render approvals as tool call parts in the thread**: Map `pending_approvals` to tool messages in the runtime adapter, so they appear inline in the message thread. The toolkit renderers handle them. - -Go with **Approach A** — it's simpler and matches the existing UX where approvals appear as a bar between the messages and the composer. The components just change from custom MUI to Tool UI. - -### 3. Create approval wrapper components - -#### SingleApproval → ApprovalCard - -Map from `ApprovalRequest` to `ApprovalCard` props: - -```tsx -import { ApprovalCard } from '@/components/tool-ui/approval-card'; - -export const ToolApproval: React.FC<{ - request: ApprovalRequest; - onApprove: (id: string) => void; - onDeny: (id: string, message?: string) => void; -}> = ({ request, onApprove, onDeny }) => { - const parsedTool = parseMcpToolName(request.tool_name); - - return ( - onApprove(request.id)} - onCancel={() => onDeny(request.id)} - /> - ); -}; -``` - -The `metadata` prop accepts key-value pairs, which maps well to showing the tool input args: -```typescript -function buildMetadata(toolInput: Record): Array<{ key: string; value: string }> { - return Object.entries(toolInput) - .filter(([, v]) => v != null) - .slice(0, 5) - .map(([key, value]) => ({ - key, - value: typeof value === 'string' ? value.slice(0, 100) : JSON.stringify(value).slice(0, 100), - })); -} -``` - -#### QuestionForm → QuestionFlow + OptionList - -When `tool_name === 'AskUserQuestion'`, render a `QuestionFlow` (if multi-step) or `OptionList` (if single question with options): - -```tsx -import { OptionList } from '@/components/tool-ui/option-list'; -import { QuestionFlow } from '@/components/tool-ui/question-flow'; - -export const ToolQuestion: React.FC<{ - request: ApprovalRequest; - onApprove: (id: string, updatedInput?: Record) => void; - onDeny: (id: string) => void; -}> = ({ request, onApprove, onDeny }) => { - const { question, options, allow_multiple, allow_free_text } = request.tool_input; - - if (options?.length > 0) { - return ( - ({ - id: opt.id || opt.value || opt.label, - label: opt.label || opt.text, - description: opt.description, - }))} - selectionMode={allow_multiple ? 'multi' : 'single'} - actions={[ - { id: 'confirm', label: 'Submit' }, - { id: 'cancel', label: 'Skip', variant: 'secondary' }, - ]} - onAction={(actionId, selection) => { - if (actionId === 'confirm') { - onApprove(request.id, { answer: selection }); - } else { - onDeny(request.id); - } - }} - /> - ); - } - - // Free text question without options — render a simple card - // (or use a text input approach) -}; -``` - -#### BatchApprovalBar → composed ApprovalCards - -When `session.pending_approvals.length > 1`, render a batch approval UI. Options: -- Render multiple `ApprovalCard` instances stacked -- Add a "Approve All" / "Deny All" header above them - -### 4. Port MCP tool metadata - -The existing `approvalUtils.tsx` has `parseMcpToolName()` and `useMcpToolMeta()` that look up MCP tool metadata (integration icons, descriptions) from the Redux store. Port these utility functions into the approval-tools file or a local helper. - -**Important**: `parseMcpToolName` is also used by Agent 6 (for MCP service cards) and Agent 4 (in toolCallUtils). If you extract it, put it in a shared location that other agents can access, or duplicate the logic since it's small. - -### 5. Handle ToolPreview replacement - -`ToolPreview.tsx` renders a preview of tool arguments for native tools (showing the bash command, file path, edit diff, etc.). With Tool UI, this preview is naturally handled by the metadata display in `ApprovalCard`. - -For more detailed previews (like showing the full bash command or code diff), you can nest a Tool UI `Terminal` or `CodeBlock` inside the approval area. But for now, the metadata key-value pairs should be sufficient. - -## Files Created / Modified - -| File | Action | Description | -|------|--------|-------------| -| `toolkit/approval-tools.tsx` | **Fill in** (was stub) | Approval renderers: `ToolApproval`, `ToolQuestion`, batch wrapper | - -## Files Deleted (by this agent) - -| File | Lines | Replaced By | -|------|-------|------------| -| `ApprovalBar.tsx` | 195 | `ToolApproval` using `ApprovalCard` | -| `BatchApprovalBar.tsx` | 179 | Batch wrapper using composed `ApprovalCard` | -| `QuestionForm.tsx` | 238 | `ToolQuestion` using `OptionList` / `QuestionFlow` | -| `ToolPreview.tsx` | 140 | `ApprovalCard` metadata + optional nested Terminal/CodeBlock | -| `approvalUtils.tsx` | 139 | Utility functions ported to `approval-tools.tsx` | - -**Important**: Before deleting `approvalUtils.tsx`, check if other files import `parseMcpToolName` from it. If `ToolCallBubble.tsx` or `McpServiceCards.tsx` imports it, leave the file until Agent 7 cleanup, or extract the shared function. - -**Total deleted: ~891 lines** - -## Files NOT Modified - -- `AgentChat.tsx` — Agent 7 wires the new approval components into the approval area -- `thread/`, `composer/` — owned by Agents 2 and 3 -- Other toolkit files — owned by Agents 4 and 6 - -## Verification Checklist - -- [ ] `toolkit/approval-tools.tsx` exports `approvalToolkit` and approval wrapper components -- [ ] `ToolApproval` renders an `ApprovalCard` with title, description, metadata -- [ ] Native tool approvals show tool name and key input args -- [ ] MCP tool approvals show the integration name and action -- [ ] Destructive variant is used for dangerous tools (e.g. Bash with `rm`, `delete`) -- [ ] `ToolQuestion` renders `OptionList` for questions with options -- [ ] Multi-select works for `allow_multiple: true` questions -- [ ] Free-text questions render an input field -- [ ] Approve/Deny callbacks work correctly -- [ ] Batch approval renders multiple cards or a batch wrapper -- [ ] No TypeScript errors -- [ ] Deleted files don't break other imports diff --git a/ASSISTANT_UI_MIGRATION/MIGRATION_AGENT_6.md b/ASSISTANT_UI_MIGRATION/MIGRATION_AGENT_6.md deleted file mode 100644 index c9c5b852..00000000 --- a/ASSISTANT_UI_MIGRATION/MIGRATION_AGENT_6.md +++ /dev/null @@ -1,369 +0,0 @@ -# Migration Agent 6: Tool Toolkit — MCP Services, Browser Feed & Custom Tools - -## Objective - -Register Tool UI components for MCP service results (`MessageDraft` for Gmail, `DataTable`/`ItemCarousel` for Calendar/Drive, `ProgressTracker` for browser feed). Wire existing `AgentToolBubble` (sub-agent rendering) and `ViewBubble` (iframe previews) into the toolkit as custom entries. Replace `GmailCard`, `McpServiceCards`, `BrowserAgentInlineFeed`, `BrowserFeedEntryRow`, `browserFeedUtils`, and `DiffViewer`. - -## Prerequisites - -- **Agent 1 must be complete.** Tool UI components must be installed in `src/components/tool-ui/`, and the toolkit skeleton must exist at `toolkit/mcp-tools.tsx` and `toolkit/custom-tools.tsx`. - -## Constraints - -- **No custom styling**: Use default Tool UI appearance for MessageDraft, DataTable, ProgressTracker. -- **Only modify `toolkit/mcp-tools.tsx` and `toolkit/custom-tools.tsx`**. Do NOT modify `AgentChat.tsx`, `thread/`, `composer/`, or other toolkit files. -- **Keep `AgentToolBubble.tsx` and `ViewBubble.tsx` as-is** — just wire them into the custom toolkit as renderers. Do not rewrite them. -- **The assistant-ui MCP docs server is available** in `.cursor/mcp.json`. - -## Key Files to Read First - -Understand what you're replacing: - -- `frontend/src/app/pages/AgentChat/McpServiceCards.tsx` — dispatches to Gmail/Calendar/Drive/Generic cards (158 lines) -- `frontend/src/app/pages/AgentChat/GmailCard.tsx` — email list/detail rendering (136 lines) -- `frontend/src/app/pages/AgentChat/BrowserAgentInlineFeed.tsx` — browser automation feed (164 lines) -- `frontend/src/app/pages/AgentChat/BrowserFeedEntryRow.tsx` — feed entry rendering (120 lines) -- `frontend/src/app/pages/AgentChat/browserFeedUtils.ts` — browser feed utilities (129 lines) -- `frontend/src/app/pages/AgentChat/DiffViewer.tsx` — git diff panel (140 lines) - -Understand what you're keeping and wiring: - -- `frontend/src/app/pages/AgentChat/AgentToolBubble.tsx` — InvokeAgent/CreateAgent (193 lines) — KEEP -- `frontend/src/app/pages/AgentChat/ViewBubble.tsx` — iframe app preview (194 lines) — KEEP -- `frontend/src/app/pages/AgentChat/ViewBubbleParts.tsx` — iframe parts (136 lines) — KEEP - -Also read the installed Tool UI schemas: - -- `src/components/tool-ui/message-draft/schema.ts` -- `src/components/tool-ui/data-table/schema.tsdi` -- `src/components/tool-ui/progress-tracker/schema.ts` -- `src/components/tool-ui/item-carousel/schema.ts` (if installed) - -## Background: How MCP Tool Results Work - -MCP tools have names like `mcp__google-gmail__search`, `mcp__google-calendar__listEvents`, `mcp__google-drive__listFiles`. The existing `parseMcpToolName()` function (in `toolCallUtils.ts`) parses these into: - -```typescript -{ - isMcp: true, - serverSlug: 'google-gmail', - service: 'gmail', - actionName: 'search', - displayName: 'Search', -} -``` - -The `McpResultCard` component dispatches based on `service`: - -- `gmail` → `GmailCard` -- `calendar` → `CalendarCard` -- `drive` / `sheets` → `DriveCard` -- anything else → `GenericMcpCard` - -## Step-by-Step - -### 1. Read Tool UI component schemas - -Read the installed schemas: - -- `src/components/tool-ui/message-draft/schema.ts` — email/message rendering -- `src/components/tool-ui/data-table/schema.ts` — table rendering -- `src/components/tool-ui/progress-tracker/schema.ts` — step-by-step progress - -### 2. Implement mcp-tools.tsx — Gmail → MessageDraft - -The `GmailCard` renders: - -- **Email list**: Multiple email cards with subject, from, date, snippet -- **Single email**: Subject header, from/to/date fields, labels, body (markdown), attachments - -Map to Tool UI's `MessageDraft`: - -```tsx -import { MessageDraft } from '@/components/tool-ui/message-draft'; - -// For single email results -function renderGmailResult(data: any, action: string) { - const email = extractEmailFields(data); - return ( - - ); -} -``` - -For **email list results** (search/list), use `DataTable`: - -```tsx -import { DataTable } from '@/components/tool-ui/data-table'; - -function renderGmailList(messages: any[]) { - return ( - ({ - id: String(i), - ...extractEmailFields(msg), - }))} - /> - ); -} -``` - -### 3. Implement mcp-tools.tsx — Calendar → DataTable - -The `CalendarCard` renders: - -- **Event list**: Cards with summary + date -- **Single event**: Summary, start, end, location, description - -Map event lists to `DataTable`: - -```tsx -function renderCalendarList(items: any[]) { - return ( - ({ - id: String(i), - summary: item.summary || '(no title)', - start: item.start?.dateTime || item.start?.date || '', - end: item.end?.dateTime || item.end?.date || '', - location: item.location || '', - }))} - /> - ); -} -``` - -### 4. Implement mcp-tools.tsx — Drive → DataTable - -The `DriveCard` renders file lists with name and mimeType. Map to `DataTable`: - -```tsx -function renderDriveFiles(files: any[]) { - return ( - ({ - id: String(i), - name: f.name || f.id, - mimeType: f.mimeType?.split('/').pop() || '', - }))} - /> - ); -} -``` - -### 5. Implement mcp-tools.tsx — Generic MCP fallback - -For MCP tools without a specific handler, render a simple key-value display. Use `DataTable` with two columns (key, value) or create a minimal fallback component. - -### 6. Register MCP tool renderers in the toolkit - -The challenge: MCP tool names are dynamic (`mcp____`). You can't register every possible tool name in the toolkit. - -**Solution**: Use a catch-all pattern. Check if assistant-ui supports a `ToolFallback` component or a wildcard toolkit entry. Look up `ui/tool-fallback` in the docs. - -If the toolkit supports a fallback/default renderer: - -```tsx -export const mcpToolkit = { - // Specific MCP tools can be registered by name if desired - // Generic fallback handles all MCP tools - __fallback__: { - type: "backend", - render: ({ result, args, toolName }) => { - const mcpInfo = parseMcpToolName(toolName); - if (!mcpInfo.isMcp) return null; // Let other handlers deal with it - - switch (mcpInfo.service) { - case 'gmail': return renderGmailResult(result, mcpInfo.actionName); - case 'calendar': return renderCalendarResult(result, mcpInfo.actionName); - case 'drive': return renderDriveResult(result); - default: return renderGenericMcp(result); - } - }, - }, -}; -``` - -If assistant-ui doesn't support a fallback, register a `ToolFallback` component that checks if the tool name starts with `mcp__` and routes accordingly. - -### 7. Implement mcp-tools.tsx — Browser Feed → ProgressTracker - -The `BrowserAgentInlineFeed` renders a compact activity log of browser automation steps (navigate, click, type, screenshot) with status indicators. - -Map to `ProgressTracker`: - -```tsx -import { ProgressTracker } from '@/components/tool-ui/progress-tracker'; - -function renderBrowserFeed(entries: BrowserFeedEntry[]) { - return ( - ({ - id: entry.id, - label: entry.action, // "navigate", "click", "type" - description: entry.detail, // URL, selector, text - status: entry.status === 'done' ? 'completed' - : entry.status === 'error' ? 'failed' - : entry.status === 'running' ? 'in-progress' - : 'pending', - }))} - /> - ); -} -``` - -The browser feed data comes from Redux (via `useBrowserActivity` or similar). The renderer needs to read this data. Since it's rendered inside a tool call bubble (for `BrowserAgent` tool), the data should be accessible from the tool result or via Redux selector. - -### 8. Implement mcp-tools.tsx — DiffViewer → CodeDiff - -The `DiffViewer.tsx` fetches a git worktree diff from the API and renders colorized diff output. Replace with Tool UI's `CodeDiff`: - -```tsx -import { CodeDiff } from '@/components/tool-ui/code-diff'; - -// DiffViewer fetches diff text from /api/agents/sessions/{id}/worktree-diff -// This is rendered as a side panel, not a tool call result. -// It may need to stay as a standalone component, just using CodeDiff internally. -``` - -**Note**: DiffViewer is rendered in `ChatHeader`, not as a tool call. It may not fit the toolkit pattern. Two options: - -- Replace the rendering logic inside DiffViewer to use `CodeDiff` component but keep the wrapper -- Or just replace the internals - -Go with replacing the internals: keep a thin wrapper that fetches the diff and passes it to `CodeDiff`. - -### 9. Implement custom-tools.tsx — Wire AgentToolBubble and ViewBubble - -These are kept as-is but registered in the toolkit so assistant-ui knows how to render them: - -```tsx -import { InvokeAgentBubble, CreateAgentBubble } from '../AgentToolBubble'; -import ViewBubble from '../ViewBubble'; - -export const customToolkit = { - InvokeAgent: { - type: "backend", - render: ({ result, args, toolCallId }) => ( - - ), - }, - CreateAgent: { - type: "backend", - render: ({ result, args, toolCallId }) => ( - - ), - }, - RenderOutput: { - type: "backend", - render: ({ result, args, toolCallId }) => ( - - ), - }, -}; -``` - -Adapt the props to match what `AgentToolBubble` and `ViewBubble` expect. Read those files to understand their prop interfaces. - -### 10. Port `parseMcpToolName` and `extractEmailFields` - -These utility functions are needed by the MCP toolkit. Port them into `mcp-tools.tsx` or a local helper file. Keep them minimal — only port what you need. - -## Files Created / Modified - - -| File | Action | Description | -| -------------------------- | ---------------------- | ----------------------------------------------------------------------- | -| `toolkit/mcp-tools.tsx` | **Fill in** (was stub) | Gmail, Calendar, Drive, Generic MCP, Browser feed, DiffViewer renderers | -| `toolkit/custom-tools.tsx` | **Fill in** (was stub) | AgentToolBubble, ViewBubble wrappers | - - -## Files Deleted (by this agent) - - -| File | Lines | Replaced By | -| ---------------------------- | ----- | ---------------------------------------------------- | -| `GmailCard.tsx` | 136 | MessageDraft + DataTable in `mcp-tools.tsx` | -| `McpServiceCards.tsx` | 158 | Routing logic in `mcp-tools.tsx` | -| `BrowserAgentInlineFeed.tsx` | 164 | ProgressTracker in `mcp-tools.tsx` | -| `BrowserFeedEntryRow.tsx` | 120 | ProgressTracker step rendering | -| `browserFeedUtils.ts` | 129 | Simplified in `mcp-tools.tsx` | -| `DiffViewer.tsx` | 140 | CodeDiff (keep thin wrapper if needed for API fetch) | - - -**Important**: `DiffViewer` is rendered in `ChatHeader.tsx`, not as a tool call. Before deleting, check how it's used. If it's a side panel that fetches from an API, you may want to keep a thin wrapper that uses `CodeDiff` internally rather than fully deleting it. - -**Total deleted: ~847 lines** - -## Files Kept (wired as custom toolkit entries) - - -| File | Lines | Action | -| --------------------- | ----- | -------------------------------------- | -| `AgentToolBubble.tsx` | 193 | Kept, registered in `custom-tools.tsx` | -| `ViewBubble.tsx` | 194 | Kept, registered in `custom-tools.tsx` | -| `ViewBubbleParts.tsx` | 136 | Kept (dependency of ViewBubble) | - - -## Files NOT Modified - -- `AgentChat.tsx` — Agent 7 handles integration -- `thread/`, `composer/` — owned by Agents 2 and 3 -- Other toolkit files — owned by Agents 4 and 5 - -## Verification Checklist - -- `toolkit/mcp-tools.tsx` exports `mcpToolkit` with MCP tool renderers -- Gmail email results render as `MessageDraft` (single) or `DataTable` (list) -- Calendar events render as `DataTable` -- Drive files render as `DataTable` -- Unknown MCP tools render a generic key-value fallback -- Browser agent activity renders as `ProgressTracker` steps -- `toolkit/custom-tools.tsx` exports `customToolkit` with `InvokeAgent`, `CreateAgent`, `RenderOutput` -- `AgentToolBubble` and `ViewBubble` render correctly through the toolkit -- DiffViewer rendering uses `CodeDiff` internally -- No TypeScript errors -- Deleted files don't break other imports - diff --git a/ASSISTANT_UI_MIGRATION/MIGRATION_AGENT_7.md b/ASSISTANT_UI_MIGRATION/MIGRATION_AGENT_7.md deleted file mode 100644 index 1b3b4b4e..00000000 --- a/ASSISTANT_UI_MIGRATION/MIGRATION_AGENT_7.md +++ /dev/null @@ -1,257 +0,0 @@ -# Migration Agent 7: Integration, SkillBuilderChat & Cleanup - -## Objective - -Wire all Phase 2 outputs together in `AgentChat.tsx`. Migrate `SkillBuilderChat.tsx`. Delete all remaining dead code. Verify the build passes and the chat is functional end-to-end. - -**This agent runs last, after ALL Phase 2 agents (2–6) are complete.** - -## Prerequisites - -- **Agents 1–6 must ALL be complete.** - - Agent 1: Runtime adapter, toolkit skeleton, providers exist - - Agent 2: `thread/OpenSwarmThread.tsx` exists and works - - Agent 3: `composer/OpenSwarmComposer.tsx` exists and works - - Agent 4: `toolkit/native-tools.tsx` is filled in - - Agent 5: `toolkit/approval-tools.tsx` is filled in - - Agent 6: `toolkit/mcp-tools.tsx` and `toolkit/custom-tools.tsx` are filled in - -## Constraints - -- **No custom styling**: Do not add CSS to make assistant-ui/Tool UI match MUI. -- **The assistant-ui MCP docs server is available** in `.cursor/mcp.json`. - -## Step-by-Step - -### 1. Rewrite AgentChat.tsx - -This is the main integration task. Replace the current `AgentChat.tsx` (250 lines) with the new structure. - -The current structure: -``` -AgentChat -├── ChatHeader (keep) -├── Scroll container with message render loop (REPLACE → OpenSwarmThread) -│ ├── renderItems.map (messages, tool groups, tool pairs) (REPLACE) -│ ├── Streaming message (REPLACE — handled by runtime) -│ ├── ThinkingBubble (REPLACE — handled by Thread) -│ ├── Resume bubble (KEEP) -│ └── Scroll-to-bottom button (REPLACE — Thread handles this) -├── ApprovalBar / BatchApprovalBar (REPLACE → approval-tools components) -├── Glow CTA (keep) -└── MessageQueue > ChatInput (REPLACE ChatInput → OpenSwarmComposer) -``` - -The new structure: -```tsx -import { AssistantRuntimeProvider } from '@assistant-ui/react'; -import { useOpenSwarmRuntime } from './runtime/useOpenSwarmRuntime'; -import { toolkit } from './toolkit'; -import { OpenSwarmThread } from './thread/OpenSwarmThread'; -import { OpenSwarmComposer } from './composer/OpenSwarmComposer'; -import { ToolApproval, ToolQuestion, BatchApproval } from './toolkit/approval-tools'; -import ChatHeader from './ChatHeader'; -import MessageQueue from './MessageQueue'; - -const AgentChat = ({ sessionId, ... }) => { - const runtime = useOpenSwarmRuntime(sessionId); - const { session, handleApprove, handleDeny, handleStop, ... } = useAgentChat({ sessionId }); - - return ( - - - - {!embedded && } - - {/* Thread replaces the scroll container + message loop */} - - - {/* Resume bubble (keep existing) */} - {showResumeBubble && session.status === 'stopped' && ( - - )} - - {/* Approvals area — now using Tool UI components */} - {session.pending_approvals.length > 1 ? ( - - ) : ( - session.pending_approvals.map((req) => ( - req.tool_name === 'AskUserQuestion' - ? - : - )) - )} - - {/* Composer replaces ChatInput */} - {isGlowing ? ( - - ) : ( - - - - )} - - - - ); -}; -``` - -### 2. Register the toolkit - -Check how assistant-ui registers the toolkit. It may be via: -- `Tools` component or `useAui` hook (check current docs) -- Props on `AssistantRuntimeProvider` -- A separate context provider - -Use `assistantUIDocs` to look up: -- `copilots/model-context` — how to provide tools/toolkit -- `guides/tool-ui` — how toolkit is registered - -Make sure ALL toolkit entries (native + approvals + MCP + custom) are registered so tool calls render correctly. - -### 3. Clean up useAgentChat hook - -The `useAgentChat` hook (`hooks/useAgentChat.ts`) currently manages: -- Session state and WebSocket lifecycle (KEEP) -- Scroll container ref + scroll handlers (REMOVE — Thread handles scroll) -- `handleSend` (KEEP — still used by runtime adapter or Composer) -- Approval handlers (KEEP) -- Stop/resume handlers (KEEP) - -Remove the scroll-related code: -- `scrollContainerRef` — Thread manages its own scroll -- `showScrollButton` — Thread has built-in scroll-to-bottom -- `handleScroll`, `scrollToBottom` — Thread handles these - -Also remove `chatInputRef` if the Composer no longer uses it (replaced by ComposerRuntime API). - -### 4. Migrate SkillBuilderChat - -Read `frontend/src/app/pages/Skills/SkillBuilderChat.tsx` (403 lines). - -It embeds `AgentChat` in `embedded` mode: -```tsx - -``` - -Since `AgentChat` is being rewritten with assistant-ui, `SkillBuilderChat` should work as-is **if** the `embedded` prop behavior is preserved. Verify: -- `embedded` hides the `ChatHeader` -- The Composer still renders in embedded mode -- The Thread fills the available space - -If `SkillBuilderChat` used `ChatInputHandle` ref to programmatically set content: -- Check if Agent 3 created a `useComposerHandle` hook -- Update `SkillBuilderChat` to use the new API (ComposerRuntime's `setText()`) - -### 5. Delete all remaining dead files - -After integration, these files should no longer be imported anywhere: - -**Already deleted by Phase 2 agents** (verify they're gone): -- Agent 2: `MessageBubble`, `UserBubbleContent`, `AssistantBubbleContent`, `MessageActionBar`, `BranchNavigator`, `ThinkingBubble`, `MessageImageThumbnails`, `messageBubbleUtils`, `useMessageRendering` -- Agent 3: `ChatInput`, `useChatSubmit`, `CommandPicker`, `commandPickerTypes`, `useCommandPickerItems`, `CommandPickerIcons`, `SlashCommandPicker`, `AttachmentChips`, `ImageAttachments` -- Agent 4: `ToolCallBubble`, `toolCallColors`, `ElapsedTimer`, `ToolGroupBubble` -- Agent 5: `ApprovalBar`, `BatchApprovalBar`, `QuestionForm`, `ToolPreview`, `approvalUtils` -- Agent 6: `GmailCard`, `McpServiceCards`, `BrowserAgentInlineFeed`, `BrowserFeedEntryRow`, `browserFeedUtils`, `DiffViewer` - -**Files that may have been left due to shared imports** — delete now if no longer needed: -- `toolCallUtils.ts` — functions may have been ported to toolkit files -- `richEditorUtils.ts` — functions may have been moved to RichPromptEditor -- `AttachedContextSection.tsx` — check if Thread's UserMessage uses it - -Run a grep/search for any remaining imports of deleted files. Fix any broken imports. - -### 6. Remove unused npm packages - -After the migration, some packages may no longer be needed: -- `react-markdown` — if assistant-ui's Markdown component replaces all usage. **Check**: `RichPromptEditor` or other non-chat pages may still use it. -- `remark-gfm` — same as above - -Only remove if truly unused across the entire codebase. - -### 7. Clean up agentsSlice streaming reducers - -The `streamStart`, `streamDelta`, `streamEnd` reducers in the agents Redux slice may be simplified since the ExternalStoreRuntime adapter handles streaming display state. However, keep them if: -- The WebSocketManager still dispatches them -- The runtime adapter reads `session.streamingMessage` from Redux - -Don't remove them if removing would break the data flow. Just leave them — they're small. - -### 8. Verify the build - -```bash -cd frontend -npm run build -``` - -Fix any TypeScript errors, missing imports, or broken references. - -### 9. Manual smoke test - -Start the dev server and verify: -- [ ] Chat page loads -- [ ] Existing sessions show their message history -- [ ] Can type in the Composer and send a message -- [ ] `@` trigger opens the mention popover with categories -- [ ] `/` trigger opens command categories (templates, skills, modes) -- [ ] Streaming responses animate token-by-token -- [ ] Tool calls render with Terminal/CodeBlock/CodeDiff components -- [ ] MCP tool results render (Gmail, Calendar, Drive) -- [ ] Approval requests show ApprovalCard -- [ ] AskUserQuestion shows OptionList/QuestionFlow -- [ ] Branch navigation works -- [ ] Copy/edit/regenerate actions work -- [ ] Scroll-to-bottom works -- [ ] SkillBuilderChat works in embedded mode -- [ ] Dashboard page still works (no regressions) -- [ ] Settings/Modes/Templates pages still work - -## Files Modified - -| File | Change | -|------|--------| -| `AgentChat.tsx` | Rewritten to use assistant-ui Thread + Composer + toolkit | -| `hooks/useAgentChat.ts` | Removed scroll-related code | -| `SkillBuilderChat.tsx` | Updated to work with new AgentChat (if needed) | - -## Files Deleted (remaining dead code) - -| File | Reason | -|------|--------| -| `toolCallUtils.ts` | Functions ported to toolkit files | -| `richEditorUtils.ts` | Functions moved to RichPromptEditor (if still needed) | -| `AttachedContextSection.tsx` | Context rendering moved to UserMessage in Thread | -| Any other orphaned files | Verify with import search | - -## Verification Checklist - -- [ ] `npm run build` succeeds with no errors -- [ ] `npm run dev` starts the dev server -- [ ] Chat page renders with assistant-ui Thread -- [ ] Messages display correctly (user + assistant) -- [ ] Tool calls render with Tool UI components -- [ ] Approvals render with ApprovalCard -- [ ] Questions render with OptionList/QuestionFlow -- [ ] Composer works with Mention popover -- [ ] Streaming works -- [ ] Branching works -- [ ] SkillBuilderChat works -- [ ] No console errors -- [ ] No broken imports (grep for imports of deleted files) -- [ ] Non-chat pages (Dashboard, Settings, Modes, Templates, Tools, Views) still work diff --git a/ASSISTANT_UI_MIGRATION/OVERVIEW.md b/ASSISTANT_UI_MIGRATION/OVERVIEW.md deleted file mode 100644 index 48b7ab8c..00000000 --- a/ASSISTANT_UI_MIGRATION/OVERVIEW.md +++ /dev/null @@ -1,165 +0,0 @@ -# assistant-ui + Tool UI Migration Plan - -## Goal - -Replace the custom chat UI in `frontend/src/app/pages/AgentChat/` with [assistant-ui](https://www.assistant-ui.com/) (React chat primitives) and [Tool UI](https://www.tool-ui.com/) (tool call rendering components). This replaces ~5,800 lines of custom chat code with maintained, accessible, schema-driven components. - -## Decisions - -- **Tailwind + MUI coexist**: Tailwind is added for assistant-ui/Tool UI; MUI stays for non-chat pages (Dashboard, Settings, etc.) -- **Clean swap**: No feature flag — old code is replaced directly -- **No custom styling**: Use default assistant-ui/Tool UI appearance as-is (no theming to match MUI) -- **SkillBuilderChat**: Included in migration -- **Sub-agent rendering**: Kept as-is, wired into toolkit as custom tool UIs - -## Architecture: Before → After - -### Before -``` -AgentChat.tsx -├── useAgentChat.ts (WS lifecycle, session state) -├── useMessageRendering.ts (branch resolution, render items) -├── ChatInput.tsx + CommandPicker (custom contentEditable + @/slash picker) -├── MessageBubble → UserBubbleContent / AssistantBubbleContent (custom markdown) -├── MessageActionBar (copy/edit/regen/branch) -├── BranchNavigator (custom branch picker) -├── ToolCallBubble + toolCallColors + toolCallUtils (custom terminal rendering) -├── ToolGroupBubble (custom accordion) -├── ApprovalBar / BatchApprovalBar / QuestionForm (custom HITL) -├── GmailCard / McpServiceCards (custom MCP rendering) -├── BrowserAgentInlineFeed (custom progress feed) -├── DiffViewer (custom diff rendering) -└── ThinkingBubble (custom reasoning display) -``` - -### After -``` -AgentChat.tsx -├── AssistantRuntimeProvider + ExternalStoreRuntime (bridges Redux + WS) -├── OpenSwarmThread (assistant-ui Thread + Message + ActionBar + BranchPicker) -├── OpenSwarmComposer (assistant-ui Composer + ComposerMentionPopover) -├── Toolkit registry -│ ├── native-tools.tsx → Tool UI Terminal, CodeBlock, CodeDiff -│ ├── approval-tools.tsx → Tool UI ApprovalCard, QuestionFlow, OptionList -│ ├── mcp-tools.tsx → Tool UI MessageDraft, DataTable, ProgressTracker -│ └── custom-tools.tsx → AgentToolBubble, ViewBubble (kept as-is) -├── ModelModeSelector (kept, MUI) -├── MessageQueue (kept, MUI) -└── ChatHeader (kept, MUI) -``` - -## Phases & Dependency Graph - -``` -┌─────────────────────────────────────┐ -│ PHASE 1 — Sequential (blocking) │ -│ Agent 1: Foundation & Packages │ -└──────────────┬──────────────────────┘ - │ - ┌─────────┼──────────┬──────────────┬──────────────┐ - │ │ │ │ │ - ▼ ▼ ▼ ▼ ▼ -┌─────────┐┌─────────┐┌──────────┐┌──────────┐┌──────────────┐ -│ Agent 2 ││ Agent 3 ││ Agent 4 ││ Agent 5 ││ Agent 6 │ -│ Thread ││Composer ││ Tool UI: ││ Tool UI: ││ Tool UI: │ -│ & ││ & ││ Native ││Approvals ││ MCP Cards │ -│Messages ││Mentions ││ Tools ││ & ││ & Browser │ -│ ││ ││ ││Questions ││ Feed │ -└────┬────┘└────┬────┘└────┬─────┘└────┬─────┘└──────┬───────┘ - │ │ │ │ │ - │ PHASE 2 — All 5 agents run in parallel │ - │ │ │ │ │ - └─────────┴──────────┴────────────┴─────────────┘ - │ - ┌───────────▼───────────────┐ - │ PHASE 3 — Sequential │ - │ Agent 7: Integration, │ - │ SkillBuilderChat & │ - │ Cleanup │ - └───────────────────────────┘ -``` - -## Agent Summary - -| Agent | Plan File | Phase | What It Does | -|-------|-----------|-------|-------------| -| 1 | `MIGRATION_AGENT_1.md` | 1 (sequential) | Install Tailwind, shadcn, assistant-ui, Tool UI packages. Create ExternalStoreRuntime adapter, toolkit skeleton, new directory structure. Scaffold `AgentChat.tsx` with provider wrapper. | -| 2 | `MIGRATION_AGENT_2.md` | 2 (parallel) | Replace message list with `Thread`. Replace bubbles with `Message` primitives. Replace `MessageActionBar` → `ActionBar`, `BranchNavigator` → `BranchPicker`, `ThinkingBubble` → `Reasoning`. | -| 3 | `MIGRATION_AGENT_3.md` | 2 (parallel) | Replace `ChatInput` + `CommandPicker` with `Composer` + `ComposerMentionPopover`. Create `MentionAdapter` for templates/skills/modes/tools/files. | -| 4 | `MIGRATION_AGENT_4.md` | 2 (parallel) | Register Tool UI components for native tools: `Terminal` (bash), `CodeBlock` (file read), `CodeDiff` (edit/diff). Replace `ToolCallBubble`, `toolCallColors`, `toolCallUtils`. | -| 5 | `MIGRATION_AGENT_5.md` | 2 (parallel) | Register Tool UI components for approvals: `ApprovalCard`, `QuestionFlow`, `OptionList`. Replace `ApprovalBar`, `BatchApprovalBar`, `QuestionForm`, `ToolPreview`. | -| 6 | `MIGRATION_AGENT_6.md` | 2 (parallel) | Register Tool UI for MCP services: `MessageDraft` (Gmail), `DataTable` (Calendar/Drive), `ProgressTracker` (browser feed). Wire `AgentToolBubble` and `ViewBubble` as custom toolkit entries. | -| 7 | `MIGRATION_AGENT_7.md` | 3 (sequential) | Wire all pieces in `AgentChat.tsx`. Migrate `SkillBuilderChat`. Delete all dead files. Verify build. | - -## File Ownership (Parallel Safety) - -During Phase 2, each agent only touches files it owns. No conflicts. - -| Agent | Creates | Modifies | Deletes | -|-------|---------|----------|---------| -| 2 | `thread/` directory | Nothing shared | `MessageBubble`, `UserBubbleContent`, `AssistantBubbleContent`, `MessageActionBar`, `BranchNavigator`, `ThinkingBubble`, `MessageImageThumbnails`, `messageBubbleUtils`, `useMessageRendering` | -| 3 | `composer/` directory | Nothing shared | `ChatInput`, `useChatSubmit`, `CommandPicker`, `commandPickerTypes`, `useCommandPickerItems`, `CommandPickerIcons`, `SlashCommandPicker`, `AttachmentChips`, `ImageAttachments`, `richEditorUtils`, `RichPromptEditor` | -| 4 | `toolkit/native-tools.tsx` | Nothing shared | `ToolCallBubble`, `toolCallColors`, `toolCallUtils`, `ElapsedTimer`, `ToolGroupBubble` | -| 5 | `toolkit/approval-tools.tsx` | Nothing shared | `ApprovalBar`, `BatchApprovalBar`, `QuestionForm`, `ToolPreview`, `approvalUtils` | -| 6 | `toolkit/mcp-tools.tsx`, `toolkit/custom-tools.tsx` | Nothing shared | `GmailCard`, `McpServiceCards`, `BrowserAgentInlineFeed`, `BrowserFeedEntryRow`, `browserFeedUtils`, `DiffViewer` | -| 7 | (integration) | `AgentChat.tsx`, `SkillBuilderChat.tsx` | Remaining dead imports/files | - -## Key Technical Notes - -### ExternalStoreRuntime Adapter - -The runtime adapter (`runtime/useOpenSwarmRuntime.ts`) bridges Redux ↔ assistant-ui: -- **Messages**: Read from `session.messages` + `session.streamingMessage` in Redux -- **isRunning**: Derived from `session.status === 'running'` -- **onNew**: Dispatches `sendMessage` thunk → WebSocket -- **onEdit**: Dispatches `editMessage` thunk → REST API -- **onCancel**: Dispatches `stopAgent` thunk → REST API -- **Branches**: Mapped from `session.branches` + `session.active_branch_id` - -### Message Format Conversion - -Redux `AgentMessage` → assistant-ui format: -- `role: 'user'` → `{ role: 'user', content: [{ type: 'text', text }] }` -- `role: 'assistant'` → `{ role: 'assistant', content: [{ type: 'text', text }] }` -- `role: 'tool_call'` → `{ role: 'assistant', content: [{ type: 'tool-call', toolCallId, toolName, args }] }` -- `role: 'tool_result'` → `{ role: 'tool', content: [{ type: 'tool-result', toolCallId, result }] }` - -### Tailwind + MUI Coexistence - -Tailwind is configured with a prefix or scoped to assistant-ui components to avoid conflicts with MUI's global styles. The `important` selector strategy or Tailwind's `prefix` option may be needed. - -### Tool UI Registration Pattern - -All Tool UI components are registered in toolkit files. Each toolkit file exports a partial `Toolkit` object. The `toolkit/index.ts` merges them: - -```typescript -import { nativeToolkit } from './native-tools'; -import { approvalToolkit } from './approval-tools'; -import { mcpToolkit } from './mcp-tools'; -import { customToolkit } from './custom-tools'; - -export const toolkit: Toolkit = { - ...nativeToolkit, - ...approvalToolkit, - ...mcpToolkit, - ...customToolkit, -}; -``` - -## Lines of Code Impact (Estimated) - -| Category | Lines | -|----------|-------| -| Deleted (old components) | ~5,800 | -| New bridge/adapter code | ~600–800 | -| Tool UI components (installed, not authored) | ~2,000 (maintained externally) | -| **Net reduction in authored code** | **~5,000** | - -## Estimated Effort - -| Phase | Agents | Estimated Time | -|-------|--------|---------------| -| Phase 1 | Agent 1 | 1–2 hours | -| Phase 2 | Agents 2–6 (parallel) | 2–4 hours each, ~4 hours wall clock | -| Phase 3 | Agent 7 | 1–2 hours | -| **Total wall clock** | | **~6–8 hours** |