1092 lines
29 KiB
Markdown
1092 lines
29 KiB
Markdown
# Assistant Model Selector Implementation Plan
|
|
|
|
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
|
|
|
|
**Goal:** Add a provider-discovered assistant model selector that saves each user's selected model and uses it for chat completions.
|
|
|
|
**Architecture:** Keep model discovery server-side in a focused LLM helper, expose it through `/api/agent/models`, and persist the selected non-default model on the `users` row. The chat route resolves the effective model from the posted model, saved preference, and `LLM_MODEL` fallback before creating the OpenAI-compatible client.
|
|
|
|
**Tech Stack:** Next.js 15 App Router, TypeScript, Drizzle, PostgreSQL, shadcn/ui Base UI Select, Tailwind v4, Node test runner via `tsx --test`, Playwright.
|
|
|
|
---
|
|
|
|
## File Map
|
|
|
|
- Create: `src/lib/llm/models.ts` — fetch, normalize, validate, and resolve LLM model choices.
|
|
- Create: `src/app/api/agent/models/route.ts` — authenticated model-discovery endpoint for the chat UI.
|
|
- Modify: `src/lib/llm/config.ts` — centralize fallback model shape through existing config.
|
|
- Modify: `src/lib/llm/index.ts` — allow `createLlmClient({ model })` overrides.
|
|
- Modify: `src/modules/agent/messages.ts` — accept optional chat request `model`.
|
|
- Modify: `src/modules/agent/server/run.ts` — pass the effective model to the LLM client.
|
|
- Modify: `src/app/api/agent/chat/route.ts` — validate and resolve requested/saved model choices.
|
|
- Modify: `src/modules/_core/schema.ts` — add nullable `assistantModel` user column.
|
|
- Create: `drizzle/0025_assistant_model.sql` — add `assistant_model` column.
|
|
- Modify: `drizzle/meta/_journal.json` — add the migration entry for `0025_assistant_model`.
|
|
- Modify: `src/lib/assistant-preference.ts` — load the saved assistant model.
|
|
- Modify: `src/app/settings/assistant-actions.ts` — add server action to save the selected assistant model.
|
|
- Modify: `src/app/layout.tsx` — pass initial saved model into the assistant bubble.
|
|
- Modify: `src/modules/agent/components/assistant-bubble.tsx` — pass initial saved model into the panel.
|
|
- Modify: `src/modules/agent/components/assistant-panel.tsx` — render the selector, load models, save changes, and include selected model on chat requests.
|
|
- Create: `tests/unit/llm-models.test.ts` — pure model discovery and validation tests.
|
|
- Modify: `tests/unit/agent-chat.test.ts` — LLM override and chat runner coverage.
|
|
- Modify: `tests/unit/agent-messages.test.ts` — request schema coverage for `model`.
|
|
- Modify: `tests/e2e/assistant.spec.ts` — selector render smoke.
|
|
|
|
---
|
|
|
|
### Task 1: LLM Model Discovery Helper
|
|
|
|
**Files:**
|
|
|
|
- Create: `tests/unit/llm-models.test.ts`
|
|
- Create: `src/lib/llm/models.ts`
|
|
|
|
- [ ] **Step 1: Write the failing model helper tests**
|
|
|
|
Create `tests/unit/llm-models.test.ts`:
|
|
|
|
```ts
|
|
import assert from "node:assert/strict";
|
|
import { describe, it } from "node:test";
|
|
import {
|
|
isValidLlmModelId,
|
|
listLlmModels,
|
|
normalizeLlmModelsPayload,
|
|
resolveAssistantModel,
|
|
} from "../../src/lib/llm/models";
|
|
import type { LlmConfig } from "../../src/lib/llm/config";
|
|
|
|
const openAiConfig: LlmConfig = {
|
|
provider: "openai",
|
|
baseUrl: "https://llm.example.test/v1",
|
|
apiKey: "secret",
|
|
model: "llama3.2",
|
|
};
|
|
|
|
describe("normalizeLlmModelsPayload", () => {
|
|
it("normalizes OpenAI-compatible data arrays", () => {
|
|
const models = normalizeLlmModelsPayload({
|
|
data: [{ id: "qwen2.5-coder" }, { id: "llama3.2" }, { id: "qwen2.5-coder" }],
|
|
});
|
|
|
|
assert.deepEqual(models, [
|
|
{ id: "llama3.2", label: "llama3.2" },
|
|
{ id: "qwen2.5-coder", label: "qwen2.5-coder" },
|
|
]);
|
|
});
|
|
|
|
it("ignores invalid or empty model rows", () => {
|
|
const models = normalizeLlmModelsPayload({
|
|
data: [{ id: "" }, { id: " " }, { id: "bad model" }, { object: "model" }],
|
|
});
|
|
|
|
assert.deepEqual(models, []);
|
|
});
|
|
});
|
|
|
|
describe("isValidLlmModelId", () => {
|
|
it("accepts common provider model IDs", () => {
|
|
assert.equal(isValidLlmModelId("llama3.2"), true);
|
|
assert.equal(isValidLlmModelId("qwen2.5-coder:latest"), true);
|
|
assert.equal(isValidLlmModelId("hf.co/ginnoir/model-v1"), true);
|
|
});
|
|
|
|
it("rejects empty, whitespace, and overlong model IDs", () => {
|
|
assert.equal(isValidLlmModelId(""), false);
|
|
assert.equal(isValidLlmModelId("bad model"), false);
|
|
assert.equal(isValidLlmModelId("x".repeat(129)), false);
|
|
});
|
|
});
|
|
|
|
describe("listLlmModels", () => {
|
|
it("fetches provider models with API key auth and includes the fallback model", async () => {
|
|
const requests: Request[] = [];
|
|
const result = await listLlmModels({
|
|
config: openAiConfig,
|
|
fetchImpl: async (input, init) => {
|
|
requests.push(new Request(input, init));
|
|
return Response.json({ data: [{ id: "qwen2.5-coder" }] });
|
|
},
|
|
});
|
|
|
|
assert.equal(requests[0]?.url, "https://llm.example.test/v1/models");
|
|
assert.equal(requests[0]?.headers.get("authorization"), "Bearer secret");
|
|
assert.deepEqual(result.models, [
|
|
{ id: "llama3.2", label: "llama3.2" },
|
|
{ id: "qwen2.5-coder", label: "qwen2.5-coder" },
|
|
]);
|
|
assert.equal(result.fallbackModel, "llama3.2");
|
|
assert.equal(result.degraded, false);
|
|
});
|
|
|
|
it("falls back to LLM_MODEL when provider discovery fails", async () => {
|
|
const result = await listLlmModels({
|
|
config: openAiConfig,
|
|
fetchImpl: async () => new Response("nope", { status: 500 }),
|
|
});
|
|
|
|
assert.deepEqual(result.models, [{ id: "llama3.2", label: "llama3.2" }]);
|
|
assert.equal(result.fallbackModel, "llama3.2");
|
|
assert.equal(result.degraded, true);
|
|
});
|
|
|
|
it("uses fallback only for mock provider config", async () => {
|
|
const result = await listLlmModels({
|
|
config: { provider: "mock", baseUrl: null, apiKey: null, model: "llama3.2" },
|
|
fetchImpl: async () => {
|
|
throw new Error("fetch should not run for mock config");
|
|
},
|
|
});
|
|
|
|
assert.deepEqual(result.models, [{ id: "llama3.2", label: "llama3.2" }]);
|
|
assert.equal(result.degraded, false);
|
|
});
|
|
});
|
|
|
|
describe("resolveAssistantModel", () => {
|
|
it("uses a valid requested model before saved and fallback values", () => {
|
|
const resolved = resolveAssistantModel({
|
|
requestedModel: "qwen2.5-coder",
|
|
savedModel: "llama3.2",
|
|
fallbackModel: "llama3.2",
|
|
models: [
|
|
{ id: "llama3.2", label: "llama3.2" },
|
|
{ id: "qwen2.5-coder", label: "qwen2.5-coder" },
|
|
],
|
|
});
|
|
|
|
assert.deepEqual(resolved, { ok: true, model: "qwen2.5-coder" });
|
|
});
|
|
|
|
it("rejects invalid requested models", () => {
|
|
const resolved = resolveAssistantModel({
|
|
requestedModel: "missing",
|
|
savedModel: null,
|
|
fallbackModel: "llama3.2",
|
|
models: [{ id: "llama3.2", label: "llama3.2" }],
|
|
});
|
|
|
|
assert.deepEqual(resolved, {
|
|
ok: false,
|
|
model: "llama3.2",
|
|
error: "Invalid assistant model",
|
|
});
|
|
});
|
|
|
|
it("silently falls back when a saved model is gone", () => {
|
|
const resolved = resolveAssistantModel({
|
|
requestedModel: null,
|
|
savedModel: "old-model",
|
|
fallbackModel: "llama3.2",
|
|
models: [{ id: "llama3.2", label: "llama3.2" }],
|
|
});
|
|
|
|
assert.deepEqual(resolved, { ok: true, model: "llama3.2" });
|
|
});
|
|
});
|
|
```
|
|
|
|
- [ ] **Step 2: Run the tests to verify they fail**
|
|
|
|
Run:
|
|
|
|
```powershell
|
|
pnpm exec tsx --test tests/unit/llm-models.test.ts
|
|
```
|
|
|
|
Expected: fail with a module resolution error for `src/lib/llm/models.ts`.
|
|
|
|
- [ ] **Step 3: Implement the model helper**
|
|
|
|
Create `src/lib/llm/models.ts`:
|
|
|
|
```ts
|
|
import { getLlmConfig, type LlmConfig } from "./config";
|
|
|
|
export type LlmModelOption = {
|
|
id: string;
|
|
label: string;
|
|
};
|
|
|
|
export type LlmModelsResult = {
|
|
models: LlmModelOption[];
|
|
fallbackModel: string;
|
|
degraded: boolean;
|
|
};
|
|
|
|
export type AssistantModelResolution =
|
|
| { ok: true; model: string }
|
|
| { ok: false; model: string; error: string };
|
|
|
|
const MODEL_ID_PATTERN = /^[A-Za-z0-9._:/-]+$/;
|
|
const MAX_MODEL_ID_LENGTH = 128;
|
|
|
|
export function isValidLlmModelId(value: string): boolean {
|
|
const trimmed = value.trim();
|
|
return (
|
|
trimmed.length > 0 &&
|
|
trimmed.length <= MAX_MODEL_ID_LENGTH &&
|
|
trimmed === value &&
|
|
MODEL_ID_PATTERN.test(trimmed)
|
|
);
|
|
}
|
|
|
|
export function normalizeLlmModelsPayload(payload: unknown): LlmModelOption[] {
|
|
const data =
|
|
typeof payload === "object" && payload !== null && "data" in payload
|
|
? (payload as { data?: unknown }).data
|
|
: null;
|
|
|
|
if (!Array.isArray(data)) return [];
|
|
|
|
const ids = new Set<string>();
|
|
for (const row of data) {
|
|
if (typeof row !== "object" || row === null || !("id" in row)) continue;
|
|
const id = (row as { id?: unknown }).id;
|
|
if (typeof id !== "string") continue;
|
|
const trimmed = id.trim();
|
|
if (!isValidLlmModelId(trimmed)) continue;
|
|
ids.add(trimmed);
|
|
}
|
|
|
|
return [...ids].sort((a, b) => a.localeCompare(b)).map((id) => ({ id, label: id }));
|
|
}
|
|
|
|
export async function listLlmModels(options?: {
|
|
config?: LlmConfig;
|
|
fetchImpl?: typeof fetch;
|
|
}): Promise<LlmModelsResult> {
|
|
const config = options?.config ?? getLlmConfig();
|
|
const fetchImpl = options?.fetchImpl ?? fetch;
|
|
const fallbackModel = config.model;
|
|
const fallbackOption = { id: fallbackModel, label: fallbackModel };
|
|
|
|
if (config.provider === "mock" || !config.baseUrl) {
|
|
return { models: [fallbackOption], fallbackModel, degraded: false };
|
|
}
|
|
|
|
try {
|
|
const headers: Record<string, string> = {};
|
|
if (config.apiKey) headers.Authorization = `Bearer ${config.apiKey}`;
|
|
|
|
const response = await fetchImpl(`${config.baseUrl.replace(/\/$/, "")}/models`, {
|
|
method: "GET",
|
|
headers,
|
|
});
|
|
|
|
if (!response.ok) {
|
|
return { models: [fallbackOption], fallbackModel, degraded: true };
|
|
}
|
|
|
|
const models = normalizeLlmModelsPayload(await response.json());
|
|
const merged = new Map<string, LlmModelOption>();
|
|
merged.set(fallbackModel, fallbackOption);
|
|
for (const model of models) merged.set(model.id, model);
|
|
|
|
return {
|
|
models: [...merged.values()].sort((a, b) => a.id.localeCompare(b.id)),
|
|
fallbackModel,
|
|
degraded: models.length === 0,
|
|
};
|
|
} catch {
|
|
return { models: [fallbackOption], fallbackModel, degraded: true };
|
|
}
|
|
}
|
|
|
|
export function resolveAssistantModel(options: {
|
|
requestedModel: string | null | undefined;
|
|
savedModel: string | null | undefined;
|
|
fallbackModel: string;
|
|
models: LlmModelOption[];
|
|
}): AssistantModelResolution {
|
|
const available = new Set(options.models.map((model) => model.id));
|
|
const fallback = available.has(options.fallbackModel)
|
|
? options.fallbackModel
|
|
: (options.models[0]?.id ?? options.fallbackModel);
|
|
|
|
if (options.requestedModel) {
|
|
if (!available.has(options.requestedModel)) {
|
|
return { ok: false, model: fallback, error: "Invalid assistant model" };
|
|
}
|
|
return { ok: true, model: options.requestedModel };
|
|
}
|
|
|
|
if (options.savedModel && available.has(options.savedModel)) {
|
|
return { ok: true, model: options.savedModel };
|
|
}
|
|
|
|
return { ok: true, model: fallback };
|
|
}
|
|
```
|
|
|
|
- [ ] **Step 4: Run the model helper tests**
|
|
|
|
Run:
|
|
|
|
```powershell
|
|
pnpm exec tsx --test tests/unit/llm-models.test.ts
|
|
```
|
|
|
|
Expected: pass.
|
|
|
|
- [ ] **Step 5: Commit the helper**
|
|
|
|
Run:
|
|
|
|
```powershell
|
|
git add src/lib/llm/models.ts tests/unit/llm-models.test.ts
|
|
git commit -m "feat(agent): add llm model discovery helper"
|
|
```
|
|
|
|
---
|
|
|
|
### Task 2: Persist Assistant Model Preference
|
|
|
|
**Files:**
|
|
|
|
- Modify: `src/modules/_core/schema.ts`
|
|
- Create: `drizzle/0025_assistant_model.sql`
|
|
- Modify: `drizzle/meta/_journal.json`
|
|
- Modify: `src/lib/assistant-preference.ts`
|
|
- Modify: `src/app/settings/assistant-actions.ts`
|
|
|
|
- [ ] **Step 1: Update the Drizzle user schema**
|
|
|
|
In `src/modules/_core/schema.ts`, add the nullable text column next to the other assistant fields:
|
|
|
|
```ts
|
|
assistantEnabled: boolean("assistant_enabled").notNull().default(false),
|
|
assistantName: text("assistant_name").notNull().default("Assistant"),
|
|
assistantSystemPrompt: text("assistant_system_prompt"),
|
|
assistantModel: text("assistant_model"),
|
|
defaultEventReminderOffsets: jsonb("default_event_reminder_offsets")
|
|
```
|
|
|
|
- [ ] **Step 2: Add the migration SQL**
|
|
|
|
Create `drizzle/0025_assistant_model.sql`:
|
|
|
|
```sql
|
|
ALTER TABLE "users" ADD COLUMN "assistant_model" text;
|
|
```
|
|
|
|
Add this entry to the end of the `entries` array in `drizzle/meta/_journal.json`:
|
|
|
|
```json
|
|
{
|
|
"idx": 25,
|
|
"version": "7",
|
|
"when": 1783560000000,
|
|
"tag": "0025_assistant_model",
|
|
"breakpoints": true
|
|
}
|
|
```
|
|
|
|
- [ ] **Step 3: Load the saved model preference**
|
|
|
|
Update `src/lib/assistant-preference.ts` so the type, select, and return object include `model`:
|
|
|
|
```ts
|
|
export type AssistantPreferences = {
|
|
enabled: boolean;
|
|
name: string;
|
|
systemPrompt: string | null;
|
|
model: string | null;
|
|
};
|
|
```
|
|
|
|
```ts
|
|
const [row] = await db
|
|
.select({
|
|
assistantEnabled: users.assistantEnabled,
|
|
assistantName: users.assistantName,
|
|
assistantSystemPrompt: users.assistantSystemPrompt,
|
|
assistantModel: users.assistantModel,
|
|
})
|
|
.from(users)
|
|
.where(eq(users.id, userId))
|
|
.limit(1);
|
|
```
|
|
|
|
```ts
|
|
return {
|
|
enabled: row?.assistantEnabled ?? false,
|
|
name: row?.assistantName?.trim() || DEFAULT_ASSISTANT_NAME,
|
|
systemPrompt: row?.assistantSystemPrompt ?? null,
|
|
model: row?.assistantModel?.trim() || null,
|
|
};
|
|
```
|
|
|
|
- [ ] **Step 4: Add a server action for saving the model**
|
|
|
|
In `src/app/settings/assistant-actions.ts`, import the helper:
|
|
|
|
```ts
|
|
import { isValidLlmModelId, listLlmModels } from "@/lib/llm/models";
|
|
```
|
|
|
|
Add the server action:
|
|
|
|
```ts
|
|
export async function setAssistantModel(model: string | null): Promise<void> {
|
|
const { user } = await getCurrentSession();
|
|
const normalized = model?.trim() || null;
|
|
|
|
if (normalized !== null && !isValidLlmModelId(normalized)) {
|
|
throw new Error("Invalid assistant model");
|
|
}
|
|
|
|
const available = await listLlmModels();
|
|
const requested = normalized === available.fallbackModel ? null : normalized;
|
|
|
|
if (requested !== null && !available.models.some((option) => option.id === requested)) {
|
|
throw new Error("Invalid assistant model");
|
|
}
|
|
|
|
await db.update(users).set({ assistantModel: requested }).where(eq(users.id, user.id));
|
|
revalidateAssistantSurfaces();
|
|
}
|
|
```
|
|
|
|
- [ ] **Step 5: Run typecheck**
|
|
|
|
Run:
|
|
|
|
```powershell
|
|
pnpm typecheck
|
|
```
|
|
|
|
Expected: pass with `users.assistantModel` recognized from `src/modules/_core/schema.ts`.
|
|
|
|
- [ ] **Step 6: Commit persistence**
|
|
|
|
Run:
|
|
|
|
```powershell
|
|
git add src/modules/_core/schema.ts drizzle/0025_assistant_model.sql drizzle/meta/_journal.json src/lib/assistant-preference.ts src/app/settings/assistant-actions.ts
|
|
git commit -m "feat(agent): persist assistant model preference"
|
|
```
|
|
|
|
---
|
|
|
|
### Task 3: Use the Effective Model in Chat
|
|
|
|
**Files:**
|
|
|
|
- Modify: `tests/unit/agent-messages.test.ts`
|
|
- Modify: `tests/unit/agent-chat.test.ts`
|
|
- Modify: `src/modules/agent/messages.ts`
|
|
- Modify: `src/lib/llm/index.ts`
|
|
- Modify: `src/modules/agent/server/run.ts`
|
|
- Modify: `src/app/api/agent/chat/route.ts`
|
|
|
|
- [ ] **Step 1: Add request schema tests**
|
|
|
|
Append to `tests/unit/agent-messages.test.ts`:
|
|
|
|
```ts
|
|
it("accepts an optional model ID", () => {
|
|
const parsed = clientChatInputSchema.safeParse({
|
|
model: "qwen2.5-coder",
|
|
messages: [{ role: "user", content: "hello" }],
|
|
});
|
|
|
|
assert.equal(parsed.success, true);
|
|
});
|
|
|
|
it("rejects invalid model IDs", () => {
|
|
const parsed = clientChatInputSchema.safeParse({
|
|
model: "bad model",
|
|
messages: [{ role: "user", content: "hello" }],
|
|
});
|
|
|
|
assert.equal(parsed.success, false);
|
|
});
|
|
```
|
|
|
|
- [ ] **Step 2: Add an LLM client override test**
|
|
|
|
Append to `tests/unit/agent-chat.test.ts`:
|
|
|
|
```ts
|
|
it("passes a model override to the OpenAI-compatible client", async () => {
|
|
const originalBaseUrl = process.env.LLM_BASE_URL;
|
|
const originalModel = process.env.LLM_MODEL;
|
|
const originalProvider = process.env.LLM_PROVIDER;
|
|
const originalFetch = globalThis.fetch;
|
|
let requestBody: unknown = null;
|
|
|
|
process.env.LLM_BASE_URL = "https://llm.example.test/v1";
|
|
process.env.LLM_MODEL = "llama3.2";
|
|
delete process.env.LLM_PROVIDER;
|
|
|
|
globalThis.fetch = (async (_input: RequestInfo | URL, init?: RequestInit) => {
|
|
requestBody = JSON.parse(String(init?.body));
|
|
return Response.json({
|
|
choices: [{ message: { role: "assistant", content: "done" }, finish_reason: "stop" }],
|
|
});
|
|
}) as typeof fetch;
|
|
|
|
const { createLlmClient } = await import("../../src/lib/llm/index");
|
|
const client = createLlmClient({ model: "qwen2.5-coder" });
|
|
await client.chatCompletion({ messages: [{ role: "user", content: "hello" }] });
|
|
|
|
assert.equal((requestBody as { model?: string }).model, "qwen2.5-coder");
|
|
|
|
globalThis.fetch = originalFetch;
|
|
if (originalBaseUrl === undefined) delete process.env.LLM_BASE_URL;
|
|
else process.env.LLM_BASE_URL = originalBaseUrl;
|
|
if (originalModel === undefined) delete process.env.LLM_MODEL;
|
|
else process.env.LLM_MODEL = originalModel;
|
|
if (originalProvider === undefined) delete process.env.LLM_PROVIDER;
|
|
else process.env.LLM_PROVIDER = originalProvider;
|
|
});
|
|
```
|
|
|
|
- [ ] **Step 3: Run the tests to verify they fail**
|
|
|
|
Run:
|
|
|
|
```powershell
|
|
pnpm exec tsx --test tests/unit/agent-messages.test.ts tests/unit/agent-chat.test.ts
|
|
```
|
|
|
|
Expected: `agent-messages` fails because `model` is not accepted, and `agent-chat` fails because `createLlmClient` does not accept the override object yet.
|
|
|
|
- [ ] **Step 4: Extend the chat input schema**
|
|
|
|
In `src/modules/agent/messages.ts`, import the model validator:
|
|
|
|
```ts
|
|
import { isValidLlmModelId } from "@/lib/llm/models";
|
|
```
|
|
|
|
Add a reusable schema:
|
|
|
|
```ts
|
|
export const clientChatModelSchema = z
|
|
.string()
|
|
.trim()
|
|
.refine((value) => isValidLlmModelId(value), "Invalid assistant model");
|
|
```
|
|
|
|
Update `clientChatInputSchema`:
|
|
|
|
```ts
|
|
export const clientChatInputSchema = z.object({
|
|
stream: z.boolean().optional(),
|
|
model: clientChatModelSchema.optional(),
|
|
messages: z.array(clientChatMessageSchema).min(1).max(40),
|
|
});
|
|
```
|
|
|
|
- [ ] **Step 5: Add LLM client model override support**
|
|
|
|
Replace `createLlmClient` in `src/lib/llm/index.ts` with:
|
|
|
|
```ts
|
|
export function createLlmClient(options?: { model?: string; override?: LlmClient }): LlmClient {
|
|
if (options?.override) return options.override;
|
|
|
|
const config = getLlmConfig();
|
|
if (config.provider === "mock" || !config.baseUrl) {
|
|
return createMockLlmClient();
|
|
}
|
|
|
|
return createOpenAiCompatibleClient({
|
|
baseUrl: config.baseUrl,
|
|
apiKey: config.apiKey,
|
|
model: options?.model ?? config.model,
|
|
});
|
|
}
|
|
```
|
|
|
|
- [ ] **Step 6: Pass model through the agent runner**
|
|
|
|
In `src/modules/agent/server/run.ts`, extend options and client creation:
|
|
|
|
```ts
|
|
export async function runAgentChat(options: {
|
|
messages: ClientChatMessage[];
|
|
request: Request;
|
|
systemPrompt?: string;
|
|
model?: string;
|
|
llm?: LlmClient;
|
|
executeTool?: ToolExecutor;
|
|
onProgress?: AgentProgressHandler;
|
|
}): Promise<AgentChatResult> {
|
|
const llm = options.llm ?? createLlmClient({ model: options.model });
|
|
```
|
|
|
|
- [ ] **Step 7: Resolve and validate model in the chat route**
|
|
|
|
In `src/app/api/agent/chat/route.ts`, import:
|
|
|
|
```ts
|
|
import { listLlmModels, resolveAssistantModel } from "@/lib/llm/models";
|
|
```
|
|
|
|
Before the stream branch, add:
|
|
|
|
```ts
|
|
const modelList = await listLlmModels();
|
|
const modelResolution = resolveAssistantModel({
|
|
requestedModel: parsed.data.model,
|
|
savedModel: assistant.model,
|
|
fallbackModel: modelList.fallbackModel,
|
|
models: modelList.models,
|
|
});
|
|
|
|
if (!modelResolution.ok) {
|
|
return apiError(modelResolution.error, 400);
|
|
}
|
|
```
|
|
|
|
Pass `model: modelResolution.model` in both `runAgentChat` calls:
|
|
|
|
```ts
|
|
const result = await runAgentChat({
|
|
messages: parsed.data.messages,
|
|
request,
|
|
systemPrompt,
|
|
model: modelResolution.model,
|
|
onProgress: send,
|
|
});
|
|
```
|
|
|
|
```ts
|
|
const result = await runAgentChat({
|
|
messages: parsed.data.messages,
|
|
request,
|
|
systemPrompt,
|
|
model: modelResolution.model,
|
|
});
|
|
```
|
|
|
|
- [ ] **Step 8: Run targeted tests**
|
|
|
|
Run:
|
|
|
|
```powershell
|
|
pnpm exec tsx --test tests/unit/agent-messages.test.ts tests/unit/agent-chat.test.ts tests/unit/llm-models.test.ts
|
|
```
|
|
|
|
Expected: pass.
|
|
|
|
- [ ] **Step 9: Commit chat model wiring**
|
|
|
|
Run:
|
|
|
|
```powershell
|
|
git add src/modules/agent/messages.ts src/lib/llm/index.ts src/modules/agent/server/run.ts src/app/api/agent/chat/route.ts tests/unit/agent-messages.test.ts tests/unit/agent-chat.test.ts
|
|
git commit -m "feat(agent): route chat through selected model"
|
|
```
|
|
|
|
---
|
|
|
|
### Task 4: Add Model Discovery API
|
|
|
|
**Files:**
|
|
|
|
- Create: `src/app/api/agent/models/route.ts`
|
|
|
|
- [ ] **Step 1: Implement the route**
|
|
|
|
Create `src/app/api/agent/models/route.ts`:
|
|
|
|
```ts
|
|
import { apiError, apiJson } from "@/lib/api-handler";
|
|
import { resolveApiAuth } from "@/lib/api-auth";
|
|
import { getAssistantPreferences } from "@/lib/assistant-preference";
|
|
import { listLlmModels, resolveAssistantModel } from "@/lib/llm/models";
|
|
|
|
export async function GET(request: Request) {
|
|
const auth = await resolveApiAuth(request);
|
|
if (!auth?.userId) {
|
|
return apiError("Unauthorized", 401);
|
|
}
|
|
|
|
const assistant = await getAssistantPreferences(auth.userId);
|
|
if (!assistant.enabled) {
|
|
return apiError("Assistant not enabled", 403);
|
|
}
|
|
|
|
const modelList = await listLlmModels();
|
|
const resolved = resolveAssistantModel({
|
|
requestedModel: null,
|
|
savedModel: assistant.model,
|
|
fallbackModel: modelList.fallbackModel,
|
|
models: modelList.models,
|
|
});
|
|
|
|
return apiJson({
|
|
models: modelList.models,
|
|
selectedModel: resolved.model,
|
|
fallbackModel: modelList.fallbackModel,
|
|
degraded: modelList.degraded,
|
|
});
|
|
}
|
|
```
|
|
|
|
- [ ] **Step 2: Run typecheck**
|
|
|
|
Run:
|
|
|
|
```powershell
|
|
pnpm typecheck
|
|
```
|
|
|
|
Expected: pass.
|
|
|
|
- [ ] **Step 3: Commit the route**
|
|
|
|
Run:
|
|
|
|
```powershell
|
|
git add src/app/api/agent/models/route.ts
|
|
git commit -m "feat(agent): expose available assistant models"
|
|
```
|
|
|
|
---
|
|
|
|
### Task 5: Add the Chat Panel Selector
|
|
|
|
**Files:**
|
|
|
|
- Modify: `src/app/layout.tsx`
|
|
- Modify: `src/modules/agent/components/assistant-bubble.tsx`
|
|
- Modify: `src/modules/agent/components/assistant-panel.tsx`
|
|
- Modify: `tests/e2e/assistant.spec.ts`
|
|
|
|
- [ ] **Step 1: Pass saved model from layout to the panel**
|
|
|
|
In `src/app/layout.tsx`, add local state:
|
|
|
|
```ts
|
|
let assistantModel: string | null = null;
|
|
```
|
|
|
|
Select it:
|
|
|
|
```ts
|
|
assistantModel: users.assistantModel,
|
|
```
|
|
|
|
Assign it when the row exists:
|
|
|
|
```ts
|
|
assistantModel = row.assistantModel?.trim() || null;
|
|
```
|
|
|
|
Pass it into `AssistantBubble`:
|
|
|
|
```tsx
|
|
<AssistantBubble
|
|
configured={isLlmConfigured()}
|
|
userId={session.user.id}
|
|
assistantName={assistantName}
|
|
assistantModel={assistantModel}
|
|
/>
|
|
```
|
|
|
|
- [ ] **Step 2: Thread the prop through `AssistantBubble`**
|
|
|
|
In `src/modules/agent/components/assistant-bubble.tsx`, update props:
|
|
|
|
```ts
|
|
type Props = {
|
|
configured: boolean;
|
|
userId: string;
|
|
assistantName: string;
|
|
assistantModel: string | null;
|
|
};
|
|
```
|
|
|
|
Update the component signature:
|
|
|
|
```ts
|
|
export function AssistantBubble({ configured, userId, assistantName, assistantModel }: Props) {
|
|
```
|
|
|
|
Pass it to `AssistantPanel`:
|
|
|
|
```tsx
|
|
<AssistantPanel
|
|
key={userId}
|
|
configured={configured}
|
|
userId={userId}
|
|
assistantName={assistantName}
|
|
assistantModel={assistantModel}
|
|
/>
|
|
```
|
|
|
|
- [ ] **Step 3: Add selector state and fetch helpers to `AssistantPanel`**
|
|
|
|
In `src/modules/agent/components/assistant-panel.tsx`, import the Select pieces and server action:
|
|
|
|
```ts
|
|
import {
|
|
Select,
|
|
SelectContent,
|
|
SelectGroup,
|
|
SelectItem,
|
|
SelectTrigger,
|
|
SelectValue,
|
|
} from "@/components/ui/select";
|
|
import { setAssistantModel } from "@/app/settings/assistant-actions";
|
|
```
|
|
|
|
Add types:
|
|
|
|
```ts
|
|
type LlmModelOption = {
|
|
id: string;
|
|
label: string;
|
|
};
|
|
|
|
type ModelsResponse = {
|
|
models: LlmModelOption[];
|
|
selectedModel: string;
|
|
fallbackModel: string;
|
|
degraded: boolean;
|
|
};
|
|
```
|
|
|
|
Extend props:
|
|
|
|
```ts
|
|
type Props = {
|
|
configured: boolean;
|
|
userId: string;
|
|
assistantName: string;
|
|
assistantModel: string | null;
|
|
};
|
|
```
|
|
|
|
Update the function signature:
|
|
|
|
```ts
|
|
export function AssistantPanel({ configured, userId, assistantName, assistantModel }: Props) {
|
|
```
|
|
|
|
Add state after the pending/error state:
|
|
|
|
```ts
|
|
const [modelOptions, setModelOptions] = useState<LlmModelOption[]>([]);
|
|
const [selectedModel, setSelectedModel] = useState(assistantModel ?? "");
|
|
const [fallbackModel, setFallbackModel] = useState("");
|
|
const [modelsDegraded, setModelsDegraded] = useState(false);
|
|
const [modelsLoading, setModelsLoading] = useState(true);
|
|
```
|
|
|
|
Add the model loading effect:
|
|
|
|
```ts
|
|
useEffect(() => {
|
|
let cancelled = false;
|
|
|
|
async function loadModels() {
|
|
setModelsLoading(true);
|
|
try {
|
|
const response = await fetch("/api/agent/models");
|
|
if (!response.ok) throw new Error("Model discovery unavailable");
|
|
const payload = (await response.json()) as ModelsResponse;
|
|
if (cancelled) return;
|
|
setModelOptions(payload.models);
|
|
setSelectedModel(payload.selectedModel);
|
|
setFallbackModel(payload.fallbackModel);
|
|
setModelsDegraded(payload.degraded);
|
|
} catch {
|
|
if (cancelled) return;
|
|
setModelsDegraded(true);
|
|
} finally {
|
|
if (!cancelled) setModelsLoading(false);
|
|
}
|
|
}
|
|
|
|
void loadModels();
|
|
return () => {
|
|
cancelled = true;
|
|
};
|
|
}, []);
|
|
```
|
|
|
|
Add the save handler:
|
|
|
|
```ts
|
|
function changeModel(nextModel: string) {
|
|
setSelectedModel(nextModel);
|
|
setError(null);
|
|
|
|
startTransition(async () => {
|
|
try {
|
|
await setAssistantModel(nextModel === fallbackModel ? null : nextModel);
|
|
} catch (err) {
|
|
setError(err instanceof Error ? err.message : "Could not save assistant model");
|
|
}
|
|
});
|
|
}
|
|
```
|
|
|
|
Update the React import:
|
|
|
|
```ts
|
|
import { useEffect, useRef, useState, useTransition } from "react";
|
|
```
|
|
|
|
Then add:
|
|
|
|
```ts
|
|
const [savingModel, startTransition] = useTransition();
|
|
```
|
|
|
|
- [ ] **Step 4: Render the selector above the message list**
|
|
|
|
Replace the current top row in `AssistantPanel` with a two-column responsive row:
|
|
|
|
```tsx
|
|
<div className="flex items-start justify-between gap-3">
|
|
<div className="min-w-0 flex-1">
|
|
<p className="muted text-[12px] leading-relaxed">
|
|
{configured
|
|
? "Type, talk, or send a photo — I can update lists, calendar, notes, and more."
|
|
: "Mock provider active — set LLM_BASE_URL for your homelab model."}
|
|
</p>
|
|
{modelsDegraded ? (
|
|
<p className="muted mt-1 text-[11px]">Model discovery unavailable; using fallback.</p>
|
|
) : null}
|
|
</div>
|
|
<div className="flex shrink-0 items-center gap-2">
|
|
{modelOptions.length > 0 ? (
|
|
<Select
|
|
items={modelOptions.map((model) => ({ value: model.id, label: model.label }))}
|
|
value={selectedModel}
|
|
onValueChange={changeModel}
|
|
disabled={modelsLoading || savingModel || isPending}
|
|
>
|
|
<SelectTrigger size="sm" className="max-w-36" aria-label="Assistant model">
|
|
<SelectValue>{selectedModel || "Model"}</SelectValue>
|
|
</SelectTrigger>
|
|
<SelectContent align="end">
|
|
<SelectGroup>
|
|
{modelOptions.map((model) => (
|
|
<SelectItem key={model.id} value={model.id}>
|
|
{model.label}
|
|
</SelectItem>
|
|
))}
|
|
</SelectGroup>
|
|
</SelectContent>
|
|
</Select>
|
|
) : null}
|
|
{messages.length > 0 ? (
|
|
<button
|
|
type="button"
|
|
onClick={clearChat}
|
|
disabled={isPending}
|
|
className="shrink-0 text-[11px] text-muted-foreground transition-colors hover:text-foreground disabled:opacity-50"
|
|
>
|
|
Clear
|
|
</button>
|
|
) : null}
|
|
</div>
|
|
</div>
|
|
```
|
|
|
|
- [ ] **Step 5: Include the selected model in chat requests**
|
|
|
|
In `sendMessage`, update the JSON body:
|
|
|
|
```ts
|
|
body: JSON.stringify({
|
|
messages: nextMessages.map(toClientChatMessage),
|
|
model: selectedModel || undefined,
|
|
stream: true,
|
|
}),
|
|
```
|
|
|
|
- [ ] **Step 6: Update the E2E smoke**
|
|
|
|
In `tests/e2e/assistant.spec.ts`, after the dialog assertion, add:
|
|
|
|
```ts
|
|
await expect(page.getByRole("combobox", { name: "Assistant model" })).toBeVisible();
|
|
```
|
|
|
|
- [ ] **Step 7: Run typecheck and targeted E2E**
|
|
|
|
Run:
|
|
|
|
```powershell
|
|
pnpm typecheck
|
|
pnpm test:e2e -- tests/e2e/assistant.spec.ts
|
|
```
|
|
|
|
Expected: typecheck passes and both assistant E2E tests pass. Playwright config owns dev server startup; do not leave a manual server running.
|
|
|
|
- [ ] **Step 8: Commit the UI**
|
|
|
|
Run:
|
|
|
|
```powershell
|
|
git add src/app/layout.tsx src/modules/agent/components/assistant-bubble.tsx src/modules/agent/components/assistant-panel.tsx tests/e2e/assistant.spec.ts
|
|
git commit -m "feat(agent): add assistant model selector"
|
|
```
|
|
|
|
---
|
|
|
|
### Task 6: Final Verification and Cleanup
|
|
|
|
**Files:**
|
|
|
|
- No planned edits.
|
|
|
|
- [ ] **Step 1: Run all targeted unit tests**
|
|
|
|
Run:
|
|
|
|
```powershell
|
|
pnpm exec tsx --test tests/unit/llm-models.test.ts tests/unit/agent-messages.test.ts tests/unit/agent-chat.test.ts
|
|
```
|
|
|
|
Expected: all tests pass.
|
|
|
|
- [ ] **Step 2: Run typecheck**
|
|
|
|
Run:
|
|
|
|
```powershell
|
|
pnpm typecheck
|
|
```
|
|
|
|
Expected: pass.
|
|
|
|
- [ ] **Step 3: Run lint**
|
|
|
|
Run:
|
|
|
|
```powershell
|
|
pnpm lint
|
|
```
|
|
|
|
Expected: exit 0.
|
|
|
|
- [ ] **Step 4: Run assistant E2E**
|
|
|
|
Run:
|
|
|
|
```powershell
|
|
pnpm test:e2e -- tests/e2e/assistant.spec.ts
|
|
```
|
|
|
|
Expected: assistant opt-in and chat smoke pass.
|
|
|
|
- [ ] **Step 5: Check git status**
|
|
|
|
Run:
|
|
|
|
```powershell
|
|
git status --short
|
|
```
|
|
|
|
Expected: clean worktree.
|