diff --git a/docs/superpowers/specs/2026-07-08-assistant-model-selector-design.md b/docs/superpowers/specs/2026-07-08-assistant-model-selector-design.md new file mode 100644 index 0000000..cc85146 --- /dev/null +++ b/docs/superpowers/specs/2026-07-08-assistant-model-selector-design.md @@ -0,0 +1,151 @@ +# Assistant model selector design + +Date: 2026-07-08 +Status: approved for planning + +## Context + +The AI assistant chat currently uses one environment-configured model through `LLM_MODEL`. +The chat UI posts messages to `/api/agent/chat`, and the server creates the OpenAI-compatible +client without any request-time model choice. + +ginnoir wants a model selector in the assistant chat. The selector should discover available +models from the configured OpenAI-compatible provider and save the selected model as the user's +default. + +## Goals + +- Show a compact model selector in the assistant chat panel. +- Discover models from the provider's `/models` endpoint server-side. +- Persist the selected model per user so it works across browser sessions and devices. +- Keep `LLM_MODEL` as the fallback when discovery fails, no model is saved, or the saved model + is no longer available. +- Preserve mock-provider behavior in CI and local setups without `LLM_BASE_URL`. + +## Non-goals + +- Model hosting, training, or fine-tuning. +- Multiple LLM providers in the same deployment. +- Per-message experimental settings beyond selecting the model ID. +- Exposing arbitrary browser-supplied model IDs to the provider. + +## User experience + +When the assistant bubble opens, the chat panel loads available model IDs from the server. +The selector appears near the existing assistant status and clear-chat controls. It should be +visible but compact enough not to reduce the message area materially. + +Changing the selector immediately saves the user's default model. The next message uses that +model, and future assistant sessions start with the saved selection when it is still available. + +If model discovery fails, the panel remains usable with the `LLM_MODEL` fallback and shows a +muted status that model discovery is unavailable. If the saved model has disappeared from the +provider, the server and UI fall back to `LLM_MODEL`. + +## Architecture + +### Configuration + +`getLlmConfig()` remains the source for provider, base URL, API key, and fallback model. +No additional allowlist environment variable is required because model IDs come from the +provider's OpenAI-compatible `/models` endpoint. + +### Persistence + +Add nullable `assistant_model` storage to `users`. + +The existing assistant preference loader should return: + +- assistant enabled state +- assistant display name +- assistant system prompt +- saved assistant model ID + +The value is nullable. `null` means "use the environment fallback model." + +### Model discovery API + +Add `GET /api/agent/models`. + +Behavior: + +- Require the same authenticated user/session or API auth shape as the chat endpoint. +- Require assistant access to be enabled for the user. +- If `LLM_BASE_URL` is missing or the provider is mock, return the fallback model as the only + available model. +- Fetch `${LLM_BASE_URL}/models` with `Authorization: Bearer ${LLM_API_KEY}` when configured. +- Accept OpenAI-style payloads with a top-level `data` array. +- Normalize each model to `{ id: string, label: string }`, using the ID as the label. +- Deduplicate, sort consistently, and include the fallback model if the provider omitted it. +- If discovery fails, return the fallback model plus a degraded status instead of failing the + chat UI. + +The response should include enough metadata for the UI: + +```json +{ + "models": [{ "id": "llama3.2", "label": "llama3.2" }], + "selectedModel": "llama3.2", + "fallbackModel": "llama3.2", + "degraded": false +} +``` + +### Saving the default model + +Add a server action for updating the user's assistant model, matching the existing assistant +settings actions. The update path must: + +- Accept a model ID string or `null`. +- Validate length and basic shape before touching the database. +- Validate the requested model against the current discovered model list. +- Save `null` when the selected model matches the fallback so `LLM_MODEL` changes take effect for + users who have not chosen a non-default model. +- Revalidate assistant surfaces after saving. + +### Chat request flow + +Extend `clientChatInputSchema` with optional `model`. + +The chat route should: + +- Parse `model` from the request body. +- Resolve the effective model from request model, saved user default, and fallback model. +- Validate request model and saved user default against discovered models. +- Reject an invalid request model with `400`. +- Silently fall back when the saved user default is no longer available. +- Pass the effective model into `runAgentChat`. + +`runAgentChat` should accept an optional model override. `createLlmClient` should support an +override object or equivalent path that replaces only the model while preserving the configured +provider, base URL, and API key. + +## Error handling + +- Missing auth: `401`. +- Assistant disabled: `403`. +- Invalid posted model: `400`. +- Provider `/models` failure: return fallback model from the model-discovery API with + `degraded: true`; do not block chat startup. +- LLM completion failure after a valid model is selected: keep the existing chat error behavior. + +## Testing + +Unit tests: + +- Model discovery normalizes OpenAI-compatible `/models` responses. +- Discovery falls back to `LLM_MODEL` for mock or failed provider states. +- Chat input schema accepts an optional valid model string and rejects invalid shapes. +- Chat route rejects a model not returned by discovery. +- `runAgentChat` passes the effective model override into the LLM client path. + +E2E smoke: + +- After assistant opt-in, opening the assistant shows the model selector. +- Sending a message still renders the user message and assistant response with the mock provider. + +## Rollout notes + +This is additive. Existing deployments without a provider `/models` endpoint continue to use +`LLM_MODEL`. The database migration is nullable, so existing users keep current behavior until +they choose a model.