famapp/docs/superpowers/specs/2026-07-08-assistant-model-selector-design.md

Assistant model selector design

Date: 2026-07-08 Status: approved for planning

Context

The AI assistant chat currently uses one environment-configured model through LLM_MODEL. The chat UI posts messages to /api/agent/chat, and the server creates the OpenAI-compatible client without any request-time model choice.

ginnoir wants a model selector in the assistant chat. The selector should discover available models from the configured OpenAI-compatible provider and save the selected model as the user's default.

Goals

• Show a compact model selector in the assistant chat panel.
• Discover models from the provider's /models endpoint server-side.
• Persist the selected model per user so it works across browser sessions and devices.
• Keep LLM_MODEL as the fallback when discovery fails, no model is saved, or the saved model is no longer available.
• Preserve mock-provider behavior in CI and local setups without LLM_BASE_URL.

Non-goals

• Model hosting, training, or fine-tuning.
• Multiple LLM providers in the same deployment.
• Per-message experimental settings beyond selecting the model ID.
• Exposing arbitrary browser-supplied model IDs to the provider.

User experience

When the assistant bubble opens, the chat panel loads available model IDs from the server. The selector appears near the existing assistant status and clear-chat controls. It should be visible but compact enough not to reduce the message area materially.

Changing the selector immediately saves the user's default model. The next message uses that model, and future assistant sessions start with the saved selection when it is still available.

If model discovery fails, the panel remains usable with the LLM_MODEL fallback and shows a muted status that model discovery is unavailable. If the saved model has disappeared from the provider, the server and UI fall back to LLM_MODEL.

Architecture

Configuration

getLlmConfig() remains the source for provider, base URL, API key, and fallback model. No additional allowlist environment variable is required because model IDs come from the provider's OpenAI-compatible /models endpoint.

Persistence

Add nullable assistant_model storage to users.

The existing assistant preference loader should return:

• assistant enabled state
• assistant display name
• assistant system prompt
• saved assistant model ID

The value is nullable. null means "use the environment fallback model."

Model discovery API

Add GET /api/agent/models.

Behavior:

• Require the same authenticated user/session or API auth shape as the chat endpoint.
• Require assistant access to be enabled for the user.
• If LLM_BASE_URL is missing or the provider is mock, return the fallback model as the only available model.
• Fetch ${LLM_BASE_URL}/models with Authorization: Bearer ${LLM_API_KEY} when configured.
• Accept OpenAI-style payloads with a top-level data array.
• Normalize each model to { id: string, label: string }, using the ID as the label.
• Deduplicate, sort consistently, and include the fallback model if the provider omitted it.
• If discovery fails, return the fallback model plus a degraded status instead of failing the chat UI.

The response should include enough metadata for the UI:

{
  "models": [{ "id": "llama3.2", "label": "llama3.2" }],
  "selectedModel": "llama3.2",
  "fallbackModel": "llama3.2",
  "degraded": false
}

Saving the default model

Add a server action for updating the user's assistant model, matching the existing assistant settings actions. The update path must:

• Accept a model ID string or null.
• Validate length and basic shape before touching the database.
• Validate the requested model against the current discovered model list.
• Save null when the selected model matches the fallback so LLM_MODEL changes take effect for users who have not chosen a non-default model.
• Revalidate assistant surfaces after saving.

Chat request flow

Extend clientChatInputSchema with optional model.

The chat route should:

• Parse model from the request body.
• Resolve the effective model from request model, saved user default, and fallback model.
• Validate request model and saved user default against discovered models.
• Reject an invalid request model with 400.
• Silently fall back when the saved user default is no longer available.
• Pass the effective model into runAgentChat.

runAgentChat should accept an optional model override. createLlmClient should support an override object or equivalent path that replaces only the model while preserving the configured provider, base URL, and API key.

Error handling

• Missing auth: 401.
• Assistant disabled: 403.
• Invalid posted model: 400.
• Provider /models failure: return fallback model from the model-discovery API with degraded: true; do not block chat startup.
• LLM completion failure after a valid model is selected: keep the existing chat error behavior.

Testing

Unit tests:

• Model discovery normalizes OpenAI-compatible /models responses.
• Discovery falls back to LLM_MODEL for mock or failed provider states.
• Chat input schema accepts an optional valid model string and rejects invalid shapes.
• Chat route rejects a model not returned by discovery.
• runAgentChat passes the effective model override into the LLM client path.

E2E smoke:

• After assistant opt-in, opening the assistant shows the model selector.
• Sending a message still renders the user message and assistant response with the mock provider.

Rollout notes

This is additive. Existing deployments without a provider /models endpoint continue to use LLM_MODEL. The database migration is nullable, so existing users keep current behavior until they choose a model.