Compare commits
18
Commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
ea371b8085 | ||
|
|
c8a7160ef2 | ||
|
|
49732b02f3 | ||
|
|
ec3f96dab1 | ||
|
|
b279f16ba0 | ||
|
|
4dc04b21d6 | ||
|
|
bb679d02be | ||
|
|
716bca0fcb | ||
|
|
1092941c45 | ||
|
|
4c82d551ea | ||
|
|
27db599444 | ||
|
|
3983c30fe1 | ||
|
|
9b7a04431c | ||
|
|
bf7e07ead9 | ||
|
|
876e72671a | ||
|
|
0bf63cf8a8 | ||
|
|
eb8e562565 | ||
|
|
1a58ef993e |
@@ -1,5 +1,6 @@
|
|||||||
node_modules
|
node_modules
|
||||||
.next
|
.next
|
||||||
|
.worktrees
|
||||||
.git
|
.git
|
||||||
deploy
|
deploy
|
||||||
docs
|
docs
|
||||||
|
|||||||
@@ -1,5 +1,31 @@
|
|||||||
# Changelog
|
# Changelog
|
||||||
|
|
||||||
|
## [0.6.2](https://github.com/ginnoir/famapp/compare/v0.6.1...v0.6.2) (2026-07-09)
|
||||||
|
|
||||||
|
### Features
|
||||||
|
|
||||||
|
- **agent:** add assistant model selector ([4c82d55](https://github.com/ginnoir/famapp/commit/4c82d551ea1f0c8091a3ffc31935a8d22855bbc0))
|
||||||
|
- **agent:** add llm model discovery helper ([0bf63cf](https://github.com/ginnoir/famapp/commit/0bf63cf8a83a1adca58fa9e8bf42f16dac60687b))
|
||||||
|
- **agent:** expose available assistant models ([27db599](https://github.com/ginnoir/famapp/commit/27db599444fc0bb80d5a1650e2f5b5ba4f695c8e))
|
||||||
|
- **agent:** let users choose model route ([ec3f96d](https://github.com/ginnoir/famapp/commit/ec3f96dab170fb2333780438bdec3628cfc58116))
|
||||||
|
- **agent:** persist assistant model preference ([bf7e07e](https://github.com/ginnoir/famapp/commit/bf7e07ead91dd8c428f06551afb2113ce838cc82))
|
||||||
|
- **agent:** refresh assistant model catalog ([b279f16](https://github.com/ginnoir/famapp/commit/b279f16ba0e1537d2752d30bf5e565f4ede400af))
|
||||||
|
- **agent:** route chat through selected model ([9b7a044](https://github.com/ginnoir/famapp/commit/9b7a04431cb12609cbcd766e05fbc411599e0b34))
|
||||||
|
|
||||||
|
### Bug Fixes
|
||||||
|
|
||||||
|
- **agent:** hide unrelated models for alias fallback ([4dc04b2](https://github.com/ginnoir/famapp/commit/4dc04b21d63aaef7c773f238602f06edbbf03509))
|
||||||
|
- **agent:** improve model selector legibility ([bb679d0](https://github.com/ginnoir/famapp/commit/bb679d02be28440acbc860146cbcb3809224aa09))
|
||||||
|
- **agent:** move route selector to settings ([49732b0](https://github.com/ginnoir/famapp/commit/49732b02f31cbef57e6cd3b577bee6dbd998e029))
|
||||||
|
- **agent:** reject padded assistant model ids ([3983c30](https://github.com/ginnoir/famapp/commit/3983c30fe1b8a324f7d5826df0b24506fe054e50))
|
||||||
|
- **agent:** use native model selector ([716bca0](https://github.com/ginnoir/famapp/commit/716bca0fcba8f2026c814f668bf873cf61a0ad16))
|
||||||
|
- **agent:** validate assistant model requests ([876e726](https://github.com/ginnoir/famapp/commit/876e72671a0b82b579a9783eb86f452a4a026a52))
|
||||||
|
|
||||||
|
### Documentation
|
||||||
|
|
||||||
|
- plan assistant model selector ([eb8e562](https://github.com/ginnoir/famapp/commit/eb8e5625656a1e3fe97e19b0bb2514660cf0c5f6))
|
||||||
|
- specify assistant model selector ([1a58ef9](https://github.com/ginnoir/famapp/commit/1a58ef993e62ea2855d7d5aafc17446a0e91c33c))
|
||||||
|
|
||||||
## [0.6.1](https://github.com/ginnoir/famapp/compare/v0.6.0...v0.6.1) (2026-07-05)
|
## [0.6.1](https://github.com/ginnoir/famapp/compare/v0.6.0...v0.6.1) (2026-07-05)
|
||||||
|
|
||||||
### Features
|
### Features
|
||||||
|
|||||||
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,151 @@
|
|||||||
|
# Assistant model selector design
|
||||||
|
|
||||||
|
Date: 2026-07-08
|
||||||
|
Status: approved for planning
|
||||||
|
|
||||||
|
## Context
|
||||||
|
|
||||||
|
The AI assistant chat currently uses one environment-configured model through `LLM_MODEL`.
|
||||||
|
The chat UI posts messages to `/api/agent/chat`, and the server creates the OpenAI-compatible
|
||||||
|
client without any request-time model choice.
|
||||||
|
|
||||||
|
ginnoir wants a model selector in the assistant chat. The selector should discover available
|
||||||
|
models from the configured OpenAI-compatible provider and save the selected model as the user's
|
||||||
|
default.
|
||||||
|
|
||||||
|
## Goals
|
||||||
|
|
||||||
|
- Show a compact model selector in the assistant chat panel.
|
||||||
|
- Discover models from the provider's `/models` endpoint server-side.
|
||||||
|
- Persist the selected model per user so it works across browser sessions and devices.
|
||||||
|
- Keep `LLM_MODEL` as the fallback when discovery fails, no model is saved, or the saved model
|
||||||
|
is no longer available.
|
||||||
|
- Preserve mock-provider behavior in CI and local setups without `LLM_BASE_URL`.
|
||||||
|
|
||||||
|
## Non-goals
|
||||||
|
|
||||||
|
- Model hosting, training, or fine-tuning.
|
||||||
|
- Multiple LLM providers in the same deployment.
|
||||||
|
- Per-message experimental settings beyond selecting the model ID.
|
||||||
|
- Exposing arbitrary browser-supplied model IDs to the provider.
|
||||||
|
|
||||||
|
## User experience
|
||||||
|
|
||||||
|
When the assistant bubble opens, the chat panel loads available model IDs from the server.
|
||||||
|
The selector appears near the existing assistant status and clear-chat controls. It should be
|
||||||
|
visible but compact enough not to reduce the message area materially.
|
||||||
|
|
||||||
|
Changing the selector immediately saves the user's default model. The next message uses that
|
||||||
|
model, and future assistant sessions start with the saved selection when it is still available.
|
||||||
|
|
||||||
|
If model discovery fails, the panel remains usable with the `LLM_MODEL` fallback and shows a
|
||||||
|
muted status that model discovery is unavailable. If the saved model has disappeared from the
|
||||||
|
provider, the server and UI fall back to `LLM_MODEL`.
|
||||||
|
|
||||||
|
## Architecture
|
||||||
|
|
||||||
|
### Configuration
|
||||||
|
|
||||||
|
`getLlmConfig()` remains the source for provider, base URL, API key, and fallback model.
|
||||||
|
No additional allowlist environment variable is required because model IDs come from the
|
||||||
|
provider's OpenAI-compatible `/models` endpoint.
|
||||||
|
|
||||||
|
### Persistence
|
||||||
|
|
||||||
|
Add nullable `assistant_model` storage to `users`.
|
||||||
|
|
||||||
|
The existing assistant preference loader should return:
|
||||||
|
|
||||||
|
- assistant enabled state
|
||||||
|
- assistant display name
|
||||||
|
- assistant system prompt
|
||||||
|
- saved assistant model ID
|
||||||
|
|
||||||
|
The value is nullable. `null` means "use the environment fallback model."
|
||||||
|
|
||||||
|
### Model discovery API
|
||||||
|
|
||||||
|
Add `GET /api/agent/models`.
|
||||||
|
|
||||||
|
Behavior:
|
||||||
|
|
||||||
|
- Require the same authenticated user/session or API auth shape as the chat endpoint.
|
||||||
|
- Require assistant access to be enabled for the user.
|
||||||
|
- If `LLM_BASE_URL` is missing or the provider is mock, return the fallback model as the only
|
||||||
|
available model.
|
||||||
|
- Fetch `${LLM_BASE_URL}/models` with `Authorization: Bearer ${LLM_API_KEY}` when configured.
|
||||||
|
- Accept OpenAI-style payloads with a top-level `data` array.
|
||||||
|
- Normalize each model to `{ id: string, label: string }`, using the ID as the label.
|
||||||
|
- Deduplicate, sort consistently, and include the fallback model if the provider omitted it.
|
||||||
|
- If discovery fails, return the fallback model plus a degraded status instead of failing the
|
||||||
|
chat UI.
|
||||||
|
|
||||||
|
The response should include enough metadata for the UI:
|
||||||
|
|
||||||
|
```json
|
||||||
|
{
|
||||||
|
"models": [{ "id": "llama3.2", "label": "llama3.2" }],
|
||||||
|
"selectedModel": "llama3.2",
|
||||||
|
"fallbackModel": "llama3.2",
|
||||||
|
"degraded": false
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
### Saving the default model
|
||||||
|
|
||||||
|
Add a server action for updating the user's assistant model, matching the existing assistant
|
||||||
|
settings actions. The update path must:
|
||||||
|
|
||||||
|
- Accept a model ID string or `null`.
|
||||||
|
- Validate length and basic shape before touching the database.
|
||||||
|
- Validate the requested model against the current discovered model list.
|
||||||
|
- Save `null` when the selected model matches the fallback so `LLM_MODEL` changes take effect for
|
||||||
|
users who have not chosen a non-default model.
|
||||||
|
- Revalidate assistant surfaces after saving.
|
||||||
|
|
||||||
|
### Chat request flow
|
||||||
|
|
||||||
|
Extend `clientChatInputSchema` with optional `model`.
|
||||||
|
|
||||||
|
The chat route should:
|
||||||
|
|
||||||
|
- Parse `model` from the request body.
|
||||||
|
- Resolve the effective model from request model, saved user default, and fallback model.
|
||||||
|
- Validate request model and saved user default against discovered models.
|
||||||
|
- Reject an invalid request model with `400`.
|
||||||
|
- Silently fall back when the saved user default is no longer available.
|
||||||
|
- Pass the effective model into `runAgentChat`.
|
||||||
|
|
||||||
|
`runAgentChat` should accept an optional model override. `createLlmClient` should support an
|
||||||
|
override object or equivalent path that replaces only the model while preserving the configured
|
||||||
|
provider, base URL, and API key.
|
||||||
|
|
||||||
|
## Error handling
|
||||||
|
|
||||||
|
- Missing auth: `401`.
|
||||||
|
- Assistant disabled: `403`.
|
||||||
|
- Invalid posted model: `400`.
|
||||||
|
- Provider `/models` failure: return fallback model from the model-discovery API with
|
||||||
|
`degraded: true`; do not block chat startup.
|
||||||
|
- LLM completion failure after a valid model is selected: keep the existing chat error behavior.
|
||||||
|
|
||||||
|
## Testing
|
||||||
|
|
||||||
|
Unit tests:
|
||||||
|
|
||||||
|
- Model discovery normalizes OpenAI-compatible `/models` responses.
|
||||||
|
- Discovery falls back to `LLM_MODEL` for mock or failed provider states.
|
||||||
|
- Chat input schema accepts an optional valid model string and rejects invalid shapes.
|
||||||
|
- Chat route rejects a model not returned by discovery.
|
||||||
|
- `runAgentChat` passes the effective model override into the LLM client path.
|
||||||
|
|
||||||
|
E2E smoke:
|
||||||
|
|
||||||
|
- After assistant opt-in, opening the assistant shows the model selector.
|
||||||
|
- Sending a message still renders the user message and assistant response with the mock provider.
|
||||||
|
|
||||||
|
## Rollout notes
|
||||||
|
|
||||||
|
This is additive. Existing deployments without a provider `/models` endpoint continue to use
|
||||||
|
`LLM_MODEL`. The database migration is nullable, so existing users keep current behavior until
|
||||||
|
they choose a model.
|
||||||
@@ -0,0 +1 @@
|
|||||||
|
ALTER TABLE "users" ADD COLUMN "assistant_model" text;
|
||||||
@@ -0,0 +1,5 @@
|
|||||||
|
ALTER TABLE "users" ADD COLUMN "assistant_model_route" text;--> statement-breakpoint
|
||||||
|
UPDATE "users"
|
||||||
|
SET "assistant_model_route" = "assistant_model",
|
||||||
|
"assistant_model" = NULL
|
||||||
|
WHERE "assistant_model" IN ('auto', 'uncensored');
|
||||||
@@ -176,6 +176,20 @@
|
|||||||
"when": 1780394000000,
|
"when": 1780394000000,
|
||||||
"tag": "0024_user_assistant_customization",
|
"tag": "0024_user_assistant_customization",
|
||||||
"breakpoints": true
|
"breakpoints": true
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"idx": 25,
|
||||||
|
"version": "7",
|
||||||
|
"when": 1783560000000,
|
||||||
|
"tag": "0025_assistant_model",
|
||||||
|
"breakpoints": true
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"idx": 26,
|
||||||
|
"version": "7",
|
||||||
|
"when": 1783561000000,
|
||||||
|
"tag": "0026_assistant_model_route",
|
||||||
|
"breakpoints": true
|
||||||
}
|
}
|
||||||
]
|
]
|
||||||
}
|
}
|
||||||
@@ -8,6 +8,7 @@ export default tseslint.config(
|
|||||||
ignores: [
|
ignores: [
|
||||||
"node_modules/**",
|
"node_modules/**",
|
||||||
".next/**",
|
".next/**",
|
||||||
|
".worktrees/**",
|
||||||
".claude/**",
|
".claude/**",
|
||||||
".design-tmp/**",
|
".design-tmp/**",
|
||||||
"dist/**",
|
"dist/**",
|
||||||
|
|||||||
+8
-1
@@ -178,7 +178,14 @@ const nextConfig: NextConfig = {
|
|||||||
output: "standalone",
|
output: "standalone",
|
||||||
// Keep pino and pino-pretty as native Node.js requires so their worker-thread
|
// Keep pino and pino-pretty as native Node.js requires so their worker-thread
|
||||||
// transport and stream internals work correctly inside the standalone bundle.
|
// transport and stream internals work correctly inside the standalone bundle.
|
||||||
serverExternalPackages: ["pino", "pino-pretty", "drizzle-orm", "postgres"],
|
serverExternalPackages: [
|
||||||
|
"pino",
|
||||||
|
"pino-pretty",
|
||||||
|
"drizzle-orm",
|
||||||
|
"postgres",
|
||||||
|
"isomorphic-dompurify",
|
||||||
|
"jsdom",
|
||||||
|
],
|
||||||
};
|
};
|
||||||
|
|
||||||
export default nextConfig;
|
export default nextConfig;
|
||||||
|
|||||||
+1
-1
@@ -1,6 +1,6 @@
|
|||||||
{
|
{
|
||||||
"name": "famapp",
|
"name": "famapp",
|
||||||
"version": "0.6.1",
|
"version": "0.6.2",
|
||||||
"private": true,
|
"private": true,
|
||||||
"type": "module",
|
"type": "module",
|
||||||
"packageManager": "pnpm@10.33.3",
|
"packageManager": "pnpm@10.33.3",
|
||||||
|
|||||||
@@ -2,6 +2,7 @@ import { apiError, apiJson } from "@/lib/api-handler";
|
|||||||
import { resolveApiAuth } from "@/lib/api-auth";
|
import { resolveApiAuth } from "@/lib/api-auth";
|
||||||
import { getAssistantPreferences, resolveAssistantSystemPrompt } from "@/lib/assistant-preference";
|
import { getAssistantPreferences, resolveAssistantSystemPrompt } from "@/lib/assistant-preference";
|
||||||
import { isLlmConfigured } from "@/lib/llm";
|
import { isLlmConfigured } from "@/lib/llm";
|
||||||
|
import { listLlmModels, resolveAssistantModel } from "@/lib/llm/models";
|
||||||
import { clientChatInputSchema } from "@/modules/agent/messages";
|
import { clientChatInputSchema } from "@/modules/agent/messages";
|
||||||
import { encodeSseEvent } from "@/modules/agent/server/progress";
|
import { encodeSseEvent } from "@/modules/agent/server/progress";
|
||||||
import { runAgentChat } from "@/modules/agent/server/run";
|
import { runAgentChat } from "@/modules/agent/server/run";
|
||||||
@@ -31,6 +32,18 @@ export async function POST(request: Request) {
|
|||||||
return apiError(parsed.error.issues[0]?.message ?? "Validation error", 400);
|
return apiError(parsed.error.issues[0]?.message ?? "Validation error", 400);
|
||||||
}
|
}
|
||||||
|
|
||||||
|
const modelList = await listLlmModels({ route: assistant.modelRoute });
|
||||||
|
const modelResolution = resolveAssistantModel({
|
||||||
|
requestedModel: parsed.data.model,
|
||||||
|
savedModel: assistant.model,
|
||||||
|
fallbackModel: modelList.fallbackModel,
|
||||||
|
models: modelList.models,
|
||||||
|
});
|
||||||
|
|
||||||
|
if (!modelResolution.ok) {
|
||||||
|
return apiError(modelResolution.error, 400);
|
||||||
|
}
|
||||||
|
|
||||||
if (parsed.data.stream) {
|
if (parsed.data.stream) {
|
||||||
const stream = new ReadableStream<Uint8Array>({
|
const stream = new ReadableStream<Uint8Array>({
|
||||||
async start(controller) {
|
async start(controller) {
|
||||||
@@ -44,6 +57,7 @@ export async function POST(request: Request) {
|
|||||||
messages: parsed.data.messages,
|
messages: parsed.data.messages,
|
||||||
request,
|
request,
|
||||||
systemPrompt,
|
systemPrompt,
|
||||||
|
model: modelResolution.model,
|
||||||
onProgress: send,
|
onProgress: send,
|
||||||
});
|
});
|
||||||
|
|
||||||
@@ -78,6 +92,7 @@ export async function POST(request: Request) {
|
|||||||
messages: parsed.data.messages,
|
messages: parsed.data.messages,
|
||||||
request,
|
request,
|
||||||
systemPrompt,
|
systemPrompt,
|
||||||
|
model: modelResolution.model,
|
||||||
});
|
});
|
||||||
|
|
||||||
return apiJson({
|
return apiJson({
|
||||||
|
|||||||
@@ -0,0 +1,47 @@
|
|||||||
|
import { apiError, apiJson } from "@/lib/api-handler";
|
||||||
|
import { resolveApiAuth } from "@/lib/api-auth";
|
||||||
|
import { getAssistantPreferences } from "@/lib/assistant-preference";
|
||||||
|
import { isValidAssistantModelRoute, listLlmModels, resolveAssistantModel } from "@/lib/llm/models";
|
||||||
|
|
||||||
|
export const dynamic = "force-dynamic";
|
||||||
|
|
||||||
|
function noStore(response: Response): Response {
|
||||||
|
response.headers.set("Cache-Control", "no-store");
|
||||||
|
return response;
|
||||||
|
}
|
||||||
|
|
||||||
|
export async function GET(request: Request) {
|
||||||
|
const auth = await resolveApiAuth(request);
|
||||||
|
if (!auth?.userId) {
|
||||||
|
return noStore(apiError("Unauthorized", 401));
|
||||||
|
}
|
||||||
|
|
||||||
|
const assistant = await getAssistantPreferences(auth.userId);
|
||||||
|
if (!assistant.enabled) {
|
||||||
|
return noStore(apiError("Assistant not enabled", 403));
|
||||||
|
}
|
||||||
|
|
||||||
|
const requestedRoute = new URL(request.url).searchParams.get("route");
|
||||||
|
if (requestedRoute !== null && !isValidAssistantModelRoute(requestedRoute)) {
|
||||||
|
return noStore(apiError("Invalid assistant model route", 400));
|
||||||
|
}
|
||||||
|
|
||||||
|
const modelRoute = requestedRoute ?? assistant.modelRoute;
|
||||||
|
const modelList = await listLlmModels({ route: modelRoute });
|
||||||
|
const resolved = resolveAssistantModel({
|
||||||
|
requestedModel: null,
|
||||||
|
savedModel: requestedRoute === null ? assistant.model : null,
|
||||||
|
fallbackModel: modelList.fallbackModel,
|
||||||
|
models: modelList.models,
|
||||||
|
});
|
||||||
|
|
||||||
|
return noStore(
|
||||||
|
apiJson({
|
||||||
|
models: modelList.models,
|
||||||
|
selectedModel: resolved.model,
|
||||||
|
fallbackModel: modelList.fallbackModel,
|
||||||
|
route: modelList.route,
|
||||||
|
degraded: modelList.degraded,
|
||||||
|
}),
|
||||||
|
);
|
||||||
|
}
|
||||||
@@ -104,6 +104,7 @@ export default async function RootLayout({ children }: { children: React.ReactNo
|
|||||||
let signedIn = false;
|
let signedIn = false;
|
||||||
let assistantEnabled = false;
|
let assistantEnabled = false;
|
||||||
let assistantName = DEFAULT_ASSISTANT_NAME;
|
let assistantName = DEFAULT_ASSISTANT_NAME;
|
||||||
|
let assistantModel: string | null = null;
|
||||||
|
|
||||||
const session = await auth();
|
const session = await auth();
|
||||||
if (session?.user?.id) {
|
if (session?.user?.id) {
|
||||||
@@ -117,6 +118,7 @@ export default async function RootLayout({ children }: { children: React.ReactNo
|
|||||||
themeNavStyle: users.themeNavStyle,
|
themeNavStyle: users.themeNavStyle,
|
||||||
assistantEnabled: users.assistantEnabled,
|
assistantEnabled: users.assistantEnabled,
|
||||||
assistantName: users.assistantName,
|
assistantName: users.assistantName,
|
||||||
|
assistantModel: users.assistantModel,
|
||||||
})
|
})
|
||||||
.from(users)
|
.from(users)
|
||||||
.where(eq(users.id, session.user.id))
|
.where(eq(users.id, session.user.id))
|
||||||
@@ -129,6 +131,7 @@ export default async function RootLayout({ children }: { children: React.ReactNo
|
|||||||
navStyle = row.themeNavStyle as NavStyle;
|
navStyle = row.themeNavStyle as NavStyle;
|
||||||
assistantEnabled = row.assistantEnabled;
|
assistantEnabled = row.assistantEnabled;
|
||||||
assistantName = row.assistantName?.trim() || DEFAULT_ASSISTANT_NAME;
|
assistantName = row.assistantName?.trim() || DEFAULT_ASSISTANT_NAME;
|
||||||
|
assistantModel = row.assistantModel?.trim() || null;
|
||||||
}
|
}
|
||||||
userDashboards = await db
|
userDashboards = await db
|
||||||
.select({
|
.select({
|
||||||
@@ -186,6 +189,7 @@ export default async function RootLayout({ children }: { children: React.ReactNo
|
|||||||
configured={isLlmConfigured()}
|
configured={isLlmConfigured()}
|
||||||
userId={session.user.id}
|
userId={session.user.id}
|
||||||
assistantName={assistantName}
|
assistantName={assistantName}
|
||||||
|
assistantModel={assistantModel}
|
||||||
/>
|
/>
|
||||||
) : null}
|
) : null}
|
||||||
<AppToaster position="bottom-right" />
|
<AppToaster position="bottom-right" />
|
||||||
|
|||||||
@@ -9,8 +9,15 @@ import {
|
|||||||
MAX_ASSISTANT_NAME_LENGTH,
|
MAX_ASSISTANT_NAME_LENGTH,
|
||||||
MAX_ASSISTANT_SYSTEM_PROMPT_LENGTH,
|
MAX_ASSISTANT_SYSTEM_PROMPT_LENGTH,
|
||||||
} from "@/lib/assistant-config";
|
} from "@/lib/assistant-config";
|
||||||
|
import {
|
||||||
|
isValidAssistantModelRoute,
|
||||||
|
isValidLlmModelId,
|
||||||
|
listLlmModels,
|
||||||
|
type AssistantModelRoute,
|
||||||
|
} from "@/lib/llm/models";
|
||||||
import { users } from "@/modules/_core/schema";
|
import { users } from "@/modules/_core/schema";
|
||||||
import { getCurrentSession } from "@/lib/session";
|
import { getCurrentSession } from "@/lib/session";
|
||||||
|
import { getAssistantPreferences } from "@/lib/assistant-preference";
|
||||||
|
|
||||||
const assistantNameSchema = z
|
const assistantNameSchema = z
|
||||||
.string()
|
.string()
|
||||||
@@ -50,6 +57,39 @@ export async function setAssistantSystemPrompt(prompt: string | null): Promise<v
|
|||||||
revalidateAssistantSurfaces();
|
revalidateAssistantSurfaces();
|
||||||
}
|
}
|
||||||
|
|
||||||
|
export async function setAssistantModelRoute(route: AssistantModelRoute): Promise<void> {
|
||||||
|
if (!isValidAssistantModelRoute(route)) {
|
||||||
|
throw new Error("Invalid assistant model route");
|
||||||
|
}
|
||||||
|
|
||||||
|
const { user } = await getCurrentSession();
|
||||||
|
await db
|
||||||
|
.update(users)
|
||||||
|
.set({ assistantModelRoute: route, assistantModel: null })
|
||||||
|
.where(eq(users.id, user.id));
|
||||||
|
revalidateAssistantSurfaces();
|
||||||
|
}
|
||||||
|
|
||||||
|
export async function setAssistantModel(model: string | null): Promise<void> {
|
||||||
|
const { user } = await getCurrentSession();
|
||||||
|
const normalized = model?.trim() || null;
|
||||||
|
|
||||||
|
if (normalized !== null && !isValidLlmModelId(normalized)) {
|
||||||
|
throw new Error("Invalid assistant model");
|
||||||
|
}
|
||||||
|
|
||||||
|
const assistant = await getAssistantPreferences(user.id);
|
||||||
|
const available = await listLlmModels({ route: assistant.modelRoute });
|
||||||
|
const requested = normalized === available.fallbackModel ? null : normalized;
|
||||||
|
|
||||||
|
if (requested !== null && !available.models.some((option) => option.id === requested)) {
|
||||||
|
throw new Error("Invalid assistant model");
|
||||||
|
}
|
||||||
|
|
||||||
|
await db.update(users).set({ assistantModel: requested }).where(eq(users.id, user.id));
|
||||||
|
revalidateAssistantSurfaces();
|
||||||
|
}
|
||||||
|
|
||||||
export async function resetAssistantName(): Promise<void> {
|
export async function resetAssistantName(): Promise<void> {
|
||||||
await setAssistantName(DEFAULT_ASSISTANT_NAME);
|
await setAssistantName(DEFAULT_ASSISTANT_NAME);
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -324,6 +324,7 @@ function AppearanceSection({
|
|||||||
themeNavStyle: string;
|
themeNavStyle: string;
|
||||||
assistantEnabled: boolean;
|
assistantEnabled: boolean;
|
||||||
assistantName: string;
|
assistantName: string;
|
||||||
|
assistantModelRoute: string | null;
|
||||||
assistantSystemPrompt: string | null;
|
assistantSystemPrompt: string | null;
|
||||||
};
|
};
|
||||||
}) {
|
}) {
|
||||||
@@ -361,6 +362,7 @@ function AppearanceSection({
|
|||||||
<AssistantSettings
|
<AssistantSettings
|
||||||
enabled={user.assistantEnabled}
|
enabled={user.assistantEnabled}
|
||||||
name={user.assistantName}
|
name={user.assistantName}
|
||||||
|
modelRoute={user.assistantModelRoute}
|
||||||
systemPrompt={user.assistantSystemPrompt}
|
systemPrompt={user.assistantSystemPrompt}
|
||||||
defaultSystemPrompt={AGENT_SYSTEM_PROMPT}
|
defaultSystemPrompt={AGENT_SYSTEM_PROMPT}
|
||||||
/>
|
/>
|
||||||
|
|||||||
@@ -7,9 +7,11 @@ import {
|
|||||||
resetAssistantSystemPrompt,
|
resetAssistantSystemPrompt,
|
||||||
setAssistantEnabled,
|
setAssistantEnabled,
|
||||||
setAssistantName,
|
setAssistantName,
|
||||||
|
setAssistantModelRoute,
|
||||||
setAssistantSystemPrompt,
|
setAssistantSystemPrompt,
|
||||||
} from "@/app/settings/assistant-actions";
|
} from "@/app/settings/assistant-actions";
|
||||||
import { DEFAULT_ASSISTANT_NAME, MAX_ASSISTANT_NAME_LENGTH } from "@/lib/assistant-config";
|
import { DEFAULT_ASSISTANT_NAME, MAX_ASSISTANT_NAME_LENGTH } from "@/lib/assistant-config";
|
||||||
|
import type { AssistantModelRoute } from "@/lib/llm/models";
|
||||||
import { Button } from "@/components/ui/button";
|
import { Button } from "@/components/ui/button";
|
||||||
import { Input } from "@/components/ui/input";
|
import { Input } from "@/components/ui/input";
|
||||||
import { Switch } from "@/components/ui/switch";
|
import { Switch } from "@/components/ui/switch";
|
||||||
@@ -17,15 +19,29 @@ import { Switch } from "@/components/ui/switch";
|
|||||||
type Props = {
|
type Props = {
|
||||||
enabled: boolean;
|
enabled: boolean;
|
||||||
name: string;
|
name: string;
|
||||||
|
modelRoute: string | null;
|
||||||
systemPrompt: string | null;
|
systemPrompt: string | null;
|
||||||
defaultSystemPrompt: string;
|
defaultSystemPrompt: string;
|
||||||
};
|
};
|
||||||
|
|
||||||
export function AssistantSettings({ enabled, name, systemPrompt, defaultSystemPrompt }: Props) {
|
function normalizeAssistantModelRoute(value: string | null): AssistantModelRoute {
|
||||||
|
return value === "uncensored" ? "uncensored" : "auto";
|
||||||
|
}
|
||||||
|
|
||||||
|
export function AssistantSettings({
|
||||||
|
enabled,
|
||||||
|
name,
|
||||||
|
modelRoute,
|
||||||
|
systemPrompt,
|
||||||
|
defaultSystemPrompt,
|
||||||
|
}: Props) {
|
||||||
const [isPending, startTransition] = useTransition();
|
const [isPending, startTransition] = useTransition();
|
||||||
const router = useRouter();
|
const router = useRouter();
|
||||||
|
|
||||||
const effectivePrompt = systemPrompt ?? defaultSystemPrompt;
|
const effectivePrompt = systemPrompt ?? defaultSystemPrompt;
|
||||||
|
const [selectedRoute, setSelectedRoute] = useState<AssistantModelRoute>(() =>
|
||||||
|
normalizeAssistantModelRoute(modelRoute),
|
||||||
|
);
|
||||||
const [savedName, setSavedName] = useState(name);
|
const [savedName, setSavedName] = useState(name);
|
||||||
const [draftName, setDraftName] = useState(name);
|
const [draftName, setDraftName] = useState(name);
|
||||||
const [savedPrompt, setSavedPrompt] = useState(systemPrompt);
|
const [savedPrompt, setSavedPrompt] = useState(systemPrompt);
|
||||||
@@ -45,6 +61,16 @@ export function AssistantSettings({ enabled, name, systemPrompt, defaultSystemPr
|
|||||||
});
|
});
|
||||||
}
|
}
|
||||||
|
|
||||||
|
function changeModelRoute(nextRoute: string) {
|
||||||
|
const route = normalizeAssistantModelRoute(nextRoute);
|
||||||
|
setSelectedRoute(route);
|
||||||
|
|
||||||
|
startTransition(async () => {
|
||||||
|
await setAssistantModelRoute(route);
|
||||||
|
router.refresh();
|
||||||
|
});
|
||||||
|
}
|
||||||
|
|
||||||
function saveName() {
|
function saveName() {
|
||||||
const next = draftName.trim();
|
const next = draftName.trim();
|
||||||
if (!next) return;
|
if (!next) return;
|
||||||
@@ -105,6 +131,29 @@ export function AssistantSettings({ enabled, name, systemPrompt, defaultSystemPr
|
|||||||
/>
|
/>
|
||||||
</label>
|
</label>
|
||||||
|
|
||||||
|
<div className="flex flex-col gap-2">
|
||||||
|
<div className="min-w-0">
|
||||||
|
<label htmlFor="assistant-model-route" className="text-sm font-medium">
|
||||||
|
Model route
|
||||||
|
</label>
|
||||||
|
<p className="muted text-[12px] mt-0.5">
|
||||||
|
Choose the default router family for your assistant. Individual models can still be
|
||||||
|
picked from the chat panel.
|
||||||
|
</p>
|
||||||
|
</div>
|
||||||
|
<select
|
||||||
|
id="assistant-model-route"
|
||||||
|
aria-label="Assistant model route"
|
||||||
|
value={selectedRoute}
|
||||||
|
disabled={isPending}
|
||||||
|
onChange={(event) => changeModelRoute(event.target.value)}
|
||||||
|
className="input h-10"
|
||||||
|
>
|
||||||
|
<option value="auto">Auto</option>
|
||||||
|
<option value="uncensored">Uncensored</option>
|
||||||
|
</select>
|
||||||
|
</div>
|
||||||
|
|
||||||
<div className="flex flex-col gap-2">
|
<div className="flex flex-col gap-2">
|
||||||
<div className="flex items-end justify-between gap-3">
|
<div className="flex items-end justify-between gap-3">
|
||||||
<div className="min-w-0 flex-1">
|
<div className="min-w-0 flex-1">
|
||||||
|
|||||||
@@ -1,6 +1,7 @@
|
|||||||
import { eq } from "drizzle-orm";
|
import { eq } from "drizzle-orm";
|
||||||
import { db } from "@/lib/db";
|
import { db } from "@/lib/db";
|
||||||
import { DEFAULT_ASSISTANT_NAME } from "@/lib/assistant-config";
|
import { DEFAULT_ASSISTANT_NAME } from "@/lib/assistant-config";
|
||||||
|
import { isValidAssistantModelRoute, type AssistantModelRoute } from "@/lib/llm/models";
|
||||||
import { users } from "@/modules/_core/schema";
|
import { users } from "@/modules/_core/schema";
|
||||||
|
|
||||||
export {
|
export {
|
||||||
@@ -14,6 +15,8 @@ export type AssistantPreferences = {
|
|||||||
enabled: boolean;
|
enabled: boolean;
|
||||||
name: string;
|
name: string;
|
||||||
systemPrompt: string | null;
|
systemPrompt: string | null;
|
||||||
|
modelRoute: AssistantModelRoute | null;
|
||||||
|
model: string | null;
|
||||||
};
|
};
|
||||||
|
|
||||||
export async function getAssistantPreferences(userId: string): Promise<AssistantPreferences> {
|
export async function getAssistantPreferences(userId: string): Promise<AssistantPreferences> {
|
||||||
@@ -22,6 +25,8 @@ export async function getAssistantPreferences(userId: string): Promise<Assistant
|
|||||||
assistantEnabled: users.assistantEnabled,
|
assistantEnabled: users.assistantEnabled,
|
||||||
assistantName: users.assistantName,
|
assistantName: users.assistantName,
|
||||||
assistantSystemPrompt: users.assistantSystemPrompt,
|
assistantSystemPrompt: users.assistantSystemPrompt,
|
||||||
|
assistantModelRoute: users.assistantModelRoute,
|
||||||
|
assistantModel: users.assistantModel,
|
||||||
})
|
})
|
||||||
.from(users)
|
.from(users)
|
||||||
.where(eq(users.id, userId))
|
.where(eq(users.id, userId))
|
||||||
@@ -31,6 +36,11 @@ export async function getAssistantPreferences(userId: string): Promise<Assistant
|
|||||||
enabled: row?.assistantEnabled ?? false,
|
enabled: row?.assistantEnabled ?? false,
|
||||||
name: row?.assistantName?.trim() || DEFAULT_ASSISTANT_NAME,
|
name: row?.assistantName?.trim() || DEFAULT_ASSISTANT_NAME,
|
||||||
systemPrompt: row?.assistantSystemPrompt ?? null,
|
systemPrompt: row?.assistantSystemPrompt ?? null,
|
||||||
|
modelRoute:
|
||||||
|
row?.assistantModelRoute && isValidAssistantModelRoute(row.assistantModelRoute)
|
||||||
|
? row.assistantModelRoute
|
||||||
|
: null,
|
||||||
|
model: row?.assistantModel?.trim() || null,
|
||||||
};
|
};
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|||||||
@@ -13,8 +13,8 @@ export type {
|
|||||||
export { getLlmConfig, isLlmConfigured } from "./config";
|
export { getLlmConfig, isLlmConfigured } from "./config";
|
||||||
export { createMockLlmClient } from "./mock";
|
export { createMockLlmClient } from "./mock";
|
||||||
|
|
||||||
export function createLlmClient(override?: LlmClient): LlmClient {
|
export function createLlmClient(options?: { model?: string; override?: LlmClient }): LlmClient {
|
||||||
if (override) return override;
|
if (options?.override) return options.override;
|
||||||
|
|
||||||
const config = getLlmConfig();
|
const config = getLlmConfig();
|
||||||
if (config.provider === "mock" || !config.baseUrl) {
|
if (config.provider === "mock" || !config.baseUrl) {
|
||||||
@@ -24,6 +24,6 @@ export function createLlmClient(override?: LlmClient): LlmClient {
|
|||||||
return createOpenAiCompatibleClient({
|
return createOpenAiCompatibleClient({
|
||||||
baseUrl: config.baseUrl,
|
baseUrl: config.baseUrl,
|
||||||
apiKey: config.apiKey,
|
apiKey: config.apiKey,
|
||||||
model: config.model,
|
model: options?.model ?? config.model,
|
||||||
});
|
});
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -0,0 +1,148 @@
|
|||||||
|
import { getLlmConfig, type LlmConfig } from "./config";
|
||||||
|
|
||||||
|
export type LlmModelOption = {
|
||||||
|
id: string;
|
||||||
|
label: string;
|
||||||
|
};
|
||||||
|
|
||||||
|
export const ASSISTANT_MODEL_ROUTES = ["auto", "uncensored"] as const;
|
||||||
|
|
||||||
|
export type AssistantModelRoute = (typeof ASSISTANT_MODEL_ROUTES)[number];
|
||||||
|
|
||||||
|
export type LlmModelsResult = {
|
||||||
|
models: LlmModelOption[];
|
||||||
|
fallbackModel: string;
|
||||||
|
route: AssistantModelRoute;
|
||||||
|
degraded: boolean;
|
||||||
|
};
|
||||||
|
|
||||||
|
export type AssistantModelResolution =
|
||||||
|
| { ok: true; model: string }
|
||||||
|
| { ok: false; model: string; error: string };
|
||||||
|
|
||||||
|
const MODEL_ID_PATTERN = /^[A-Za-z0-9._:/-]+$/;
|
||||||
|
const MAX_MODEL_ID_LENGTH = 128;
|
||||||
|
|
||||||
|
export function isValidLlmModelId(value: string): boolean {
|
||||||
|
const trimmed = value.trim();
|
||||||
|
return (
|
||||||
|
trimmed.length > 0 &&
|
||||||
|
trimmed.length <= MAX_MODEL_ID_LENGTH &&
|
||||||
|
trimmed === value &&
|
||||||
|
MODEL_ID_PATTERN.test(trimmed)
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
|
export function isValidAssistantModelRoute(value: string): value is AssistantModelRoute {
|
||||||
|
return ASSISTANT_MODEL_ROUTES.some((route) => route === value);
|
||||||
|
}
|
||||||
|
|
||||||
|
function resolveModelRoute(
|
||||||
|
route: AssistantModelRoute | null | undefined,
|
||||||
|
config: LlmConfig,
|
||||||
|
): { route: AssistantModelRoute; fallbackModel: string } {
|
||||||
|
if (route) {
|
||||||
|
return { route, fallbackModel: route };
|
||||||
|
}
|
||||||
|
|
||||||
|
const defaultRoute: AssistantModelRoute = config.model === "uncensored" ? "uncensored" : "auto";
|
||||||
|
return { route: defaultRoute, fallbackModel: config.model };
|
||||||
|
}
|
||||||
|
|
||||||
|
export function normalizeLlmModelsPayload(payload: unknown): LlmModelOption[] {
|
||||||
|
const data =
|
||||||
|
typeof payload === "object" && payload !== null && "data" in payload
|
||||||
|
? (payload as { data?: unknown }).data
|
||||||
|
: null;
|
||||||
|
|
||||||
|
if (!Array.isArray(data)) return [];
|
||||||
|
|
||||||
|
const ids = new Set<string>();
|
||||||
|
for (const row of data) {
|
||||||
|
if (typeof row !== "object" || row === null || !("id" in row)) continue;
|
||||||
|
const id = (row as { id?: unknown }).id;
|
||||||
|
if (typeof id !== "string") continue;
|
||||||
|
if (!isValidLlmModelId(id)) continue;
|
||||||
|
ids.add(id);
|
||||||
|
}
|
||||||
|
|
||||||
|
return [...ids].sort((a, b) => a.localeCompare(b)).map((id) => ({ id, label: id }));
|
||||||
|
}
|
||||||
|
|
||||||
|
export async function listLlmModels(options?: {
|
||||||
|
config?: LlmConfig;
|
||||||
|
fetchImpl?: typeof fetch;
|
||||||
|
route?: AssistantModelRoute | null;
|
||||||
|
}): Promise<LlmModelsResult> {
|
||||||
|
const config = options?.config ?? getLlmConfig();
|
||||||
|
const fetchImpl = options?.fetchImpl ?? fetch;
|
||||||
|
const { route, fallbackModel } = resolveModelRoute(options?.route, config);
|
||||||
|
const fallbackOption = { id: fallbackModel, label: fallbackModel };
|
||||||
|
|
||||||
|
if (config.provider === "mock" || !config.baseUrl) {
|
||||||
|
return { models: [fallbackOption], fallbackModel, route, degraded: false };
|
||||||
|
}
|
||||||
|
|
||||||
|
try {
|
||||||
|
const headers: Record<string, string> = {};
|
||||||
|
if (config.apiKey) headers.Authorization = `Bearer ${config.apiKey}`;
|
||||||
|
|
||||||
|
const modelsUrl = new URL(`${config.baseUrl.replace(/\/$/, "")}/models`);
|
||||||
|
if (route === "uncensored") {
|
||||||
|
modelsUrl.searchParams.set("type", "uncensored");
|
||||||
|
}
|
||||||
|
|
||||||
|
const response = await fetchImpl(modelsUrl, {
|
||||||
|
method: "GET",
|
||||||
|
headers,
|
||||||
|
});
|
||||||
|
|
||||||
|
if (!response.ok) {
|
||||||
|
return { models: [fallbackOption], fallbackModel, route, degraded: true };
|
||||||
|
}
|
||||||
|
|
||||||
|
const models = normalizeLlmModelsPayload(await response.json());
|
||||||
|
|
||||||
|
if (models.length === 0) {
|
||||||
|
return { models: [fallbackOption], fallbackModel, route, degraded: true };
|
||||||
|
}
|
||||||
|
|
||||||
|
if (!models.some((model) => model.id === fallbackModel)) {
|
||||||
|
return { models: [fallbackOption], fallbackModel, route, degraded: false };
|
||||||
|
}
|
||||||
|
|
||||||
|
return {
|
||||||
|
models,
|
||||||
|
fallbackModel,
|
||||||
|
route,
|
||||||
|
degraded: false,
|
||||||
|
};
|
||||||
|
} catch {
|
||||||
|
return { models: [fallbackOption], fallbackModel, route, degraded: true };
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
export function resolveAssistantModel(options: {
|
||||||
|
requestedModel: string | null | undefined;
|
||||||
|
savedModel: string | null | undefined;
|
||||||
|
fallbackModel: string;
|
||||||
|
models: LlmModelOption[];
|
||||||
|
}): AssistantModelResolution {
|
||||||
|
const available = new Set(options.models.map((model) => model.id));
|
||||||
|
const fallback = available.has(options.fallbackModel)
|
||||||
|
? options.fallbackModel
|
||||||
|
: (options.models[0]?.id ?? options.fallbackModel);
|
||||||
|
|
||||||
|
if (options.requestedModel !== null && options.requestedModel !== undefined) {
|
||||||
|
if (!isValidLlmModelId(options.requestedModel) || !available.has(options.requestedModel)) {
|
||||||
|
return { ok: false, model: fallback, error: "Invalid assistant model" };
|
||||||
|
}
|
||||||
|
return { ok: true, model: options.requestedModel };
|
||||||
|
}
|
||||||
|
|
||||||
|
if (options.savedModel && available.has(options.savedModel)) {
|
||||||
|
return { ok: true, model: options.savedModel };
|
||||||
|
}
|
||||||
|
|
||||||
|
return { ok: true, model: fallback };
|
||||||
|
}
|
||||||
@@ -38,6 +38,8 @@ export const users = pgTable("users", {
|
|||||||
assistantEnabled: boolean("assistant_enabled").notNull().default(false),
|
assistantEnabled: boolean("assistant_enabled").notNull().default(false),
|
||||||
assistantName: text("assistant_name").notNull().default("Assistant"),
|
assistantName: text("assistant_name").notNull().default("Assistant"),
|
||||||
assistantSystemPrompt: text("assistant_system_prompt"),
|
assistantSystemPrompt: text("assistant_system_prompt"),
|
||||||
|
assistantModelRoute: text("assistant_model_route"),
|
||||||
|
assistantModel: text("assistant_model"),
|
||||||
defaultEventReminderOffsets: jsonb("default_event_reminder_offsets")
|
defaultEventReminderOffsets: jsonb("default_event_reminder_offsets")
|
||||||
.notNull()
|
.notNull()
|
||||||
.$type<number[]>()
|
.$type<number[]>()
|
||||||
|
|||||||
@@ -8,9 +8,10 @@ type Props = {
|
|||||||
configured: boolean;
|
configured: boolean;
|
||||||
userId: string;
|
userId: string;
|
||||||
assistantName: string;
|
assistantName: string;
|
||||||
|
assistantModel: string | null;
|
||||||
};
|
};
|
||||||
|
|
||||||
export function AssistantBubble({ configured, userId, assistantName }: Props) {
|
export function AssistantBubble({ configured, userId, assistantName, assistantModel }: Props) {
|
||||||
const [open, setOpen] = useState(false);
|
const [open, setOpen] = useState(false);
|
||||||
|
|
||||||
return (
|
return (
|
||||||
@@ -41,6 +42,7 @@ export function AssistantBubble({ configured, userId, assistantName }: Props) {
|
|||||||
configured={configured}
|
configured={configured}
|
||||||
userId={userId}
|
userId={userId}
|
||||||
assistantName={assistantName}
|
assistantName={assistantName}
|
||||||
|
assistantModel={assistantModel}
|
||||||
/>
|
/>
|
||||||
</div>
|
</div>
|
||||||
) : null}
|
) : null}
|
||||||
|
|||||||
@@ -1,7 +1,8 @@
|
|||||||
"use client";
|
"use client";
|
||||||
|
|
||||||
import { useEffect, useRef, useState } from "react";
|
import { useCallback, useEffect, useRef, useState, useTransition } from "react";
|
||||||
import { ImagePlus, Loader2, Mic, Send, Square } from "lucide-react";
|
import { ChevronDown, ImagePlus, Loader2, Mic, RefreshCw, Send, Square } from "lucide-react";
|
||||||
|
import { setAssistantModel } from "@/app/settings/assistant-actions";
|
||||||
import { Button } from "@/components/ui/button";
|
import { Button } from "@/components/ui/button";
|
||||||
import { Input } from "@/components/ui/input";
|
import { Input } from "@/components/ui/input";
|
||||||
import { consumeAgentChatStream } from "../assistant-chat-stream";
|
import { consumeAgentChatStream } from "../assistant-chat-stream";
|
||||||
@@ -18,12 +19,25 @@ type Props = {
|
|||||||
configured: boolean;
|
configured: boolean;
|
||||||
userId: string;
|
userId: string;
|
||||||
assistantName: string;
|
assistantName: string;
|
||||||
|
assistantModel: string | null;
|
||||||
};
|
};
|
||||||
|
|
||||||
type PendingImage = {
|
type PendingImage = {
|
||||||
url: string;
|
url: string;
|
||||||
};
|
};
|
||||||
|
|
||||||
|
type LlmModelOption = {
|
||||||
|
id: string;
|
||||||
|
label: string;
|
||||||
|
};
|
||||||
|
|
||||||
|
type ModelsResponse = {
|
||||||
|
models: LlmModelOption[];
|
||||||
|
selectedModel: string;
|
||||||
|
fallbackModel: string;
|
||||||
|
degraded: boolean;
|
||||||
|
};
|
||||||
|
|
||||||
async function uploadAssistantImage(file: File): Promise<string> {
|
async function uploadAssistantImage(file: File): Promise<string> {
|
||||||
const formData = new FormData();
|
const formData = new FormData();
|
||||||
formData.append("file", file);
|
formData.append("file", file);
|
||||||
@@ -36,7 +50,7 @@ async function uploadAssistantImage(file: File): Promise<string> {
|
|||||||
return payload.url;
|
return payload.url;
|
||||||
}
|
}
|
||||||
|
|
||||||
export function AssistantPanel({ configured, userId, assistantName }: Props) {
|
export function AssistantPanel({ configured, userId, assistantName, assistantModel }: Props) {
|
||||||
const [messages, setMessages] = useState<AssistantChatMessage[]>(() => loadAssistantChat(userId));
|
const [messages, setMessages] = useState<AssistantChatMessage[]>(() => loadAssistantChat(userId));
|
||||||
const [input, setInput] = useState("");
|
const [input, setInput] = useState("");
|
||||||
const [pendingImage, setPendingImage] = useState<PendingImage | null>(null);
|
const [pendingImage, setPendingImage] = useState<PendingImage | null>(null);
|
||||||
@@ -44,6 +58,12 @@ export function AssistantPanel({ configured, userId, assistantName }: Props) {
|
|||||||
const [error, setError] = useState<string | null>(null);
|
const [error, setError] = useState<string | null>(null);
|
||||||
const [isPending, setIsPending] = useState(false);
|
const [isPending, setIsPending] = useState(false);
|
||||||
const [activityLabel, setActivityLabel] = useState<string | null>(null);
|
const [activityLabel, setActivityLabel] = useState<string | null>(null);
|
||||||
|
const [modelOptions, setModelOptions] = useState<LlmModelOption[]>([]);
|
||||||
|
const [selectedModel, setSelectedModel] = useState(assistantModel ?? "");
|
||||||
|
const [fallbackModel, setFallbackModel] = useState("");
|
||||||
|
const [modelsDegraded, setModelsDegraded] = useState(false);
|
||||||
|
const [modelsLoading, setModelsLoading] = useState(true);
|
||||||
|
const [savingModel, startTransition] = useTransition();
|
||||||
const listRef = useRef<HTMLDivElement>(null);
|
const listRef = useRef<HTMLDivElement>(null);
|
||||||
const abortRef = useRef<AbortController | null>(null);
|
const abortRef = useRef<AbortController | null>(null);
|
||||||
const imageInputRef = useRef<HTMLInputElement>(null);
|
const imageInputRef = useRef<HTMLInputElement>(null);
|
||||||
@@ -67,6 +87,48 @@ export function AssistantPanel({ configured, userId, assistantName }: Props) {
|
|||||||
};
|
};
|
||||||
}, []);
|
}, []);
|
||||||
|
|
||||||
|
const loadModels = useCallback(async (options?: { refresh?: boolean; signal?: AbortSignal }) => {
|
||||||
|
if (options?.signal?.aborted) return;
|
||||||
|
|
||||||
|
setModelsLoading(true);
|
||||||
|
try {
|
||||||
|
const params = new URLSearchParams();
|
||||||
|
if (options?.refresh) params.set("refresh", "1");
|
||||||
|
const query = params.size > 0 ? `?${params.toString()}` : "";
|
||||||
|
|
||||||
|
const response = await fetch(`/api/agent/models${query}`, {
|
||||||
|
cache: "no-store",
|
||||||
|
signal: options?.signal,
|
||||||
|
});
|
||||||
|
if (!response.ok) throw new Error("Model discovery unavailable");
|
||||||
|
const payload = (await response.json()) as ModelsResponse;
|
||||||
|
if (options?.signal?.aborted) return;
|
||||||
|
setModelOptions(payload.models);
|
||||||
|
setSelectedModel(payload.selectedModel);
|
||||||
|
setFallbackModel(payload.fallbackModel);
|
||||||
|
setModelsDegraded(payload.degraded);
|
||||||
|
setError(null);
|
||||||
|
} catch (err) {
|
||||||
|
if (err instanceof Error && err.name === "AbortError") return;
|
||||||
|
setModelsDegraded(true);
|
||||||
|
setError("Model discovery unavailable");
|
||||||
|
} finally {
|
||||||
|
if (!options?.signal?.aborted) setModelsLoading(false);
|
||||||
|
}
|
||||||
|
}, []);
|
||||||
|
|
||||||
|
useEffect(() => {
|
||||||
|
const controller = new AbortController();
|
||||||
|
|
||||||
|
queueMicrotask(() => {
|
||||||
|
void loadModels({ signal: controller.signal });
|
||||||
|
});
|
||||||
|
|
||||||
|
return () => {
|
||||||
|
controller.abort();
|
||||||
|
};
|
||||||
|
}, [loadModels]);
|
||||||
|
|
||||||
function scrollToBottom() {
|
function scrollToBottom() {
|
||||||
requestAnimationFrame(() => {
|
requestAnimationFrame(() => {
|
||||||
const node = listRef.current;
|
const node = listRef.current;
|
||||||
@@ -102,6 +164,21 @@ export function AssistantPanel({ configured, userId, assistantName }: Props) {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
function changeModel(nextModel: string | null) {
|
||||||
|
if (!nextModel) return;
|
||||||
|
|
||||||
|
setSelectedModel(nextModel);
|
||||||
|
setError(null);
|
||||||
|
|
||||||
|
startTransition(async () => {
|
||||||
|
try {
|
||||||
|
await setAssistantModel(nextModel === fallbackModel ? null : nextModel);
|
||||||
|
} catch (err) {
|
||||||
|
setError(err instanceof Error ? err.message : "Could not save assistant model");
|
||||||
|
}
|
||||||
|
});
|
||||||
|
}
|
||||||
|
|
||||||
async function sendMessage() {
|
async function sendMessage() {
|
||||||
const text = input.trim();
|
const text = input.trim();
|
||||||
const hasImage = pendingImage !== null;
|
const hasImage = pendingImage !== null;
|
||||||
@@ -133,6 +210,7 @@ export function AssistantPanel({ configured, userId, assistantName }: Props) {
|
|||||||
headers: { "Content-Type": "application/json" },
|
headers: { "Content-Type": "application/json" },
|
||||||
body: JSON.stringify({
|
body: JSON.stringify({
|
||||||
messages: nextMessages.map(toClientChatMessage),
|
messages: nextMessages.map(toClientChatMessage),
|
||||||
|
model: selectedModel || undefined,
|
||||||
stream: true,
|
stream: true,
|
||||||
}),
|
}),
|
||||||
signal: controller.signal,
|
signal: controller.signal,
|
||||||
@@ -168,6 +246,7 @@ export function AssistantPanel({ configured, userId, assistantName }: Props) {
|
|||||||
const inputDisabled = isPending || voiceState === "transcribing" || uploadingImage;
|
const inputDisabled = isPending || voiceState === "transcribing" || uploadingImage;
|
||||||
const canSend =
|
const canSend =
|
||||||
!isPending &&
|
!isPending &&
|
||||||
|
!savingModel &&
|
||||||
voiceState === "idle" &&
|
voiceState === "idle" &&
|
||||||
!uploadingImage &&
|
!uploadingImage &&
|
||||||
(input.trim().length > 0 || pendingImage !== null);
|
(input.trim().length > 0 || pendingImage !== null);
|
||||||
@@ -175,21 +254,63 @@ export function AssistantPanel({ configured, userId, assistantName }: Props) {
|
|||||||
return (
|
return (
|
||||||
<div className="flex min-h-0 flex-1 flex-col gap-3">
|
<div className="flex min-h-0 flex-1 flex-col gap-3">
|
||||||
<div className="flex items-start justify-between gap-3">
|
<div className="flex items-start justify-between gap-3">
|
||||||
<p className="muted min-w-0 text-[12px] leading-relaxed">
|
<div className="min-w-0 flex-1">
|
||||||
{configured
|
<p className="muted text-[12px] leading-relaxed">
|
||||||
? "Type, talk, or send a photo — I can update lists, calendar, notes, and more."
|
{configured
|
||||||
: "Mock provider active — set LLM_BASE_URL for your homelab model."}
|
? "Type, talk, or send a photo — I can update lists, calendar, notes, and more."
|
||||||
</p>
|
: "Mock provider active — set LLM_BASE_URL for your homelab model."}
|
||||||
{messages.length > 0 ? (
|
</p>
|
||||||
<button
|
{modelsDegraded ? (
|
||||||
|
<p className="muted mt-1 text-[11px]">Model discovery unavailable; using fallback.</p>
|
||||||
|
) : null}
|
||||||
|
</div>
|
||||||
|
<div className="flex shrink-0 items-center gap-2">
|
||||||
|
{modelOptions.length > 0 ? (
|
||||||
|
<div className="relative max-w-36">
|
||||||
|
<select
|
||||||
|
aria-label="Assistant model"
|
||||||
|
value={selectedModel}
|
||||||
|
onChange={(event) => changeModel(event.target.value)}
|
||||||
|
disabled={modelsLoading || savingModel || isPending}
|
||||||
|
className="h-9 w-full max-w-36 appearance-none truncate rounded-[min(var(--radius-md),10px)] border border-input bg-[var(--card)] py-0 pr-8 pl-3 text-[13px] leading-9 text-[var(--ink)] outline-none transition-colors focus-visible:border-ring focus-visible:ring-3 focus-visible:ring-ring/50 disabled:cursor-not-allowed disabled:opacity-50"
|
||||||
|
>
|
||||||
|
{modelOptions.map((model) => (
|
||||||
|
<option key={model.id} value={model.id}>
|
||||||
|
{model.label}
|
||||||
|
</option>
|
||||||
|
))}
|
||||||
|
</select>
|
||||||
|
<ChevronDown
|
||||||
|
className="pointer-events-none absolute top-1/2 right-2 size-4 -translate-y-1/2 text-muted-foreground"
|
||||||
|
aria-hidden="true"
|
||||||
|
/>
|
||||||
|
</div>
|
||||||
|
) : null}
|
||||||
|
<Button
|
||||||
type="button"
|
type="button"
|
||||||
onClick={clearChat}
|
size="sm"
|
||||||
disabled={isPending}
|
variant="outline"
|
||||||
className="shrink-0 text-[11px] text-muted-foreground transition-colors hover:text-foreground disabled:opacity-50"
|
aria-label="Refresh assistant models"
|
||||||
|
disabled={modelsLoading || savingModel || isPending}
|
||||||
|
onClick={() => void loadModels({ refresh: true })}
|
||||||
>
|
>
|
||||||
Clear
|
{modelsLoading ? (
|
||||||
</button>
|
<Loader2 className="size-4 animate-spin" />
|
||||||
) : null}
|
) : (
|
||||||
|
<RefreshCw className="size-4" />
|
||||||
|
)}
|
||||||
|
</Button>
|
||||||
|
{messages.length > 0 ? (
|
||||||
|
<button
|
||||||
|
type="button"
|
||||||
|
onClick={clearChat}
|
||||||
|
disabled={isPending}
|
||||||
|
className="shrink-0 text-[11px] text-muted-foreground transition-colors hover:text-foreground disabled:opacity-50"
|
||||||
|
>
|
||||||
|
Clear
|
||||||
|
</button>
|
||||||
|
) : null}
|
||||||
|
</div>
|
||||||
</div>
|
</div>
|
||||||
|
|
||||||
<div
|
<div
|
||||||
|
|||||||
@@ -1,4 +1,5 @@
|
|||||||
import { z } from "zod";
|
import { z } from "zod";
|
||||||
|
import { isValidLlmModelId } from "@/lib/llm/models";
|
||||||
|
|
||||||
export const clientChatAttachmentSchema = z.object({
|
export const clientChatAttachmentSchema = z.object({
|
||||||
type: z.literal("image"),
|
type: z.literal("image"),
|
||||||
@@ -11,8 +12,13 @@ export const clientChatMessageSchema = z.object({
|
|||||||
attachments: z.array(clientChatAttachmentSchema).max(3).optional(),
|
attachments: z.array(clientChatAttachmentSchema).max(3).optional(),
|
||||||
});
|
});
|
||||||
|
|
||||||
|
export const clientChatModelSchema = z
|
||||||
|
.string()
|
||||||
|
.refine((value) => isValidLlmModelId(value), "Invalid assistant model");
|
||||||
|
|
||||||
export const clientChatInputSchema = z.object({
|
export const clientChatInputSchema = z.object({
|
||||||
stream: z.boolean().optional(),
|
stream: z.boolean().optional(),
|
||||||
|
model: clientChatModelSchema.optional(),
|
||||||
messages: z.array(clientChatMessageSchema).min(1).max(40),
|
messages: z.array(clientChatMessageSchema).min(1).max(40),
|
||||||
});
|
});
|
||||||
|
|
||||||
|
|||||||
@@ -45,11 +45,12 @@ export async function runAgentChat(options: {
|
|||||||
messages: ClientChatMessage[];
|
messages: ClientChatMessage[];
|
||||||
request: Request;
|
request: Request;
|
||||||
systemPrompt?: string;
|
systemPrompt?: string;
|
||||||
|
model?: string;
|
||||||
llm?: LlmClient;
|
llm?: LlmClient;
|
||||||
executeTool?: ToolExecutor;
|
executeTool?: ToolExecutor;
|
||||||
onProgress?: AgentProgressHandler;
|
onProgress?: AgentProgressHandler;
|
||||||
}): Promise<AgentChatResult> {
|
}): Promise<AgentChatResult> {
|
||||||
const llm = options.llm ?? createLlmClient();
|
const llm = options.llm ?? createLlmClient({ model: options.model });
|
||||||
const executeTool = options.executeTool ?? createApiToolExecutor(options.request);
|
const executeTool = options.executeTool ?? createApiToolExecutor(options.request);
|
||||||
const onProgress = options.onProgress;
|
const onProgress = options.onProgress;
|
||||||
const systemPrompt = options.systemPrompt ?? AGENT_SYSTEM_PROMPT;
|
const systemPrompt = options.systemPrompt ?? AGENT_SYSTEM_PROMPT;
|
||||||
|
|||||||
@@ -1,4 +1,13 @@
|
|||||||
import { expect, test } from "@playwright/test";
|
import { expect, test, type Page } from "@playwright/test";
|
||||||
|
|
||||||
|
async function ensureSignedIn(page: Page) {
|
||||||
|
await page.goto("/");
|
||||||
|
const devLogin = page.getByRole("button", { name: "Dev login" });
|
||||||
|
if (await devLogin.isVisible().catch(() => false)) {
|
||||||
|
await devLogin.click();
|
||||||
|
await page.waitForURL((url) => !url.pathname.startsWith("/login"));
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
test("assistant bubble is hidden until opted in", async ({ page }) => {
|
test("assistant bubble is hidden until opted in", async ({ page }) => {
|
||||||
await page.goto("/");
|
await page.goto("/");
|
||||||
@@ -6,15 +15,35 @@ test("assistant bubble is hidden until opted in", async ({ page }) => {
|
|||||||
});
|
});
|
||||||
|
|
||||||
test("assistant chat smoke after opt-in", async ({ page }) => {
|
test("assistant chat smoke after opt-in", async ({ page }) => {
|
||||||
|
await ensureSignedIn(page);
|
||||||
await page.goto("/settings?s=appearance");
|
await page.goto("/settings?s=appearance");
|
||||||
const assistantSwitch = page.getByRole("switch", { name: "AI assistant" });
|
const assistantSwitch = page.getByRole("switch", { name: "AI assistant" });
|
||||||
if (!(await assistantSwitch.isChecked())) {
|
if (!(await assistantSwitch.isChecked())) {
|
||||||
await assistantSwitch.click();
|
await assistantSwitch.click();
|
||||||
}
|
}
|
||||||
|
await expect(assistantSwitch).toBeChecked();
|
||||||
|
const routeSelector = page.getByRole("combobox", { name: "Assistant model route" });
|
||||||
|
await expect(routeSelector).toBeVisible();
|
||||||
|
await routeSelector.selectOption("auto");
|
||||||
|
await expect(routeSelector).toHaveValue("auto");
|
||||||
|
await expect(routeSelector).toBeEnabled();
|
||||||
|
await routeSelector.selectOption("uncensored");
|
||||||
|
await expect(routeSelector).toHaveValue("uncensored");
|
||||||
|
await expect(routeSelector).toBeEnabled();
|
||||||
|
|
||||||
await page.goto("/");
|
await page.goto("/");
|
||||||
await page.getByRole("button", { name: "Open assistant" }).click();
|
await page.getByRole("button", { name: "Open assistant" }).click();
|
||||||
await expect(page.getByRole("dialog", { name: "Assistant" })).toBeVisible();
|
await expect(page.getByRole("dialog", { name: "Assistant" })).toBeVisible();
|
||||||
|
await expect(page.getByRole("combobox", { name: "Assistant model route" })).toHaveCount(0);
|
||||||
|
const modelSelector = page.getByRole("combobox", { name: "Assistant model", exact: true });
|
||||||
|
await expect(modelSelector).toBeVisible();
|
||||||
|
const refreshModels = page.getByRole("button", { name: "Refresh assistant models" });
|
||||||
|
await expect(refreshModels).toBeVisible();
|
||||||
|
await refreshModels.click();
|
||||||
|
await expect.poll(() => modelSelector.evaluate((node) => node.tagName)).toBe("SELECT");
|
||||||
|
await expect
|
||||||
|
.poll(() => modelSelector.evaluate((node) => node.getBoundingClientRect().height))
|
||||||
|
.toBeGreaterThanOrEqual(36);
|
||||||
|
|
||||||
await page.getByLabel("Message for Assistant").fill("hello assistant");
|
await page.getByLabel("Message for Assistant").fill("hello assistant");
|
||||||
await page.getByRole("button", { name: "Send" }).click();
|
await page.getByRole("button", { name: "Send" }).click();
|
||||||
|
|||||||
@@ -84,3 +84,36 @@ describe("runAgentChat", () => {
|
|||||||
assert.ok(result.message.content.length > 0);
|
assert.ok(result.message.content.length > 0);
|
||||||
});
|
});
|
||||||
});
|
});
|
||||||
|
|
||||||
|
it("passes a model override to the OpenAI-compatible client", async () => {
|
||||||
|
const originalBaseUrl = process.env.LLM_BASE_URL;
|
||||||
|
const originalModel = process.env.LLM_MODEL;
|
||||||
|
const originalProvider = process.env.LLM_PROVIDER;
|
||||||
|
const originalFetch = globalThis.fetch;
|
||||||
|
let requestBody: unknown = null;
|
||||||
|
|
||||||
|
process.env.LLM_BASE_URL = "https://llm.example.test/v1";
|
||||||
|
process.env.LLM_MODEL = "llama3.2";
|
||||||
|
delete process.env.LLM_PROVIDER;
|
||||||
|
|
||||||
|
globalThis.fetch = (async (_input: RequestInfo | URL, init?: RequestInit) => {
|
||||||
|
requestBody = JSON.parse(String(init?.body));
|
||||||
|
return Response.json({
|
||||||
|
choices: [{ message: { role: "assistant", content: "done" }, finish_reason: "stop" }],
|
||||||
|
});
|
||||||
|
}) as typeof fetch;
|
||||||
|
|
||||||
|
const { createLlmClient } = await import("../../src/lib/llm/index");
|
||||||
|
const client = createLlmClient({ model: "qwen2.5-coder" });
|
||||||
|
await client.chatCompletion({ messages: [{ role: "user", content: "hello" }] });
|
||||||
|
|
||||||
|
assert.equal((requestBody as { model?: string }).model, "qwen2.5-coder");
|
||||||
|
|
||||||
|
globalThis.fetch = originalFetch;
|
||||||
|
if (originalBaseUrl === undefined) delete process.env.LLM_BASE_URL;
|
||||||
|
else process.env.LLM_BASE_URL = originalBaseUrl;
|
||||||
|
if (originalModel === undefined) delete process.env.LLM_MODEL;
|
||||||
|
else process.env.LLM_MODEL = originalModel;
|
||||||
|
if (originalProvider === undefined) delete process.env.LLM_PROVIDER;
|
||||||
|
else process.env.LLM_PROVIDER = originalProvider;
|
||||||
|
});
|
||||||
|
|||||||
@@ -25,4 +25,31 @@ describe("clientChatInputSchema", () => {
|
|||||||
|
|
||||||
assert.equal(parsed.success, false);
|
assert.equal(parsed.success, false);
|
||||||
});
|
});
|
||||||
|
|
||||||
|
it("accepts an optional model ID", () => {
|
||||||
|
const parsed = clientChatInputSchema.safeParse({
|
||||||
|
model: "qwen2.5-coder",
|
||||||
|
messages: [{ role: "user", content: "hello" }],
|
||||||
|
});
|
||||||
|
|
||||||
|
assert.equal(parsed.success, true);
|
||||||
|
});
|
||||||
|
|
||||||
|
it("rejects invalid model IDs", () => {
|
||||||
|
const parsed = clientChatInputSchema.safeParse({
|
||||||
|
model: "bad model",
|
||||||
|
messages: [{ role: "user", content: "hello" }],
|
||||||
|
});
|
||||||
|
|
||||||
|
assert.equal(parsed.success, false);
|
||||||
|
});
|
||||||
|
|
||||||
|
it("rejects whitespace-padded model IDs", () => {
|
||||||
|
const parsed = clientChatInputSchema.safeParse({
|
||||||
|
model: " qwen2.5-coder ",
|
||||||
|
messages: [{ role: "user", content: "hello" }],
|
||||||
|
});
|
||||||
|
|
||||||
|
assert.equal(parsed.success, false);
|
||||||
|
});
|
||||||
});
|
});
|
||||||
|
|||||||
@@ -0,0 +1,17 @@
|
|||||||
|
import assert from "node:assert/strict";
|
||||||
|
import { describe, it } from "node:test";
|
||||||
|
|
||||||
|
describe("GET /api/agent/models", () => {
|
||||||
|
it("is dynamic and returns no-store responses", async () => {
|
||||||
|
process.env.DATABASE_URL ??= "postgres://famapp:famapp@localhost:5432/famapp";
|
||||||
|
|
||||||
|
const { GET, dynamic } = await import("../../src/app/api/agent/models/route");
|
||||||
|
|
||||||
|
assert.equal(dynamic, "force-dynamic");
|
||||||
|
|
||||||
|
const response = await GET(new Request("http://localhost/api/agent/models?refresh=1"));
|
||||||
|
|
||||||
|
assert.equal(response.status, 401);
|
||||||
|
assert.equal(response.headers.get("Cache-Control"), "no-store");
|
||||||
|
});
|
||||||
|
});
|
||||||
@@ -0,0 +1,241 @@
|
|||||||
|
import assert from "node:assert/strict";
|
||||||
|
import { describe, it } from "node:test";
|
||||||
|
import {
|
||||||
|
isValidAssistantModelRoute,
|
||||||
|
isValidLlmModelId,
|
||||||
|
listLlmModels,
|
||||||
|
normalizeLlmModelsPayload,
|
||||||
|
resolveAssistantModel,
|
||||||
|
} from "../../src/lib/llm/models";
|
||||||
|
import type { LlmConfig } from "../../src/lib/llm/config";
|
||||||
|
|
||||||
|
const openAiConfig: LlmConfig = {
|
||||||
|
provider: "openai",
|
||||||
|
baseUrl: "https://llm.example.test/v1",
|
||||||
|
apiKey: "secret",
|
||||||
|
model: "llama3.2",
|
||||||
|
};
|
||||||
|
|
||||||
|
describe("normalizeLlmModelsPayload", () => {
|
||||||
|
it("normalizes OpenAI-compatible data arrays", () => {
|
||||||
|
const models = normalizeLlmModelsPayload({
|
||||||
|
data: [{ id: "qwen2.5-coder" }, { id: "llama3.2" }, { id: "qwen2.5-coder" }],
|
||||||
|
});
|
||||||
|
|
||||||
|
assert.deepEqual(models, [
|
||||||
|
{ id: "llama3.2", label: "llama3.2" },
|
||||||
|
{ id: "qwen2.5-coder", label: "qwen2.5-coder" },
|
||||||
|
]);
|
||||||
|
});
|
||||||
|
|
||||||
|
it("ignores invalid or empty model rows", () => {
|
||||||
|
const models = normalizeLlmModelsPayload({
|
||||||
|
data: [
|
||||||
|
{ id: "" },
|
||||||
|
{ id: " " },
|
||||||
|
{ id: "bad model" },
|
||||||
|
{ id: " llama3.2 " },
|
||||||
|
{ object: "model" },
|
||||||
|
],
|
||||||
|
});
|
||||||
|
|
||||||
|
assert.deepEqual(models, []);
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
|
describe("isValidLlmModelId", () => {
|
||||||
|
it("accepts common provider model IDs", () => {
|
||||||
|
assert.equal(isValidLlmModelId("llama3.2"), true);
|
||||||
|
assert.equal(isValidLlmModelId("qwen2.5-coder:latest"), true);
|
||||||
|
assert.equal(isValidLlmModelId("hf.co/ginnoir/model-v1"), true);
|
||||||
|
});
|
||||||
|
|
||||||
|
it("rejects empty, whitespace, and overlong model IDs", () => {
|
||||||
|
assert.equal(isValidLlmModelId(""), false);
|
||||||
|
assert.equal(isValidLlmModelId("bad model"), false);
|
||||||
|
assert.equal(isValidLlmModelId("x".repeat(129)), false);
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
|
describe("isValidAssistantModelRoute", () => {
|
||||||
|
it("accepts the supported route families", () => {
|
||||||
|
assert.equal(isValidAssistantModelRoute("auto"), true);
|
||||||
|
assert.equal(isValidAssistantModelRoute("uncensored"), true);
|
||||||
|
});
|
||||||
|
|
||||||
|
it("rejects unsupported or padded route families", () => {
|
||||||
|
assert.equal(isValidAssistantModelRoute("bogus"), false);
|
||||||
|
assert.equal(isValidAssistantModelRoute(" uncensored "), false);
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
|
describe("listLlmModels", () => {
|
||||||
|
it("fetches provider models with API key auth when the fallback is advertised", async () => {
|
||||||
|
const requests: Request[] = [];
|
||||||
|
const result = await listLlmModels({
|
||||||
|
config: openAiConfig,
|
||||||
|
fetchImpl: async (input, init) => {
|
||||||
|
requests.push(new Request(input, init));
|
||||||
|
return Response.json({ data: [{ id: "qwen2.5-coder" }, { id: "llama3.2" }] });
|
||||||
|
},
|
||||||
|
});
|
||||||
|
|
||||||
|
assert.equal(requests[0]?.url, "https://llm.example.test/v1/models");
|
||||||
|
assert.equal(requests[0]?.headers.get("authorization"), "Bearer secret");
|
||||||
|
assert.deepEqual(result.models, [
|
||||||
|
{ id: "llama3.2", label: "llama3.2" },
|
||||||
|
{ id: "qwen2.5-coder", label: "qwen2.5-coder" },
|
||||||
|
]);
|
||||||
|
assert.equal(result.fallbackModel, "llama3.2");
|
||||||
|
assert.equal(result.degraded, false);
|
||||||
|
});
|
||||||
|
|
||||||
|
it("falls back to LLM_MODEL when provider discovery fails", async () => {
|
||||||
|
const result = await listLlmModels({
|
||||||
|
config: openAiConfig,
|
||||||
|
fetchImpl: async () => new Response("nope", { status: 500 }),
|
||||||
|
});
|
||||||
|
|
||||||
|
assert.deepEqual(result.models, [{ id: "llama3.2", label: "llama3.2" }]);
|
||||||
|
assert.equal(result.fallbackModel, "llama3.2");
|
||||||
|
assert.equal(result.degraded, true);
|
||||||
|
});
|
||||||
|
|
||||||
|
it("fetches the typed uncensored catalog when uncensored is the selected route", async () => {
|
||||||
|
const requests: Request[] = [];
|
||||||
|
const result = await listLlmModels({
|
||||||
|
config: { ...openAiConfig, model: "auto" },
|
||||||
|
route: "uncensored",
|
||||||
|
fetchImpl: async (input, init) => {
|
||||||
|
requests.push(new Request(input, init));
|
||||||
|
return Response.json({
|
||||||
|
data: [
|
||||||
|
{ id: "uncensored" },
|
||||||
|
{ id: "gemma4-uncensored:26b" },
|
||||||
|
{ id: "dolphin-mistral:latest" },
|
||||||
|
],
|
||||||
|
});
|
||||||
|
},
|
||||||
|
});
|
||||||
|
|
||||||
|
assert.equal(requests[0]?.url, "https://llm.example.test/v1/models?type=uncensored");
|
||||||
|
assert.deepEqual(result.models, [
|
||||||
|
{ id: "dolphin-mistral:latest", label: "dolphin-mistral:latest" },
|
||||||
|
{ id: "gemma4-uncensored:26b", label: "gemma4-uncensored:26b" },
|
||||||
|
{ id: "uncensored", label: "uncensored" },
|
||||||
|
]);
|
||||||
|
assert.equal(result.fallbackModel, "uncensored");
|
||||||
|
assert.equal(result.route, "uncensored");
|
||||||
|
assert.equal(result.degraded, false);
|
||||||
|
});
|
||||||
|
|
||||||
|
it("fetches the default catalog when auto is selected over an uncensored deployment default", async () => {
|
||||||
|
const requests: Request[] = [];
|
||||||
|
const result = await listLlmModels({
|
||||||
|
config: { ...openAiConfig, model: "uncensored" },
|
||||||
|
route: "auto",
|
||||||
|
fetchImpl: async (input, init) => {
|
||||||
|
requests.push(new Request(input, init));
|
||||||
|
return Response.json({
|
||||||
|
data: [{ id: "auto" }, { id: "qwen3:8b" }],
|
||||||
|
});
|
||||||
|
},
|
||||||
|
});
|
||||||
|
|
||||||
|
assert.equal(requests[0]?.url, "https://llm.example.test/v1/models");
|
||||||
|
assert.deepEqual(result.models, [
|
||||||
|
{ id: "auto", label: "auto" },
|
||||||
|
{ id: "qwen3:8b", label: "qwen3:8b" },
|
||||||
|
]);
|
||||||
|
assert.equal(result.fallbackModel, "auto");
|
||||||
|
assert.equal(result.route, "auto");
|
||||||
|
assert.equal(result.degraded, false);
|
||||||
|
});
|
||||||
|
|
||||||
|
it("uses fallback only for mock provider config", async () => {
|
||||||
|
const result = await listLlmModels({
|
||||||
|
config: { provider: "mock", baseUrl: null, apiKey: null, model: "llama3.2" },
|
||||||
|
fetchImpl: async () => {
|
||||||
|
throw new Error("fetch should not run for mock config");
|
||||||
|
},
|
||||||
|
});
|
||||||
|
|
||||||
|
assert.deepEqual(result.models, [{ id: "llama3.2", label: "llama3.2" }]);
|
||||||
|
assert.equal(result.degraded, false);
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
|
describe("resolveAssistantModel", () => {
|
||||||
|
it("uses a valid requested model before saved and fallback values", () => {
|
||||||
|
const resolved = resolveAssistantModel({
|
||||||
|
requestedModel: "qwen2.5-coder",
|
||||||
|
savedModel: "llama3.2",
|
||||||
|
fallbackModel: "llama3.2",
|
||||||
|
models: [
|
||||||
|
{ id: "llama3.2", label: "llama3.2" },
|
||||||
|
{ id: "qwen2.5-coder", label: "qwen2.5-coder" },
|
||||||
|
],
|
||||||
|
});
|
||||||
|
|
||||||
|
assert.deepEqual(resolved, { ok: true, model: "qwen2.5-coder" });
|
||||||
|
});
|
||||||
|
|
||||||
|
it("rejects invalid requested models", () => {
|
||||||
|
const resolved = resolveAssistantModel({
|
||||||
|
requestedModel: "bad model",
|
||||||
|
savedModel: null,
|
||||||
|
fallbackModel: "llama3.2",
|
||||||
|
models: [
|
||||||
|
{ id: "llama3.2", label: "llama3.2" },
|
||||||
|
{ id: "bad model", label: "bad model" },
|
||||||
|
],
|
||||||
|
});
|
||||||
|
|
||||||
|
assert.deepEqual(resolved, {
|
||||||
|
ok: false,
|
||||||
|
model: "llama3.2",
|
||||||
|
error: "Invalid assistant model",
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
|
it("rejects empty requested models", () => {
|
||||||
|
const resolved = resolveAssistantModel({
|
||||||
|
requestedModel: "",
|
||||||
|
savedModel: null,
|
||||||
|
fallbackModel: "llama3.2",
|
||||||
|
models: [{ id: "llama3.2", label: "llama3.2" }],
|
||||||
|
});
|
||||||
|
|
||||||
|
assert.deepEqual(resolved, {
|
||||||
|
ok: false,
|
||||||
|
model: "llama3.2",
|
||||||
|
error: "Invalid assistant model",
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
|
it("rejects unavailable requested models", () => {
|
||||||
|
const resolved = resolveAssistantModel({
|
||||||
|
requestedModel: "missing",
|
||||||
|
savedModel: null,
|
||||||
|
fallbackModel: "llama3.2",
|
||||||
|
models: [{ id: "llama3.2", label: "llama3.2" }],
|
||||||
|
});
|
||||||
|
|
||||||
|
assert.deepEqual(resolved, {
|
||||||
|
ok: false,
|
||||||
|
model: "llama3.2",
|
||||||
|
error: "Invalid assistant model",
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
|
it("silently falls back when a saved model is gone", () => {
|
||||||
|
const resolved = resolveAssistantModel({
|
||||||
|
requestedModel: null,
|
||||||
|
savedModel: "old-model",
|
||||||
|
fallbackModel: "llama3.2",
|
||||||
|
models: [{ id: "llama3.2", label: "llama3.2" }],
|
||||||
|
});
|
||||||
|
|
||||||
|
assert.deepEqual(resolved, { ok: true, model: "llama3.2" });
|
||||||
|
});
|
||||||
|
});
|
||||||
Reference in New Issue
Block a user