llm-router/docs/AUDIO.md

Audio / voice

Ollama can understand audio with speech-capable models (e.g. Gemma 4 e4b/e2b/ 12b), but only via its native /api/chat with the audio base64 in the images field — the OpenAI input_audio format is silently dropped, and there's no native speech-to-text endpoint. This router bridges that gap.

Set the model with ROUTER_AUDIO_MODEL (default gemma4:e4b). It must be a model whose ollama show capabilities include audio. Note: some fine-tunes/quants have broken audio even if the flag is set — test before relying on one.

Speech-to-text (Whisper-shaped)

curl http://<host>:8080/v1/audio/transcriptions \
  -F file=@recording.wav \
  -F model=whisper-1
# -> {"text": "the transcription"}

• response_format=text returns plain text instead of JSON.
• /v1/audio/translations does the same but translates to English.
• Works with the OpenAI SDK: set base_url to the router and call audio.transcriptions.create(...). The model field is ignored — the router always uses ROUTER_AUDIO_MODEL.

Audio inside a chat

Send OpenAI input_audio content to /v1/chat/completions and ask about it:

{"model":"auto","messages":[{"role":"user","content":[
  {"type":"text","text":"answer the question in this audio"},
  {"type":"input_audio","input_audio":{"data":"<base64-wav>","format":"wav"}}
]}]}

The router detects the audio, translates it to Ollama's native call, runs it on ROUTER_AUDIO_MODEL, and returns a normal OpenAI response (streaming supported).

Limitations

• WAV is verified. Other formats (mp3/m4a/ogg) depend on Ollama's decoding — transcode to WAV if they fail.
• Not streaming STT — it transcribes a complete clip per request (like Whisper's file API), not a live mic stream. Perfect for record-then-send.
• Audio only works through the router's native bridge, so audio requests always target ROUTER_AUDIO_MODEL (they bypass the normal model-selection map).