Add ComfyUI image generation (/v1/images/generations)
OpenAI-compatible image endpoint that proxies to a ComfyUI server: fills a
workflow template with per-request params (prompt/size/n/seed/steps/checkpoint),
submits to ComfyUI, polls history, fetches images, returns OpenAI image shape.
- router.py: _handle_image + fill_workflow + route; env ROUTER_COMFYUI /
ROUTER_COMFY_CHECKPOINT / ROUTER_COMFY_WORKFLOW
- comfyui-workflow.json: default SD/SDXL text2img template with {{SENTINELS}}
- docs/IMAGES.md + README: config, usage, custom workflows
- verified: templating + clean error when ComfyUI is down; end-to-end pending
a live ComfyUI
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
co-authored by
Claude Opus 4.8
parent
5bd8142d30
commit
2f95e6be96
@@ -0,0 +1,62 @@
|
||||
# Image generation (ComfyUI)
|
||||
|
||||
The router exposes an OpenAI-compatible **`POST /v1/images/generations`** endpoint
|
||||
backed by a [ComfyUI](https://github.com/comfyanonymous/ComfyUI) server. It submits
|
||||
a workflow to ComfyUI, polls until it finishes, fetches the images, and returns them
|
||||
in OpenAI's image-response shape — so any OpenAI image client works.
|
||||
|
||||
## Configure
|
||||
|
||||
| Env var | Default | Purpose |
|
||||
|---|---|---|
|
||||
| `ROUTER_COMFYUI` | `http://127.0.0.1:8188` | ComfyUI server URL |
|
||||
| `ROUTER_COMFY_CHECKPOINT` | *(unset)* | your checkpoint filename, e.g. `sd_xl_base_1.0.safetensors` |
|
||||
| `ROUTER_COMFY_WORKFLOW` | `comfyui-workflow.json` | the workflow template (see below) |
|
||||
|
||||
At minimum, **set `ROUTER_COMFY_CHECKPOINT`** to a model that exists in your
|
||||
ComfyUI (`ComfyUI/models/checkpoints/`), or hardcode it into the template's
|
||||
`ckpt_name`. Then restart the router.
|
||||
|
||||
## Use
|
||||
|
||||
```bash
|
||||
curl http://<host>:8080/v1/images/generations -H 'content-type: application/json' -d '{
|
||||
"prompt": "a red fox in a snowy forest, cinematic lighting",
|
||||
"n": 1, "size": "1024x1024"
|
||||
}'
|
||||
# -> {"created": ..., "data": [{"b64_json": "<png>"}]}
|
||||
```
|
||||
- `response_format`: `b64_json` (default) or `url` (returns a `data:` URL).
|
||||
- Extra (non-OpenAI) knobs the router honors: `negative_prompt`, `steps`.
|
||||
- Works with the OpenAI SDK: `images.generate(prompt=..., size=..., n=...)`.
|
||||
|
||||
## The workflow template
|
||||
|
||||
`comfyui-workflow.json` is a ComfyUI **API-format** workflow with string sentinels
|
||||
the router substitutes per request:
|
||||
|
||||
| Sentinel | Filled with |
|
||||
|---|---|
|
||||
| `{{PROMPT}}` | the request prompt |
|
||||
| `{{NEGATIVE}}` | `negative_prompt` |
|
||||
| `{{WIDTH}}` / `{{HEIGHT}}` | parsed from `size` |
|
||||
| `{{BATCH}}` | `n` |
|
||||
| `{{SEED}}` | a fresh seed |
|
||||
| `{{STEPS}}` | `steps` (default 25) |
|
||||
| `{{CHECKPOINT}}` | `ROUTER_COMFY_CHECKPOINT` |
|
||||
|
||||
The default is a standard SD/SDXL text-to-image graph. **To use your own** (Flux,
|
||||
a fancier pipeline, LoRAs, upscalers, etc.): build it in ComfyUI, export via
|
||||
**Save (API Format)**, then replace the values you want driven by requests with the
|
||||
sentinels above. Point `ROUTER_COMFY_WORKFLOW` at your file. Anything without a
|
||||
sentinel stays fixed.
|
||||
|
||||
## Notes
|
||||
|
||||
- **Not yet verified end-to-end** — it was written before ComfyUI was installed.
|
||||
Once your ComfyUI is up (`ROUTER_COMFYUI` reachable, checkpoint set), test with
|
||||
the curl above; if a node name/shape differs, adjust the template.
|
||||
- Generation can take a while; the router polls up to ~10 min per request.
|
||||
- Video: ComfyUI video workflows (AnimateDiff, SVD, etc.) also export to API format
|
||||
and return frames/files — the same endpoint can drive them, but the response
|
||||
mapping for multi-frame/video output may need a tweak. Ask when you get there.
|
||||
Reference in New Issue
Block a user