Providers & models

Supported providers

The API bases are built in; you normally only supply a key. All listed endpoints (except xai_oauth and openai_codex) speak the OpenAI Chat-Completions wire protocol. xai_oauth and openai_codex both use a Responses-API wire format instead.

Configuring providers (BYOK)

BYOK keys go under providers.<name> in ~/.flowly/config.json. On-disk keys are camelCase:

{
  "providers": {
    "openrouter": {
      "apiKey": "sk-or-...",
      "apiBase": "https://openrouter.ai/api/v1",
      "fallbackKeys": ["sk-or-...", "sk-or-..."]
    }
  }
}

You can enter a BYOK key via the setup wizard:

flowly setup byok <slot> --key <...>

Flowly hosted

The hosted provider has no API key. Sign in with your account and Flowly uses an account-derived bearer token (serverId:gatewayAuthToken):

flowly login

Login uses a device-code flow (a one-click browser URL plus polling) and stores your account in the keychain or ~/.flowly/credentials/account.json (mode 0600). After a fresh login Flowly registers the machine, wires the relay channel, and auto-selects providers.active = "flowly" only if nothing is set yet. The hosted provider is gated on providers.flowly.enabled (default true) and a usable account.

flowly login --repair            # re-register + re-wire without a browser
flowly login --repair --dry-run
flowly logout                    # clears account; preserves BYOK keys

xAI / Grok OAuth

For a Grok subscription, authenticate with xAI's OAuth (PKCE) flow:

flowly xai login          # sets active provider + default Grok model
flowly xai status
flowly xai logout
flowly xai test           # hits /v1/models

Tokens are stored in the keychain or ~/.flowly/credentials/xai_oauth.json (mode 0600), not in config.json. The client id is fixed (xAI has no self-service client registration). Use flowly xai login --no-set-active to authenticate without switching the active provider.

ChatGPT subscription (Codex OAuth)

For a ChatGPT Plus / Pro / Team plan, authenticate with OpenAI's Codex "Sign in with ChatGPT" OAuth (PKCE) flow — no API key, usage is billed against your plan:

flowly codex login          # sets active provider + default model (gpt-5.6-sol)
flowly codex login --device # headless / no-browser: prints a code to enter at auth.openai.com/codex/device
flowly codex status         # shows both codex_session tool AND ChatGPT subscription state
flowly codex logout

Tokens are stored in the keychain or ~/.flowly/credentials/openai_codex.json (mode 0600), not in config.json. The client id is fixed (the public Codex CLI client — OpenAI has no self-service client registration for this OAuth scope).

Switching providers and models at runtime

The active provider is resolved in this priority order:

1. providers.active, if that provider is currently usable (sticky; falls through if not).
2. flowly hosted, if enabled and signed in.
3. The BYOK cascade — first usable of openrouter, anthropic, openai, openai_codex, zai_coding, xai, xai_oauth, gemini, groq, zhipu, sakana, vllm.

Because openai_codex and zai_coding can read a login you made in another tool (the Codex CLI, OpenCode), the cascade may find a provider you never configured in Flowly. flowly setup surfaces that as a choice rather than using it silently, and picking anything writes providers.active — after which the cascade no longer applies.

Switch live from the TUI:

/provider [name]   # write providers.active, then hot-reload the gateway
/model [id]        # write agents.defaults.model, then hot-reload

Both open a picker if you omit the argument. /model's picker loads the live catalog for the active provider.

When Flowly changes the model for you

Only in one situation: switching providers. If the provider you switch to can't serve the model you're on (Anthropic can't serve an OpenRouter-only id, say), Flowly moves you to that provider's curated default in the same write and tells you — model → claude-haiku-4-5 — rather than letting your next message fail.

Your choice is never rewritten in the background otherwise. A model that a catalogue lookup can't confirm stays exactly as you set it; if a plan really can't serve it, the request says so, which is information you can act on.

Hot-reload

/provider and /model write config and tell the running gateway to reload — no restart. The gateway re-reads config, re-resolves the active provider, and builds the new provider before swapping, so a build error (for example an empty key) leaves the old provider in place. If the gateway is offline, the TUI reports "gateway offline — restart to apply".

Choosing the model

Each provider ships with a sensible default model, so the first run never asks you to pick one — you can start chatting and change it whenever you have an opinion. Choose one with /model in the chat, or by taking the Full path in flowly setup. It is stored as agents.defaults.model in config.json:

{
  "agents": {
    "defaults": {
      "model": "openrouter/some-model-id"
    }
  }
}

Key rotation

When a provider slot has more than one key (apiKey plus fallbackKeys), Flowly creates a key rotator:

• On an auth/rate-limit/overload failure, the current key is marked failed with a 60-second cooldown and the next available key is picked round-robin. If every key is in cooldown, the one expiring soonest is used rather than failing outright.
• Rotation only happens with more than one key — a single key never rotates.
• Rotation does not happen during streaming. Streaming picks a key once; on a stream-open failure it yields an error without rotating. Only non-streaming calls rotate.
• xai_oauth and openai_codex have no rotator — each does a single token refresh and one retry on HTTP 401. openai_codex treats a 403 as a plan-entitlement error (not authenticated to use Codex) and doesn't retry it.
• Flowly hosted does not use fallback keys — it uses a single refreshable account token.

Prompt caching

• Strategy: up to 4 cache breakpoints — one on the system prompt plus up to 3 on the most recent non-system, non-tool messages.

• TTL: default 1h. Supported values are 5m and 1h. Override at process start:

  Copy

  FLOWLY_CLAUDE_CACHE_TTL=5m flowly ...

  Invalid values fall back to 1h.

Model catalog (live vs empty)

Flowly builds the model picker from a live catalog — each provider's own /models endpoint where it has one, or the models.dev community catalogue otherwise (cached locally, served stale on network failure). Providers differ in whether a catalog is available:

Related

• Sandbox & exec approvals
• Codex runtime
• Channels overview
• CLI commands
• Slash commands
• Environment variables
• Setup wizard