Providers & models

Supported providers

The API bases are built in; you normally only supply a key. All listed endpoints (except xai_oauth) speak the OpenAI Chat-Completions wire protocol. The xai_oauth provider uses xAI's Responses API instead.

Configuring providers (BYOK)

BYOK keys go under providers.<name> in ~/.flowly/config.json. On-disk keys are camelCase:

{
  "providers": {
    "openrouter": {
      "apiKey": "sk-or-...",
      "apiBase": "https://openrouter.ai/api/v1",
      "fallbackKeys": ["sk-or-...", "sk-or-..."]
    }
  }
}

You can enter a BYOK key via the setup wizard:

flowly setup byok <slot> --key <...>

Flowly hosted

The hosted provider has no API key. Sign in with your account and Flowly uses an account-derived bearer token (serverId:gatewayAuthToken):

flowly login

Login uses a device-code flow (a one-click browser URL plus polling) and stores your account in the keychain or ~/.flowly/credentials/account.json (mode 0600). After a fresh login Flowly registers the machine, wires the relay channel, and auto-selects providers.active = "flowly" only if nothing is set yet. The hosted provider is gated on providers.flowly.enabled (default true) and a usable account.

flowly login --repair            # re-register + re-wire without a browser
flowly login --repair --dry-run
flowly logout                    # clears account; preserves BYOK keys

xAI / Grok OAuth

For a Grok subscription, authenticate with xAI's OAuth (PKCE) flow:

flowly xai login          # sets active provider + default Grok model
flowly xai status
flowly xai logout
flowly xai test           # hits /v1/models

Tokens are stored in the keychain or ~/.flowly/credentials/xai_oauth.json (mode 0600), not in config.json. The client id is fixed (xAI has no self-service client registration). Use flowly xai login --no-set-active to authenticate without switching the active provider.

Switching providers and models at runtime

The active provider is resolved in this priority order:

1. providers.active, if that provider is currently usable (sticky; falls through if not).
2. flowly hosted, if enabled and signed in.
3. The BYOK cascade — first usable of openrouter, anthropic, openai, xai, xai_oauth, gemini, groq, zhipu, sakana, vllm.

Switch live from the TUI:

/provider [name]   # write providers.active, then hot-reload the gateway
/model [id]        # write agents.defaults.model, then hot-reload

Both open a picker if you omit the argument. /model's picker loads the live catalog for the active provider.

Hot-reload

/provider and /model write config and tell the running gateway to reload — no restart. The gateway re-reads config, re-resolves the active provider, and builds the new provider before swapping, so a build error (for example an empty key) leaves the old provider in place. If the gateway is offline, the TUI reports "gateway offline — restart to apply".

Choosing the model

The model is chosen during flowly setup and changed later via /model (or /provider to switch providers). It is stored as agents.defaults.model in config.json:

{
  "agents": {
    "defaults": {
      "model": "openrouter/some-model-id"
    }
  }
}

Key rotation

When a provider slot has more than one key (apiKey plus fallbackKeys), Flowly creates a key rotator:

• On an auth/rate-limit/overload failure, the current key is marked failed with a 60-second cooldown and the next available key is picked round-robin. If every key is in cooldown, the one expiring soonest is used rather than failing outright.
• Rotation only happens with more than one key — a single key never rotates.
• Rotation does not happen during streaming. Streaming picks a key once; on a stream-open failure it yields an error without rotating. Only non-streaming calls rotate.
• xai_oauth has no rotator — it does a single token refresh and one retry on HTTP 401.
• Flowly hosted does not use fallback keys — it uses a single refreshable account token.

Prompt caching

• Strategy: up to 4 cache breakpoints — one on the system prompt plus up to 3 on the most recent non-system, non-tool messages.

• TTL: default 1h. Supported values are 5m and 1h. Override at process start:

  Copy

  FLOWLY_CLAUDE_CACHE_TTL=5m flowly ...

  Invalid values fall back to 1h.

Model catalog (live vs empty)

Flowly builds the model picker from a live catalog — each provider's own /models endpoint where it has one, or the models.dev community catalogue otherwise (cached locally, served stale on network failure). Providers differ in whether a catalog is available:

Related

• Sandbox & exec approvals
• Codex runtime
• Channels overview
• CLI commands
• Slash commands
• Environment variables
• Setup wizard