# Providers & models

> Flowly talks to LLMs through pluggable providers — the hosted Flowly proxy or your own keys (BYOK) for OpenAI-compatible and native providers. This page covers the supported providers, configuration, runtime switching, key rotation, prompt caching, and the model catalog.

Source: https://useflowlyapp.com/en/docs/using-flowly/providers-and-models
Language: en

## Supported providers

| Provider key | Auth | Canonical API base |
| --- | --- | --- |
| `flowly` | Hosted — account token (`serverId:gatewayAuthToken`) | `https://useflowlyapp.com/api/v1` |
| `openrouter` | BYOK `api_key` | `https://openrouter.ai/api/v1` |
| `anthropic` | BYOK `api_key` | `https://api.anthropic.com/v1` |
| `openai` | BYOK `api_key` | `https://api.openai.com/v1` |
| `openai_codex` | OAuth (`flowly codex login`) | `https://chatgpt.com/backend-api/codex` |
| `xai` | BYOK `api_key` | `https://api.x.ai/v1` |
| `xai_oauth` | OAuth (`flowly xai login`) | `https://api.x.ai/v1` |
| `gemini` | BYOK `api_key` | `https://generativelanguage.googleapis.com/v1beta/openai` |
| `groq` | BYOK `api_key` | `https://api.groq.com/openai/v1` |
| `zhipu` | BYOK `api_key` | `https://open.bigmodel.cn/api/paas/v4` |
| `sakana` | BYOK `api_key` | `https://api.sakana.ai/v1` (Fugu / Fugu Ultra, OpenAI-compat) |
| `vllm` | BYOK `api_key` | none (self-hosted — you must set `apiBase`) |

The API bases are built in; you normally only supply a key. All listed endpoints (except `xai_oauth` and `openai_codex`) speak the OpenAI Chat-Completions wire protocol. `xai_oauth` and `openai_codex` both use a Responses-API wire format instead.

> [!NOTE]
> On direct `anthropic` BYOK: the canonical base is Anthropic's native API, which is not OpenAI-Chat-Completions-shaped, so direct Anthropic BYOK is a documented-but-questionable path. To run Claude reliably, route it through OpenRouter or the Flowly hosted proxy.

## Configuring providers (BYOK)

BYOK keys go under `providers.<name>` in `~/.flowly/config.json`. On-disk keys are camelCase:

```json
{
  "providers": {
    "openrouter": {
      "apiKey": "sk-or-...",
      "apiBase": "https://openrouter.ai/api/v1",
      "fallbackKeys": ["sk-or-...", "sk-or-..."]
    }
  }
}
```

| Field | Purpose |
| --- | --- |
| `apiKey` | The provider key (whitespace-stripped). |
| `apiBase` | Overrides the built-in base if set (required for `vllm`). |
| `fallbackKeys` | Extra keys for rotation (see below). |

You can enter a BYOK key via the setup wizard:

```bash
flowly setup byok <slot> --key <...>
```

## Flowly hosted

The hosted provider has no API key. Sign in with your account and Flowly uses an account-derived bearer token (`serverId:gatewayAuthToken`):

```bash
flowly login
```

Login uses a device-code flow (a one-click browser URL plus polling) and stores your account in the keychain or `~/.flowly/credentials/account.json` (mode `0600`). After a fresh login Flowly registers the machine, wires the relay channel, and auto-selects `providers.active = "flowly"` **only if nothing is set yet**. The hosted provider is gated on `providers.flowly.enabled` (default true) and a usable account.

```bash
flowly login --repair            # re-register + re-wire without a browser
flowly login --repair --dry-run
flowly logout                    # clears account; preserves BYOK keys
```

## xAI / Grok OAuth

For a Grok subscription, authenticate with xAI's OAuth (PKCE) flow:

```bash
flowly xai login          # sets active provider + default Grok model
flowly xai status
flowly xai logout
flowly xai test           # hits /v1/models
```

Tokens are stored in the keychain or `~/.flowly/credentials/xai_oauth.json` (mode `0600`), not in `config.json`. The client id is fixed (xAI has no self-service client registration). Use `flowly xai login --no-set-active` to authenticate without switching the active provider.

## ChatGPT subscription (Codex OAuth)

For a ChatGPT Plus / Pro / Team plan, authenticate with OpenAI's Codex "Sign in with ChatGPT" OAuth (PKCE) flow — no API key, usage is billed against your plan:

```bash
flowly codex login          # sets active provider + default model (gpt-5.6-sol)
flowly codex login --device # headless / no-browser: prints a code to enter at auth.openai.com/codex/device
flowly codex status         # shows both codex_session tool AND ChatGPT subscription state
flowly codex logout
```

Tokens are stored in the keychain or `~/.flowly/credentials/openai_codex.json` (mode `0600`), not in `config.json`. The client id is fixed (the public Codex CLI client — OpenAI has no self-service client registration for this OAuth scope).

> [!TIP]
> If you've already run `codex login` for the [Codex runtime](https://useflowlyapp.com/en/docs/features/codex-runtime) tool, Flowly picks up `~/.codex/auth.json` automatically as a fallback — no separate sign-in needed. Flowly's own store (if you've run `flowly codex login`) always takes priority when both exist, and Flowly writes refreshed tokens back to `~/.codex/auth.json` too, so the Codex CLI keeps working.

> [!NOTE]
> The `openai_codex` **provider** (this section) and the `codex_session` **tool** ([Codex runtime](https://useflowlyapp.com/en/docs/features/codex-runtime)) are unrelated features that happen to share the "Codex" name and the `flowly codex` CLI namespace. The provider makes Flowly's *own* agent loop run on GPT-5.x via your ChatGPT plan. The tool *delegates* a coding turn to a separate `codex app-server` subprocess. You can use either, both, or neither.

> [!NOTE]
> The `openai_codex` picker reads the signed-in account's live ChatGPT Codex model catalogue. The current default is `gpt-5.6-sol`; the catalogue also exposes other available GPT-5.x variants for the account. See [Environment variables](https://useflowlyapp.com/en/docs/reference/environment-variables) to override the default model or the system instructions sent as `instructions`.

## Switching providers and models at runtime

The active provider is resolved in this priority order:

1. `providers.active`, if that provider is currently usable (sticky; falls through if not).
2. `flowly` hosted, if enabled and signed in.
3. The BYOK cascade — first usable of `openrouter`, `anthropic`, `openai`, `openai_codex`, `zai_coding`, `xai`, `xai_oauth`, `gemini`, `groq`, `zhipu`, `sakana`, `vllm`.

Because `openai_codex` and `zai_coding` can read a login you made in another tool (the Codex CLI, OpenCode), the cascade may find a provider you never configured in Flowly. `flowly setup` surfaces that as a choice rather than using it silently, and picking anything writes `providers.active` — after which the cascade no longer applies.

Switch live from the TUI:

```text
/provider [name]   # write providers.active, then hot-reload the gateway
/model [id]        # write agents.defaults.model, then hot-reload
```

Both open a picker if you omit the argument. `/model`'s picker loads the live catalog for the active provider.

### When Flowly changes the model for you

Only in one situation: **switching providers**. If the provider you switch to
can't serve the model you're on (Anthropic can't serve an OpenRouter-only id,
say), Flowly moves you to that provider's curated default in the same write
and tells you — `model → claude-haiku-4-5` — rather than letting your next
message fail.

Your choice is never rewritten in the background otherwise. A model that a
catalogue lookup can't confirm stays exactly as you set it; if a plan really
can't serve it, the request says so, which is information you can act on.

### Hot-reload

`/provider` and `/model` write config and tell the running gateway to reload — no restart. The gateway re-reads config, re-resolves the active provider, and **builds the new provider before swapping**, so a build error (for example an empty key) leaves the old provider in place. If the gateway is offline, the TUI reports "gateway offline — restart to apply".

### Choosing the model

Each provider ships with a sensible default model, so the first run never asks you to pick one — you can start chatting and change it whenever you have an opinion. Choose one with `/model` in the chat, or by taking the **Full** path in `flowly setup`. It is stored as `agents.defaults.model` in `config.json`:

```json
{
  "agents": {
    "defaults": {
      "model": "openrouter/some-model-id"
    }
  }
}
```

> [!TIP]
> Set the model interactively with `/model <id>` rather than hand-editing when possible, so the picker can validate against the live catalog.

## Key rotation

When a provider slot has **more than one key** (`apiKey` plus `fallbackKeys`), Flowly creates a key rotator:

- On an auth/rate-limit/overload failure, the current key is marked failed with a **60-second cooldown** and the next available key is picked round-robin. If every key is in cooldown, the one expiring soonest is used rather than failing outright.
- Rotation only happens with more than one key — a single key never rotates.
- **Rotation does not happen during streaming.** Streaming picks a key once; on a stream-open failure it yields an error without rotating. Only non-streaming calls rotate.
- **`xai_oauth` and `openai_codex` have no rotator** — each does a single token refresh and one retry on HTTP 401. `openai_codex` treats a 403 as a plan-entitlement error (not authenticated to use Codex) and doesn't retry it.
- **Flowly hosted does not use fallback keys** — it uses a single refreshable account token.

## Prompt caching

> [!NOTE]
> Prompt caching is **Anthropic / Claude only** — it is applied solely to models whose id contains `claude`. Other providers and models are unaffected (the xAI OAuth and ChatGPT subscription providers never cache this way).

- **Strategy:** up to 4 cache breakpoints — one on the system prompt plus up to 3 on the most recent non-system, non-tool messages.
- **TTL:** default **`1h`**. Supported values are `5m` and `1h`. Override at process start:

  ```bash
  FLOWLY_CLAUDE_CACHE_TTL=5m flowly ...
  ```

  Invalid values fall back to `1h`.

## Model catalog (live vs empty)

Flowly builds the model picker from a live catalog — each provider's own `/models` endpoint where it has one, or the [models.dev](https://models.dev) community catalogue otherwise (cached locally, served stale on network failure). Providers differ in whether a catalog is available:

| Provider | Catalog source |
| --- | --- |
| `openrouter` | Live `GET /models` (public, filtered to tool-capable models, free-first). |
| `flowly` | Live `GET {base}/models` (plan-filtered with `allowed`/`locked` tags; degrades to OpenRouter on no-account/network/401). |
| `xai` | Live `GET /v1/models` with your BYOK key. |
| `xai_oauth` | Live `GET /v1/models` with the OAuth token. |
| `openai_codex` | Live authenticated `GET /models` from the ChatGPT Codex backend, filtered to picker-visible entries and kept in server priority order. Falls back to a curated list on network/schema failure; only shown once signed in. |
| `anthropic`, `openai`, `gemini`, `groq`, `zhipu` | The [models.dev](https://models.dev) community catalogue — cached locally, filtered to tool-capable models (no per-provider fetcher needed). |
| `sakana`, `vllm` | No catalog — set the model id directly. |

> [!NOTE]
> Only `sakana` and `vllm` have nothing for the `/model` picker to enumerate; for those, set the model id directly with `/model <id>`.

## Related

- [Sandbox & exec approvals](https://useflowlyapp.com/en/docs/using-flowly/sandbox-and-approvals)
- [Codex runtime](https://useflowlyapp.com/en/docs/features/codex-runtime)
- [Channels overview](https://useflowlyapp.com/en/docs/channels/overview)
- [CLI commands](https://useflowlyapp.com/en/docs/reference/cli-commands)
- [Slash commands](https://useflowlyapp.com/en/docs/reference/slash-commands)
- [Environment variables](https://useflowlyapp.com/en/docs/reference/environment-variables)
- [Setup wizard](https://useflowlyapp.com/en/docs/getting-started/setup-wizard)