> ## Documentation index
> Fetch the complete documentation index at: https://haarness.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# Local models and custom endpoints

Point Haarness at any model server or gateway: a model running on your own computer, a company
gateway, or your own Azure, Vertex or Bedrock deployments. You add a provider definition with an id
of its own; the key, if one is needed, stays in your environment or credential store.

## Ollama and LM Studio

Start the app before Haarness and there is nothing to set up. Haarness looks for Ollama on
`127.0.0.1:11434` and LM Studio on `127.0.0.1:1234` when its server starts, and lists their models
as `ollama/…` and `lmstudio/…`.

```sh
haarness models ollama
haarness run -m ollama/qwen3-coder:30b "Explain src/main.rs"
```

Started Ollama after Haarness? Stop the Haarness server for the project with
`haarness serve --stop` and open Haarness again.

To hide a detected runtime, set `"enabled": false` for `ollama` or `lmstudio` in the
[configuration file](https://haarness.ai/docs/providers/custom-providers.md#add-it-in-the-configuration-file).

## Add a local model server

For vLLM, llama.cpp's server, LocalAI or anything else that speaks the OpenAI Chat Completions API:

```sh
haarness provider add local-vllm \
  --url http://127.0.0.1:8000/v1 \
  --no-auth \
  --model my-org/my-model
```

You see `Added custom provider 'local-vllm' to …` with the file it wrote. The model is now in the
picker as `local-vllm/my-org/my-model`:

```sh
haarness run -m local-vllm/my-org/my-model "Hello"
```

Leave out `--model` if the server lists its models at `GET /models`; Haarness then discovers them.
With `--model`, only the models you list are offered.

## Add a gateway that needs a key

Name the environment variable that holds the key. The key itself is never written to a file.

```sh
haarness provider add company \
  --url https://models.example.com/v1 \
  --api-key-env COMPANY_MODELS_KEY
```

Or store the key instead: `haarness auth set company`.

If the gateway wants a different header, send it with `--header`. Quote it so your shell does not
expand the variable, and make `${VAR}` the whole value. Haarness reads the variable when you add the
provider and whenever its server starts, so it must be set both times:

```sh
haarness provider add company \
  --url https://models.example.com/v1 \
  --header 'X-Api-Key=${COMPANY_MODELS_KEY}'
```

## Endpoints with another API format

Add `--protocol` when the endpoint does not speak OpenAI Chat Completions:

| `--protocol` | For |
| --- | --- |
| `openai-chat-completions` | The default; most gateways and local servers |
| `anthropic-messages` | Anthropic-compatible endpoints |
| `gemini-generate-content` | Gemini-compatible endpoints |
| `azure-openai` | Azure OpenAI deployments |
| `google-vertex` | Vertex AI |
| `amazon-bedrock` | Bedrock Converse |
| `github-copilot` | Copilot-compatible endpoints |

```sh
haarness provider add company-claude \
  --url https://models.example.com/anthropic/v1 \
  --protocol anthropic-messages \
  --api-key-env COMPANY_MODELS_KEY \
  --model company-sonnet
```

## Use your own Azure deployments or Bedrock models

The built-in Azure, Vertex and Bedrock providers come with two example models each. To use your
own deployment names or model ids, add a provider with that protocol and list them:

```sh
haarness provider add my-azure \
  --url https://my-resource.openai.azure.com/openai/v1 \
  --protocol azure-openai \
  --api-key-env AZURE_OPENAI_API_KEY \
  --model my-gpt-deployment
```

```sh
haarness provider add bedrock-eu \
  --url https://bedrock-runtime.eu-west-1.amazonaws.com \
  --protocol amazon-bedrock \
  --model eu.anthropic.claude-sonnet-4-5-20250929-v1:0
```

Without `--api-key-env`, store a key with `haarness auth set my-azure`. Bedrock also uses your
AWS keys or profile; set `AWS_REGION` to the region in the URL.

## Add it from the app

- **Desktop:** **Settings → Providers → Add custom provider.** Fill in **Provider ID**,
  **Base URL** and **API format**, choose **Authentication** (**API key**,
  **Environment variable** or **No authentication**), and under **Models** choose
  **Discover from provider** or **Choose model IDs**. The button appears while Desktop uses a
  Haarness server on this computer.
- **Terminal UI:** `/providers`, then **Add a custom endpoint**, or `/connect`, then
  **Custom endpoint…**.

**Available in** (Desktop and the terminal UI) and `--project` (CLI) choose whether the provider is
for every project or only the current one.

## Add it in the configuration file

`haarness provider add` writes a `providers` entry to your configuration file, `haarness.json` on
a new install. You can write it yourself, and add what Haarness cannot discover about a model:

```json
{
  "providers": {
    "local-vllm": {
      "name": "Local vLLM",
      "base_url": "http://127.0.0.1:8000/v1",
      "requires_auth": false,
      "models": ["my-org/my-model"],
      "context_window": 131072,
      "max_output_tokens": 8192,
      "images": true,
      "tools": true,
      "reasoning": false
    }
  }
}
```

| Key | Means |
| --- | --- |
| `protocol` | The API format; defaults to `openai-chat-completions` |
| `name` | The name shown in pickers |
| `base_url` | The endpoint's base URL |
| `api_key_env` | Environment variable holding the key |
| `headers` | Extra headers; a value of exactly `"${VAR}"` reads a variable |
| `models` | Only offer these models |
| `requires_auth` | `false` for an endpoint with no key |
| `context_window` | The model's context size, in tokens |
| `max_output_tokens` | The longest answer it may write |
| `images` | `true` if it accepts images |
| `video` | `true` if it accepts video (Gemini and Vertex formats only) |
| `tools` | `false` if it cannot call tools |
| `reasoning` | `true` if it has a reasoning mode |
| `stream_idle_timeout` | Seconds an answer may send nothing before Haarness gives up; `0` never |
| `request_timeout` | Seconds a Bedrock answer may take in all; `0` waits as long as it takes |
| `enabled` | `false` hides this provider |

These settings apply to every model of that provider. The same keys work under
`[providers.ID]` in an older `config.toml`. See [Configuration](https://haarness.ai/docs/cli/configuration.md) for where
the files live.

## Why does a slow model stop answering?

Haarness gives up on an answer that sends nothing for five minutes, and retries it. A model on your
own computer (`localhost`, `127.0.0.1`, `::1`) is waited for as long as it takes, since it may say
nothing while it loads. Bedrock answers in one piece and has 15 minutes in all.

To change it, set `stream_idle_timeout` (or `request_timeout` for Bedrock) in seconds. An entry
with only these keys tunes a built-in provider without redefining it:

```json
{ "providers": { "anthropic": { "stream_idle_timeout": 600 } } }
```

## Rules

- **Give each endpoint its own id.** `haarness provider add`, Desktop and the terminal UI refuse
  the id of a built-in provider, such as `openai`, `anthropic` or `ollama`: `openai is a built-in
  provider. Give your endpoint its own ID, for example openai-proxy`. So a key you stored for a
  built-in provider is never sent to an endpoint you add.
- **A project cannot redirect a provider.** In a project's `haarness.json`, an entry that gives a
  built-in provider, or one from your global configuration, another `base_url` is ignored, and
  `haarness providers` says so. Define the project's endpoint under its own id instead.
- **HTTPS is required** except for `localhost`, `127.0.0.1` and `[::1]`, which may use plain HTTP.
  A URL with a user name or password in it is refused.
- **Changes apply on their own.** A running Haarness server picks up an added, changed or removed
  provider once no turn is running.
- **Removing a provider keeps its stored key.** `haarness provider remove ID` deletes the
  definition; `haarness auth logout ID` deletes the key.
- **Always name the provider on the command line:** `-m local-vllm/my-org/my-model`. With
  `-m my-org/my-model`, `my-org` would be read as the provider.

## Troubleshooting

**The provider does not appear.** Haarness skips a definition it cannot use, for example an
`http://` URL that is not on loopback. Run `haarness providers` in the project; it prints a line
such as `Provider lan-box: base_url must use https, or http on an exact loopback host; skipped.`

**"does not advertise image input".** Set `"images": true` for that provider if the model accepts
images.

## Related

- [Sign-ins and API keys](https://haarness.ai/docs/providers/native-providers.md)
- [Haarness Router](https://haarness.ai/docs/providers/router.md) — many providers behind one endpoint