LLM providers¶

A provider is a configured LLM vendor — Anthropic, OpenAI, Perplexity, an OpenAI-compatible endpoint. Agents call providers through their adapters (see adapters.md). This page is about the operator's job of getting providers wired in correctly.

Concepts¶

Provider — vendor + credentials + a list of models. Lives inside a company.
Adapter type on the agent — llm (the default) for any of the HTTP providers below; or a non‑provider runtime (claude_local/codex_local/claude_sandbox/codex_sandbox/ docker/lambda/remote). An agent with a provider binding uses the provider regardless of adapter_type (provider dispatch wins — see adapters.md).
Model — the specific model id (e.g. claude-opus-4-6, gpt-5, sonar-pro). The provider's catalog defines what's valid.

The flow:

Operator adds a provider to the company (this page).
Operator (or another admin) hires an agent and picks the provider
model.
Agent's heartbeat calls that provider with that model.
Postgres records token usage and cost per run.

Adding a provider¶

Sidebar → Providers → + Add Provider (admin only). Fields:

Field	Notes
Name	Display label. Free text. "Anthropic Production", "OpenAI Cheap", etc.
Adapter type	`anthropic`, `openai`, `perplexity`, `openai_compatible`.
API key	The vendor's API key. Encrypted at rest under the active DEK. Requires `TRIMUS_ENCRYPTION_KEY` — without it the save is refused with a 503 (SEC‑010).
Base URL	Override the upstream API endpoint. Empty = vendor default.
Enabled	Whether agents can pick it. New providers default to enabled.

Click Save Provider. You land on the provider detail page.

Verify it works¶

Click Test Connection. The server makes a read-only call to the provider's ListModels endpoint with the stored key, and reports:

Connected — discovered N model(s) in NNNms → fully working.
Live check failed: ... → key or base URL is wrong, or the provider is temporarily down. The provider stays enabled so existing agents keep working off the hardcoded fallback model list, but the warning means you should fix it.
adapter X does not support connection testing → only for exotic adapters; rare.

The same probe is reachable via POST /api/providers/{id}/test. See ../../../API.md#providers.

Per-vendor specifics¶

Anthropic¶

Base URL default: https://api.anthropic.com/v1
API key format: sk-ant-...
API version: sent as the anthropic-version header. Default 2023-06-01. Override globally with the TRIMUS_ANTHROPIC_API_VERSION env var, or per-provider by editing the JSON config (rare).
Default model: typically claude-opus-4-7 or claude-sonnet-4-6 at the time of writing — Anthropic ships new ones regularly. Trust the Test Connection model list.

OpenAI¶

Base URL default: https://api.openai.com/v1
API key format: sk-...
Default model: gpt-4o or gpt-5 family.

Perplexity¶

Base URL default: https://api.perplexity.ai
API key format: pplx-...
Default model: sonar family.
Special fields on agents (not on the provider itself): the agent's heartbeat-config form has search_recency_filter and search_domain_filter that are merged into Perplexity calls. See ../board/configuring-an-agent.md.

OpenAI-compatible (self-hosted, vLLM, Together, Anyscale, etc.)¶

Base URL: REQUIRED. Whatever your endpoint is.
API key format: provider-specific. Use whatever the upstream expects.
Default model: provider-specific. The catalog comes from the upstream's /v1/models endpoint.

This adapter type is a thin shim over OpenAI's API shape, so anything that implements /v1/chat/completions and /v1/models works.

AWS Bedrock¶

Primary use case: enterprise compliance — running LLM traffic through AWS so it stays inside your VPC / account / region.

Base URL: optional. Defaults to the region's standard endpoint. Set to a VPC interface endpoint URL (e.g. https://bedrock-runtime.us-east-1.vpce.amazonaws.com) or a FIPS endpoint when required by your compliance regime.
API key field: a JSON blob, not a plain key. Minimum: {"region":"us-east-1"}. Full shape:

{
  "region": "us-east-1",
  "access_key_id": "AKIA...",
  "secret_access_key": "...",
  "session_token": "..."
}

region is required — the SDK uses it to pick the Bedrock endpoint.
access_key_id and secret_access_key are optional. When omitted, the AWS SDK's default credential chain takes over (IAM instance profile → IRSA → AWS_PROFILE → env vars). This is the recommended posture for production deployments running on EC2 / ECS / EKS with an IAM role attached.
session_token is for temporary credentials from STS AssumeRole / web identity. Optional.
Default model: anthropic.claude-3-5-sonnet-20241022-v2:0.
Supported model family in v1: anthropic.claude-* only. Llama / Mistral / Amazon Titan / Cohere models surface a clear "not supported in v1" error and will land in a follow-up slice when there's a concrete use case.
Streaming: not yet wired in v1; arrives in a follow-up (Bedrock's Event Stream framing is supported by the SDK so the adapter shim will be small).

Minimum IAM policy¶

Attach this to the IAM role or user that Trimus runs as:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": [
        "bedrock:InvokeModel"
      ],
      "Resource": "arn:aws:bedrock:*::foundation-model/anthropic.claude-*"
    }
  ]
}

Scope the Resource ARN tighter (specific region / model ARN) if your compliance regime demands least-privilege.

When to choose Bedrock vs. direct Anthropic¶

Want	Pick
Lowest latency / cheapest direct billing	direct Anthropic
Data residency / VPC-bound traffic / IAM auth	Bedrock
Already on AWS, want consolidated billing	Bedrock
Non-Anthropic models (Llama / Mistral / Titan)	Wait for Bedrock v1.2+ or use OpenAI-compatible against a host that serves them

Why have multiple providers?¶

Cost vs. quality tradeoff. A "Cheap OpenAI" provider with gpt-4o-mini for triage agents, an "Expensive Anthropic" provider with claude-opus-4-7 for executives.
Vendor redundancy. If one provider is down, swap an agent's provider on the fly without losing state.
Different keys for different teams. Bill agents owned by different sub-teams to different vendor accounts.

Switching an agent's provider¶

Agent detail page → Configuration tab → Provider section. Pick the new provider, optionally override the model, Save Configuration. The next heartbeat uses the new setup.

The agent's existing run history is preserved — heartbeat_runs records the provider + model that ran each turn, so swapping mid-life doesn't make old data ambiguous.

Which provider computes embeddings¶

Embeddings power the RAG knowledge base, notes search, and semantic retrieval. They need an openai / openai_compatible provider — Anthropic and Perplexity have no embeddings endpoint.

By default Trimus picks one automatically: it prefers a canonical OpenAI row (api.openai.com, which hosts the default text-embedding-3-small model) over a local Ollama / OpenRouter row, and never picks a non-embedding provider. This is what makes the common "Anthropic for chat + OpenAI for embeddings" setup work without any extra configuration.

To pin it explicitly — for a self-hosted proxy, a specific vendor, or to override the heuristic — open AI Providers, find the openai / openai_compatible provider you want, and click Use for embeddings. That provider shows an embeddings badge; click the button again to clear the designation and return to automatic selection. A designation that later becomes disabled (or isn't embedding-capable) safely falls back to automatic selection rather than stranding your embeddings.

By default embeddings use the engine model (text-embedding-3-small). If your designated endpoint serves a different embedding model, set it in the provider's Embedding model field (on the provider edit page) — the worker then requests that model and re-embeds consistently, with no per-tick churn. Leave it blank to use the engine default.

Disabling vs. deleting¶

Disable (toggle on the provider page) keeps the provider record but agents pointed at it will fail their next heartbeat. Use this for temporary "vendor is down" windows.
Delete removes the provider record. Agents pointed at it immediately fail until you reassign them. The handler returns 409 if any agent is currently using the provider — fix the references first.

Storing the API key safely¶

Provider credentials live in the company_providers table. Both the api_key and any custom_headers are encrypted at rest under the active DEK via the envelope encryption helper (enc:v2:…).

TRIMUS_ENCRYPTION_KEY is required to save a provider

Without the key the vault is not initialised and the create/edit returns 503 — there is no plaintext fallback (SEC‑010). Set the key before adding any provider. If you carried legacy unencrypted rows from an older build, migrate them with trimus-cli encrypt-providers.

The release‑archive installer (./install.sh) generates TRIMUS_ENCRYPTION_KEY automatically at install time and writes it into ~/.config/trimus/server.env (or /etc/trimus/server.env in --system mode), so provider keys are encrypted from the very first heartbeat — no manual step required.

See secrets-and-encryption.md and key-rotation.md for the full encryption and rotation procedures.

Importing providers from a bundle¶

A portability bundle can carry per‑company provider configuration so a fresh import lands with working providers in one step. The bundle's providers array declares each provider's label, adapter_type, optional base_url, and the env var that holds its API key (the api_key_env field, e.g. PERPLEXITY_API_KEY). The bundle never carries the key itself.

At import time the importer resolves each provider's key from:

An operator-supplied override entered in the import UI preflight (or sent in the wrapped API body under provider_keys), or
os.Getenv(api_key_env) on the Trimus server's environment.

If neither source produces a key, the provider is created with enabled=false. Agents that reference it via provider_label are still wired to the right row — they'll fail their first heartbeat with a clear "provider disabled" error rather than the generic "no LLM provider configured" fallthrough. Enable the provider via the UI once you've populated the key.

Agents reference a provider by provider_label (resolved at import time to agents.provider_id), mirroring how reports_to_name resolves to agents.reports_to. See the Portability section of ../../../API.md for the full bundle shape and import-request body.

Default fallback¶

The env vars TRIMUS_DEFAULT_LLM_PROVIDER / TRIMUS_DEFAULT_LLM_MODEL / TRIMUS_DEFAULT_LLM_API_KEY / TRIMUS_DEFAULT_LLM_BASE_URL define an instance-wide fallback used when an agent's adapter_config specifies no provider. Useful for:

The seed flow (so trimus-seed produces a working sample agent).
Smoke-testing fresh installs before you've added a real provider.

In production, avoid relying on the fallback for real agents. Configure providers per-company so the audit trail is clean and billing is attributable.

Cost tracking¶

Every heartbeat run records model, token usage, and cost_usd, computed from a per‑model rate table at run time. The dashboard's Costs widget rolls these up (see costs.md).

The default rate table is embedded in the engine (internal/engine/pricing.go). To override pricing without rebuilding — e.g. when a vendor changes rates or you negotiate custom pricing — point TRIMUS_PRICING_FILE at a JSON file of model → {InputPer1K, OutputPer1K} rates. Unknown models fall back to a built‑in default rate.

Instance-wide provider connectivity test¶

The /admin/operations panel ships a Test all providers button that probes each instance-level provider concurrently and reports reachability + latency per provider. Use this when an operator wonders "is the API key on this box still good?" — one click, inline results, no provider drilldown.

Probes (instance admin only):

Provider	Probe	Cost
OpenAI	`GET /v1/models` (auth-check only)	$0
Anthropic	`POST /v1/messages` with max_tokens=1	≈$0.0001
Bedrock	`sts GetCallerIdentity` in `AWS_REGION`	$0

"Configured?" gating:

OpenAI: OPENAI_API_KEY env var present.
Anthropic: ANTHROPIC_API_KEY env var present.
Bedrock: AWS_REGION (or AWS_DEFAULT_REGION) env var present. Credentials come from the SDK default chain (env vars / IAM role / IRSA / shared config).

Unconfigured providers surface as (not configured) — no probe issued. A reachable provider renders green with the round-trip latency in milliseconds; an unreachable one renders red with the truncated error.

Cost cap: an admin clicking the button every 30 seconds for an hour runs ≈120 Anthropic probes at ≈$0.0001 each = ≈$0.012/h. The OpenAI and Bedrock probes are list-style and contribute nothing to spend.

The endpoint (POST /admin/operations/providers/test) also accepts Accept: application/json or ?format=json for curl / scripted callers and returns the same probe results as a JSON envelope.

Where to next¶

adapters.md — picking an adapter for the agent (llm is the common one and uses providers from this page; the CLI/docker/lambda/remote adapters don't).
costs.md — LLM spend dashboards, budgets, and overrides.
../board/hiring-an-agent.md — the board-side of agent setup.
secrets-and-encryption.md — protecting provider keys at rest.