Skip to content
vmctl

AI configuration (operators)

Where the LLM config lives, and how per-group keys work in a hosted deployment. User-facing rationale: Bring your own AI. Step-by-step setup: group-key runbook.

The autopsy analyst — the server default

The analyst speaks an OpenAI-compatible API, configured under AUTOPSY_LLM__*:

Terminal window
AUTOPSY_LLM__API_KEY=sk-…                              # the server-default key (present / absent / "sk-nokey")
AUTOPSY_LLM__BASE_URL=https://your-litellm.example/v1  # the gateway (a LiteLLM proxy, or any OpenAI-compat)
AUTOPSY_LLM__ALIAS_AGENT=your-provider/model           # the model/alias the analyst drives
  • The analyst runs with no token/step caps — the spending limit is the key’s budget (enforced by LiteLLM), which is the point of per-group keys below.
  • The key is never logged (only present/absent/sentinel).
  • With no real key (sk-nokey/empty) autopsy falls back to a stub reasoner (deterministic evidence only, no AI judgment).

Per-group keys (hosted) — how BYO-AI actually works

All budget/model config lives in LiteLLM; rvmc’s only job is to map an identity-provider group to a LiteLLM key and hand it to the analysis. The mechanism:

  1. A budget-bearing group in your identity provider (IdP) carries a litellm_key attribute — the group’s LiteLLM key value (created + budgeted in LiteLLM by you). One key per group, shared.
  2. A project always has at least one group: the groups granted at provision, or — when none are given — the convention member group proj-<pid> that provisioning seeds.
  3. On dispatch, rvmc resolves project → its groups (or [proj-<pid>] when the stored list is empty) → the first group carrying a litellm_key (read from the IdP, cached ~5 min) and sends it as llm_api_key on POST /jobs/drive. autopsy runs that job on the group’s key — only the key swaps; base URL and model aliases stay the server’s.

If no group carries a key (or there is no IdP — e.g. RVMC_AUTH_MODE=local), rvmc sends no key and autopsy uses the AUTOPSY_LLM__* server default. So self-host needs nothing here.

Your OIDC provider

The reference deployment uses Authentik, but any OIDC provider works as long as a group can carry a custom attribute the controller can read. The contract rvmc needs is just two things:

  • groups you can attach a litellm_key string attribute to, and
  • an API (or forward-auth claim) the controller can read that attribute from.

With Keycloak, add litellm_key as a group attribute and expose it via a group-membership + attribute mapper so it’s readable the same way; other OIDC providers with group attributes are equivalent. The runbook shows the Authentik steps — in another IdP it’s the same two moves: mint the key in LiteLLM, store it on the group.

The chat agent (OWUI) — a separate key path

The in-workbench chat agent runs through Open WebUI, which is the LLM client for that path — it owns the model catalog and calls LiteLLM with its own connection. rvmc passes an OWUI bearer, not a LiteLLM key, so the per-group litellm_key does not flow to chat. Chat usage is metered on the OWUI side (LiteLLM tags spend per user via X-OpenWebUI-User-Id). If you want per-group chat budgets, configure that in OWUI/LiteLLM (per-group OWUI connections) — it’s out of rvmc’s dispatch. The expensive path — the autopsy analysis — is the one the group key covers.

The gateway (a LiteLLM proxy)

A LiteLLM proxy fronts many providers and exposes stable model aliases. It’s where keys, budgets, and model scoping live (master-key admin API; per-consumer virtual keys with max_budget/budget_duration). rvmc never holds a budget — it only reads a group’s key and passes it.

Local / offline models

Any OpenAI-compatible local server works as the endpoint:

Terminal window
AUTOPSY_LLM__BASE_URL=http://localhost:11434/v1   # Ollama
AUTOPSY_LLM__API_KEY=ollama                        # any non-empty value
AUTOPSY_LLM__ALIAS_AGENT=qwen2.5-coder:32b

Nothing leaves the machine — the air-gapped conservation setup.