Skip to content

Recipes & walkthroughs

End-to-end tasks, start to finish. Each recipe states the goal, what you need, the exact steps (with nav paths and the API where it helps), and how to verify it worked. They assume the core concepts.

Conventions

Nav paths look like Operate ▸ Catalog. API calls are the same endpoints the screens use, so you can script any of this. Where a step needs a role or tier, it says so.

1. Onboard a new tenant

Goal: stand up an organization with a team, a teammate, and a working key. You need: platform admin (to create the org) or a fresh account (to self-onboard); MANAGE_TEAMS / MANAGE_MEMBERS after.

  1. Create or join the org. A brand-new account with no org lands on Welcome: create an org (pick its base currency now, it is permanent) or accept an invite. A platform admin can also create orgs from Operate ▸ Organizations.
  2. Make a team. Operate ▸ Organizations ▸ Teams → New team (e.g. "Platform"). Optionally appoint a lead.
  3. Invite a teammate. Organizations ▸ Members → Invite by email, choosing a role at or below your own. They get an accept-link that becomes a membership.
  4. Issue a key for an app. Operate ▸ Virtual keys ▸ Issue key, place it in the team (a teamless org-level key needs budget authority), label it, and copy the secret now (shown once).

Verify: paste the key into Operate ▸ Playground, pick a model, and send a message. A reply means auth, scope, and routing all work end to end.

Why it works: the key carries its scope chain from the moment it is issued, so every later control (budgets, limits, guardrails) already knows where it belongs.

2. Connect your first models (a vendor and a local one)

Goal: offer one alias backed by OpenAI and another backed by a local Ollama. You need: MANAGE_PROVIDERS (vendor + key), MANAGE_DEPLOYMENTS (alias + pricing).

  1. Add the vendor provider. Operate ▸ Catalog ▸ Add provider: protocol openai, the OpenAI base URL, and your master key (encrypted at rest, never seen by apps).
  2. Add a deployment. Add deployment: alias gpt-4o-mini, provider = the one above, upstream model gpt-4o-mini. Set pricing (input/output per token) so calls get a money cost.
  3. Add the local provider. Add provider: protocol openai, base URL http://ollama:11434/v1, any placeholder key.
  4. Add its deployment. alias local-llama, upstream model gemma3:4b (or whatever Ollama serves). Leave pricing empty (local inference is free); usage still meters in tokens.

Verify: in the Playground, switch between gpt-4o-mini and local-llama and confirm both reply. The same app code calls either by changing only the model field.

Why it works: the canonical OpenAI shape means your app never changes; only the catalog does.

3. Cost-aware failover (cap the premium vendor, spill to the cheap one)

Goal: one alias served by a premium and a cheap backend, where traffic moves to the cheap one once the premium's budget is spent. You need: MANAGE_DEPLOYMENTS and MANAGE_BUDGETS.

  1. Two deployments, one alias. Under alias chat, add deployment A (provider openai-premium, priced high) and deployment B (provider local-or-cheap, priced low or free). The catalog now shows chat as load-balanced ×2.
  2. Cap the premium backend per-deployment. Operate ▸ Budgets ▸ Manage → add a budget on the scope, model = chat, provider = openai-premium (this restricts it to that one deployment), a monthly cost limit, period MONTHLY.
  3. Leave aggregate headroom. Do not put a provider-less chat budget so low that it blocks the alias outright (that aggregate cap is checked before routing and would deny rather than fail over).

Verify: drive traffic until the premium budget is exhausted (or set it tiny), then watch new calls land on the cheap backend in Govern ▸ Audit & usage (the served provider is recorded per call).

Why it works: a per-deployment budget is applied by the per-candidate filter during routing, so an exhausted backend is dropped from the candidate list and the request fails over instead of being denied. A 429 only happens when every candidate is over budget. See aggregate vs per-deployment.

4. Stop PII and secrets leaking

Goal: redact emails in prompts and block API keys in replies. You need: MANAGE_GATEWAY. Regex detectors work on all editions; semantic PII needs Pro AI.

  1. Seed a starter set. Govern ▸ Guardrails. On a fresh org, click apply recommended to plant PII/secret categories, detectors, and policies.
  2. Tune the request policy. Policies tab → category PII → set the Request action to REDACT with a reversible TOKEN strategy (so the model still gets coherent text, restored for the client at the end).
  3. Tune the response policy. category SECRET → set the Response action to BLOCK, fail-closed on (if the detector cannot finish, block rather than risk a leak).
  4. Dry-run it. Test tab → type a prompt containing an email, watch it get redacted on the way in; edit the hypothetical model reply to include a fake API key and re-screen to see it blocked on the way out. No model is called.

Verify: repeat in the Playground against a real model to see it act on a live call (the Playground is fully governed).

Do not redact tool calls

If your apps use tool/function calling, keep tool-call content on FLAG/BLOCK, never REDACT. Mutating tool-call arguments breaks the tool. See guardrails.

5. Cap a team's spend and throttle a noisy project

Goal: a hard monthly dollar cap on a team, plus a per-minute request cap on one project. You need: MANAGE_BUDGETS (budget), MANAGE_GATEWAY (rate limit).

  1. Team budget. Operate ▸ Budgets ▸ Manage → pick the TEAM scope → add an all-models, MONTHLY cost limit (hard), plus an optional soft limit that warns earlier.
  2. Project rate limit. Operate ▸ Rate limits → pick the PROJECT scope → set RPM (and/or TPM).

Verify: the Budgets Manage meters show live spend against the cap; the Rate limits Limits meters show RPM/TPM against the effective cap. Exceeding RPM returns 429 with Retry-After.

Why it works: both are enforced across the whole scope chain at once, so the team cap and project limit apply on top of anything set at the org or key level. The tightest wins.

6. Ship a weekly cost report

Goal: a branded PDF cost summary, emailed every Monday, password-protected. You need: MANAGE_REPORTS (operator and up); an SMTP server for email delivery.

  1. Mail server (once). Settings ▸ Mail server → set host/port/credentials/from, send-test to confirm.
  2. New report. Govern ▸ Reports ▸ New report: dataset = spend by model, scope = your org, window = last 7 days with period-over-period deltas, format = PDF.
  3. Schedule + deliver. every Monday 08:00, deliver to a notification channel or a secure link.
  4. Password (optional). set one so the emailed PDF needs it to open (native PDF password; the stored artifact is always encrypted at rest regardless).
  5. Run now. the card shows "Generating…" then "Last generated …" with a Download.

Verify: download the PDF, confirm the branding, the KPIs, and (if set) the password prompt.

Why it works: reports read the same usage event stream as the dashboard, render via the object store with a TTL, and stream through the gated download. See reports.

7. Detect and auto-contain a leaked key (Pro)

Goal: when one key repeatedly trips secret-guardrails, alert and quarantine it automatically. You need: Pro (OpenSearch on), VIEW_SECURITY + MANAGE_GATEWAY.

  1. Write a detection rule. Investigate ▸ SIEM ▸ Detection rules: match category = guardrail.triggered AND action = BLOCK AND category = SECRET, group by key, threshold ≥ 5 in 10 minutes.
  2. Route an alert. Govern ▸ Notifications ▸ Routing: send siem.detection events to an email or webhook channel.
  3. Add a SOAR rule. Investigate ▸ SOAR ▸ Rules: same match, group by key, threshold ≥ 10 in 5 minutes, cooldown 30 minutes, action quarantine.
  4. Review. SOAR ▸ Actions (platform admin): when it fires, the key goes SUSPENDED; mark the action CONFIRMED or un-suspend if it was a false positive.

Verify: drive blocked-secret calls from one key (the guardrails Test plus the Playground) and watch the detection fire, the alert arrive, and the key get quarantined.

Why it works: SOAR consumes the normalized event stream and takes reversible actions (quarantine/throttle); permanent revoke stays manual. See SOAR.

8. Grow from one box to a cluster

Goal: move from a single standalone instance to multiple replicas without losing data or relearning the UI. You need: infra access; the same image.

  1. Externalize the object store. Switch aim.objectstore.backend to s3 and point it at an S3-compatible store, so every replica can serve a report download.
  2. Move rate-limit counters to Valkey. Set aim.ratelimit.backend=valkey and aim.cluster.backend=valkey so counters and config-broadcasts are shared across replicas.
  3. Run replicas behind an LB. Use deploy/compose/distributed.yml (single host, --scale app=N) or the Helm chart (multi-machine).
  4. (Already handled) scheduled maintenance becomes a cluster singleton via ShedLock automatically.

Verify: issue a report on one replica and download it from another; hammer RPM and confirm the cap holds across replicas (not N times the cap).

Why it works: the data plane is stateless by design, so only shared state (counters, artifacts, the SIEM projection) moves to external stores. See How it is deployed.

Back to the guide landing, or the section map.