Recently I realised every little project I touch wants an LLM, and each one wanted its own API key, its own billing and its own way to blow my budget at 3am. So I built one front door: an OpenAI-compatible endpoint on the cluster, virtual keys I hand out, and a spending cap I could actually trust. That last part is the interesting bit, because the cap is the thing most people hand-wave.
This post will cover running Bifrost on a Pi, keyless Azure AI Foundry models, friendly names, how Claude fits in (and why not my subscription), the layered $100 cap with real timings, and one honest dead end. Let’s dive in!
This is Part 8 of the Raspberry Co-PI-lot series. It leans on the keyless Azure access from Part 3 (Workload Identity Without AKS) and the pod-to-LAN guard from Part 6 (Hardening a Home Cluster, and How I Broke It).
Note: names, prices and policies are as I checked them in mid November 2026, and they move quickly.
In English Please?
Picture a hotel reception desk. Guests (my apps) never get the room keys (the model providers). They show a room card (a virtual key), and reception decides which rooms it opens and how much can go on the tab. If a guest misbehaves I cancel the card, not the master key.
Bifrost is that desk: an open source gateway with an OpenAI-compatible API that adds virtual keys, budgets and rate limits in front of whichever providers sit behind it. Callers point any OpenAI SDK at https://llm.lab.example.com/v1, use a virtual key as the API key and ask for a model by friendly name. It is reachable on my LAN and over remote access (Part 5) only.
Bifrost on a Pi
Bifrost runs as a Flux HelmRelease. Here is the sanitised core:
apiVersion: helm.toolkit.fluxcd.io/v2
kind: HelmRelease
metadata:
name: bifrost
namespace: llm-gateway
spec:
interval: 1h
releaseName: bifrost
chart:
spec:
chart: bifrost
version: 2.1.48
sourceRef:
kind: HelmRepository
name: bifrost
namespace: flux-system
values:
image:
tag: v2.5.1
replicaCount: 1
resources:
requests: { cpu: 100m, memory: 192Mi }
limits: { memory: 384Mi }
storage:
mode: sqlite
persistence:
enabled: true
storageClass: local-path
size: 2Gi
configStore: { enabled: true }
logsStore: { enabled: true }
extraEnv:
GOMEMLIMIT: "300MiB"
bifrost:
sourceOfTruth: "config.json"
client:
disableContentLogging: true
enforceAuthOnInference: true
Memory is tight on a 3.8 GB Pi, so the pod is capped at 384 Mi. Bifrost measured 167 Mi after start, peaking near 281 Mi. enforceAuthOnInference means every request needs a virtual key, and disableContentLogging keeps prompts and responses out of the logs. sourceOfTruth: config.json makes Git authoritative, so keys created in the dashboard vanish on the next restart.
Be warned: my plan was file-only mode, no database, no volume. Testing showed Bifrost exits with config store is required when governance (budgets and virtual keys) is on without a store, which matches an open upstream issue. So it gets a 2 Gi SQLite volume, which pins the pod to one node. When kured reboots that node the gateway is briefly down, so callers should retry.
Foundry with No Keys, and Friendly Names
For models I used an Azure AI Foundry account in Sweden Central. I prefer my data processed in Europe, and per the deployment types docs, a Data Zone deployment processes data only within the zone (the EU here). These are the four DataZoneStandard deployments, with the Data Zone prices I saw on the Azure meters at build time (per million tokens; check the pricing page for today’s):
| Name | Model | In / out |
|---|---|---|
mini |
gpt-6-luna |
$0.12 / $0.60 |
chat |
gpt-6.1-terra |
$1.20 / $6.00 |
open |
DeepSeek-V4-Flash |
$0.21 / $0.56 |
embed |
text-embedding-3-large |
$0.143 |
The Foundry Models sold by Azure page lists the GPT models, and notes GPT-6 tool calling wants the Responses API, or Chat Completions with reasoning_effort set to none. Qwen did not make the cut: the Qwen models in the catalogue were fine-tuning bases only, and hosting a tuned one would have cost roughly $219 a month.
No API keys, anywhere
The account has disableLocalAuth: true, so no API key exists to leak. Bifrost uses the Part 3 workload identity: a projected service account token federated to a managed identity holding only Cognitive Services User on this one account.
resource account 'Microsoft.CognitiveServices/accounts@2026-09-01' = {
name: accountName
location: 'swedencentral'
kind: 'AIServices'
sku: { name: 'S0' }
identity: { type: 'SystemAssigned' }
properties: {
customSubDomainName: accountName
allowProjectManagement: true
disableLocalAuth: true
publicNetworkAccess: 'Enabled'
}
}
resource fic 'Microsoft.ManagedIdentity/userAssignedIdentities/federatedIdentityCredentials@2023-01-31' = {
parent: uami
name: 'cluster-llm-gateway-bifrost'
properties: {
issuer: oidcIssuer
subject: 'system:serviceaccount:llm-gateway:bifrost'
audiences: ['api://AzureADTokenExchange']
}
}
On the Bifrost side an empty key value means "use the default Azure credential chain", which the Azure provider docs describe as DefaultAzureCredential. It is the cert-manager trick from Part 3: a projected token volume and three environment variables, no webhook.
Friendly names through routing rules
Nobody wants azure/gpt-6.1-terra in a config file. Bifrost routing rules evaluate a CEL expression and rewrite the target:
routingRules:
- id: alias-chat
name: alias chat
enabled: true
cel_expression: 'model == "chat"'
scope: global
priority: 1
targets:
- { provider: azure, model: gpt-6.1-terra, weight: 1.0 }
There is one per name, so swapping the model behind chat is a one-line Git change.
Claude, and Why Not My Subscription
I pay for Claude’s Ultra plan, and my first instinct was to expose that subscription through the gateway. The agent’s research said no.
Anthropic’s Usage Policy now lists, under "Do Not Abuse Our Platform": "Resell, proxy, or otherwise provide access to Claude through unauthorized means, including services that route requests through consumer subscriptions or misrepresent the product or client being used". The Claude Code legal page says developers building services should use Console API keys. In my opinion, risking my main Claude account for a convenience feature is a bad trade.
Thankfully there is a supported route: Max, Ultra and Team plans include a monthly API credit, $200 on Ultra. You link one Console organisation, the credit does not roll over, and it covers the API and Agent SDK but not interactive Claude Code. The line I cared about: with no purchased credit or auto-reload, "API requests stop until your next monthly credits arrive".
That is a vendor-enforced hard stop, so the Console organisation has no card and no auto-reload, the key sits in a workspace called gateway, and Bifrost exposes it as claude-fast, claude and claude-best. The pools are separate on purpose: the general virtual key cannot call Claude and the claude key cannot call Azure (I tested both), so one pool running dry never spills into the other.
The Cap, in Layers
Most cost caps are alerts wearing a trench coat. The Foundry cost docs say Azure OpenAI has no OpenAI-style hard limit and suggest action groups with custom automation. So I stacked layers and tested each with real traffic:
| Layer | Setting | Measured |
|---|---|---|
| 1. Bifrost budget | $80 per calendar month | Real time, 402 at the limit |
| 2. Capacity | TPM per deployment | Caps burn rate, not total |
| 3. Foundry cost control | Block at $90 |
Enforced 10 to 15 minutes after spend |
| 4. Tripwire | over $15 per 24 hours pauses everything | Refused calls about 3.5 minutes after trigger |
| 5. Cost Management | $100, emails at 50, 80, 100 percent | Alerts only, lags up to 72 hours |
Layer 1: why Bifrost overshoots
governance:
customers:
- id: capped
name: capped
budgets:
- id: capped-monthly
max_limit: 80
reset_duration: "1M"
calendar_aligned: true
Bifrost’s budget is check-then-charge, not a reservation. Its docs show a $2 request passing with $4 of $5 used, and only the next one refused. In a test against a mock backend, six concurrent $0.15 requests against a $0.20 cap all succeeded and charged $0.90. It is a soft ceiling.
Also, Bifrost prices at Global rates, so Data Zone spend (about 20 percent dearer for GPT-6) is under-counted. That is why the cap is $80, not $100.
Layer 3: Foundry’s cost control
Foundry has a preview cost control with a Block action. Be warned: the Bicep reference for costControls documents a rules-based shape on 2026-09-15-preview, while the call I used was on 2026-11-15-preview, which has no published Bicep types. This is what worked for me:
ACC=/subscriptions/<subscription-id>/resourceGroups/rg-<name>/providers/Microsoft.CognitiveServices/accounts/aif-<name>
az rest --method put \
--url "https://management.azure.com$ACC/costControls/monthly-cap?api-version=2026-11-15-preview" \
--body '{
"properties": { "category": "Cost", "amount": 90, "timeGrain": "Monthly",
"timePeriod": { "startDate": "2026-11-01T00:00:00Z", "endDate": "2027-12-01T00:00:00Z" },
"providerProperties": { "actions": { "stop": { "type": "Block",
"operator": "GreaterThanOrEqualTo", "thresholdType": "Percentage", "threshold": 100 } } } } }'
A temporary control with a $0.0001 limit, a fraction of a cent spent, and Azure began refusing every model with 429 budget_exceeded 10 to 15 minutes later. Deleting the control cleared it in about 3.5 minutes. That lag is Azure’s estimated cost catching up, the honest limit of this layer. The control ends on a fixed date (2027-12-01 in mine), so extend it before then.
Layer 4: the tripwire
Ten minutes suits a slow leak, not a runaway loop. A metric alert watches Foundry’s estimated cost over the trailing 24 hours and fires at $15 (normal pace is about $3.30 a day). It emails me and calls a Logic App that pauses every deployment:
pause_each: {
type: 'Foreach'
foreach: '@parameters(\'deployments\')'
// One at a time: concurrent writes to deployments on one account return 409.
runtimeConfiguration: { concurrency: { repetitions: 1 } }
actions: {
pause: {
type: 'Http'
inputs: {
method: 'PATCH'
uri: '@{concat(parameters(\'armUrl\'), substring(parameters(\'accountId\'), 1), \'/deployments/\', item(), \'?api-version=2026-09-15-preview\')}'
headers: { 'Content-Type': 'application/json' }
body: { properties: { deploymentState: 'Paused' } }
authentication: { type: 'ManagedServiceIdentity', audience: environment().resourceManager }
retryPolicy: { type: 'exponential', count: 6, interval: 'PT20S' }
}
}
}
}
The concurrency: 1 line is a scar: the first test failed because pausing four deployments in parallel returned 409. After serialising it, a simulated alert paused all four and the gateway refused calls about 3.5 minutes later. Resuming took about 9 minutes. Resume is manual on purpose, because I want to know why it fired.
I rejected Azure API Management, at about $48 a month for the Developer tier (half the cap, spent on enforcing it). At full saturation (about $13 an hour) the 10 to 15 minute lag is worth a few dollars, so even a total Bifrost failure stops near $90, not the roughly $1,260 my research feared for billing-lag-only designs.
The Dashboard SSO Dead End
After Part 4 everything with a UI here signs in through Entra, so I asked the agent to put the Bifrost dashboard behind oauth2-proxy. On paper it worked: the proxy handled Entra and injected an Authorization: Basic header with the admin credentials, which Bifrost’s auth docs say works for API calls. Then Bifrost’s own login page appeared anyway.
I told the agent: "I am not seeing the proxy present the password, instead being challenged with a login prompt." On v2.5.1 of the open source edition, the UI trusts only its own session cookie (HttpOnly, called token, issued after a password login) and ignores an injected Basic header. Native OIDC sign-in is documented for Enterprise only.
My options were two logins for no gain, a CronJob faking the cookie, or a password. I picked the password and removed the proxy, redirect URI and client secret. A word from the wise: the admin API returns virtual key values, so that password is as sensitive as a provider key.
Working with the Agent
I made the calls on scope, money and policy; the agent did the research, Bicep and testing. My opening ask was one paragraph: Bifrost as a general gateway, Foundry in a new resource group, a $100 token maximum unless I exempt something, Europe preferred, and use my Claude plan’s credit. The agent fanned research out to sub-agents that wrote cited reports, catching the file-only crash, Qwen being unusable, the overshoot and Anthropic’s policy change four days earlier.
It got right what mattered most: testing claims instead of trusting them. The 10 to 15 minute lag and the 3.5 minute tripwire are measurements, and it diagnosed the Logic App 409s itself.
It got the SSO assumption wrong (I spotted the login prompt). I also pasted a user-scoped Anthropic key that needed an extra workspace header, so I made a workspace key, which the agent validated without printing it. Whether to touch my subscription at all was my call.
My laptop also died mid-revert of the SSO attempt, with the Key Vault delete timed out and manifests uncommitted. The agent rebuilt the state from git status, the cluster and Azure, and finished the job.
Wrapping Up
One endpoint now serves everything at home, with keyless Azure access, a vendor-enforced stop on Claude and five layers of cap I have timed rather than assumed. Next is the capstone on what building all this with an AI agent was actually like.
As always, feel free to reach out with any questions or comments!
Up Next
- Rebuilding My Desk Kubernetes Cluster, Five Years On
- GitOps and Secrets with Flux and SOPS
- Workload Identity Without AKS
- Entra ID for kubectl, Five Years After Okta
- Ingress, Certificates and Getting Home Through CGNAT
- Hardening a Home Cluster, and How I Broke It
- Keeping It Patched Without Babysitting It
- One LLM Gateway for Everything, with a Hard $100 Cap (this post)
- What I Learnt Building It with an AI Agent
Until next time, stay cloudy!