Langfuse launches AI Gateway Learn more

FinOps for AI: attribute LLM spend to teams, cost centers, and features

AI spend arrives as a few large invoices from model providers, while the decisions that drive it are made by dozens of teams, services, and developers. FinOps for AI is the practice of closing that gap: measuring spend at the level of the call, attributing it to whoever made the call, and putting the numbers in front of the people who can change them. This guide shows how to set that up with LLM tracing and an AI gateway.

TL;DR: Record every model call with its cost, then attribute it along the dimensions finance cares about. Put cost centers and teams on gateway keys so every call carries them, tag application traces with the feature that made them, chart spend per team and feature in dashboards, alert on spikes, and export the raw records to your data warehouse. Provider invoices stay the source of truth for what you paid. Langfuse explains where it went.

Langfuse AI Gateway analytics with total cost, cost over time, and cost per user, API key, and model

Why AI spend is hard to attribute

Cloud FinOps works because every resource carries tags. AI spend has three properties that break that model:

  1. One key, many consumers. A single provider API key often serves several services and developers, so the invoice can't be split by owner.
  2. Cost per call varies by orders of magnitude. The model, the context size, cached tokens, and reasoning tokens change the price of a request, so request counts say little about spend.
  3. Spend is created in code and in tools. Production features, batch jobs, and developers' coding agents all draw from the same providers, through different paths.

The fix is to attach ownership to each call at the moment it is made, and to price it at the same time.

Step 1: record cost for every call

There are two ways to get calls into Langfuse with their cost, and most companies use both:

  • Application tracing with the Langfuse SDKs or OpenTelemetry records each model call inside the trace of the feature that made it.
  • The Langfuse AI Gateway records every call that passes through it, including traffic from apps nobody instrumented and from coding agents.

In both cases, Langfuse stores token usage by type (input, output, cached, reasoning) and computes cost from model prices. Add custom prices for fine-tuned or negotiated models, so the numbers match what you pay.

Step 2: attach ownership

Pick the dimensions your finance team reports on, then make sure every call carries them:

DimensionThrough the gatewayIn application traces
PersonAutomatic: a user key records its ownerUser ID on the trace
Team, cost centerKey metadata such as cost_center: 4711Trace metadata or tags
ServiceOne service account key per serviceTrace name or tag
Featurelangfuse-tags or langfuse-metadata request headerTrace name or tag per feature
Customerlangfuse-user-id header with a service account keyUser ID set to the customer, or customer metadata

Key metadata is the strongest option for cost centers: it is stored on the key, so request headers can't override it. Whoever manages a key can edit its metadata, so put admin-approved cost centers on service account keys, which only admins and owners create, and review the metadata developers set on their own user keys. Request headers and SDK tags are set by the client, so they suit feature-level detail that engineers own.

Step 3: report spend

Gateway admins get an analytics page with total cost, requests, and tokens, cost over time, and the top users, keys, and models. For finance reporting, build custom dashboards on the dimensions from step 2:

  1. Spend per cost center per month, grouped on langfuse.gateway.api_key.metadata.cost_center.
  2. Spend per feature, grouped on trace name or tag.
  3. Spend per model, to see where switching models would save the most.
  4. Unit cost, such as cost per conversation or per resolved ticket, by dividing cost by a count of sessions or traces.

Unit cost is the number that turns AI spend into a business conversation. A feature whose total cost doubled may be fine if its cost per resolved ticket fell.

Step 4: catch spikes early

Alerts watch a cost metric over a time window, filtered by model, tag, or user, and notify Slack, PagerDuty, a GitHub Action, or a webhook when it crosses a threshold. Typical alerts are hourly cost for one feature, daily cost per developer for coding agents, and cost per call for a new model.

Alerts notify, they don't block requests. Hard spend caps at the gateway are on the gateway roadmap, not available yet. Until then, combine alerts with provider-side limits on your provider account.

Step 5: export to the warehouse

Finance and data teams usually want AI spend next to revenue and headcount data. Blob storage export writes observations, including model, tokens, cost, and metadata, to S3, GCS, or Azure Blob Storage on a schedule. Load the files into your warehouse and join on cost center or customer.

Reconciling with provider invoices

Langfuse cost is computed from tokens and prices, so small differences from your invoice are expected: negotiated discounts, credits, and pricing changes all live on the provider side. Treat the provider invoice as the total and Langfuse as the breakdown. If the two drift apart, check for models without a price definition, which show tokens but no cost, and for the known gateway cost bugs during the beta.

For coding agents specifically, see coding agent cost tracking. For general cost reduction techniques, see LLM cost management.

FAQ

What is FinOps for AI?

FinOps for AI applies cloud financial management to AI spend: measuring the cost of model calls, attributing it to the teams and features that create it, and using that data to make tradeoffs between cost, speed, and quality. The core requirement is per-call cost data with ownership attached.

How do I charge AI spend back to teams?

Issue gateway keys per team or service with a cost-center metadata field, and report spend grouped on that field. For application traffic, tag traces with the owning team. Export the monthly totals per cost center to your finance system.

Can I set a budget that stops spending?

Not in the Langfuse AI Gateway today: hard spend caps are on the roadmap. You can set alerts on cost thresholds in Langfuse and spending limits on your provider account in the meantime.


Was this page helpful?