How to Build an LLM Spend Audit by Department in 7 Steps (2026)

Quick Guide: How to Build an LLM Spend Audit by Department in 7 Steps

1. Define your cost attribution dimensions — Decide whether you'll track by department, team, repository, or individual developer.
2. Set up token telemetry collection — Install an SDK wrapper or proxy that captures every LLM API call with metadata.
3. Configure department tagging — Map each request to a department using git context, API keys, or manual labels.
4. Aggregate costs by time window — Build hourly, daily, and monthly rollups so you can spot trends and spikes.
5. Add prompt compression middleware — Use Lineman to cut token costs by 40%+ while maintaining output quality.
6. Create budget thresholds and alerts — Set per-department limits that notify you before overspending.
7. Export to your FinOps dashboard — Push data to CloudWatch, Prometheus, or your existing observability stack.

How to Build a Department-Level LLM Spend Audit

1. Define Your Cost Attribution Dimensions

Start by mapping how your organization actually uses LLM APIs. Most engineering teams find that repository-level attribution gives the clearest picture of spend.

You have several options here: track by department (Engineering, Marketing, Product), by team (Backend, Frontend, DevOps), by repository, or by individual developer. The dimension you choose shapes everything else.

For AI coding tools, repository-level tracking often works better than department-level. A single repository can be touched by multiple teams, but the cost still maps cleanly to a project budget.

2. Set Up Token Telemetry Collection

Token telemetry captures every LLM API call with the metadata you need for attribution. This is the foundation of your audit system.

Open source tools like llm-cost-telemetry wrap provider SDKs to emit per-call cost spans. The wrapper intercepts each request, records input and output tokens, calculates cost using bundled pricing tables, and attaches your custom metadata.

A typical setup looks like this: install the wrapper, configure it to emit OpenTelemetry spans, and route those spans to your collector. The wrapper handles pricing differences between models automatically.

3. Configure Department Tagging

Once telemetry is flowing, you need a consistent way to tag each request with its department. The mechanics here depend on your infrastructure.

For AI coding assistants, git context is the cleanest source of truth. The telemetry layer can read the current repository, branch, and developer email from the environment. A lookup table then maps those values to departments.

For backend services, use the API key or a custom header. Each department gets its own key, or each request includes a department tag in the payload. This approach requires no changes to how developers prompt the model.

4. Aggregate Costs by Time Window

Raw telemetry spans give you per-call granularity. For auditing, you need rollups across time windows.

Build three aggregation levels: hourly (for spike detection), daily (for developer and team visibility), and monthly (for budget reconciliation). Store each level separately so queries stay fast.

The aggregation layer sums costs by your chosen dimensions. A department that spent nothing yesterday but burned through hundreds of dollars today shows up immediately in the daily rollup.

5. Add Prompt Compression Middleware

Collecting telemetry reveals where spend goes. Cutting that spend requires a different lever: prompt compression middleware.

The mechanics are straightforward. Large tool outputs, file reads, build logs, and search results dominate most LLM bills. Lineman intercepts these data-heavy calls and hands the model a compact summary instead of the verbose original.

On Lineman's benchmarks, this approach cuts token spend by 40-53% while holding output quality steady at 98.3% of baseline. Because the compressed content never enters context, it never gets re-billed on subsequent turns.

6. Create Budget Thresholds and Alerts

Visibility without controls is just expensive reporting. Set up budget thresholds that notify you before a department exceeds its allocation.

Configure cascading alert levels: log at 50% of budget, notify the team lead at 80%, and escalate to finance at 100%. The alerting system should include root-cause context: which developer, which repository, which model caused the spend.

Some tools block further requests once a budget is exhausted. Others allow overage with explicit approval. Choose the approach that fits your organization's risk tolerance.

7. Export to Your FinOps Dashboard

The final step connects your LLM spend audit to your existing financial operations tooling.

Export cost metrics to CloudWatch, Prometheus, or your internal BI platform. Use the same dimensions (department, team, repository) so LLM spend appears alongside your other cloud costs.

This integration enables quarterly planning conversations. Engineering leadership can see which departments are heavy LLM consumers and allocate budget accordingly. Finance gets a line item they can forecast rather than a surprise invoice.

Why Do LLM Costs Compound Throughout a Session?

LLM APIs are stateless. Every turn re-sends the entire conversation, including all earlier tool output, as input. Bulky content read once is re-billed on every subsequent turn.

This mechanic, called context compounding, explains why sessions get more expensive as they run longer. A file read that costs a few cents on turn one might cost dollars by turn fifty.

The implication for auditing: per-call cost is less meaningful than per-session cost. Your telemetry system should group calls by session ID and report total session spend alongside individual call costs.

What Makes Open Source LLM Spend Tools Different from Enterprise Platforms?

Enterprise platforms like Oracle bundle LLM cost management into broader cloud offerings. You get a dashboard, but you're locked into their ecosystem and pricing model.

Open source tools give you control over the data pipeline. You can run the telemetry collector on your own infrastructure, export to any backend, and modify the code to fit your attribution model.

The trade-off is setup time. Enterprise platforms work out of the box. Open source tools require configuration. For teams with existing observability infrastructure, the open source path often integrates more cleanly with what you already have.

How Lineman Helps You Build Department-Level LLM Spend Audits

Lineman provides two capabilities that accelerate department-level spend auditing: automatic attribution and token compression.

The attribution layer reads git context from every Claude Code session and maps it to a developer, repository, branch, and ticket. No manual tagging required. This data flows into Cost Explorer, where you can drill down from total spend to individual features.

The compression layer cuts your bill before it accumulates. Lineman intercepts data-heavy tool calls and returns a task-relevant summary. On Lineman's benchmarks, this delivers 40-53% token savings with minimal quality degradation.

Together, these capabilities give engineering leaders the visibility they need to manage AI coding costs and the levers to reduce them. Start with the free plan and see your first attributed cost breakdown in under a minute.

FAQs About Building LLM Spend Audits by Department

What is token telemetry?

Token telemetry is the practice of capturing metadata about every LLM API call. This includes input tokens, output tokens, model name, latency, and custom tags like department or repository. Lineman captures this automatically from Claude Code sessions.

How do I calculate LLM API costs accurately?

Multiply input tokens by the input price per million and output tokens by the output price per million. Account for cache reads if your provider supports prompt caching. Lineman's Cost Explorer handles this calculation automatically using current provider pricing.

Can I track LLM spend without changing my workflow?

Yes. Tools like Lineman install as a plugin and capture telemetry from every call without requiring changes to your prompts or code. Attribution happens automatically based on git context.

What is prompt compression middleware?

Prompt compression middleware intercepts large tool outputs, file reads, and logs before they reach the LLM. It replaces verbose content with a compact, task-relevant summary. Lineman implements this for Claude Code using a secondary model that preserves meaning while cutting tokens.

How much can prompt compression reduce LLM costs?

On Lineman's benchmarks, prompt compression reduces token spend by 40-53% while maintaining 98.3% of baseline output quality. Savings vary by workload. Data-heavy tasks like code search and log triage see the highest reductions.

What departments typically have the highest LLM spend?

Engineering teams using AI coding assistants tend to drive the majority of LLM spend. Lineman's data shows that repository-level differences matter more than team-level differences. A complex backend service can easily cost more than an entire frontend team.

How often should I review department LLM spend?

Weekly reviews catch spikes before they hit the monthly invoice. Monthly reviews inform budget allocation for the next period. Lineman's real-time Cost Explorer makes both reviews straightforward by showing spend as it happens.