Unit prices for AI intelligence have collapsed, yet enterprise spending continues to rise.
According to McKinsey’s QuantumBlack report, Is That AI Agent Worth It? Agentic Economics and the Modern Operating Model, 93% of enterprise leaders exceed their AI budgets, while one in five actively restricts AI adoption to control costs.
This is Jevons’ Paradox playing out in real time. As the cost of an input falls, consumption accelerates even faster.
The market has already moved beyond simple prompt-and-response tools to autonomous, agentic workflows. Many organizations, however, are still evaluating AI through the lens of traditional IT investments. They treat token consumption like a fixed SaaS license instead of what it really is: a variable operating expense that grows with every decision an AI agent makes.
That mindset creates a dangerous disconnect. Leaders focus on the cost of a single prompt while overlooking the economics of thousands or millions of autonomous decisions executed every day.
An AI agent is not a software asset. It is a variable-cost digital worker. Evaluate it like a fixed application, and you will misjudge both its cost and the value it can create.
The 60% Refinement Tax
Traditional software is relatively predictable. The same transaction consumes roughly the same amount of compute every time it runs.
Agentic AI works differently. Its cost is dynamic because the path to an answer is dynamic. According to McKinsey, identical enterprise tasks can vary in cost by as much as 30x depending on how the agent executes them.
The biggest driver is not the initial prompt. It is everything that happens afterward.
McKinsey found that nearly 60% of agentic compute spend comes from response refinement: the cycles where an agent checks its work, calls external tools, retrieves additional context, validates assumptions, and revises its output before delivering a final answer.
Every one of those steps consumes additional inference. Each tool call, context retrieval, verification loop, and correction expands the amount of work the model performs behind the scenes. A task that appears simple to the user can trigger dozens of intermediate operations before a response is returned.
That is why agentic AI cannot be managed like traditional software. Cost is driven less by the request itself than by the execution path the agent chooses.
Complexity kills scale. Well-designed workflows place boundaries around context growth, tool orchestration, and verification loops. Without those guardrails, complexity compounds, costs become unpredictable, and operating margins quickly erode.
From Unit Price to Token Yield
Falling token prices mean very little if your architecture increases token consumption by 1,000x per transaction. The metric that matters is not cost per token. It is Token Yield: the measurable business value created for every dollar of inference spend.
Improving Token Yield starts with eliminating three common sources of waste:
Concentrated consumption. AI spending rarely distributes evenly across the enterprise. Like many operating costs, it follows a Pareto pattern: a small number of users, workflows, and agents drive the majority of inference spend. Treat AI as a flat IT overhead expense, and you hide the biggest opportunities for optimization.
Model over-provisioning. Too many organizations run frontier reasoning models for routine, structured work because there are no routing standards. Using your most powerful model to parse text or transform data is like commuting in a commercial airliner. Match model capability to task complexity.
Architectural overhead. The LLM is only one part of the cost. Security proxies, vector databases, retrieval pipelines, orchestration layers, and monitoring services all consume resources as requests move through the stack. When those layers are poorly designed or repeatedly process the same information, complexity compounds and operating costs rise.
Operational Guardrails That Work
Governance is an execution engine, not a blocker. The objective is not to restrict autonomy. It is to ensure autonomy operates within economic and operational boundaries.
EXAMPLE: Enterprise AI Gateway (Policy, Security, Token-Capping) Architecture
Intelligent Model Router
Path A: Routine/Low-Risk
Execution: SLM/Open-Source Models
Constraint: Deterministic/Capped
Path B: High-Complexity
Execution: Frontier Engine
Constraint: Refinement Capped at N-Loops
GUARDRAILS
Force Traffic Through an Enterprise AI Gateway. Block application teams from hardcoding direct API connections. Route all AI traffic through a central gateway.
Dynamic Model Routing: Evaluate prompt complexity at runtime. Route simple extraction, classification, or formatting to Small Language Models (SLMs). Save frontier engines for multi-variable reasoning.
Semantic Caching: Cache responses for recurring enterprise queries to avoid paying twice for duplicate compute.
Cap Refinement Loops. Treat agentic iteration like a manufacturing line with hard SLAs.
Circuit Breakers: Hardcode maximum retry limits (e.g., 2 or 3 refinement loops). If an agent fails to self-correct within set thresholds, degrade the service gracefully or trigger a human-in-the-loop (HITL) handoff.
Context Pruning: Strip static system prompts, conversation history, and unused tool definitions before re-submitting context. Prompt compression cuts token volume substantially without degrading output quality.
Apply DMAIC to Token Consumption. Apply Lean Six Sigma to strip non-value-added token waste:
Define: Establish the baseline P&L return required for every agent workflow.
Measure: Track token consumption per completed transaction across inference, retrieval, and middleware.
Analyze: Target high-variance workflows showing 30x cost dispersion.
Improve: Swap open-ended agent reasoning for deterministic code or fine-tuned micro-models on repetitive tasks.
Control: Set real-time dashboards with automated spending limits per department.
Shift to Direct Chargeback. Replace seat-licensing estimates with consumption chargebacks.
Pool consumption: Pool consumption spend to secure enterprise volume discounts, but bill usage directly to the initiating business unit’s P&L.
Justify consumption: Force business unit leaders to justify AI consumption against actual revenue or labor savings.
Master Agentic Economics
Deploying AI agents without financial controls is not a strategy. It is an expensive experiment.
Competitive advantage will not belong to the organizations with the most agents or the largest AI budgets. It will belong to those that master Agentic Economics: delivering business outcomes at the lowest sustainable unit cost while maintaining quality, reliability, and governance.
That requires a shift in mindset. Stop managing AI like software. Start managing it like a variable operating expense. Every workflow should have defined cost boundaries, measurable business outcomes, and architectural guardrails that prevent unnecessary consumption before it reaches the P&L.
I highly recommend you read the McKinsey QuantumBlack report.
Next 30 Days
Audit your highest-cost workflows. Identify the users, agents, and processes driving the majority of token consumption.
Establish an enterprise AI gateway. Route AI traffic through a common control layer with centralized security, policy enforcement, intelligent model routing, and semantic caching.
Set architectural cost guardrails. Cap refinement loops, prune unnecessary context, and define maximum execution limits before costs compound.
Make AI spending financially visible. Replace flat IT allocations with consumption-based reporting and chargeback so every business unit understands both the cost it creates and the value it receives.
Great article and great cover image. Cape Town my hometown.