Cost per Task
How to define and control cost per task with budgets, caching, and routing.
Published:
Admin User
Updated:
published
Cost per Task
Cost per task is the unit metric for controlling LLM spend.
Control it with budgets, routing, caching, and monitoring-based triggers.
See also
Cost & Latency Controls Cost Spike Control (LLMOps) Cost Spike RunbookFAQ
What is cost per task?
The average cost to complete one unit of work (including retries, retrieval, and fallbacks).
How do we lower cost per task?
Routing, caching, prompt optimization, and reducing retries via better evaluation.
How do we detect cost spikes?
Monitor per-task cost distribution and alert on anomalies.
What’s a common anti-pattern?
Tracking only total spend, not unit economics per task.
What’s the first improvement?
Instrument cost per request/task and set a budget threshold.