When comparing agent workflows, cost per successful task is a more useful denominator than cost per model call. A cheap call can still lead to several searches, another generation attempt, and a result you reject. Here’s a simple way to account for it: Log the billed model calls, tool calls and compute usage for every attempt, including attempts that fail or produce an unusable result. A failed request may or may not be billed; use the provider’s actual usage record. Define an acceptable output before comparing workflows. For example: a brief with sources that pass a manual check. An HTTP 200 is not the same as a completed task. Divide the total billed cost by the number of accepted outputs. Include any applicable fees in what you actually paid. Illustrative example: 100 attempts cost $15 in total and 75 outputs pass review. That’s $0.20 per accepted output. If the $15 represents usable credit purchased with an additional 10% fee, the cash cost becomes $16.50, or $0.22 per accepted output. These are hypothetical numbers, not measured results or provider prices. Keep unused prepaid balances separate: they’re cash tied up, not usage consumed. Combining them with run costs makes it harder to tell whether your problem is inefficient execution or overfunded accounts. For comparisons, hold the task and acceptance criteria constant, then record both cost and success rate. A cheaper model with more rejected outputs can be the more expensive workflow. Do you measure accepted outputs in your agent setup, or mostly tokens and API responses? Disclosure: I’m building DreamTune Labs, a shared-balance project for AI services, so this is a problem I’m working on: https://dreamtunelabs.com/ submitted by /u/legendpizzasenpai
Originally posted by u/legendpizzasenpai on r/ArtificialInteligence