Upfront disclosure: this is my write-up (and I’ll link it below), but laying out the argument here so you can strawman/steelman it without clicking anything. Assertion 1: per token price is the wrong metric for measuring the cost of work done by LLMs/reasoning models. Users get charged the per token price regardless of whether the output/outcome was right or not. Assertion 2: real work lives in long chain processes. Reliability of agents (run through LLMs) drops geometrically in proportion to chain length. 95% per step accuracy translates to 77% process reliability for a 5-step process, 60% for 10, and under 36% for a 20 step process. This calculation holds if errors are independent, which isn’t true for real world processes, ergo real world reliability is worse than that. This adds a verification tax on top of the price of tokens the user pays. You can verify through human intervention, inference time compute (less reliable than human intervention), or swallow the decay in reliability. Argument: granted 1 & 2, you can’t reliably automate any meaningful work through LLMs/agents in a cost-effective way, because it isn’t an issue of economics but of architecture (LLMs can’t reason faithfully, which was my previous essay) Link: https://open.substack.com/pub/mauhaq/p/price-is-not-cost?r=7eoi8&utm\_campaign=post-expanded-share&utm\_medium=web submitted by /u/Sensitive_Air_5745
Originally posted by u/Sensitive_Air_5745 on r/ArtificialInteligence