I’ve been thinking about this for months, and I feel like I’m taking crazy pills. How have we collectively agreed to a pricing model where the unit of measurement is a “black box” that the seller controls entirely? When you buy a gallon of gas, a gallon is a gallon. When you buy a kWh of electricity, it’s a fixed physical constant. But a “token”? A token is whatever the company says it is. Imagine going to a car wash. The sign says $10 per 5 gallon wash. You press the button, and today the machine decides to use 50 gallons of soap instead of 5. At the exit, the attendant says, "That’ll be $100. Yeah, the machine felt like it needed more suds today. Those are the rules. This is exactly how LLM output works. I can send a “Hello” prompt today and get a 10-token response. Tomorrow, the company can tweak the system prompt or the model’s “temperature” settings internally, and suddenly that same “Hello” triggers a 500-token rambling response. They have a direct financial incentive to make models more verbose because they charge by the “word”. In any other industry, if your scale is off by 5%, you get fined by the government. In AI, if a company updates their tokenizer and your bill jumps 20% for the exact same workload, it’s just called “progress.” Am I the only one who thinks this is insane? We are building entire businesses on top of a pricing structure where the vendor controls the meter, the unit of measurement, and the amount of “product” consumed per request. submitted by /u/Matthew_Code
Originally posted by u/Matthew_Code on r/ArtificialInteligence