Anthropic already showed that models do introspection in the process of minimizing loss (as creating coherent reconstructions of the data they were trained on means thinking in advance). The issue with LLM models is that they must redundantly recompute internal representations that might be similar token to token. For example, imagine a model is trying to answer some question about math. The model needs to internally evaluate the direction in which it’s going to try to solve the math problem even before outputting the first token (Let’s not think about more modern reasoning models). The model will choose the token most likely to yield the outcome it was trained to generate, so probably succesfuly solving the math problem. The next autoregressive run will have to however see what the first token generated is, and based on that, try to guess what the model previously wanted to do, since it forgets it’s internal reasoning between token writes. Model intention -> token conversion process is very, very lossy since the model forgets its intuition and intention from the last token. Each fresh token generated means a lot of redundant calculation to understand the direction of the model’s previous token generations before it can even generate outputs. This likely serves as error correction, but my feeling is that it is very expensive. I know there are many research directions that try to approach cognition (from test-time computation via reasoning models to looping models), but these mainly solve the problem of hard tokens instead of solving the problem of losing compute cycles. AI is very expensive at this point, so having to waste 10-20% (could be even more, but i’m just throwing numbers I can’t validate) compute seems super wasteful, especially at scale. I guess some research directions may solve this problem indirectly. submitted by /u/incorporo
Originally posted by u/incorporo on r/ArtificialInteligence