I’ve been using Ai a lot the last few years and over the last few months I’m increasingly convinced that we are just a few architectural changes away from real machine intelligence. LLM’s are “just next word predictors” is a phrase you’ll hear a lot. It’s a stochastic parrot. And while there are a lot of things that might defend that point of view I think it should be very obvious that it’s a hand wave that doesn’t even make an attempt at understand what these models are doing. There has been a lot of interesting research especially over the last year that does go a lot further in explaining how the models work and I think the most interesting research is the ones examining the topology and the geometry of them. I think the proof that this is moving in the right way is that the models have internal configurations that show us that the models represents concepts across languages with the same internal geometry. A horse has the same internal representation in the model in all langues. It’s not a difference concept in french or English. To me that’s prof that there is something way deeper going on here than simple token prediction or stochastic. If you map the dimensional complexity layer by layer, it follows a distinct curve. The early layers handle the surface-level stuff (token identities, basic syntax). Then, in the middle layers, the intrinsic dimensionality expands. This is where the model is doing the heavy lifting, mapping concepts into complex, high-dimensional spaces to figure out latent relationships, logic, and context. Only in the final layers does it compress that space back down to make a deterministic choice about which specific token to emit. We can see similar things in real human brains. If we think about square, we have an internal representation of that geometry that we map on to all kinds of things. We have internal abstractions and its becoming very clear that LLM’s also have internal abstractions. It seems the deeper we dig in both real human cognition and machine intelligence, the more we are converging on concepts we know from theoretical physics regarding topology and geometry that just fit very neatly. There is extremely interesting research in these areas, trying out different things like phase state representations, flat topologies, repayable deterministic reasoning and lots of other things. I think we are really on the cusp of discovering how cognition works, and we are all ready doing pretty good approximations on it with LLMs. I think when we do finally crack this, it will be orders of magnitude cheaper than the current transformers and it will completely wipe out the value of investment made in to a lot of these data centers but that’s besides the point. The Geometry of Multilingual Language Model Representations (Chang, Tu, & Bergen) Geometry of Decision Making in Language Models (October 2025 / OpenReview 2026) The Lattice Representation Hypothesis of Large Language Models (Xiong et al., March 2026) submitted by /u/Claptraposoid
Originally posted by u/Claptraposoid on r/ArtificialInteligence