We're Learning Backwards | Please Don't Cite Me
pleasedontcite.me
LLMs are brilliant at many things, but still fail in unreasonable ways. Is that an issue of scale, or something more fundamental?

Original Reddit post

Following the recent release of ARC-AGI-3 and the performance of SOTA models on it, I’ve been thinking a lot about what intelligence is. Why do LLMs feel so smart yet occasionally do unequivocally dumb things? Why are humans so sample-efficient? Are LLMs the path to AGI? I argue that LLMs are learning backwards, starting with all the knowledge in the world and trying to distill intelligence out of it. Essays like Sutton’s Bitter Lesson and Gwern’s Scaling Hypothesis may remain true at the limit, but we only have finite data and I don’t think this approach will bring us AGI without significant innovation. submitted by /u/preyneyv

Originally posted by u/preyneyv on r/ArtificialInteligence