Original Reddit post

I have been wondering whether one of AI’s deepest problems is simply that we named it after the wrong thing. “Artificial intelligence” defines the technology through its resemblance to human intelligence. It encourages us to see every capability as an imitation of something humans do, and every difference as a defect awaiting repair. Imagine if drum machines had been called fake drummers . We might have spent decades judging them by how convincingly they reproduced a human drummer. Perfect imitation would be success. Mechanical repetition would be failure. Sounds, rhythms and timings impossible for a human performer would look like bugs or gimmicks. Instead, the drum machine became an instrument. Its inhuman precision and repetition became the point. Entire forms of music grew from properties that would have counted as failures in an “artificial drummer”. I suspect we are making the inverse mistake with AI. We call these systems intelligent, then organise the entire conversation around whether they really think, understand, know, remember or reason. The machine remains an unfinished person. Its native properties are interpreted as either human abilities or broken versions of human abilities. “Hallucination” may be the clearest example. A hallucinating person normally perceives something which is absent. The word implies a mind with an ordinary relation to reality that has suffered a temporary perceptual failure. An LLM is doing nothing comparable. It generates plausible continuations. A correct citation and an invented citation emerge through the same basic process. “Hallucination” is the name we apply afterwards, when we compare the output with an external source and discover that we had silently required correspondence with reality. The model did not briefly depart from its normal operation. It operated normally in a context where normal generation was insufficient. Of course models can encode signals correlated with truth and uncertainty. They can sometimes recognise contradictions, estimate confidence or detect that an answer is unstable. But this is different from possessing a dependable, general boundary between “language that fits” and “language that corresponds to the present world”. So perhaps hallucination is less a pathology inside the model than a category error at the system boundary. We placed a generator in the role of witness, database or oracle, then gave its predictable failures a psychiatric name. This framing affects engineering. If hallucination is a disease of the artificial mind, we search for a cure inside the model. If it is unsupported generation crossing a boundary where factual correspondence matters, we build provenance, retrieval, verification, observability and constrained effects around it. The same problem appears in words such as: memory for stored claims about previous states; reasoning for generated intermediate tokens; agent for a model inside a control loop; confidence for properties of a probability distribution; autonomy for a system operating without immediate human input. These metaphors can be useful. The trouble begins when the metaphor quietly becomes the specification. There is already serious work on anthropomorphic AI language. Murray Shanahan has written about the hazards of saying that language models “know” or “believe”. Other researchers describe AI terminology as strategic polysemy : words retain a narrow technical meaning while borrowing the emotional and philosophical force of their human meaning. But I think there is a further issue. Anthropomorphic language does not merely exaggerate these machines. It may also impoverish our imagination of them . If we treat AI as a deficient mind, we ask how to make it more human. If we treat it as a new computational instrument, we can ask what becomes possible precisely because it is alien: excessive association, cheap variation, disposable cognition, parallel exploration, strange compression, tireless response to observable state. Those may be more interesting than producing a convincing office worker in a glass box. We named a new instrument after the thing it superficially resembles. Now we treat everything unique to the instrument as either a miraculous human ability or a failed imitation. The drum machine escaped the drummer. Has AI escaped “intelligence” yet? Further reading: Murray Shanahan, Talking About Large Language Models : https://arxiv.org/abs/2212.03551 LaCroix, Mallory and Luccioni, Strategic Polysemy in AI Discourse : https://arxiv.org/abs/2604.21043 Liane Potter, Naming Is Framing : https://arxiv.org/abs/2504.13957 Ji et al., Survey of Hallucination in Natural Language Generation : https://arxiv.org/abs/2202.03629 submitted by /u/Genaforvena

Originally posted by u/Genaforvena on r/ArtificialInteligence