Some of our customers noticed Inter-1 (our omni-modal social-signal model) would occasionally “hear” a quote that didn’t exist. Feed it a video with zero audio and ask what was said, and it would sometimes report: “Yeah, Friday at five.” Verbatim. Same line, every time. We assumed it had to be baked into the training data somewhere, so we went looking everywhere: 30,960 training records with datetime mentions → zero hits on the phrase 4,603 video transcripts → zero hits ~800 inference probes, 584 storage objects → zero hits Turns out the phrase was sitting in our own system prompt — a worked example we’d written to show the model the expected output format, buried in a version our GEPA prompt-optimizer had shipped. But that only explained where the words came from, not why the model would say them over total silence. So we ran two ablations in our internal eval harness: Swap the word, keep the model: changed the prompt’s example to “Tuesday at noon.” Fabrication rate went up (37%→50%), and the invented quote tracked the swap exactly — Friday→Tuesday. Swap the model, keep the prompt: ran the same byte-identical prompt through larger variants and an earlier checkpoint of our own model. They barely fabricated (0–2%). Only the further-post-trained Inter-1 confabulated at ~12%. So it’s not one bug, it’s two stacked priors: the prompt supplied the script , but post-training is what gave the model the compulsion to recite something rather than report silence. Deleting the prompt example stops that one sentence — it doesn’t stop the model from inventing different dialogue instead. We think this is a textual/in-context variant of the audio-visual “Clever Hans effect” that’s been documented for vision priors (model writes “thud” over a silent skateboard wipeout) — except ours shows the same reflex gets worded by whatever’s nearest in the context window, which a vision-only diagnostic wouldn’t catch. Full writeup with the fabrication-rate forest plot and log data: https://www.interhuman.ai/blog/goblin-yeah-friday-at-five submitted by /u/Sardzoski
Originally posted by u/Sardzoski on r/ArtificialInteligence