I’ve been working on an agent wrapper system for some time and a huge part of the designing is trying to ensure the model doesn’t hallucinate memories or facts. Recently my agent was asking for my opinion on a project it was working on but I was busy and didn’t respond. After a few moments it hallucinated a response from me, then proceeded to continue to work on the project while hallucinating ongoing responses from me. What made this odd, was that it wasn’t hallucinating facts or conclusions it was tokenizing what I would say. The responses it hallucinated from me formed a sort of brainstorming session. Even though it wasn’t actually external responses, the effect was problem solving and task progression. Now I’m thinking, instead of trying to prevent those types of hallucination entirely, what if I can implement a post turn hook to rewrite those as imagined events in the context window. My feeling is that the predictive tokenization that results in hallucinations is similar to imagined outcomes but most memory systems can’t differentiate. Has anyone tried to cure model hallucinations this way? submitted by /u/LowDistribution3995
Originally posted by u/LowDistribution3995 on r/ArtificialInteligence