eifachposte

These systems are still far from perfect! Deep research agents look impressive when you watch one plan a multi-step investigation, gather sources, and hand back a tidy cited report. A new arXiv paper from Pengyu Zhu and colleagues puts an uncomfortable number on how easily that pipeline can be steered off course, and the number is 85.5%. The setup is straightforward. The authors built a framework they call MisKnow-Agent to generate 5,933 quality-controlled misleading-knowledge instances, then injected them into two open-source deep-research frameworks, DeerFlow and WebThinker, plus the closed-source Gemini Deep Research. In a clean no-injection control, the false-conclusion adoption rate was 0%. Introducing a single misleading document raised the mean adoption rate to 54.7%. The part I find most useful is that timing dominated everything else. At cold start the rate was 40.5%. During mid-research it was 44.2%. But when the misleading knowledge arrived immediately before final synthesis, adoption jumped to 85.5%. The closer the bad evidence sat to the moment the agent wrote its answer, the more it dominated the answer, regardless of what surrounded it in the retrieved pile. The defenses the authors tested are the ones a reasonable engineer would try first: a verification-enhanced prompt at the front of the query, and a search-enabled refinement agent that goes back over the final report claim by claim. Both reduced the false-conclusion adoption rate. Neither eliminated it. The uncomfortable finding underneath that is that cross-model verification could correctly classify a document as misleading, and the agent would still adopt its conclusion anyway. Flagging is not the same as refusing.

Originally posted by u/Justgototheeffinmoon on r/ArtificialInteligence

Deep Research agents adopt false claims at 85.5% peak rate