When you turn on web search in an AI assistant and see 15 clickable links at the bottom of the screen, it is natural to assume the AI read those pages and built its answer from them. A new study tested what actually happens between the search step and the final written answer across 160,860 cleaned citations. They ran 614 questions three times each on the web and mobile app versions of four major AI assistants, which are DeepSeek, Alibaba’s Qwen, ByteDance’s Doubao and Tencent Yuanbao. The biggest takeaway is that a huge share of the source links an AI displays on screen are never actually used to write the response: A few details from the paper make this even more interesting: Traditional search engine authority barely mattered to the AI. The authors tested a classic domain authority score based on indexed pages, site weight and domain age, and it ranked 10th out of 11 factors. Repetition across separate websites mattered far more than how authoritative any single website was. Pre-trained memory still overrides live web search when prompts are broad. Overall, 13.02% of the names in the final answers could not be matched to the pages the AI cited (hitting 16.9% on DeepSeek Web vs 10.1% on Doubao Web), and on vague questions that jumped to 27.5%. The phone app and laptop browser on the exact same AI do not even search the same web. On Alibaba’s Qwen, 64.2% of identical questions shared zero exact URLs between the web browser and the mobile app. DeepSeek was the most consistent between web and mobile at 40.8% link overlap, followed by Doubao at 31.0% and Yuanbao at 9.7%. A fair caveat from the authors is that their scraper only read the main body text of cited pages and skipped pop-ups, footer cards and page code, so a slice of that 13.02% unmatched rate could come from page metadata rather than pure model memory. Their semantic scores were also graded by an LLM rubric rather than human annotators. Nearly 40% of the source links an AI search engine shows you at the bottom of the screen are just search leftovers that the model never actually used to write the answer. The study is called “What Do Chinese-Language Generative Search Engines Cite and Surface? A Large-Scale Empirical Study” (arXiv:2607.15771) by Tao Zhen, Yue Liu, Gege Zhang and Yixuan Niu, submitted July 17, 2026. When you use web search on DeepSeek, Qwen, ChatGPT or Perplexity, do you want the interface to show every page the search tool scanned, or should it only display links that directly back up a specific sentence? submitted by /u/Sanbi_Ai
Originally posted by u/Sanbi_Ai on r/ArtificialInteligence