TL;DR EverOS (70.1%) and Hindsight (67.8%) are the best. They scored as well as giving the model the whole conversation (69.5%) while reading 24–35x less text, and were the only two to clearly beat plain keyword search. EverOS saves chats as dated notes, which kept it strong on instructions, timing and facts that change. Hindsight was the best at recalling exact details. With only 60 questions, ranks 3–8 aren’t clearly separated. Results Score by ability (%) Only 6 questions per ability, so treat these as signals. Setup Benchmark: BEAM 100K (Tavakoli et al., Mohammadta/BEAM on Hugging Face). 3 conversations fixed in advance: #6 (Writing), #1 (Coding) and #13 (Recommendation). That’s 345K tokens and 60 rubric-graded questions across 10 abilities. Models: Claude Haiku 5.5 (effort xhigh) ran each system’s LLM calls and answered every question. Claude Opus 5.5 (xhigh) judged the answers. Seven systems used OpenAI text-embedding-3-small for embeddings; memsearch, Hindsight and claude-mem used their own. Answer step: the same prompt for all, given each system’s full retrieved memory, with every system on default settings. Full context gets the whole conversation; BM25 fills an 8K-token budget by keyword search. Scoring: BEAM’s judge prompt and rubric, scoring each item 1, 0.5 or 0, with 95% bootstrap CIs. submitted by /u/Mundane-Educator75
Originally posted by u/Mundane-Educator75 on r/ClaudeCode