I’ve been reading quite a few projects that try to deal with long conversation histories in LLM applications. Most seem to rely on some combination of truncation, recency windows or summarisation. I came across an open source project called Foveance that takes a slightly different approach. Instead of assuming the most recent messages are the most valuable, it tries to allocate the available context based on what is expected to matter for future prompts. The repository is refreshingly careful about its claims. It explicitly distinguishes its work from earlier approaches like AFM and compares against straightforward baselines as well as LLMLingua-2. The benchmark scripts and CSVs are included, so the reported numbers are reproducible rather than screenshots. From the README, the practical idea is fairly simple: • use it as a Python library to shrink an OpenAI-style message list • run it as a proxy in front of OpenAI, Anthropic or Ollama-compatible clients • or wrap existing tools like Claude Code or Codex without changing application code What I found more interesting than the token savings was the motivation. The project argues that longer contexts are not always better because important information can become buried under less relevant conversation history. The benchmark is built around that problem instead of simply measuring compression ratios. I haven’t tried it in production yet, so I’m more interested in hearing from people who’ve worked on long-context memory systems. A few things I’d be interested in discussing: Does future-relevance allocation seem like a better direction than recency heuristics? Are there workloads where this kind of approach would obviously break down? Has anyone compared similar ideas against retrieval-based memory instead of context compression? Repo: ) https://github.com/Aimaghsoodi/foveance I thought it was worth sharing because it’s one of the few repositories I’ve seen recently that makes a serious effort to define exactly where its contribution starts and where previous work already exists. (Edit to fix link) submitted by /u/KrakenJiuJitsu
Originally posted by u/KrakenJiuJitsu on r/ArtificialInteligence