Personal Lemmy
  • Communities
  • Create Post
  • Create Community
  • heart
    Support Lemmy
  • search
    Search
  • Login
  • Sign Up
eifachposteMB to AI (Reddit RSS)English · 16 hours ago

Can it be that cache miss is making Claude dumber??

www.linkedin.com

external-link
message-square
0
link
fedilink
1
external-link

Can it be that cache miss is making Claude dumber??

www.linkedin.com

eifachposteMB to AI (Reddit RSS)English · 16 hours ago
message-square
0
link
fedilink
Want to hear something that blew my mind today? You know how when you come back to a Claude Code session after a few hours, the model feels noticeably dumber? Slower, clunkier, worse responses than… | Yoav Alon
www.linkedin.com
external-link
Want to hear something that blew my mind today? You know how when you come back to a Claude Code session after a few hours, the model feels noticeably dumber? Slower, clunkier, worse responses than you'd expect given where you left off? I assumed this was about the KV cache. It's not. It's something more deliberate. After digging into the Claude Code leaked source, here's what's actually happening. Anthropic's prompt cache has a 1-hour TTL. When you're idle long enough for it to expire, Claude Code fires what it calls a time-based microcompact. It recognizes the cache is cold, and proactively strips out the content of every tool result in the conversation except the last 5. File reads, grep output, shell results, all replaced with [Old tool result content cleared]. Why? Cost. There's no point re-uploading tool results that won't benefit from the cache anymore. But the side effect is that the model literally can no longer see the actual data it worked with. Only the summaries it produced during compaction. So you're not getting a dumber Claude. You're getting a Claude that's effectively starting a new conversation with a sketch of the old one. It's a reasonable engineering tradeoff, but it explains the felt experience exactly

Original Reddit post

Originally posted by u/smulikHakipod on r/ClaudeCode

alert-triangle
You must log in or # to comment.

AI (Reddit RSS)

ai_reddit

Subscribe from Remote Instance

Create a post
You are not logged in. However you can subscribe from another Fediverse account, for example Lemmy or Mastodon. To do this, paste the following into the search field of your instance: !ai_reddit@lemmy.durstig.online

AI (Reddit RSS Feed)

Visibility: Public
globe

This community can be federated to other instances and be posted/commented in by their users.

  • 19 users / day
  • 112 users / week
  • 260 users / month
  • 701 users / 6 months
  • 1 local subscriber
  • 29 subscribers
  • 14.5K Posts
  • 229 Comments
  • Modlog
  • mods:
  • eifachposte
  • BE: 0.19.15
  • Modlog
  • Instances
  • Docs
  • Code
  • join-lemmy.org