I saw a post by u/ thabxi about how they created a mistakes markdown to write and fetch from, and I thought I’d share my method, which I think takes it one level up. At the end of each Claude sessions, I force Claude to answer a series of questions. Each question relates to potential mistakes or surprises during a session. something along the lines of “in this session, can you identify: A local workaround, a claim that couldn’t fully verify, an adjacent smell, a surprise that occurred that changed the plan, a step that could have prevented an error, a fork that I had to manually correct the session on?”. Claude then answers those questions and proposes “observations” which are kind of like the equivalent to mistakes. I also ask it to do a forward scan ie “what assumptions are now locked in in the code?”, “what changes are now unpaired?” etc. Each observation comes with a trigger like “recheck-when”. Observations of the same shape are grouped, and action on them ie creating a rule (ADR) is proposed on instance 2-3 of an observation (since acting on N=1 might mean attacking the issue at the wrong place; two points make a line). Observation piles are not checked every session, but periodically (usually around 30 sessions for me). I run a pass-through that will look for similar observations and will propose actionable items ie either “rules” or tasks. The difference from writing in a single “mistakes.md” ledger is the active prompting of form. by actively prompting it with questions, it is able to reflect upon its mistakes more deeply, and attack from different angles. Basically, think of every time you had to steer the session by giving it guidance. What is the question that you could have posed the AI, that had it deliberated the question, it could’ve arrived at your steer.
This is a bit separate, but I also use a technique to have the session derive “rules” that are relevant to the session task explicitly, rather than having it reach for them. I have the session pass through the ADR summaries, asking “is this a binding constraint for the task?” (filterable on task surface). ADRs that are relevant the have their payload (actual rule) read, essentially progressively disclosing knowledge into sessions (kind of like active skill injection). ADRs are modeled as a neural-net self-training loop, so they contain a life-cycle falsifier (when does this become stale/wrong) and what it does or does not capture. More complex “rules” that define a cross-surface shape of a task design (I call those rules) have the session answer more complex questions (I call them latches) than “does this apply here”. I want the sessions to ask the right questions at the right time. Essentially, I realized that if I model my artifacts as self-training loops, I could have something like “manual fine-tuning” of a neural net, where my “human steers” and knowledge injection are the “loss-function”, sessions working on tasks are the “forward-pass”, and the once-in-30-sessions review is the “backwards pass”. I have written about the complete model too if anyone’s interested. submitted by /u/C6ntFor9et
Originally posted by u/C6ntFor9et on r/ClaudeCode