Opus 5 and GPT 5.6 are very intelligent and clearly capable of spotting bugs that previous versions would not. However, they are also prone to overthinking and over engineering simple problems. For instance, instead of a simple general parsing rule, they might instead produce a gigantic lookup table with hundreds of tests written against it. Same function in the end, but slower, arguably less robust (what edge cases are NOT in its lookup table?), but generates lots of code, lots of tests, and requires more time to produce. This suggests models are were RL-trained to optimize for completeness (test coverage) and long-horizon tasks (side effect is that model can churn for hours optimizing unnecessary edge cases which increases wall time but looks good to automated classifiers). I think this also causes progressive deviation from the original scope as the model tries to overgeneralize but fails because instead of being parsimonious it just tries to find edge cases. This leads to making assumptions not visible to the user, and the output prose based on those assumptions becomes incomprehensible. TLDR: models are post-trained for test coverage completeness, robustness, and long-horizon tasks, with likely automated classifiers that reward test coverage growth at the expense of parsimony. Parsimony is much harder to quantify and classify - so it will lose out in an RLAIF setting. So, what’s the best way to use these models? how can this clearly trained tendency be managed? One obvious solution is to not use them until they’re tuned better, or just use them as adversarial reviewers. Another option is to (over?)engineer your harness to constrain these models. submitted by /u/ZhopaRazzi
Originally posted by u/ZhopaRazzi on r/ClaudeCode
You must log in or # to comment.