Blue = the fix (split the ask into small pieces). Red = the normal way, asking for the whole thing at once. Every big model scores 0% the red way; a small open model gets 85% the blue way. (Frontier numbers are the published single-call baseline.) Spent months on this and the answer surprised me, so sharing it here. There is a public benchmark of real documents with human-checked answers. One task asks an AI to pull 369 specific values out of a 60-page financial filing in a single pass. Six of the best models around (GPT-5, Claude Opus, Gemini 3 Pro, and more) were tested. Every one scored 0%. Not close. Zero. Every document failed. Here is the counterintuitive part: they are not too dumb. The request is the problem. Asking for hundreds of answers as one giant structured form (a big block of JSON) makes the model find the answers AND type the whole form perfectly at once. On a wide form it runs out of room, the output cuts off halfway, and the whole reply is thrown away, even the answers it got right. What fixes it is not a bigger model. It is asking in a smaller way: one answer per line instead of a giant nested block, and splitting the big form into pieces that fit. Done that way, a small open model gets 85% on the exact task the giants failed at. This is large-scale structured output / data extraction, and the win is the method, not the model. To be fair: nobody re-ran the big models with this smaller-pieces approach, their 0% is the standard one-shot result. So this is not a claim the small model is smarter, it is that small asks are easier, and the same approach would likely lift the big ones too. I built this into an open-source tool (link in a comment) and would genuinely love feedback. Makes me wonder how much of what we call an “AI capability ceiling” is really just us asking for too much in one breath. Where else do you think this shows up, coding, agents, long outputs? submitted by /u/Admirable-Ease-6470
Originally posted by u/Admirable-Ease-6470 on r/ArtificialInteligence