We run gpt and claude at work plus two open weight models depending what i am doing, been at the same place four years so ive watched the slack react to every big release in that time. Gpt-5.6 sol drops from openai, then grok 4.5 from xai basically the same day, $2/$6 per million and people calling it opus-class. Year ago that combination would have had the channel going till midnight. I counted five messages. one was a meme about elon that had nothing to do with the actual model. And i cant get a straight answer out of anyone about what grok 4.5 even improved. not because they are lazy, they just can not point at the thing that matters for what we ship. we could sit down and benchmark it properly but who has the afternoon, so the stuff we already have keeps running till it falls over or my manager asks why were behind. The last release that actually did something to me was sonnet 3.5. i had this json parsing prompt, nested mess, 3 kept mangling it and i would been poking at it on and off for like two days, and 3.5 just returned it clean like it was nothing. that was what, mid 2024. everything since is tuning. tool calling gets a bit tighter, some cheaper mid variant shows up. and look im not saying thats fake work, i know the effort that goes into it, but it does not land in your hands the way 3.5 did or gpt-4 before that. What actually changed is underneath. I have had Glm-5.2 doing the grunt work for a few weeks, long context bug hunts across our services, refactors that pull in more files than i want to think about. Terminal-bench 2.1 has it at 81, Grok 4.5 at 83.3, opus floating around the same spot and the price is $1.40/$4.40 versus grok $2/$6 or opus 4.8 at $5/$25. The hard reasoning stuff, Opus and sol still pull away and you feel it inside a single session, i am not going to sit here and tell you Glm matches them there. but most of my week is not the hard stuff. It is the middle. and the middle has three or four models clearing the bar now where a year ago it had one, so the token math starts making the call before i do. my inference bill is somewhere behind rent and groceries and frankly too much coffee this quarter which is its own kind of funny. So you got a ceiling that is barely moving and a floor that is climbing. either the labs are handing us small stuff on purpose and sitting on the real jumps, or everyone hit the same wall at the same time and nobodys going to be the one to admit it out loud. i genuinely dont know which and i go back and forth. Both of them are a bad look in a launch blog so obviously neither ever shows up in one. Honestly at this point i would respect a lab just saying maintenance release, nothing exciting, take it or leave it. instead of another paradigm shift nobody on my team can even remember by the end of the week. submitted by /u/ServeAccomplished485
Originally posted by u/ServeAccomplished485 on r/ArtificialInteligence