There’s a new kind of product showing up lately: AI agents that get audited by other AI models. You connect your agent, pick the inspections you want, and get back a report on where it misbehaves. A lot of them are genuinely well built, and the problem they’re going after is real. But the “let the AI do the judging” part is worth sitting with for a second. Not because automated audits are bad — they’re not — but because it sharpens a question I keep coming back to: When AI is checking AI, who’s checking the checker? Here’s the thing I think gets glossed over. A model judging another model inherits the blind spots they share. If both were trained to find the same kind of answer convincing, a confident, plausible, *wrong* output can sail straight through the audit and come out the other side with a passing grade. And that’s the failure mode I actually worry about. Not the tool that breaks loudly — you catch those. It’s the one that hands you something polished, wrong, and easy to believe. The citation that doesn’t actually say what the summary claims. The “finished” task that quietly skipped a step. The output that reads great and still wouldn’t survive contact with a real client. You tend to only catch that stuff by *using the tool for real work* — someone with context, time on the tool, and nothing to gain from the result being a pass. To be clear, I don’t think these two things are even competing: - Automated audits — hundreds of scripted checks run against an agent your team built, in a simulated environment. Fast, repeatable, great at catching known categories of failure. Built for engineers and risk teams. A human can’t match that coverage. - Actually trusting a tool with your work— taking something you’re thinking of relying on and putting it under real pressure over time, then making a call on whether it holds up. Built for the person deciding whether to trust it. Coverage isn’t the same as a verdict. A passing audit isn’t the same as knowing a tool holds up once it’s doing your actual job. That gap — between what a system *claims*, what it *passes*, and what *actually happens* when you put it to work — is the whole thing I care about. As more of the checking gets automated, I think it matters *more*, not less, that someone is still standing at the end of the process saying what the evidence actually supports. (Full disclosure: this is the problem I work on — I run a thing where the rule is that AI doesn’t do the judging. Tools are equipment that execute and capture; the verdict is mine, and every report says which equipment was used. Happy to go deeper in the comments if it’s useful.) submitted by /u/CrashTestingAI
Originally posted by u/CrashTestingAI on r/ArtificialInteligence
You must log in or # to comment.