We all know, most news you read is factually true and still built to make you feel a certain way about it. A headline that overreaches what the article actually proves. A number with no baseline to compare it against. A quote from whoever benefits most from the story landing a certain way. Fact checking catches lies. It doesn’t catch that. Nobody really has a name for the second problem, so I’ve been calling it a “framing-check.” That’s the itch that got me building this. Not “replace journalists,” I don’t think that’s possible or desirable. I wanted to see if a pipeline of AI agents, each with a narrow job and none of them trusting the previous one blindly, could catch the framing problem the way fact checkers catch the factual one. Here’s how it works. A cheap model triages RSS feeds each day and picks a handful of candidate stories. A researcher agent goes only to primary sources, papers, filings, company announcements, and builds a dossier where every claim is tagged to where it came from. A writer agent drafts the article, but it only ever sees that dossier, never the original coverage anywhere else. It literally can’t copy someone else’s framing because it’s never read it. A separate mechanical pass then checks every quote against the dossier word for word, every number against its label, every link, with zero authority to touch the actual story, just flag mismatches. Then an editor agent approves or rejects the draft against a public style guide. If it’s approved, it gets an illustration, a translation, and only then do I do the final human check before anything goes live. One week in, the numbers are humbling in a good way. Most rejections come down to three things. Execution errors, like a quote’s capitalization getting changed to fit a sentence, or a number getting swapped for a similarly sized one that means something different. Single source stories with nothing independent to confirm them. And conflicts of interest, like a security company “discovering” a vulnerability that its own product happens to fix. The one that actually stuck with me happened by accident. The editor had already approved an article. I was moving it toward publishing when the same file got run through the editor a second time. It rejected it. Same text, not one byte changed. The second pass caught a valuation figure with no basis for comparison that the first pass had missed completely. That’s not something you patch with better code. It’s just what happens when you use a model’s judgment as a gate. It isn’t perfectly consistent, the same way a human editor catches different things depending on the day. So the actual fix isn’t “trust the AI more.” It’s pulling as much as possible out of judgment and into things that run the same way every time, like counting sources cited or checking every link resolves to something real in the dossier, and keeping a human as the genuine last check before anything publishes. That part isn’t optional and never will be. Every article ends with a section spelling out exactly what the evidence does not support, and a small mechanical count under it, sources cited, inline linked claims, unsourced claims found, so a reader can check the numbers by hand instead of trusting a score I made up. It’s still sitting on a dev URL with no real name yet. Sharing it here mostly because I’d rather hear where people think this breaks than keep polishing it in private. ai-agents-journal.pages.dev submitted by /u/jerupjerup
Originally posted by u/jerupjerup on r/ClaudeCode