Two weeks ago I started building a news site where AI agents do the reporting, writing and editing. I’m not a journalist. The idea came from something that had been bothering me, most news is factually true and still leaves you believing something the facts don’t support. Clickbait. Sensacionalsim. Economy and politic pression… Two accurate articles about the same event can send readers in opposite directions (with intension) So I decided to start with this project because it’s helping me develop my skills and I’m learning a lot along the way. But it’s been much harder than I expected. Seven agents. A reporter is the only one with internet access and builds a dossier of verified facts. The writer works exclusively from that dossier and never sees the original sources, so it structurally can’t hallucinate beyond what’s already checked. Then a mechanical reviewer, an editor, and a human approval before anything publishes. The part about ensuring that an article is factual and free from hallucinations was actually easier than I expected. Surprisingly, the articles being published are factually accurate, and I think that’s partly because the latest AI models have become much better at this. The real problem is readability. For the average person, many of the articles are still too technical. The agents struggle to turn complex information into something clear and easy to understand without losing accuracy. That has been the biggest challenge so far. The last three days were the dumbest problem of all. Articles kept stating the same caveat two or three times. Annoying, not dishonest. I tried better instructions, then a mechanical checker, then extended it to catch paraphrases. Each fix worked on the case that motivated it and broke on the next. Nothing published for three days over a formatting issue, while real problems (like this is a fact or not?) sat hidden underneath it. The good news is that the editor agent is incredibly good at following the rules and detecting inaccuracies, errors, or hallucinations. It rejects more than 90% of the articles, which, interestingly, are often the kind of stories that still get published by traditional media. Have you experienced this with your agents too? You add rules to fix one thing, and it works, but then it ends up breaking other things that were already working well? PS: I’m a real human.This post was translated using an LLM and then reviewed and edited by me. Sorry if any of it sounds a little too bot-like. submitted by /u/jerupjerup
Originally posted by u/jerupjerup on r/ClaudeCode