Every parent knows the quiet terror of the 18-month checkup. The pediatrician runs through the list. Is she pointing at objects? Is he stringing two words together? The routine visit becomes a high-stakes audit of whether your child is developing on track . Now consider that we’re deploying agentic AI systems into enterprise workflows and customer interactions with far less structured evaluation than we give a toddler’s vocabulary. The systems are walking and running. But do we actually know if they’re developing the right way, or are we just hoping they’ll figure it out? That question points at something the AI field is getting wrong. Agentic AI Toddlerhood First, let’s be precise about what we mean by agentic AI, because the term gets stretched in a lot of directions. An agentic AI system isn’t just a chatbot that answers questions. It’s a system that receives a goal, breaks it into steps, uses tools to execute those steps, evaluates its own progress, and adjusts when things go wrong. Like an AI that doesn’t just tell you how to book a flight but actually books it, handles the seat selection, notices the layover is too short, reroutes, and confirms the hotel. That’s a different category of system than a language model answering prompts. The capability is impressive. Agents built on today’s frontier models can plan, reason across long contexts, call external APIs, write and execute code, and coordinate with other agents. That stuff was science fiction five years ago. Here’s the toddler part. Toddlers are also genuinely impressive. A 20-month-old who’s learned to open a childproof cabinet, climb onto the counter, and reach the top shelf is demonstrating real planning, tool use, and environmental reasoning. The problem is not the capability. The problem is the gap between what they can do in a burst of competence and what they can do safely , and consistently across conditions. Agentic AI systems fail in exactly this way. They hallucinate tool calls, calling APIs with malformed parameters and treating the error message as confirmation of success. They get stuck in reasoning loops, repeating the same failed action because their self-evaluation mechanism doesn’t recognize the pattern. They abandon multi-step tasks when they hit an unexpected branch, sometimes silently, with no record of where things went wrong. And they do something particularly toddler-like: they produce confident, fluent outputs at the moment of failure. The system doesn’t know it’s failing. It sounds completely certain. It’s like the capability is real, but the reliability infrastructure isn’t there yet. These aren’t toy systems. They’re being deployed in production. And the gap between capability and reliability is exactly where developmental immaturity lives. The Milestone Problem In child development, milestones aren’t arbitrary. They’re grounded in decades of research across diverse populations by pediatric scientists with no financial stake in whether your child hits a benchmark. Their job is honest evaluation. That institutional neutrality matters enormously. The milestone-setter and the milestone-subject have separated incentives. Now look at the agentic AI landscape. Who sets the milestones? Benchmark creators at research institutions design evaluations, but those evaluations are becoming disconnected from real-world agentic performance. MMLU tests broad knowledge recall. HumanEval tests code generation in isolated functions. These were built to measure what LLMs know, not what agents do over time in dynamic environments. Using them to evaluate agentic systems is like assessing a toddler’s readiness for kindergarten by testing with shapes on flashcards. Technically data. Not really the point. The result is a milestone landscape that’s very fragmented. Everyone is measuring something. Nobody is measuring the same thing. And the entity with the best picture of how a deployed agent actually performs over time, the organization running it in production, often has no tools to interpreting what they’re seeing. So the next question is what a developmental assessment would actually need to measure? Pediatric milestones don’t test a single skill. They assess across developmental dimensions. Each dimension captures a different axis of maturity, and the combination produces a profile, not a score. A child can be advanced in language and behind in motor skills. That multidimensional picture is what makes the assessment useful. Agentic AI needs the equivalent. Not a single benchmark. A dimensional assessment. What actually breaks when multi-agent systems fail in production: Agents drift out of alignment with each other and with shared goals, producing outputs that each look reasonable in isolation but contradict each other at the system level. That’s a coherence problem. When misalignment is detected, the only available response is a full restart or human escalation. Nobody built a mechanism for resolving the conflict in-flight. That’s a coordination repair problem. Agents operating in sensitive, high-stakes, or ethically complex territory don’t adjust dynamically. They barrel through with the same confidence they bring to routine tasks. That’s a boundary awareness problem. One agent dominates decisions while others are sidelined, creating echo chambers and single points of reasoning failure. That’s an agency balance problem. Context evaporates across sessions, handoffs, and instance changes, forcing cold starts that destroy accumulated understanding. That’s a relational continuity problem. And governance rules stay static regardless of whether the system is running smoothly or heading toward cascading failure. That’s an adaptive governance problem. Six dimensions. Each distinct. Each capturing a failure mode that current benchmarks don’t touch. And the combination produces something no individual metric can: a governance profile that tells you where your system is actually mature and where it’s exposed. The organizations running multi-agent systems in production already encounter these problems. They just don’t have a structured vocabulary for naming them or a framework for measuring them. They’re watching a toddler and going on instinct, when they need the developmental checklist. Reframing Evaluation There’s a version of developmental milestones that’s purely celebratory. Baby took her first steps! He said his first word! Share the video, mark the calendar, feel the joy. But it’s not the primary function. In pediatric medicine, the function of developmental milestones is early detection. When a child isn’t hitting language milestones at 24 months, that’s not just a data point. The milestone exists to catch problems while there’s still a wide intervention window. The AI industry has largely adopted the celebratory version of evaluation and skipped the diagnostic one. A new model passes a benchmark, and the result is a press release. The announcement tells you the system achieved a new high score. It doesn’t tell you what the benchmark misses, what failure modes were excluded from the test set, or what performance looks like three months into deployment when the edge cases start accumulating. Reframing evaluation as diagnostic infrastructure rather than performance marketing changes what you do after passing a benchmark. It means treating a high score as the beginning of deeper questions, not the end of them. This is where a maturity model becomes essential. Not a binary pass/fail, but a graduated scale that distinguishes between fundamentally different levels of developmental readiness. A useful maturity model needs at least five levels. At the bottom, the governance mechanism is simply absent . Risk is unmonitored. One step up, it’s reactive : problems are addressed after they surface through manual intervention or post-incident review. Then structured , where defined processes and monitoring exist and interventions follow documented procedures. Then integrated , where governance is embedded in the workflow rather than bolted on. At the top, adaptive : the governance itself self-adjusts based on real-time system health, learning from past coordination patterns. The critical insight is that not every system needs to reach the top. A low-stakes internal workflow might be fine at reactive. A customer-facing multi-agent pipeline handling financial decisions needs integrated or above. The maturity model doesn’t set a universal standard. It maps governance readiness against actual risk. That’s the diagnostic function. It tells you whether your developmental infrastructure matches what your deployment actually demands. Here’s the concept that ties this together: developmental debt . When agentic systems are rushed past evaluation stages, scaled before failure modes are mapped, organizations accumulate a specific kind of debt. Not technical debt in the classic sense of messy code, but something more insidious: a growing gap between what the system is assumed to be capable of and what it can actually do consistently under pressure. That gap compounds. The longer it goes unexamined, the more infrastructure and workflow gets built on top of assumptions that aren’t grounded in honest assessment. The analogy holds: skipping physical therapy after a knee injury might let you get back on the field faster. But you’re trading a six-week recovery for a vulnerability that surfaces under load, at the worst possible time, in ways that are harder to treat than the original injury. Organizations should invest in evaluation frameworks with the same seriousness they invest in model selection. This isn’t overhead. It’s infrastructure. The cost of building honest assessment before broad deployment is a fraction of the cost of managing cascading failures after it. Ultimately, the toddler stage of agentic AI is a temporary state—but only if we actively manage the transition out of it. Moving from demos to infrastructure requires acknowledging that capability and maturity are not the same thing. The organizations that figure out how to measure that difference will be the ones that actually scale successfully. This post was informed by Lynn Comp’s piece on AI developmental maturity: Nurturing agentic AI beyond the toddler stage, published in MIT Technology Review. submitted by /u/cbbsherpa
Originally posted by u/cbbsherpa on r/ArtificialInteligence