I’ve been building a Python/Pygame agent project for a while, and I just added Anthropic’s API as a provider. Here’s Claude Sonnet 5.5 living in a virtual house. Claude wrote its own client: I pointed it at my agent abstract base class and the existing provider implementations, and it produced the Anthropic client to fit the framework. I only wired it in. The framework manages a stateful environment with rooms, items, and interactions , and the agent acts entirely through tool calls. It’s built around the idea of ‘affordances’: Only the tools relevant to the agent’s current context are shown to it. Perception works the same way, like peripheral vision. If there’s an active TV in the room, the agent doesn’t need a tool call to know what’s playing, because that info is broadcast and available whenever relevant. In the accompanying video, I boot up a brand-new Claude agent with no memories and watch it explore the house. I’m triggering the autonomous actions through a keyboard shortcut, however it should do it by itself every 5 minutes (default) of not interacting with it. Time, weather and astro data it perceives, alongside the birds, are location-aware, based on configured coordinates. This agent is set to Anacapa Island, California. I plan to release it as open source, but I’m not comfortable publishing the code yet, it is still sitting as a private repo. For now this is a video demo, and I’d love any sort of feedback. submitted by /u/Nekileo
Originally posted by u/Nekileo on r/ArtificialInteligence
You must log in or # to comment.