Hi reddit, I’m thinking of building a small testbed that measures how well an LLM agent decides when to delegate work to a tool. I use Hamiltonian path as the task because it’s easy to verify but hard to solve, so the LLM’s reasoning breaks down predictably as graphs get bigger. I run the same problems through three setups: the LLM reasoning alone, the LLM with a classical solver available as a tool it can choose to call, and forced delegation to the solver. A deterministic verifier checks every answer, and I track accuracy, token cost, and how often the model chooses the tool as difficulty increases. Most tool-use benchmarks test whether a model can call a tool correctly; almost none measure whether it knows when to, which is where deployed agents actually fail. I’d appreciate any thoughts on whether the setup or metrics have blind spots. submitted by /u/Acceptable_Block_591
Originally posted by u/Acceptable_Block_591 on r/ArtificialInteligence