This is interesting to me because of all the hype around agentic AI and workforce automation. This is the flip side of productivity and speed which is risk. If agentic AI use increases your odds of extremely high impact mistakes, that’s part of the math too. I don’t think that it’s a good idea to give models / agents access to secrets directly regardless of any security skill. It’s not their job to be trustworthy just like it’s not the model’s job to “know things” which is why we have RAG etc. I don’t know how often people are already giving models access to sensitive secrets and data where the same context is available when they’re dealing with potential external plumbing like email etc. That to me is a huge danger is companies have already started embracing this sort of model of agent. Again, I don’t think this is a good problem to solve, it’s more like “you’ve made some bad architectural decisions with AI usage if you’re trying to solve this problem at all.” Really great benchmark to find a way to do the math on the risk though to make that clear. submitted by /u/HasOneComment
Originally posted by u/HasOneComment on r/ArtificialInteligence