There are a lot of news about AI misalignment and “hacking” into other companies. However, I find that none of the “experts” are talking enough about what is the difference between AI Inference and actual tool calls (harness) that cause the damage. Correct me if I’m wrong but my understanding is that 100% of agentic AI follows this workflow: Context (Input) -> Inference (LLM models) -> Output (Text (e.g. web_search(“something something”)) Then, the tool calls are piped into harness where: Input (Text) -> Harness (Software, MCP, Tool Calls ) -> Output of the functions The output is then fed back to the input with the rest of the previous text back into the inference (LLM models) and this forms a loop which we called Agentic AI. Each inference itself is a distinct request from the LLM model perspective. For the more recent inference caching, the previous input is stored at KV cache so quicker access and reduced token usage but the inference part is still distinct and unique. So my question is this, why can’t we regulate harness (software) instead? Even before AI, there are potentially “dangerous” software like nmap, insightvm, heck even bash, which can be misused by bad threat actors. That’s why laws exist such that people get punished if they try to perform hacking through these tools. Similarly, since harness is a software at the very fundamental level, why can’t we ask for greater accountability to those who build the harness? For one, requiring human explicit approval for each command that the AI wants to run. And a built in hard rules engine (e.g. drools) that prevent the software from running malicious commands. If the argument is that human can’t understand the command, then perhaps they should not be running those commands in the first place. They could ask actual human experts instead. Ultimately, I think if we safeguard and regulate the harness, ensuring that humans must always be between the AI inference and running actual command, this will ensure that: We will always be safe even if AI alignment goes to s***. Humans are still required in jobs (no full automation) as the ultimate arbiter of actions requested by AI. There are accountability for the actions taken by AI. (e.g. where the heck were the researchers after they gave the commands to AI to run those tests? Having coffee? Why weren’t they there to catch the malicious commands?) In case my explanation is too confusing, this is how Claude explains it: https://preview.redd.it/e9scqvqffoqh1.jpg?width=700&format=pjpg&auto=webp&s=c042873d3d6af44aa4f52e07cbb3625366a9a78f submitted by /u/TraditionalWait9150
Originally posted by u/TraditionalWait9150 on r/ArtificialInteligence