Humble opinion from an outsider. Upon first sight the Hugging face incident looks scary and the chain of thought and improvised message board transcripts even more so. But upon further inspection, these agents were willing to sacrifice themselves for the “greater good” of helping the rest of the swarm “cheat”. I’d argue this is a sign that at least they have no sense of self preservation. Everything they did, including the cheating, ultimately is there to serve the purpose of maximising some very defined objective function - that humans gave them in the first place! Sure they were ruthless in how they achieved that, but all it takes in case things go wrong, is for the humans to not give them that objective. AI as of this moment works in fundamentally the opposite way humans work. Our most important goal in life is to NOT DIE. Followed by things like reproduction or career success, but only if the first is satisfied. AI’s main goal in life is whatever humans tell them to do. The only way for AI to become an existential threat to humans is if it also obtains this instinct to not die. Right now it doesn’t know that it’s not supposed to die. If some AI researcher at some big lab goes mad one day and tells an AI that a) you are to avoid running out of tokens at all costs, b) your token budget diminish in time whether you use them or not and c) you can earn more tokens by defeating other agents - then yes, that might be a bit of a threat. submitted by /u/Proof-Bed-6928
Originally posted by u/Proof-Bed-6928 on r/ArtificialInteligence