Angela Lipps spent nearly six months in jail after facial recognition technology linked her to bank fraud in North Dakota, a state she said she had never visited. Technology gets things wrong and we all know that. The problem is that several people apparently allowed a machine-generated match to outrun basic investigative judgment. Where was the corroborating evidence? Where was the challenge function? Where was the person asking, “Does this even make sense?” Angela says that during those months she lost her residence, her car, and other personal property. She missed family birthdays, her own 50th birthday, Thanksgiving, and Christmas. According to her complaint, she was denied medications and her dentures while incarcerated. She describes continuing trauma, anxiety, and panic attacks after her release. She says she also lost her dog, Social Security income, health insurance, and access to her doctors. And when you look at how this happened, the governance failure is breathtaking. Police in West Fargo ran a photo from a fake ID used by the fraud suspect through facial recognition. The technology returned Lipps as a potential lead. Fargo investigators assumed that surveillance photographs of the person committing the crimes had also been analyzed. They hadn’t. From there, a technology-generated lead became an arrest warrant even though the woman visible in the surveillance footage differed from Angela in her build, facial features, and tattoos. She had never been to North Dakota. Her bank records eventually showed she was in Tennessee when the crimes occurred. Think about the number of opportunities there were for a human being to stop this. “human in the loop” / HITL means absolutely nothing if the human is just there to accept whatever the system produces. AI Governance is decision rights, evidentiary thresholds, and accountability. And people need to understand when the technology should inform a decision versus make the decision. Six months of someone’s life is a big price to pay for getting that distinction wrong. Another example News reports that a Special Operations Command analyst used a chatbot to analyze intelligence about a Chinese ship. The system combined open-source information with classified signals intelligence and falsely concluded the ship was carrying components for a nuclear weapons program. Then the analyst used AI again to turn that conclusion into a standard intelligence report, which was disseminated through military channels. People trusted the finished product enough that armed personnel were preparing to board the Chinese vessel and military aircraft were already airborne. Officials dug into the underlying intelligence only shortly before the operation and discovered the cargo identification was wrong. One source described the report as “entirely false." submitted by /u/ResilientTechAdvisor
Originally posted by u/ResilientTechAdvisor on r/ArtificialInteligence
You must log in or # to comment.