OpenAI published a breakdown of an autonomous security pipeline it ran against the Linux kernel in mid-October. Over about ten days, agents scanned 2.8M lines of security-relevant kernel code, turned 2,308 candidate findings into 1,724 reproduced vulnerabilities (the reproduction stage ran inside 30 hours), and finished with 948 working local privilege-escalation exploits plus candidate patches. Humans reviewed everything before anything went to maintainers. The detail worth your time is not the 948. In an early run the pipeline reported 200 successful exploits, and human review found 20 of them had gotten root through a completely different bug than the one assigned. OpenAI had to add an independent check that each exploit actually used its assigned vulnerability. If you run agents that grade their own work, that is the failure mode to design against: verify the path, not just the result. Second detail: they built a library of 103 exploitation techniques to help the weaker model, and the stronger model barely benefited (1.5 points), so they deleted the scaffolding. Your harness has a shelf life. Caveats: all numbers are OpenAI’s own reporting, and while the scan ran, Linux maintainers had already patched 61 of those bugs through other channels. submitted by /u/shoumikgoswami
Originally posted by u/shoumikgoswami on r/ArtificialInteligence