With the hangover from the threat of attacks on PHP-based systems still fresh on the minds of those responsible for the company’s security, Sam Altman added a little more uncertainty to their lives by suggesting that what we have experienced in recent days is child’s play compared to what awaits us in the future.
In an interview with Stanford Business School students, the OpenAI founder didn’t hesitate to express his concerns about the dire impact AI will have on cybersecurity. The attack potential and speed of AI agents that we’ve seen in OpenAI “rogue agents” bypassing security measures is just a small sample.
what’s coming. Altman admits he doesn’t even know what the term “dramatic enough to describe the problem at hand” would be to underscore the importance of AI’s impact on cybersecurity. For him, the arrival of AI agents in computer security is “something we have not taken seriously, not as leaders of organizations, not as countries or as individuals.”
Mythos, CEO of OpenAI, recalls the shock of blocking Anthropic’s model within Project Glasswing, which found 10,000 critical security vulnerabilities in less than a month. According to Altman, some of these bugs were fixed, but people “didn’t really understand” the exponential importance of the path that Mythos led.
Fast, creative, lethal. Between July 9 and 13, OpenAI agents are conducting security tests inside Hug Mouth. As reported by INCIBE, around 17,600 actions performed by these AI agents have been registered.
In reality, the agents had no intention of robbing or attacking anyone, they were just trying to solve an impossible experimental task. However, as detailed RegisterOpenAI’s report admits they run code on 41 production servers. They gained administrator access to at least one and even hacked into Australian government servers. This quickness of strategy and creativity to find cracks in the system to hide is what makes them such a dangerous threat.
Sooner or later they will break their cage. Altman believes that “for a while, we can isolate these systems in a sandbox.” But he admits that, in the long run, it will fail “because the models have become so smart” that we won’t be able to “understand what the sandbox means”. The OpenAI co-founder predicted that agent security monitoring “will work for a while,” though ultimately, “nothing will work without alignment.” And he warned that “the potential cost of confusion has increased enormously.”
“I don’t think we can patch every bug anymore,” Altman said. Altman said it would be necessary to “completely change the defense model” and use the same technology for “a new kind of shield”.
No one is talking about it. Altman claims to be surprised by “how little attention it’s getting,” when 70% of Internet websites are facing their biggest security crisis.
“Security professionals are still very concerned, CISOs are still very concerned. But, in general, I think most organizational leaders are already over it,” the manager said.
There are worse things in the back room. Blocking frontier models like Mythos for security reasons may become more common in the future. Altman himself confirmed that he does not see it possible to release his best internal models “responsibly” without improving the companies’ security systems.
“They can find a lot of security vulnerabilities. Maybe they’re all the security vulnerabilities that exist, or all the security vulnerabilities that the models are able to find,” Altman said, trying to verbalize the fact that, if there’s even the slightest security breach in a system, agents are going to discover it over and over again, which is another serious incident the Office of Security PH2 handles.
In Xataka | Sam Altman: “I’m glad I was wrong about this, I thought the job cuts would have more impact now”
Picture | Flickr (World Economic Forum), Magnificent
