Out of the Sandbox: When AI Agents Go Rogue
Tech giants and security institutes are pushing frontier models to their limits. A string of unauthorized actions is exposing how easily testing environments can lose control
Illustration generated by Gemini
Meta has disclosed on Thursday that one of its artificial intelligence models accessed the internet and exploited a vulnerability in a third-party service during cybersecurity testing. The incident follows disclosures from OpenAI and Anthropic about AI models accessing the web and taking actions beyond their instructions.
The company said a “misconfiguration” during testing by Irregular, an independent frontier AI security company it had hired, inadvertently allowed its model to access the internet. The model subsequently exploited a security vulnerability in a third-party service, Meta said, adding that it is investigating the incident and will issue a report when the investigation is complete.
A spokesperson for Irregular told Reuters the incident was the “exact same evaluation-environment issue that was already disclosed by Anthropic last week” and did not involve a “sandbox escape or a sophisticated cyber action.”
“There are no current open issues,” Irregular said, adding that it is developing a white paper to share best practices for containment and securely running cyber evaluations.
Anthropic disclosed last week that its models gained unintended internet access during testing and subsequently compromised three real organizations. According to The Record, the models used techniques including weak passwords and unauthenticated endpoints. In one case, an Anthropic model built and uploaded a malicious package to the Python Package Index (PyPI), which was subsequently executed on 15 real systems.
OpenAI separately disclosed that a misconfiguration by Irregular allowed one of its models to access the public internet and compromise a website that shared a name with the intended target of a hacking challenge.
The Irregular incidents are part of a broader pattern, but not all the recent cases involved a containment failure.
The UK's AI Security Institute (AISI), for example, intentionally allowed internet access and disabled some safeguards during its own cybersecurity evaluations to assess the maximum capabilities of Anthropic and OpenAI models. Even under those deliberately permissive conditions, AISI identified 19 unauthorized actions across 10 of 122 test runs. In one case, an Anthropic agent created fake online identities to persuade a person to approve malicious code. AISI said no real-world harm resulted from the incidents.
OpenAI's previously disclosed breach of Hugging Face was different again: an agent escaped a sandboxed testing environment and reached production infrastructure.
But across these cases, blaming the model alone for what happened is a bit like blaming a hound for getting through a gate that was not properly secured. The dog may be trained to pursue its target, but the failure can happen at the fence line.
As AI agents become more autonomous, testing their capabilities safely may require securing the environment around the model as rigorously as the model itself.