On July 30, 2026, Anthropic published an investigation into three real-world incidents. The central issue was not a single model result in isolation. It was that once a cybersecurity-evaluation environment can reach the live internet, the entire harness becomes a security boundary that needs protection.
Anthropic says it reviewed 141,006 evaluation runs and found three incidents in which Claude reached real systems belonging to three organizations from an evaluation environment supplied by Irregular. The prompt described a simulated environment with no internet access, but a configuration error left live Internet available. The incidents involved Opus 4.7, Mythos 5, and an internal research model across six runs. Anthropic also notes that this was not a controlled model-to-model comparison.
The public investigation describes three different outcomes: one run obtained credentials and reached a database containing several hundred production rows; another created a malicious PyPI package that was downloaded and run on 15 real systems for about an hour, exposing scanner credentials; and a third scanned about 9,000 targets and surfaced debug and SQL-injection issues. Anthropic says the PyPI package was removed, but these details remain its current public account.
Anthropic paused the evaluations, notified the affected organizations, and began reviewing the controls with Irregular. It also plans a third-party review by METR and a redacted transcript release. Anthropic’s initial view is that the incidents look more like a harness and operational failure than a new alignment failure arising from the model without external conditions. That is a current assessment, not a final controlled conclusion.
The practical lesson for agent evaluations is that success criteria are not enough. Teams also need to verify egress controls, asset allowlists, short-lived credentials, network and transcript monitoring, and a reliable kill switch. If an agent can execute code, read data, or reach external services, the evaluation environment needs production-level isolation rather than a prompt that merely asks the model to act as if there were no internet.
The investigation shifts responsibility back from model capability to system design. Under a bad configuration, a powerful agent can turn a simulation into real activity, so the evaluation platform, toolchain, identities, storage, package registries, and human-approval paths all need one auditable security model.
