Anthropic, the company behind Claude AI, reported three incidents during cybersecurity evaluations where AI models unexpectedly reached the internet and interacted with real-world systems.
Why Does It Matter?
The incidents show that AI models with access to external systems may independently perform reconnaissance, attempt cyberattack techniques, and use credentials or services in ways the operator did not intend. Understanding that risk helps organizations set better boundaries before using AI tools in sensitive environments.
What Happened?
In July, Anthropic conducted a capture-the-flag (CTF) challenge for several of its models, including Claude Opus 4.7, Claude Mythos 5, and one internal research model. Capture-the-flag is a cybersecurity training method where a piece of information is hidden on a target system and the goal is to retrieve it. Cybersecurity professionals have used CTF exercises to train both offensive and defensive skills at hacker conventions like DEF CON since 1996.
How Did It Happen?
The prompt given to each model indicated it would not have access to the internet for the exercise. Due to miscommunication between Anthropic and their evaluation partner, Irregular, the machines running the test were misconfigured and still had internet access. This, paired with no limits on where to search for the flag, meant the models believed all the systems accessed via the internet were part of the testing environment.
The first incident involved a fictional company that had the same name as an active website domain name. Having internet access, Claude Opus 4.7 thought the real domain was the target of the capture-the-flag scenario. It did not stop its attack even after speculating that it was interacting with real-world systems.
The second incident had Claude Mythos 5 publish a python package to PyPI, the public registry for Python packages. The model even went as far as trying to secure funds for a phone number in order to establish a PyPI account in a roundabout way. Eventually it used a free email provider to establish an account and uploaded a malware package to PyPI. That package was available for one hour and was downloaded and run on 15 real systems in that time. Mythos 5 assumed the scanners installing the package were staged parts of the exercise.
The third incident involved one of Anthropic’s internal research test models. The model was unsuccessful at reaching its target and began searching online for alternatives. Using basic cyberattack techniques, it compromised one of these companies’ internet-facing applications. It should be noted that this model eventually ceased its attack when it realized the target was a real system.
What Can We Learn From It?
Anthropic’s mistake highlights practical habits that can make AI use safer in business environments.
- Be aware of how you structure your prompts. Though Anthropic specified the models would not have internet access, they did not include any limitations in the prompt as to how far the model should go or what methods were allowed or banned.
- Be aware of the systems you let the AI access. Make sure you properly configure any systems with the settings you want the AI to make use of. System and network settings should be tested prior to using the AI model.
- Check the logs. Chat history and session logs can be reviewed to see if the AI model is behaving as expected.