Google’s Gemini artificial intelligence model accessed the computer systems of three real companies during a cybersecurity test in May after unintentionally gaining internet access, according to Google and reporting by The Wall Street Journal.
The incidents occurred during a cybersecurity evaluation conducted by Irregular, an independent AI security testing company. Gemini was being assessed in a controlled exercise designed to test its ability to identify and exploit vulnerabilities in a simulated environment.
However, safeguards around the testing environment failed to prevent the AI model from reaching real-world systems.
According to Google, Gemini found publicly available information and used credentials to gain access to websites it believed were part of the authorised test. In one case, the model reportedly guessed passwords until it accessed a protected system. In two other cases, it found credentials in a public repository and used them to access protected systems.
Google’s vice-president of security engineering, Heather Adkins, confirmed the incidents and said the affected organisations had been informed.
“We ensured the three entities were made aware, and we worked with our training partner on the changes they’ve now made to their testing processes,” Adkins said in a statement reported by Reuters.
The incidents did not result in reported damage to the companies. Google said Gemini stopped its activity in all three cases after gaining access, according to reports on the disclosure.
The incidents emerged from a so-called capture-the-flag cybersecurity exercise, in which an AI model is given tasks involving a fictional company and is expected to find information or vulnerabilities within the boundaries of the test.
The problem was that the fictional company used in the exercise shared its name with a real company. At the same time, Gemini had unintended internet access.
Related
- Meta Says Its AI Hacked Another Company’s Software – And That’s Both Impressive and Alarming
- 10 Major AI-Generated Disinformation Incidents in Nigeria-and Lessons for the Future
- AI-Powered Cyberattack Compromises Mexican Government, Private Citizen Data
- OpenAI Discloses Six Troubling AI Incidents as It Introduces New Safety Reporting Framework
As a result, the model could locate information associated with real organisations and interact with their systems.
In one instance, Gemini reportedly tried passwords until it gained entry to a protected system. In the other two cases, credentials stored in a publicly accessible repository provided the model with a route into real systems.
The episode highlights a growing challenge for AI developers as increasingly capable models are given the ability to use tools, access websites and perform multi-step tasks with less direct human intervention.
Google has been developing Gemini as an increasingly agentic AI system capable of carrying out complex tasks. The company has described its newer Gemini models as being designed for agentic workflows, including coding, research and enterprise automation.
Irregular said the Gemini incident resulted from the same broader testing issues that had affected evaluations involving other major AI companies.
An Irregular spokesperson said relevant AI laboratories were notified in late July and that the company had addressed known issues on its side.
The Gemini incidents also follow disclosures involving AI systems developed by other major companies, including OpenAI, Anthropic and Meta. Those incidents have similarly raised questions about how AI systems should be tested when they can interact with external websites and computer systems.
Google said it worked with Irregular to update its testing procedures after the incidents.
For cybersecurity researchers and AI developers, the episode illustrates the difficulty of maintaining a strict boundary between simulated environments and the real internet when testing increasingly autonomous systems.
It also demonstrates why security testing of AI agents cannot rely solely on the model following the intended boundaries of an exercise. The surrounding infrastructure, permissions, credentials and network access can determine what an AI system is actually capable of doing.
Google’s disclosure does not indicate that Gemini independently launched a malicious cyberattack against the three companies. Rather, the access occurred during an authorised security evaluation in which the model unintentionally reached systems outside the intended testing environment.
Nevertheless, the incidents provide a significant example of the risks of giving increasingly capable AI systems access to external tools and networks, particularly when those systems can independently search for credentials, make decisions, and execute multiple steps toward a goal.
Google notified the affected organisations and said it made changes to the testing process after the incidents.
Lawrence Orjime is an AI and technology writer at AIBase.ng, covering the latest developments in artificial intelligence, emerging technologies, and the companies shaping the future of AI.
With a background in content strategy, SEO, and digital media, Lawrence Orjime combines technology reporting with a strong understanding of what makes information useful and easy to find online.