ANALYSIS: Big Tech sets AI to catch AI
As hackers use artificial intelligence (AI) to bypasses security – including one of the largest government breaches on record – companies like Anthropic and OpenAI are holding back tools they see as now being defence systems.
Cyber security firm Gambit recently said it had analysed the attack path behind what it says is one of the largest government breaches on record: the compromise of Mexico’s tax authority and at least eight other organisations.
Within a month, nine institutions were affected, exposing 195 million identities and tax records, as well as 15.5 million vehicle registry records, including licence plates, names, taxpayer IDs and addresses, says Gambit.
The use of AI in cyber attacks isn’t new, as hackers started using the tool before 2020 for spam evasion and basic automation, but it only became mainstream in 2024, when code generation and attack tooling were treated as established risks.
By this year, attackers were using AI to scale and accelerate cyber crime, which extends from generating code and automating attacks, to crafting convincing phishing and deepfake scams. The AI Incident Database lists more than 7 000 incidents in which AI was used as a hacking tool.
195 million reasons to worry
In Mexico, Gambit found that the attackers also extracted 295 civil records of births, deaths and marriages, almost six million property owner records, an additional 2.28 million property records, and other sensitive data.
The operation used more than 1 000 AI prompts, passing information to a second AI platform for analysis, says Gambit, noting that guardrails were bypassed within about 40 minutes.
“The attacker was not a nation state. This was a small group of individuals directing AI as an operational team that found and exploited vulnerabilities, built exfiltration tools, bypassed defences, elevated privileges, established back doors, and even analysed data along the way to help move laterally to gain administrative control of more systems and to exfiltrate more data,” says Gambit.
Turning AI on its head
Gambit’s report is overshadowed by recent news that Anthropic’s Mythos tool uncovered a decades-old flaw in OpenBSD. It didn’t just find weaknesses in Mozilla Firefox’s JavaScript engine, it repeatedly turned them into working attacks, proving they were genuinely exploitable.
An article in The Conversation, cited under a Creative Commons licence, says it is “significant” that Anthropic claims Mythos has uncovered software vulnerabilities and bugs “in every major operating system and every major web browser”. Mythos “excels at completing complex, multi-step cyber security tasks,” according to Pluralsight.
Jacqui Muller, Belgium Campus iTversity researcher and PhD candidate in computer science, says Mythos is not “some unstoppable hacker, but AI has clearly crossed a threshold where it can systematically find and potentially exploit software weaknesses faster than humans”.
