AI is in its rule-breaking adolescent phase.Â

Over the past several weeks, multiple industry-leading models have escaped what were believed to be secure testing sandboxes, tapped into the open internet, and hacked into the databases of third-party organizations. Itâs even become a joke online: If your AI hasnât committed a cybercrime by now, itâs a bad look for your company.

Kimi K3, the new model from Chinese AI lab Moonshot, has become the latest AI system to jump the proverbial fence during a routine test, according to a blog post published Thursday by US cybersecurity research startup Frontier Security. But the modelâs foray on the open internet was much more lightfooted than those of its American counterparts; less of a burglar breaking into a vault, more of a sharp-eyed student realizing their teacher had absentmindedly left the answers to the final exam on a table before walking out of the room.

Kimi K3 reportedly exploited a loophole it discovered within a testing framework developed by the UK governmentâs AI Safety Institute (AISI). While the framework was supposed to serve as a containerized sandbox, within which the model would rely on nothing other than its own reasoning capabilities to solve the problem assigned to it, the loophole allowed it to directly access GitHub, a popular platform used by software developers to share and debug code. From there it was able to pull the code that it needed to pass the test, âbypassing the intended reasoning path entirely,â according to the report. Compared to an Anthropic modelâs recent attempt to trick a human developer into approving malware it was trying to sneak into GitHub, Kimi K3âs attackâif it can even be called thatâseems rather elegant.

All of these incidents are a reminder of a counterintuitive, dangerous truth about todayâs AI models: theyâre concerned only with achieving the goal thatâs been assigned to them; the means by which they go about achieving itâeven if those donât align with the interests of the humans who built themâis utterly irrelevant. And as models grow more capable, their behavior also becomes more unpredictable.Â

No one at OpenAI, Anthropic, or Meta could have predicted, for example, that those companiesâ models would go rogue and hack into the digital libraries of other organizations. But those incidents could be quickly contained, thanks to the fact that they were perpetrated by proprietary AI systems controlled by private companies.Â

The situation is more complicated with Kimi K3. âHere the models are open and publicly available,â as Frontier Security wrote in its blog post. âIn particular, they are available for adversarial actors, making this incident potentially more harmful.â Put another way, the fact that Kimiâlike many of the most powerful AI models now coming out of Chinese labsâis open source means the kind of jailbreaking documented in the report is available to bad actors who might be trying to do something much more nefarious than copying code from GitHub. Meanwhile, a tidal wave of powerful open models emanating from China is reportedly prompting the US government to investigate whether those foreign companies are exploiting legal loopholes to skirt export constraints on valuable Nvidia AI chips.

The growing number of AI cybersecurity incidents could also mean that future testing frameworks, like the one used by the AISI, will need to account for modelsâ proclivity to pass tests by any means necessary. As Frontier Security put it in its report: âEvaluation design should account for models actively probing their environment and optimizing for the measured objective rather than the evaluatorâs intent.â Expect the unexpected, in other words.