And the Worm turns once again. The WormGPT that is.
In the summer of 2023, WormGPT had a short run as an uncensored GenAI tool on Hack Forums before one of its creators shut it down.
But that didn’t stop its variants from springing up in BreachForums like nightcrawlers, much to an angler’s delight, after a summer rain. Now, Cato CTRL has discovered previously unreported WormGPT variants, but with a twist — they are powered by xAI’s Grok and Mistral AI’s Mixtral.
Clearly, early on, threat actors caught onto the potential of large language models (LLMs) to be used maliciously.
You may remember that WormGPT was based on the open-source LLM GPT-J and, according to Cato CTRL, pricing ranged from €60 to €100 per month, or €550 per year. Those who partook could also opt for a private setup for a cool €5,000.
In the wake of WormGPT’s emergence and shutdown, other malicious LLMs emerged. But beyond that, “the trend of threat actors attempting to jailbreak legitimate LLMs like ChatGPT and Google Bard/Gemini to circumvent their safety measures also gained traction,” says Cato CTRL. And researchers found “indications that threat actors are actively recruiting AI experts to develop their own custom uncensored LLMs tailored to specific needs and attack vectors.”
As Trey Ford, Bugcrowd CISO, says, we should “fully expect GenAI to be abused in this manner, the use of local and open-weight LLMs in a jailbroken manner, where prompts and outputs are effectively jailbroken, on the regular.”
The WormGPT variants’ proliferation clearly “LLM guardrails are not perfect,” says Margaret Cunningham, director, security & AI strategy at Darktrace. “As we’ve seen with WormGPT and similar findings like HiddenLayer’s universal jailbreak technique, threat actors will continue to find creative ways to skirt safeguards, expose system prompts and remove censorship.”
Noting that “WormGPT became a brand name for uncensored LLMs that can be leveraged by threat actors in their offensive operations,” Cato CTRL discovered two new variants—“xzin0vich” and “keanu”—that emerged in late 2024 and early 2025, more than a year after WormGPT was shuttered.
“Hundreds of uncensored LLMs exist in Dark Web communities. Many of them are labeled “WormGPT” as a means of convenience, just like Americans say Kleenex for a facial tissue, even though the first is actually a brand versus the true item — a tissue,” says Dave Tyson, Chief Intelligence Officer at Apollo Information Systems, who’s not “especially surprised” to see WormGPT source code’s “dominance as a term or as backend code.”
Virtually any model could be used behind the scenes, taking advantage of “a barrier of isolation between the AI and actual user” who has a query in a chat channel and that “allows criminals to provide a service to customers.”
As it turns out, xzin0vich-WormGPT is powered by Mixtral, while the keanu variant is powered by Grok.
Indeed, “the evolution of WormGPT reveals a significant shift from its original GPT-J-based incarnation,” the researchers wrote.
They are “not bespoke models built from the ground up, but rather the result of threat actors skillfully adapting existing LLMs.” Their creators manipulate system prompts “and potentially [employ] fine-tuning on illicit data [to] offer potent AI-driven tools for cybercriminal operations under the WormGPT brand.”
That concerns Stephen Kowski, field CTO at SlashNext Email Security, who says that organizations now “need to think beyond just blocking known bad tools and start looking at how AI-generated content behaves, regardless of which platform created it.”
Subscription models, too, are troubling because they “significantly lower the barrier to entry for threat actors, allowing them to leverage these tools without needing the technical skills to develop them themselves,” says Cunningham.
Ford cautions not to malign GPT itself — it “isn’t the malicious actor,” rather that honor falls to the “miscreant using and abusing the tooling.”
