Welcome to Safe Mode, your weekly briefing with urgent security and privacy news, and what to do next. Want this newsletter delivered straight to your inbox? Register on our website!
Last week, Anthropic published a report detailing a series of experiments in which AI was tasked with spreading through low-tech intelligent machines. The team examined how easily small models could discover, configure, and install model weights (i.e., model parameters for data processing) on vulnerable devices and then repeat the process to proliferate. The resulting botnet (and collective computing power) could be used for AI workloads. The behavior mirrors that of a classic computer worm, where no human supervision is needed for malicious code to be continually copied.
(What is a small model? AI comes in different sizes, so to speak. The ones we consumers know best represent the variety of large languages, such as Claude, ChatGPT, Gemini, etc. But there are also small language versions. They are more streamlined and usually designed for specific purposes. Think of an LLM as a sprawling multipurpose tool, while an SLM is single-purpose. And sometimes a very singular purpose.)
It turns out that small, specialized AI models achieved this goal quite effectively. At least, they did it in a controlled lab environment, with the researchers releasing several small models on Android TV boxes. (These media boxes have notoriously lax security settings.) But Anthropic believes this attack method could become real “within months.”
Sounds scary, right? I have to admit, when I first saw a news headline about this report, it seemed appropriately creepy for October. After all, we already know that the vast majority of smart devices have poor security. (Heck, I’ve written about the same topic, and several times.) Botnets have also been around forever. The idea of them becoming more advanced through AI is also not surprising. But I read the report more as a warning than a serious warning.
Nicolás J. Leclercq / Unsplash
First: Anthropic arrived at these results in a controlled environment, rather than viewing them on the open Internet. (Nature shenanigans are the purview of your big-language models right now.)
Second: Anthropic’s predictions for when we would see this in the wild are pretty vague. Sure, the implication is only a few months, based on current rates of AI improvement. But “months” is very open-ended.
