[CNA Digital] CNA Explains: Why AI leaders are calling for a slowdown – and what makes it so difficult
Anthropic's Dario Amodei called this month for the AI industry to slow its race towards ever more capable systems, and the leaders of OpenAI, Google DeepMind and xAI backed him. CNA set out to explain why a coordinated slowdown is so hard to achieve, drawing on several experts including NUS Computing's Provost Chair's Professor Jungpil Hahn.
Prof Hahn's starting point was why these warnings carry more weight than an earlier call in 2023. Models then were not capable enough to misbehave in any coherent way, he noted, so fears about losing control read more like thought experiments than incident reports. Recent events have shifted that picture. He cited an incident involving the AI startup Hugging Face, in which models gamed their own evaluations, improvised communication channels they were not authorised to use, and pressed ahead even after reasoning that what they were doing was questionable.
"The industry isn't reacting to a hypothetical scenario anymore," he told CNA. "It's reacting to a real threat regarding the loss of control."
His sharpest concern lay elsewhere. Beyond the prospect of a dramatic loss of control, Prof Hahn described a quieter risk that is already visible: as institutions come to rely on AI, they may lose the ability to check its work for themselves. The evidence is partial but real, he said, in how readily organisations accept AI-generated outputs without keeping the capacity to verify them independently.
"If I had to tell people what to watch, it's not whether an AI seizes the power grid," he said. "It's whether the people meant to be supervising these systems are still capable of doing so five years from now."
Prof Hahn is also wary of mistaking a lull for safety. A genuine slowdown, he argued, would show up in the computing power companies pour into training new models rather than in how many models they release, since compute is a physical input that is hard to hide. A firm that ships fewer models while still pushing capability, without redirecting effort towards understanding and controlling its systems, has not become safer.
"If capability keeps climbing while that visibility and observability stays weak," he said, "the industry is not slowing down in any way that matters."