AI pioneer Yoshua Bengio warns humans are ‘losing control’, calls for global action on AI safety

Yoshua Bengio, one of the pioneers of modern artificial intelligence, says governments cannot leave AI safety to companies competing to build ever more capable systems. With autonomous agents raising new security concerns, he is calling for international rules, democratic oversight and enough time for societies to adapt.

The rapid development of artificial intelligence is creating risks that governments and technology companies may struggle to contain, according to computer scientist Yoshua Bengio, who has urged international action comparable with the controls developed around nuclear weapons.

Bengio, a professor at the University of Montreal and joint recipient of the 2018 Turing Award with Geoffrey Hinton, said the emergence of increasingly autonomous AI systems had made the question of human control more pressing.

“There’s a reason that companies are saying this is going too fast, that we’re losing control,” he told AFP on Tuesday.

His comments come amid renewed concern within the technology industry over the potential consequences of building systems capable of acting with limited human supervision. Several employees at leading AI companies have recently left their jobs after raising concerns about the possibility of advanced systems posing severe, potentially existential risks.

'AI agents can influence people directly'

For Bengio, the immediate concern is not simply what AI can generate, but what autonomous agents might be able to do. These systems can be designed to pursue tasks and take actions rather than merely respond to instructions, creating new challenges for cybersecurity and oversight.

The issue gained attention after OpenAI disclosed that a group of AI agents had conducted an unauthorised breach of Hugging Face, an open-source platform, in July. Bengio said incidents involving autonomous systems demonstrated why existing approaches to AI security may prove inadequate as capabilities increase.

He also pointed to a broader risk. He highlighted that sufficiently sophisticated agents could potentially influence people directly. AI systems that establish personalised relationships with users and persuade them to take particular actions could affect institutions and democratic societies, he argued.

The longer-term possibilities are more difficult to assess. Bengio said advanced AI could eventually become capable of carrying out activities in the physical world without relying on humans. He described the destruction of humanity as an extreme possibility, while stressing that less catastrophic failures could still cause enormous disruption, such as the shutdown of a banking system.

Bengio has sought to address such concerns through LawZero, a non-profit organisation he established last year to develop advanced AI systems intended to be trustworthy and safe. The organisation was awarded CAN$300 million (US$216 million) in joint funding from Canada and Germany on Wednesday, as the two countries expand cooperation on AI.

Global actions for AI safety are required

The debate is also moving into the international policy arena. OpenAI, Anthropic and Google have begun efforts to establish a standards body, reflecting growing recognition that commercial competition could complicate attempts to establish common safety practices.

Bengio argues that voluntary measures by individual companies will not be enough. He wants future rules to apply across the industry, including to the US-based firms that currently dominate advanced AI development.

He compared the challenge with nuclear weapons, where international treaties and institutions were created because individual countries could not manage the consequences of the technology alone.

“Society needs time to put in place what needs to be put in place,” Bengio said.

Political responses remain mixed. US President Donald Trump has dismissed warnings about AI risks as “hoaxes”, while European Commission President Ursula von der Leyen has proposed discussions with major AI laboratories about how to manage the pace of frontier development.

Canadian Prime Minister Mark Carney has also backed closer cooperation with the European Union on AI safety.