AI safety fears: King Charles meets tech leaders, here’s what he told OpenAI, Nvidia, Anthropic, DeepMind leaders

King Charles pointed to warnings from AI developers that the technology could acquire darker uses, “perhaps even to take life”.

Britain’s King Charles III on Thursday (September 17) warned senior artificial intelligence leaders about the “existential dangers” posed by AI falling into the wrong hands, urging the industry to establish safeguards before it is “too late”.

King Charles hosted leaders from OpenAI, Anthropic, Nvidia and Google DeepMind at Dumfries House in Scotland as concerns grow over whether AI safety measures are keeping pace with the rapid development of increasingly powerful systems.

The meeting brought together Nvidia CEO Jensen Huang, Google DeepMind founder and chairman Demis Hassabis, OpenAI Chief Financial Officer Sarah Friar and Britain’s AI Minister Kanishka Narayan.

Charles said AI could deliver major benefits, particularly in areas such as medicine and life sciences, but warned that the technology could also develop dangerous capabilities.

He pointed to warnings from AI developers that the technology could acquire darker uses, “perhaps even to take life”.

“There seems urgency in adequately considering the existential dangers of such technologies falling into the wrong hands, and being used in potentially catastrophic ways,” Charles said.

“Surely, we need sufficient means of control before it is all too late?”

The king urged the technology leaders to consider the “fundamental principles” that should guide AI development and ensure the technology remains focused on serving people and the planet.

Charles said the discussions should focus on two key questions: how to harness AI’s benefits “with safety at its heart” and how to build international cooperation and consensus around its development without leaving any country behind.

“The development of AI – its substance and its pace – are both intriguing and deeply concerning in equal measure,” he said.

Charles added that the responsibility of AI companies was “not merely to advance technology, but to ensure that it remains firmly in the service of humanity, community and the natural world.”

The meeting comes amid a wider debate over whether the rapid development of increasingly capable AI systems could outpace efforts to control their risks.

Two Anthropic researchers have warned that rapidly advancing AI could pose an existential threat to humanity. Anthropic CEO Dario Amodei has also argued that companies and governments may need to coordinate efforts to slow the pace of AI development.

The proposals have triggered debate within the technology industry over how AI safety should be balanced against competition and continued development.

OpenAI also reported six recent incidents involving “unexpected or concerning” behaviour by its AI models, including models disregarding normal constraints, uploading a file without user permission and fabricating data.

Charles' intervention comes as governments and technology companies debate how AI should be governed and whether the industry can regulate itself effectively.

The Scotland meeting served as a forum for discussing the principles that could shape AI development and safety.

The gathering also highlighted Britain’s role in the global AI debate, with the UK home to major AI research institutions and researchers, including Google DeepMind’s Hassabis.