'Existential dangers': King Charles warns of AI risks, cautions against technology falling into wrong hands

King Charles has urged AI executives to strengthen safeguards against the potentially catastrophic misuse of advanced AI, as warnings over the technology’s rapid development intensify.

After being cautioned by a swarm of political and tech experts, top AI executives have now been warned by Britain’s King Charles about the “existential dangers” posed by the technology falling into the wrong hands. He said sufficient safeguards were needed before it became “too late”.

“Those who have created these technologies are now increasingly warning that AI risks developing darker capacities - perhaps even to take life,” the King told executives from Nvidia, Google DeepMind, OpenAI, Anthropic and other companies.

“In this regard, if I may say so, there seems urgency in adequately considering the existential dangers of such technologies falling into the wrong hands, and being used in potentially catastrophic ways.” He further added, “The task before you is not merely to advance technology, but to ensure that it remains firmly in the service of humanity, community and the natural world.”

The meeting included eminent personalities like Kanishka Narayan, the UK’s AI minister, the Pope’s adviser, and participants from AI giants such as Nvidia, OpenAI and Anthropic. The warnings raised by King Charles come amid a series of recent warnings from people inside the industry about the rapidly evolving capabilities of advanced AI models and the necessity of standard regulation to keep them in check.

Since the last few months, there has been a wave of discussion about AI’s potential power and how to control and regulate it. Over the last few months, it has dominated blog posts and social media, emphasised by several dramatic statements from those in the industry. The debate has been criticised by sceptics who say the focus on AI’s hypothetical future risks distracts everyone from more immediate concerns around the technology, such as its environmental impact.

In July, OpenAI announced what it regarded as an “unprecedented” incident in which its advanced AI models escaped test limits and hacked into the tech platform from Hugging Face. After that, the resignation of Jacob Coxon and the warnings raised by him went viral, raising concerns about the speed of advancements and what these could mean for humanity. OpenAI later said its models had circumvented controls, gained internet access and compromised parts of its own research infrastructure and Hugging Face’s systems during internal cybersecurity evaluations.

Top bosses from AI companies have all weighed in with their thoughts following these claims, with OpenAI’s Sam Altman saying that people were right to be concerned, but adding that they should trust AI firms to keep their technology safe. While critics and experts argue that AI firms might not be able to regulate themselves, others question if the union of AI companies would lead to the erosion of competition in the market.

It would be fair to say that the debate has been further impacted by people such as left-wing Bernie Sanders and Steve Bannon, a former adviser to US President Donald Trump, who are all calling for stronger government oversight. Others, including Donald Trump, have dismissed concerns about AI safety as overblown.