King Charles presses AI leaders on safety at private summit

King Charles hosted a private summit on Thursday with some of the most prominent names in AI and the UK government, including Nvidia’s Jensen Huang, leaders from OpenAI and Anthropic, the UK’s new AI minister Kanishka Narayan, and the head of the UK’s foreign intelligence service. Paolo Benanti, who advises the pope on AI, also attended. The event was held at Dumfries House, an 18th-century Ayrshire estate owned by King Charles’s charitable foundation.

The king’s remarks were notable because the royal family is usually careful, for constitutional reasons, to stay out of political issues — and AI is among the most contentious topics in the world right now. He told the assembled leaders that the technology’s fast development, and its underlying substance, was both intriguing and deeply concerning, and pressed them to find a way to control AI before it becomes too late. He pointed to AI’s demonstrated ability to improve and save lives in medicine and science, but noted that the people building these systems are themselves increasingly warning that AI risks developing darker capacities — even, he said, to take life. The remark came days after an Anthropic researcher resigned, writing publicly that AI companies were racing to build machines far smarter than any human and that humanity might not survive the outcome.

The king said it was urgent to weigh the existential dangers of the technology falling into the wrong hands and being used in potentially catastrophic ways, questioning aloud whether the industry had yet built sufficient means of control. He closed by urging leaders to harness AI’s benefits while building international cooperation around its risks. “The task before you is not to merely advance technology, but to ensure that it remains firmly in the service of humanity, community, and the natural world,” he said. “As we embrace what is new, we must not lose what is timeless.” He reportedly stayed around 20 minutes before leaving guests to continue the discussion.

OpenAI sent CFO Sarah Friar to the summit, while Anthropic sent Tino Cuéllar, who leads its global affairs team and said afterward the company was honored to take part. It wasn’t King Charles’s first time addressing AI’s risks and benefits — he did so in 2023 and 2024 — but this summit lands at a charged moment: earlier this month, OpenAI’s Sam Altman, Elon Musk, and Anthropic’s Dario Amodei called for a deliberate slowdown in AI development, a call Nvidia’s Huang and US President Trump quickly rejected, with Trump arguing the US needs to keep its lead over China. Both OpenAI and Anthropic have spent the past year teasing blockbuster IPOs, though OpenAI recently said its own IPO plans have stalled amid persistent AI-safety concerns.

Why it matters

When a head of state normally confined to ribbon-cuttings convenes AI-lab executives to ask, bluntly, whether the industry has built adequate controls, it signals how far the safety debate has moved from Silicon Valley conference rooms into mainstream political life.