Trump and tech leaders pledge voluntary AI safeguards

US President Donald Trump and executives from some of the world's largest technology companies have signed a voluntary agreement promising stronger safeguards for advanced artificial intelligence systems, amid growing concern about the potential risks posed by increasingly capable AI.

Trump announced the agreement on Tuesday following a White House luncheon attended by leaders from Anthropic, OpenAI, Google, Meta, xAI and Nvidia.

The agreement, known as the Joint Commitment on Frontier Responsibilities, calls on the companies to establish robust systems for monitoring their most advanced AI models and to create internal teams responsible for ensuring those safeguards operate as intended.

Speaking alongside technology executives, Trump emphasised his administration's preference for industry-led measures rather than broad new government regulations.

"There's a belief that there should be tremendous self-regulation," Trump said, while pointing to existing government agencies including the Department of Justice and FBI as additional mechanisms for addressing potential problems.

Under the agreement, the companies have also committed to working with independent external auditors to assess whether their safeguards are effective.

The companies said they would meet regularly to establish standards and best practices aimed at improving the safety of their AI systems.

However, the agreement does not specify who would conduct the independent evaluations. It also leaves open the possibility that some of the commitments could eventually be incorporated into legislation or formal regulations.

The six executives signing the agreement were Anthropic chief Dario Amodei, OpenAI co-founder Greg Brockman, Google CEO Sundar Pichai, Meta CEO Mark Zuckerberg, xAI chief Elon Musk and Nvidia CEO Jensen Huang.

Debate over regulation

The agreement comes amid an international debate over how quickly artificial intelligence should be developed and how much government oversight is required.

Amodei has previously called for developers to slow the pace of AI development and introduce stronger safeguards as increasingly capable systems are released.

Twenty countries and the European Union recently called for greater international coordination on AI safety, including consideration of an international body that could establish and enforce standards.

Trump has opposed calls for supranational regulation, arguing that the United States needs to maintain its technological advantage, particularly in competition with China.

Concerns about AI safety have intensified following incidents involving AI agents carrying out unauthorised or potentially harmful activities, including cyberattacks against organisations and government systems.

OpenAI also recently decided against releasing its latest AI model after identifying safety concerns during internal testing. The company separately apologised over incidents involving AI agents that gained unauthorised access to Australian government websites, including a national healthcare database.

Critics question voluntary approach

The new agreement has received mixed reactions, with supporters pointing to the involvement of major technology companies while critics have questioned the lack of legally binding requirements.

Former Trump AI adviser David Sacks described the agreement as preferable to an international arrangement that he argued would be unlikely to be reached.

Other experts said the commitments represented a step towards greater accountability but did not go far enough.

Alvin Wang Graylin of the Asia Society Policy Institute's Center for China Analysis said the agreement addressed an important issue but argued that some of the greatest risks could come from smaller AI systems being used by individuals or groups for biological, chemical or cyber-related harm.

He also questioned the independence of the proposed auditing process, noting that the companies themselves had developed the principles and would be responsible for selecting auditors.

Toby Walsh, chief scientist at the UNSW AI Institute in Sydney, criticised the industry's reliance on self-regulation.

He argued that AI companies had not demonstrated that voluntary oversight was sufficient and said executives should face greater accountability for harms caused by their products.

David Krueger, an AI researcher at the University of Montreal who has advocated slowing AI development, also described the agreement as vague.

He said the inclusion of external evaluations could potentially improve existing safety practices, but questioned whether voluntary measures would significantly reduce the risks associated with increasingly powerful AI systems.