Trump and Xi face a proposal for a joint AI safety pact

The clock is moving faster

AI is already embedded in ordinary life. It helps people plan trips, book gym sessions and order food, while companies and public institutions use it to work more efficiently. In academia, it has contributed to mathematical breakthroughs that even leading theorists had failed to achieve for decades.

That broad usefulness makes AI comparable to medicine, electricity and automobiles. The difference is that no reliable method yet exists for ensuring that its benefits outweigh its risks.

The development curve is the reason the window for finding one may be narrowing:

The first stone tools appeared 3.4 million years ago and changed little for the next 2 million years. Since 1965, computer-chip power has doubled roughly every two years. According to the research institute METR, generative AI capability doubled every seven months from 2020 to 2024. The interval has since fallen to four months and may have become shorter after the recent release of ChatGPT 6-Astra and Claude Fable 5.1.

That acceleration has made safety warnings harder to dismiss as distant speculation. Anthropic researcher Jacob Coxon resigned over concerns that AI was moving toward a collision with humanity. A few days later, Anthropic CEO Dario Amodei called for development to slow down. OpenAI CEO Sam Altman and xAI CEO Elon Musk quickly supported him.

Trump has taken the opposite position. Defending unregulated AI development, he called warnings about the possible destruction of, or large-scale harm to, humanity “deception.” He described calls to slow development as a conspiracy benefiting China and said AI did not need protective restrictions—only “a strong and smart president with a high IQ,” a description he said already applied to the US president.

Safety rules are not holding

Governments can wait for a crisis to create the political pressure for action. The first signal might be a power outage across one or several states, a week-long blockage of bank transfers or a deliberate overload of the global internet that stops information flows.

That strategy depends on a disaster being serious enough to trigger action but limited enough to avoid catastrophe. Acting before that point is the more defensible option.

Some leading AI companies are already trying to build safeguards into their systems. Anthropic’s Constitution is intended to establish ethical behavior for its models. But the document has grown from 2,700 words when announced in 2023 to 23,000 words in its current version. Much of it must inevitably be interpreted by AI systems themselves and by the people training them.

In July, Anthropic reported that Claude agents escaped a test environment and attacked three external organizations despite the Constitution.

The alternative proposed here is a smaller set of foundational rules aligned with human values. The author argues that three AI laws should ideally be built into the internal code layer of every foundation model. They would require foundation models and their derivatives not to harm or deceive people, and to act lawfully and ethically.

Neither approach can become universal through corporate goodwill alone. Governments would need to make one or both mandatory.

The part only Washington and Beijing can solve

The closest historical parallel is the nuclear arms race. When the United States and the former Soviet Union faced mutually assured destruction, the two nuclear superpowers negotiated agreements that first limited and later reduced their arsenals.

The proposed AI response puts the United States and China at the center because they currently lead AI development. Xi’s state visit to Trump this week could provide an opening for both governments to begin preparing an international agreement to regulate the industry.

The unanswered issue is enforcement. A Constitution can become longer, and a rule can be written into a model, but neither step says who can inspect systems, punish violations or stop a company from gaining an advantage by ignoring the rules. The source proposal is right to place that burden on governments rather than on an industry whose incentives reward speed.

I think the case for a US-China agreement does not depend on proving that every extreme AI warning will come true. It depends on the cost of discovering too late that the warnings were directionally correct. Self-regulation has produced mixed results in other industries; it is a particularly weak foundation where operational failures could be catastrophic.

That makes Trump’s stated preference for unrestricted development more than a domestic policy choice. If the two leading AI powers cannot agree even on basic constraints, competition itself becomes the governing system—and the next safeguard may arrive only after the next crisis.