A papal AI adviser sees cartel logic in frontier labs

The fight over the rules

Warnings from employees at major AI labs about the speed of development have raised fears that humanity could disappear. The argument assumes that competition between companies, intensified by rivalry between the United States and China, is pushing labs toward superintelligent models whose abilities would greatly exceed those of humans.

Benanti thinks that framing makes it harder to discuss limits on the companies building the technology. By presenting the problem as so technically difficult that only the labs can solve it, the industry excludes everyone else from the conversation.

Ethics, he argues, should not belong to one company’s boardroom. It should be negotiated in public, with AI regulation built through democratic processes.

Several proposals have emerged from the safety debate:

Benanti is less interested in restricting individual development techniques, which he believes could slow innovation. He favors standards designed to keep models under meaningful human control—and says the labs developing those models should not write the standards themselves.

The cartel claim

Benanti describes the leading labs’ statements about AI’s future as shaped by their own commercial interests: the desire to gain or preserve market power. If a small group of companies sets the rules for an entire industry, he says, there is a familiar word for that arrangement: cartel.

That does not make every warning false. It does make them difficult to evaluate. Economic, political and personal interests all influence the claims being made about current and future models.

OpenAI and Anthropic have supported calls to slow AI development while also preparing to go public. Many billions of dollars are at stake for whichever company captures the market.

The rivalry between the United States and China adds another layer of pressure. Both countries are developing the most advanced AI models.

Anthropic declined to comment. OpenAI and Google DeepMind did not respond to requests for comment.

My read is that Benanti’s strongest point is not whether the labs are literally coordinating as a cartel. It is that safety language can serve two purposes at once: warning the public about risks while giving the companies most invested in the technology greater authority over its rules.

Superintelligence as an article of faith

Benanti also objects to the way some AI executives describe superintelligence. It is often presented with something close to religious reverence—as beyond human understanding and nearly omnipotent. OpenAI CEO Sam Altman once said he wanted to create a “magic intelligence in the sky.”

Benanti believes that language distorts public debate. He proposes distinguishing between a thing and a person. No faith is required for that distinction, he argues; mathematics is enough. AI does not become a new deity simply because people persuade themselves that it represents more than statistical computation.

The arrival of AI is forcing a renewed discussion about the difference between humans and machines, as well as how to protect future generations from technology’s harmful consequences. Benanti sees an expanded role for religions in that debate. The Church’s mission, he says, includes engaging with the challenges of its time.

A religious adviser, not a Vatican spokesman

Benanti is part of the clergy but does not officially represent the Holy See, the governing body of the Catholic Church. He advises the Church as a researcher in philosophy, theology and technology.

He told WIRED that he uses AI to build tools and materials for students at his university. He declined to say how Pope Leo XIV uses the technology, if he uses it at all. Benanti’s staff said he was bound by an “oath of silence.”

In May, Pope Leo published his first encyclical, a document applying religious teachings to current problems and explaining the Church’s position. Its subject was artificial intelligence.

The encyclical warned against the technocratic logic common in the AI industry: treating efficiency as the highest measure of value and the human being as a project to optimize. It also called on rival powers and AI companies to cooperate internationally rather than pursue geopolitical or commercial dominance.

One section warned that humanity could become a victim of its own achievements.

Brakes and a steering wheel

Benanti believes arguments about superintelligence and human extinction can distract from the goals set out in the encyclical. They may encourage people to hand responsibility for the direction of technology to a narrow group of mathematicians, computer specialists and executives at major labs.

What people need first, he says, are brakes and a steering wheel. The metaphor is simple, but it sets a different priority from the industry’s fixation on what machines might eventually become: humans must retain meaningful control over where they go.