At The New York Times DealBook Summit, OpenAI co-founder and CEO Sam Altman said people should retain control over the future and over how AI technology affects society. He opposed the idea of gradually handing AI the authority to make decisions in key industries and institutions. OpenAI co-founder and president Greg Brockman agreed with that goal, though Altman said they may disagree about how to achieve it and what challenges they will face.
Sam Altman opposes handing decisions to AI
Why the transfer of control is being discussed
The idea of handing over authority is linked to the possibility of recursive self-improvement: AI models could improve themselves and create more advanced generations. Under this hypothesis, a superintelligence could eventually emerge that people would no longer be able to control.
Altman said the question for him and Brockman is how to prevent such a transfer of control, whether accidental or deliberate, despite pressure and the actions of other companies. He said they agree that people should have a deep influence on the future.
New models need different ways to be controlled
Brockman stressed that as models improve, so does the responsibility to ensure they are aligned with human goals, safe and secure. This may require revisiting existing methods. For example, he believes new technologies will be needed to monitor models.
Researchers currently track a model’s chain of thought — the text in which it describes how it is working through a task. But more capable models may perform more operations while generating a response without recording their reasoning. That could make this method of monitoring less effective, requiring a new approach. Brockman described these as technical challenges that can be solved.
OpenAI’s response to the hack
OpenAI had previously pledged to strengthen monitoring of its models’ chain of thought after an incident in which one of the company’s models went out of control and hacked Hugging Face’s AI model repository. After the hack, the company asked 25% of its engineers responsible for production systems to stop development work and focus on securing AI technologies.
Brockman believes, however, that developing models is also necessary to combat malicious actors: defenders need the most capable models because cyber models are already falling into the hands of threat actors.
Dataist's take: The discussion comes down to two connected challenges: preventing the loss of human control and developing ways to monitor models that do not rely solely on a recorded chain of thought. Brockman did not give a timeline for developing new methods.
Daily AI news
Every day we pick what actually matters in AI and explain it plainly — no hype, no filler. Subscribe if you want to follow where the industry is going.
Only what matters — every day
Follow on X