Over the past two years, safety specialists from leading AI labs have left several times with public warnings that commercial interests are pushing safeguards aside. As AI regulation is discussed in London, researcher Isabelle Ferreras argues that the industry needs more than external rules: companies themselves must be structured differently, giving employees a say in decisions about how AI is developed and used.
AI cannot be developed safely without workers having a voice
Why resignations are no substitute for participation
Recently, a specialist responsible for safety reports on new products left OpenAI. He called the company’s culture “broken” and said the entire industry was too careless in its approach to AI development. A month earlier, an Anthropic researcher announced his departure, warning that he believed AI could destroy humanity before the end of the decade.
The public admires such people’s courage, discusses their warnings, then moves on to other topics. But Ferreras argues that the reasons behind their resignations matter more. Political scientist Albert Hirschman drew a distinction between exit and voice: resigning does not give an employee a way to influence an organization. When an engineer can report a risk only by leaving the company, that points to a problem in corporate governance. The knowledge needed to develop systems safely leaves with them.
This week in London, government officials, technology workers, trade unions, parliamentarians and experts from international organizations, including the Vatican and the UN, are discussing AI safety. Ferreras argues that external regulation will not work while companies remain autocracies internally.
Who sets the rules for AI
In debates about regulation, the author writes, two approaches dominate: treating AI as a consumer product that should be tested for defects, or as a participant in a geopolitical race in which business requirements should be relaxed to secure victory. In her view, this logic is reflected in the EU’s digital package, proposed by Ursula von der Leyen: it delays the introduction of requirements under the EU AI Act, adopted in 2024. The law sets specific safety rules for developers of high-risk AI systems. Corporate lobbyists support the idea that safety and innovation require a trade-off, arguing that legislation places an excessive burden on businesses.
But AI, Ferreras stresses, affects more than software users: it is a technology of control. At leading labs including OpenAI, Anthropic, Microsoft and Google, a small group of executives and investors decides which values will be built into models that affect millions of people’s lives.
Oxford University lawyer Jeremias Adams-Prassl, who studies labor automation in Europe, has shown that algorithmic management already monitors workers from different social groups, evaluates them and fires them—often without human involvement.
Ferreras links this practice to how companies are structured. In a democratic society, authority should rest on the consent of those being governed. Yet corporate law still treats companies as the property of shareholders and workers as subordinates, even though workers invest their knowledge, health and lives in their jobs. To direct AI toward the public good, the author proposes combining government regulation with corporate governance reform.
Three areas for reform
The first concerns AI development and protecting the people who build models. Legislators should prohibit nondisclosure and non-disparagement agreements when they prevent employees from reporting safety risks, and provide effective whistleblower protections. Workers also need collective rights to participate in company strategy, independent channels for reporting safety violations, and formal representation at leading labs before models are released.
The second area is the use of AI in the workplace. This summer, Meta faced employee backlash after Facebook, Instagram and WhatsApp introduced a tool to collect data on computer use. It tracked keystrokes, mouse clicks and screen contents to gather data for training AI models. Workers feared that their privacy and right to consent were being disregarded, and that they could lose their jobs.
Ferreras proposes giving workers a binding right to participate in decisions about the use of AI in the workplace. Algorithmic systems that monitor, evaluate or discipline employees should not be introduced without prior negotiations. At the national level and in the EU, collective bargaining over AI adoption and the right of works councils or trade unions to veto it should be considered. They should have the support of independent technical experts, and automated firing should be strictly prohibited.
The third reform concerns corporate governance more broadly. The author proposes democratizing boards and encouraging companies not to concentrate all power at the top. Legislators should provide for the election of worker representatives to company boards and develop a system of dual majorities: strategic decisions would require the consent of both investors and workers.
Economists Daron Acemoglu and Simon Johnson, who received the 2024 Nobel Prize in Economic Sciences, have shown that technological revolutions do not in themselves guarantee broadly shared prosperity: the outcome depends on the institutions that determine who benefits and who bears the costs. Ferreras argues that extending democratic principles to companies is necessary for more considered decisions. Corporate leadership needs what Yale political theorist Hélène Landemore calls “democratic reason”—the collective intelligence of diverse people who discuss decisions and take part in making them.
Isabelle Ferreras is a research fellow in the Acceleration Program at the University of Oxford’s Institute for Ethics in AI, a research director at FNRS at the University of Louvain in Belgium, and a co-founder of DemocraticIntelligence.org.
Dataist’s take: The proposed reforms connect AI safety to who makes decisions inside companies and who can challenge them. Until the people building these systems and the workers affected by algorithmic management have effective ways to intervene in those decisions, regulation will remain dependent on companies and individual whistleblowers.
Daily AI news
Every day we pick what actually matters in AI and explain it plainly — no hype, no filler. Subscribe if you want to follow where the industry is going.
Only what matters — every day
Follow on X