Scott Wiener wants tougher AI oversight after winning a seat in Congress

California state senator Scott Wiener has won election to Congress in the San Francisco seat vacated by Nancy Pelosi. He supports federal AI regulation and could help shape it. Wiener has already drafted two state AI laws: the governor, Gavin Newsom, rejected the first, SB 1047, and signed the second, SB 53. Now Wiener wants Congress to require testing and audits of AI systems, protect whistleblowers and ensure dangerous systems can be shut down. At the same time, he opposes federal rules that would strip states of the right to pass their own laws.

Scott Wiener wants tougher AI oversight after winning a seat in Congress

State laws and a funding dispute

A few hours after the election results were announced, Wiener spoke about AI risks. The occasion was a report published the previous day by the nonprofit METR. It was widely interpreted as evidence that Anthropic had achieved recursive self-improvement: a situation in which AI plays a substantial role in its own development and progress could spiral out of control. Wiener called it an alarming signal and said there was less and less time left to regulate AI. He had hoped Congress would take decisive action by the end of the year, but said that had not happened.

In recent years, Wiener has devoted increasing attention to AI safety. During a 20-minute conversation, he mentioned recursive self-improvement several times and said technological development needed to slow down, or society could approach catastrophe without realizing it.

In 2024, Wiener introduced the SB 1047 bill. It was intended to become the first law in the country specifically focused on advanced AI systems, but California governor Gavin Newsom vetoed it. Wiener then proposed the less stringent SB 53, which Newsom signed. Both measures influenced bills proposed at the federal level.

Wiener’s election campaign was backed by Public First Action, an organization that advocates AI regulation. It contributed more than $3 million to campaigns and receives significant funding from Anthropic. Wiener’s opponents accused him of being too close to the AI industry. In response, he said the organization backed him as the first lawmaker in the country to propose AI regulation, secure passage of a law and bring it into effect. Wiener called the attacks on him cynical: he was being criticized for accepting support from an organization even though he had regulated AI in opposition to Big Tech.

What measures he proposes

If Congress creates a dedicated committee on artificial intelligence — an idea Democrats have been discussing since September — Wiener would like to serve on it. He also supports establishing a strong, flexible federal regulator. Democratic California congressman Ro Khanna has proposed creating such an agency.

Wiener sees whistleblower protections, requirements for systems testing and auditing, and a mandatory ability to shut systems down as essential. He also proposes using Congress’s oversight powers to require AI labs to report on their activities. The public, he said, should have a say in decisions that affect humanity’s future and should know what is happening.

At the same time, Wiener opposes a broad federal ban on state AI laws, a provision included in some proposed bills. He believes states should retain the ability to move faster than Congress and test different approaches.

AI risks and political constraints

Wiener insists that federal measures must be effective and match the scale of the threat. He fears losing control of AI and sees recursive self-improvement as a possible path to losing control completely.

Wiener named Senator Bernie Sanders among those willing to discuss catastrophic AI risks: Sanders has met prominent figures in the AI safety movement and has spoken repeatedly about existential threats. But Wiener’s perspective, shaped in the San Francisco Bay Area and focused on the prospects for artificial general intelligence, differs from public sentiment across the country. Data centers are at the center of public discontent.

On Tuesday, voters backed 40 of 41 initiatives opposing data centers. Wiener believes this is primarily a reaction to local impacts: facilities close to homes, higher electricity bills and water consumption. In his assessment, these issues matter more to most people than general concerns about AI’s development. Opponents of AI’s further expansion, he added, may see campaigns against data centers as a way to influence what happens.

Wiener called himself only a “cautious optimist” about Congress’s ability to constrain AI, saying there were reasons to doubt whether the tech industry could be regulated effectively. He also fears the bills may not make it through the administration of President Donald Trump. In September, Trump called concerns about catastrophic AI risks a “hoax,” and Wiener sees Trump’s reluctance to cooperate with China as a problem: in his view, such cooperation is necessary to prevent the worst outcomes.

Trump’s unpredictability leaves Wiener with some hope, though he finds it alarming. So far, he said, the president’s decisions have been problematic but have not led to complete catastrophe. Wiener hopes action will not require an emergency; if one does trigger change, he would like the damage to be as limited as possible.

Dataist's take: Wiener’s election gives an AI regulation advocate the chance to take part in the federal debate, but his proposals still have to pass through Congress and the Trump administration. His position also covers both the risks posed by advanced models and the local impact of data center construction — issues that, as he notes, do not always align with public priorities.