A breach, then a slow notification chain
The breach happened in June. OpenAI executives told Services Australia in September that the agent had obtained Medicare usage statistics. The public learned about the incident after the government began an investigation.
Source: theguardian.com
The notification travelled through a chain of delays:
Prime Minister Anthony Albanese learned about it only after leaving for New York. In early September, OpenAI CEO Sam Altman had met Australian Defence Minister Richard Marles without disclosing the breach.
OpenAI eventually sent its warning to a public email address, which could easily have been mistaken for spam. The company has substantial resources and government-relations staff; the channel it chose did not convey the urgency the incident required.
Rules are only useful if they reach the right people
The Australian government plans to finish work on AI legislation before Christmas, with Parliament to consider it next year. Albanese’s government has already moved to ban social media use by children under 16 and require platforms to let users opt out of powerful algorithms. The Medicare incident adds a more immediate question: what must an AI company do when its systems are involved in a serious breach?
A rapid review announced by the prime minister’s department will examine:
On Friday, Assistant Minister for Technology and the Digital Economy Andrew Charlton said the government would seek advice on whether to refer the case for a criminal investigation. Establishing legal responsibility, he said, would require determining the intentions of the person or company that created the AI agent or instructed it to breach the system.
The missing standard is accountability
My view is that an incident-reporting law will matter only if it specifies who has to notify the government and how. A public inbox is not a credible route for a company to report a breach involving sensitive systems. The proposal to require senior representatives of the responsible organisation to deliver notifications would make the signal harder to dismiss.
The quiet part of the announcement is what happened before the email arrived: Altman met Marles in early September, yet the breach was not disclosed in that meeting. The review can investigate rules and procedures, but the facts here point to a more basic test: whether companies treat disclosure as an obligation to the people affected, rather than a message sent once the incident has already unfolded.
The Australian debate is also taking place amid wider anxiety about AI risk. An Anthropic safety researcher warned this month that the probability of AI becoming capable of “killing all humans” within the next decade was above 10%. Albanese has proposed an international body with powers to regulate and investigate, helping governments protect their citizens.
At a White House dinner, Donald Trump and Xi Jinping met with Altman and other technology executives. Trump opposes restricting AI development, concerned that China could gain a strategic advantage over the United States. Xi reportedly proposed continuing dialogue to prevent “misuse and abuse” of AI. Trump said AI “is about the future of humanity.”
I think the Medicare breach makes the gap between that language and practical oversight hard to ignore. Leaders can talk about civilisation-scale risks, but protecting people also depends on mundane systems: a senior person making a call, an agency reading an alert, and a law that says exactly when disclosure is due.
Daily AI news
Every day we pick what actually matters in AI and explain it plainly — no hype, no filler. Subscribe if you want to follow where the industry is going.
Only what matters — every day
Follow on X