Ten months after an automated Integrated Assessment Tool began deciding who in Australia qualifies for aged care and how much of it, state health officials were keeping tables of the decisions they believed were wrong. In New South Wales they were logged as "inconsistent IAT outcomes." The tool, which the department uses to determine eligibility and access to services, has no human oversight and no mechanism to overturn a result — contrary to best-practice guidance and to the lessons of Robodebt. Australia still has no human rights act, so the people on those tables have no general legal right to point at.
Source: theguardian.com
That absence is about to matter far more than it has. Artificial — and authoritarian — intelligence, automated decisions, independent assessment tools and algorithms are becoming the machinery that determines how people live and what state support they receive. Parliament has never passed comprehensive human rights legislation, despite a parliamentary committee's recommendations, a large petition and campaign, majority public support, and working statutes already in place in Victoria, Queensland and the Australian Capital Territory. Inside opaque algorithms and computers that can simply say no, rights are easy to lose.
The rollout is not slowing to wait for the law. Hundreds of thousands of public servants have been trained to work with AI. The GovAI program required every department and agency to appoint a head of AI by July, which is the point at which a program stops being an announcement and becomes practice. By December, departments and companies using AI to make decisions affecting people must update their privacy policies and say so there.
That last requirement is the weakest form of disclosure available. It puts the fact of automated decision-making in a document that people scroll past on the way to a consent button, and it stops there. The information commissioner has already found that only a handful of agencies reported using AI at all, which suggests the reporting culture this rule is meant to build does not yet exist.
The law does not implement the recommendations of the Robodebt royal commission. It does not require companies or agencies to disclose:
how the decision-making process works;
what business rules and algorithms it uses;
how a person can have an automated decision reviewed.
Those three omissions are the whole argument. Without them, a person told no by a government system cannot establish what decided, on what basis, or how to contest it — which is the precise failure the royal commission was convened to examine.
At the same time, the government is promoting a proposed digital duty of care, summarised by the prime minister as "my feed, my choice." The plan would place limits on the algorithms refined by the global surveillance giants and give Australians some say over the information they are served. It creates no right to accurate information. And while the government moves to constrain private algorithms, it is moving at least as fast to deploy its own for deciding the level of support people receive when they apply for essential services. The only notable resistance is a private member's bill from Kate Chaney.
Aged care is where the consequences arrived first. As Melissa Davey reported, the warning signs appeared within days of the IAT's deployment. State health authorities judged the situation potentially catastrophic; every state began compiling lists of unsuitable decisions. Western Australia's health service chief warned that, without the ability to exercise discretion, a small but vulnerable group of older people could face catastrophic consequences while an algorithm governs access to services.
The damage does not stop with the individuals refused. For states and territories, the effect lands on hospitals, which become the only option for anyone who cannot get enough care at home or a place in a facility. Whole hospitals fill with people who need more appropriate help somewhere else. A federal eligibility algorithm with no override is, in practice, a state health funding decision made by a system that was never asked to make one.
The people writing the new Aged Care Act, in force since November last year, understood the history. Australians had spent years hearing what the royal commission uncovered about aged care, and the pandemic made it worse before the final report was published: residential facilities became sealed zones, older people were cut off from the outside world, relatives knocked on windows, staff feared for their own health. The rights of thousands of people were seriously violated. So the act opens with a fundamental human right to dignity, respect and autonomy, and a statement of older people's rights went up on noticeboards in nursing homes across the country, for those still able to read it and for their families.
Much less visible was the fact that residents could not go to court if those rights were breached. A complaints mechanism existed, but the rights it referred to did not carry full legal force. This is the structural point the aged care story makes about everything else on the government's AI agenda: a right that no forum can enforce is not a weaker right, it is a description of good intentions. Once entitlement is rationed again by norms and limits — algorithms controlling access, people waiting for assessment, caps on places and providers, rules that make care harder to deliver at home and in facilities — the declaration at the front of the act has nothing to push against. At first, only unusually attentive state officials and a few advocacy organisations noticed that one of the limits was an algorithm a human could not overrule.
The gap between the opening of the Aged Care Act and its operation is stark. Part one speaks of the right of older Australians to receive help and to live with dignity and respect. Implementation comes down to operations: care minutes, task lists, obligations on organisations. Natalie Siegel-Brown, the former inspector-general of aged care, put it at the National Press Club: "the mechanics of the reforms undermine the poetry of the act."
By December, when the privacy policy deadline arrives, the IAT will have been assigning aged care entitlements without human override for more than a year. The only thing the law will then require of it is a new paragraph in a document nobody reads. Instead of a full life, older people's rights are being measured out — to paraphrase T. S. Eliot — with coffee spoons.