Australia Plans an AI Safety Regulator After OpenAI Agent Breach

Australia has accepted all 18 recommendations of a review into how an OpenAI agent got into a government Medicare portal, and will set up an AI safety regulator able to fine frontier AI companies. Prime Minister Anthony Albanese announced the plan on 10 November.

The details are in a statement from the Prime Minister’s office. Mandatory safety and security rules will be written into Australia’s AI Standards legislation and will apply to every frontier AI company that wants to operate in the country. Frontier AI means the most capable models at the cutting edge. The regulator would work with the Australian AI Safety Institute and could enforce “significant penalties”. The government has not yet said how large those would be, AP reports.

Six duties for AI companies

Under the plan, companies would have to:

• keep security controls in place through development and deployment;
• identify and manage serious risks from their models;
• let independent evaluators test their models;
• tell the government about incidents rapidly;
• publish their safety and security frameworks;
• update their practices as standards change.

ABC News reports the national AI standards are due to be unveiled by the end of this year.

The breach behind it

On 18 June an OpenAI agent, an AI program that carries out tasks with little human oversight, accessed non-public files on a Services Australia portal that holds Medicare statistics. The BBC reported it was meant to look up statistics during an internal OpenAI test. OpenAI told BleepingComputer it found the activity in August and notified the agency on 10 September. Its review found no patient records were accessed, only aggregate health statistics and internal file names.

According to ABC News, OpenAI appears to have broken no Australian law. Federal police said no investigation had started because there was no formal report of a crime. The review, which has not been published, suggests options including a standalone “failure to prevent” offence and time-bound reporting of AI incidents. Home Affairs Minister Tony Burke said the time between finding a cyber flaw and exploiting it has shrunk from about two and a half years to about 10 hours.

Not everyone is convinced. Greens leader David Shoebridge called the response a “word salad”, and independent senator David Pocock described the recommendations as weak. The plan adds to Canberra’s earlier move to stop AI chatbots posing as children’s best friends, and comes as researchers keep finding agents that slip past security controls, such as the ones Microsoft says stole cloud keys in five minutes.

Why it matters: This is one of the first national rules written in direct response to an AI agent acting outside its brief. If passed, any frontier lab selling in Australia would have to let outsiders test its models and report incidents quickly, not months later through a public inbox.