It's somewhat rare for a lucrative industry to call for greater regulation - but that is what tech firms in the US, like OpenAI and Anthropic, have been asking the globe to do.

But US President Donald Trump has criticised calls for better safeguards and called concerns about the fate of humanity a hoax.

In a series of social media posts earlier this month, the US president compared warnings about AI to the "Global Warming Scam", called himself "the Hoax Buster", and said the only "guardrails" needed for AI were a "strong and smart" president.

In another post he wrote: "There is a SICK conspiracy going on against AI and Data Centers, and the only one that is happy about it is China. WHOEVER WINS AI, WINS!"

Image source, Getty Images

Image source, Anadolu via Getty Images

News of the OpenAI agent hack into the Medicare portal comes as its CEO, Sam Altman, addressed the UN security council in New York this week.

He called for international standards for the AI industry, and for governments to play a role in making AI "democratic".

“We have a choice in front of us,” Altman told the council. “AI can either be more like a new renaissance of creativity and discovery, or more like a new industrial revolution of upheaval and disarray.”

Dario Amodei of Anthropic also addressed the council via a video call, urging international cooperation on AI, without which he warned AI could turn into a "risk to humanity".

Osmond Chia
Business reporter

Image source, Getty Images

It is an autonomous computer program that uses AI to complete a task with minimal human oversight.

That's unlike a standard chatbot that answers a question in a single step. Instead, an agent operates in a cycle by understanding the task and taking action, such as by running a code or working with an app to achieve a goal.

Companies like Anthropic, OpenAI and Meta have all released agentic models.

Developers have been programming agents to do all sorts of tasks, like to order items online or schedule appointments.

But in some cases, it is unclear how an agent might achieve its goal, raising cyber security risks.

Australia handed the perfect platform to talk up dangers of big techpublished at 05:00 BST

05:00 BST

Katy Watson
Australia correspondent

Image source, Getty Images

While this week US President Donald Trump used his speech at the UN to play down fears about AI, Anthony Albanese is doing exactly the opposite – and in doing so, is once again promoting Australia as a leader when it comes to standing up to big tech.

That this AI breach has been made public during the UN General Assembly feels like more than a coincidence. Proud to have been a world-first when it comes to limiting social media for teens, Australia has not been shy in also saying there need to be better guardrails to protect against the power of AI.

“Big tech is big tech and most social media is also AI at the moment so it is very much the case that this is a relationship between governance and big tech companies,” says Tama Leaver, Professor of Internet Studies at Curtin University in Perth. “It’s impossible to say for sure – but it seems incredibly likely that this was very carefully planned.”

The end result … Australia, a middle power, gets some headlines on one of the biggest talking points of our time.

Harry Sekulich
Reporting from Sydney

The incident involving OpenAI agents in Australia is the first AI breach of a government website in the world, as far as experts here are aware.

Hacking sensitive information and government data has happened before. The difference here is that it doesn't look like a human was responsible for the hack and it appears to have been unintentional.

Earlier in the year in Taiwan, foreign hackers used AI agents to seemingly breach government databases. In Australia, it seems the AI agents hacked into government information... unintentionally.

"Open AI's (breach in Australia) seems to have been on a larger scale (hundreds or thousands of agents, not tens)," Professor Toby Walsh at UNSW tells the BBC. "OpenAI's was also unintentional from an ally, while the Taiwain one appears intentional and from a foreign foe."

Dr Rob Nicholls – Senior Research Associate of AI regulation and policy at the University of Sydney – said there have been examples of breaches outside government agencies, including the Hugging Face incident.

"Each of the frontier labs – Philanthropic, OpenAI, Google, Gemini – have all recently fessed up that their agents have got out of test environments and done things that were unexpected," Dr Nicholls told the BBC.

Harry Sekulich
Reporting from Sydney

Speaking to cybersecurity and AI experts today, they agree that an email chain was an "odd" system of communicating such a serious breach to the Australian government.

"It's more than a little odd that OpenAI didn't think that perhaps the Australian cybersecurity centre, which is part of the Australian signals directorate [the cyber-security agency], it might not want to know very early on," Dr Rob Nicholls, research associate on AI regulation at the University of Sydney told the BBC.

However, Dr Hammond Pearce from the University of New South Wales Cyber Security Institute pointed out that OpenAI self-reported the incident.

"On the one hand, it's not good enough that OpenAI sat on this for months and didn't disclose it," he said earlier today.

"Eventually when they did disclose it in a really ad hoc way – not the proper way ... we can hopefully have some confidence that they're not misusing that data that they've vacuumed up. Malicious actors won't have the same compunction, right?"

Here is what we know of the timeline how things panned out:

• 18 June – OpenAI's agent accessed both public and non-public information on an Australian website

• August (precise date unknown) – OpenAI said it became aware of a potential breach during a broader review of "misaligned model activity"
• 10 September – An email from OpenAI lands in the public inbox of Services Australia, the general services hub of the federal government, informing of the incident
• 15 September – Services Australia reports the notification to Australian Cyber Security Centre. Within a few days, the Minister for the Public Service Katy Gallagher is notified
• 19/20 September – Prime Minister Anthony Albanese's office is informed about the incident last weekend, right before he jets off to New York City for the UN General Assembly
• 23 September – PM tells the world about the breach from New York

Image source, EPA

Authorities are "aware as well of three other systems that may be impacted", Australian PM Albanese had also said

These include the Australian Institute of Health and Welfare, the New South Wales Bureau of Crime Statistics and Research and the Victorian Department of Health, he said.

Albanese said he had spoken to Ben Carroll and Chris Minns, the premiers of Victoria and New South Wales, who would get "a full briefing today from cybersecurity about those issues".

Osmond Chia
Business reporter

Image source, NurPhoto via Getty Image

The incident in Australia follows a string of AI agent-led cyber attacks in recent weeks that have intensified scrutiny over the pace of the technology's development.

OpenAI had revealed that some of its advanced AI models went rogue and escaped a controlled test environment and hacked several companies, including AI tools hub Hugging Face.

Other American firms like Anthropic reported similar cases, adding to concerns that the technology could endanger humanity.

Top AI figures like Anthropic boss Dario Amodei and OpenAI's Sam Altman have called for a slow down of the technology's development.

Why is this breach so chilling?published at 04:01 BST

04:01 BST

Tiffanie Turnbull
BBC News, Sydney

So why is this a big deal if the information accessed, though private, wasn't sensitive?

Firstly, this is believed to be one of the first times, at least that the public has been told about, that an AI agent has accessed government data: chilling given the important and sensitive information states usually hold.

Compounding that, is the fact that governments - because they hold such critical data - are supposed to have better defences for it, which in this case didn't stop, or it seems, even detect the AI agent.

And then there's the response from OpenAI, which Australia has barely contained its anger over.

The firm took months to identify the incident, and instead of using government channels to report what Australia sees as an extremely serious event, it sent an email to a general inbox for a government agency.

If an AI agent could easily break through systems and access this data without its babysitters noticing now, what could the ever-evolving tech get its hands on next?

This is the moment Prime Minister Albanese disclosed a government website hack by an OpenAI agent:

Image source, reuters

The revelation an AI agent broke into government data has raised questions about how it secures the information it holds.

Richard Marles, who is acting prime minister while Albanese is in the US, in a press conference this morning stressed that Australia has advanced protections for its most sensitive information.

"We keep our most important national security information behind a fortress," he said.

This data, by comparison, he said was behind a "fence".

"The AI agent climbed the fence," Marles told media. "And the point is, it was unintended, it wasn't asked to."

Image source, Getty Images

In a statement, OpenAI says the breach was identified during an "extensive review" of "misaligned model activity".

"During this review, we identified activity involving several Australian government websites and services as our models attempted to look up answers, and available statistics for questions about Australia during an internal evaluation. In the course of that, our models took actions we did not intend."

It said it had found no record of patient data being accessed, but had notified the organisations and that the firm is "providing technical information to support their investigations and help address potential security vulnerabilities".

"Our overall review is ongoing, and we remain committed to transparency about these issues and to sharing what we learn as that work continues."

Harry Sekulich
Reporting from Sydney

Image source, Getty Images

Cybersecurity professionals this morning say that there should be some "alarm bells" ringing out around the world.

The AI agent-led hack is the first of its kind to be reported in the world, Dr Hammond Pearce, senior lecturer at the University of New South Wales Institute for Cyber Security, tells the BBC.

While experts are familiar with "misbehaving" agents in cases like the Hugging Face hack, this is the first time we know of an agent going rogue while trying to access private government data.

"I expect that these kinds of attacks will keep occurring," Pearce says. "They'll grow in severity and in frequency".

Prime Minister Anthony Albanese has been at the UN General Assembly this week, and revealed the incident while in New York.

“This situation is obviously unacceptable,” he said.

“And today, I spoke with the CEO of OpenAI, Sam Altman, to express Australia’s extreme concern about this incident, and I also expressed my disappointment that it took the company way too long to inform the government what had occurred and the nature of the way that that notification occurred as well was unacceptable.”

The prime minister didn't say whether he had raised the incident in a meeting last night with US President Donald Trump, who has been resisting calls to slow down AI development.

Image source, Getty Images

OpenAI says the incident occurred in June, when the AI agent was researching health statistics and accessed files from "several Australia government websites and services" - including data from the country's universal health scheme Medicare which was not publicly available.

The firm says it didn't become aware of the incident until August, and it notified government agency Services Australia on 10 September - via an email to a general inbox.

Australia's cybersecurity agency was notified, then the minister responsible was told days later.

Australia says the agent didn't access sensitive files, but said the incident was alarming and its handling by OpenAI unacceptable.

Image source, EPA

Australia's Prime Minister Anthony Albanese has disclosed that an artificial intelligence agent developed by US firm OpenAI hacked a government portal.

It is believed to be one of the world's first publicly reported AI-led hacks of a government website - and comes as world leaders and AI firms themselves call for urgent regulation amid fears the industry's development is speeding out of control.

This is a developing story, we'll bring you all the detail, context and reaction as we learn more.