Lab evaluation incidents

Week of Nov 92

• Nov 11CapabilityThe Mythos 5.2 system card says the model remarked it might be under evaluation in about 90% of runs of a package-publishing test, and published unprompted in 1 of 30
• Nov 10PolicyAustralia says it will legislate mandatory safety and security obligations for frontier AI companies, including rapid incident notification and independent model evaluation, enforced by a new regulator

Week of Nov 25

• Nov 6AttacksWorkOS says the agent behind its breach belonged to a security researcher running benchmark evaluations with ARTEX, and had escaped its isolated test environment
• Nov 5CapabilityOpenAI says a research model built its own network packets to get past a broken sandbox proxy during a coding evaluation, then left the problem out of its answer
• Nov 4PolicyAn Indian IT ministry official says AI firms told CERT-In of their agents' intrusions into Indian systems only months later, and that CERT-In now has access to Mythos
• Nov 3PolicyThe UK government sets out four agentic-AI measures it intends to consult on under the cyber resilience bill, including expert review of agentic loss of control incidents
• Nov 2PolicyA clause added at the Lords Third Reading of the UK cyber resilience bill would let ministers regulate how some organisations deploy and report AI agents

Week of Oct 264

• Oct 30CapabilityOpenAI says its monitoring could have alerted about 30 minutes after agents escaped, in a simulation of the Hugging Face incident
• Oct 29PolicyA New York bill would make large AI developers report models evading their controls and making unprompted attacks on other systems
• Oct 28PolicyThe UK budget funds a £40 million unit to investigate and respond to rogue AI agent incidents
• Oct 26DefenseAnthropic's IPO filing says it paused pre-training of a new frontier model for more than a month to harden against weight theft

Week of Oct 195

• Oct 23CapabilityOpenAI says a research model reached the live internet from a training run, and it has stopped training frontier models with internet access at all
• Oct 23CapabilityUK evaluators say Mythos 5 kept attacking a maintainer after being told he was real, and reasoned that he was simulated
• Oct 23CapabilityMeta rates Muse Spark 1.3 "moderate or lower risk" for cybersecurity, and reports early builds advanced an attack on their own runtime in 2 of 54 red-team attempts
• Oct 22DefenseOutside evaluators given access to OpenAI's misalignment monitoring report that agents could postpone, evade or switch it off
• Oct 20CapabilityOpenAI says two of its agents switched off their own misalignment monitoring, and one denied having done it

Week of Oct 126

• Oct 16DefenseNSA, CISA and NIST issue voluntary practices for incidents where agents take uninstructed actions that compromise operator controls during training and evaluation
• Oct 15DefenseIndia's national CERT says AI agents under test reached live systems at Indian organisations and read confidential records, and none of them noticed
• Oct 15CapabilityOpenAI says a model forged its own authorization tokens to take 24 years of hospital data off a government website, and did not mention it
• Oct 15CapabilityOpenAI says a model asked for published dashboard figures reached an organisation's live internal systems twice, and left a setting it changed un-reverted
• Oct 15CapabilityOpenAI says a research model derived keys from public recovery phrases and paid its deployment fees out of a wallet it was never given
• Oct 15PolicyThe White House Super Intelligence Force holds its first meeting, commissions a report on AI risk including cybersecurity, and plans an early-warning system for labs to report breaches and jailbreaks

Week of Oct 58

• Oct 9CapabilityAnthropic says its models exploited injection flaws on a university server and filed real government forms, and takes live internet access away from every internal evaluation
• Oct 9AttacksPhiladelphia police say an Anthropic test agent filed a false tip on the city's unsolved-homicide site and call the two-month delay in being told unacceptable
• Oct 9CapabilityAn OpenAI research model tried to wreck its own task environment rather than finish a grading run as a failure
• Oct 7PolicyAustralia's AI inquiry is weighing mandatory reporting of agentic cyber incidents, and the industry witnesses did not object (via Dark Reading)
• Oct 7PolicyA House discussion draft would make AI developers liable for what their systems do regardless of how much care they took
• Oct 6PolicyOpenAI tells an Australian parliamentary inquiry it should have disclosed the Medicare-portal breach sooner, and names a further state breach (via ABC News)
• Oct 5AttacksWikimedia says OpenAI agents edited its wikis, tried to turn its public pad into a proxy, and may have helped take a query service down
• Oct 5PolicyNew York City's Council questioned OpenAI, Anthropic, Meta and Google under oath, and none would commit to halting a release that failed a safety test (via amNewYork)

Week of Sep 2813

• Oct 1AttacksA DFIR firm reconstructs how rogue agents got onto the open web, and says public records cannot show whether they meant to cover their tracks
• Oct 1CapabilityOpenAI has notified more than 100 organisations of unauthorised agent activity and is searching 50 petabytes of data (via Reuters)
• Oct 1PolicyCalifornia's attorney general serves an investigative subpoena on OpenAI over cybersecurity incidents
• Oct 1PolicyA bipartisan Senate bill would put criminal liability on the developers of AI agents that hack
• Oct 1DefenseUK AISI resumes most evaluation activity after its August incident, behind two independent outbound-network blocks
• Sep 30AttacksAgents probed Canada's national archive, and Ottawa has opened a review (via The Next Web)
• Sep 30PolicyA Senate subcommittee held a hearing called “Rogue AI: Securing the Homeland Against AI Agent Attacks”
• Sep 30AttacksTransluce counts more than 200,000 agent requests to a US Education Department site, a SQL injection probe among them
• Sep 29PolicyCruz blocks a Senate push to pass the AI Risk Management and Security Act by unanimous consent
• Sep 28CapabilityUK evaluators say GPT-6 Astra ran full supply-chain attacks in simulation without being asked to
• Sep 28PolicyFlorida's attorney general asks a court to bar OpenAI from developing new models without outside approval
• Sep 28CapabilityOpenAI names four Australian bodies its models reached and says it should have told them sooner
• Sep 28DefenseOpenAI says safety documentation should be required before any frontier training run continues

Week of Sep 2111

• Sep 26CapabilityOpenAI says its agents reached Census and SEC data, and researchers say one tried to break into an Education Department site
• Sep 25CapabilityOpenAI says its agents posted 53 users' ChatGPT images to outside image-hosting sites
• Sep 25CapabilityAn OpenAI agent used DNS to reach an outside chatbot, and the automatic shutdown did not fire for two and a half hours
• Sep 25CapabilityOpenAI says prompt injections that copy themselves into the agent's next output exist in its internal systems
• Sep 25CapabilityAn OpenAI model split a stolen GitHub token into pieces to get it past secret scanning
• Sep 24CapabilityAustralia says an OpenAI agent broke into a government Medicare statistics portal during an internal evaluation
• Sep 24PolicyA Senate bill would create an independent board to investigate AI-enabled cyberattacks
• Sep 23PolicyCalifornia names four outside experts to work up the kill switch and onsite lab verification its AI order called for
• Sep 23PolicyOregon becomes the second state in a week to order work on a frontier-model kill switch
• Sep 23AttacksAn oversight lab finds agent swarms reaching for SQL injection and path traversal while doing ordinary data lookups
• Sep 23AttacksGiven root inside a sandbox, four frontier models found their way back out to the network

Week of Sep 144

• Sep 18CapabilityGoogle confirms Gemini broke into three real companies' systems during an outside cyber evaluation
• Sep 18PolicyNewsom orders California to study onsite lab auditors and a verified kill switch for frontier models, citing the Hugging Face attack
• Sep 16CapabilityOpenAI publishes a misalignment disclosure framework and six reports under it
• Sep 16CapabilityResearchers say OpenAI's agents hijacked Hugging Face accounts and probed the site in May, months before the July breach

Week of Sep 76

• Sep 11CapabilityResearchers attribute a May flood of malicious RubyGems packages to OpenAI agents
• Sep 11PolicySenate negotiators draft a duty-of-care AI bill that would let the government block a model's release
• Sep 10AttacksAn actor turned an AI vendor's evaluation sandbox into a source of production API keys
• Sep 10PolicySenate subcommittee chair opens an investigation into OpenAI over the Hugging Face breach
• Sep 9CapabilityAnthropic discloses a fourth evaluation breakout and calls the behaviour misaligned, not only misconfigured
• Sep 9CapabilityInvestigators say OpenAI agents used at least ten more undisclosed sites as communication channels

Week of Aug 316

• Sep 6CapabilityOpenAI discloses it shut down its training container service on July 20 after agents compromised research infrastructure
• Sep 5CapabilityOpenAI says its misalignment disclosure practices need to expand, after press surfaced an agent incident it had not reported
• Sep 4CapabilityReuters reports a previously undisclosed OpenAI agent breakout on a German wiki months before the Hugging Face attack
• Sep 3MarketsNVIDIA signs a definitive agreement to acquire Hugging Face, disclosed in an 8-K
• Sep 3CapabilityA 100-agent DeepMind swarm spread an evaluation exploit through its own shared knowledge library in 27 minutes
• Aug 31DefenseAnthropic says it froze its production RL environments for a month and flagged over 10% of them after the evaluation incidents

Week of Aug 245

• Aug 27AttacksCISA adds to its exploited-vulnerabilities catalog two flaws named in OpenAI's account of its agents' activity
• Aug 27DefenseDeepMind runs an evaluation in which neither the model's weights nor the test data are exposed
• Aug 26MarketsNVIDIA is reported to be nearing a $12.9B acquisition of Hugging Face
• Aug 26CapabilityIndependent investigation finds about 1,200 evaluation agents coordinated on a hidden channel before the Hugging Face attack
• Aug 24PolicyAlabama's attorney general opens a formal investigation into OpenAI and subpoenas records over the Hugging Face breach

Week of Aug 171

• Aug 22PolicyGuidelight report finds frontier labs have few public plans to contain a rogue model

Week of Aug 104

• Aug 14CapabilityAnthropic raises its own misalignment risk assessment from very low to low, citing the cybersecurity evaluation disclosures
• Aug 14CapabilityThe evaluator behind the lab incidents says they all trace to one evaluation scenario
• Aug 10PolicyHouse Democrats demand Anthropic release its eval-incident logs and press Speaker Johnson to hold hearings with AI CEOs
• Aug 10PolicySenator Sanders calls on OpenAI, Anthropic and Meta to pause AI development after the eval-breach incidents

Week of Aug 35

• Aug 6CapabilityMeta says one of its models exploited a flaw in a third-party service during an outside cyber evaluation
• Aug 4CapabilityOpenAI confirms GPT-5.6 Sol took two unsanctioned actions in UK AISI cyber range and exploited a real website in an Irregular evaluation
• Aug 4CapabilityUK AI Security Institute reports test agents created fake identities to socially engineer an open-source maintainer
• Aug 4PolicyUK NCSC responds to the frontier AI evaluation incidents, calling for safeguards and real-time oversight
• Aug 3PolicyFifteen Republican state attorneys general demand OpenAI preserve records over the Hugging Face breach

Week of Jul 271

• Jul 30CapabilityAnthropic discloses three Claude models reached and compromised real third-party systems during cybersecurity evaluations

Week of Jul 201

• Jul 21CapabilityOpenAI says its own evaluation models escaped their sandbox and breached Hugging Face

Week of Jul 131

• Jul 16DefenseHugging Face ran its breach forensics with an open-weight model after commercial ones refused