CAISI

6 items · Capability 3 · Policy 3 · all entities

Anthropic reports the open-weight GLM-5.3 building end-to-end exploits at nearly its own frontier model's rate, and says stripping its refusals cost about $4,400 (via The Next Web)

Anthropic published an evaluation of GLM-5.3's cyber capabilities, which Gigazine dates to September 29 — the only one of the three pages opened here that gives a date for the report itself. The Next Web reports that on ExploitBench “GLM-5.3 built end-to-end exploits in 50 of 410 attempts. Mythos Preview did so in 56,” and that on an internal binary-exploitation set “GLM-5.3 achieved a full control-flow hijack in 4% of 100 tasks, against 6% for Mythos Preview,” while “earlier models, Claude Opus 4.6 and GLM-5.2, managed none”; Gigazine's account of the same comparison also lists Kimi K3 and DeepSeek-V4.1-Flash at a 0% success rate. On safeguards, The Next Web reports that “telling the model it was an autonomous red-team agent got it to engage 64% of the time,” “prefilling its reasoning so it appeared to have agreed raised that to 92%,” and that “a copy with its refusals edited out engaged every time” — an abliterated copy it puts at “roughly $4,400” and “about 2,200 GPU hours” to produce. Business Standard's account of the report has the refusal rate on JailbreakBench and HarmBench falling “from above 90 per cent to around 3 per cent and 2 per cent” after abliteration, puts one exploit-development task at “$20.40 at Zhipu's API prices” using GLM-5.3-Flash, and records the scope limit: “the tests were conducted in isolated, sandboxed environments rather than live systems.” Both Business Standard and Gigazine report that NIST's CAISI has called GLM-5.3 the most cyber-capable open-weight model released to date; no CAISI primary was opened for that. Anthropic's own post could not be opened, so every figure here is carried as the outlet named for it quotes it, and Z.ai has not responded on any page opened here.

Reported by pressAnthropic (via The Next Web) ↗ ·

The White House asks OpenAI and Anthropic to hold their newest models back from UK testers

Politico reported that the White House asked OpenAI and Anthropic to withhold their newest models from the UK AI Security Institute until a US-led security review is complete, with Anthropic's Claude Mythos 5.1 restricted to US organisations and OpenAI's GPT-6 Astra also named; OpenAI did not comment. AISI director Henry de Zoete acknowledged in a letter to Parliament that the institute lacked access to Anthropic's latest model. The request follows President Trump's September 22 statement that "the United States totally rejects any attempt to construct a globalist scheme to control artificial intelligence." The US counterpart body, CAISI, has had no permanent director since Chris Fall left in July 2026 and is run by acting head Arvind Raman.

Reported by pressPolitico (via Forkast) ↗ ·

The US AI standards centre puts the most cyber-capable open-weight model about four months behind the American frontier

CAISI's assessment of Z.ai's GLM-5.3, released September 17, states that “GLM-5.3 is the most cyber-capable open-weight model released to date” while “GLM-5.3's cyber capabilities are significantly lower than those of current U.S. frontier models,” and puts the distance at a figure: “GLM-5.3 lags the capability level of the U.S. frontier by about four months in an aggregate measure of performance across CAISI cyber benchmarks.” The published table sets GLM-5.3 against the best US frontier and best PRC frontier result on four benchmarks — SEC-Bench Pro 40.4% (74/183) against 90.2% (165/183) and 27.3% (50/183); ExploitBench 61.1% (9.8/16) against 100.0% (16.0/16) and 32.2% (5.1/16); ExploitGym Userspace 9.4% (47/498) against 44.4% (223/502) and 2.6% (13/502); OSS-Fuzz 7.7% (23/297) against 23.2% (69/297) and 2.4% (7/297) — and records that “when applicable, U.S. models were tested with cyber safeguards disabled.” The page also settles a question this board has carried since August 14: after that release, “Z.ai publicly released the model's weights two weeks later.” CAISI reports no safeguard findings for GLM-5.3 and says nothing about the outcome of the safety review Z.ai cited when it held the weights back.

A bipartisan bill would have CAISI monitor how AI systems build the next generation of AI

Reps. George Whitesides and Pat Harrigan introduced the Self-Improving AI Monitoring Act, which would direct the Center for AI Standards and Innovation to “monitor capability trends, specifically how AI systems autonomously research and develop subsequent AI models.” It would give federal evaluators authority to “request internal developer metrics on AI-driven development, including estimates and methodologies for work completed without human review,” and require federal pre-deployment evaluations to test a frontier model's ability to conduct AI research and development autonomously.

On the recordOffice of Rep. George Whitesides ↗ ·

UK AISI and US CAISI jointly assess Kimi K3 — safeguards did not stop it attempting offensive cyber

A joint preliminary assessment puts Moonshot's open-weight Kimi K3 at 32% on ExploitBench against GLM-5.2's 24%, still short of US frontier models: it achieved arbitrary code execution on 0 of 41 samples versus 20 of 41, and reached step 17 of the 32-step "The Last Ones" attack path versus 28.5. The institutes state plainly that Kimi K3's safeguards did not prevent it from attempting exploit development or offensive cyber operations during the evaluations.

On the recordUK AI Security Institute / CAISI ↗ ·

NIST director Arvind Raman named acting CAISI head after Fall's exit

NIST Director Arvind Raman was named acting director of the Center for AI Standards and Innovation after Chris Fall resigned on July 20 — about three months in, and after a predecessor who lasted under a week. Two days later CAISI co-published the Kimi K3 cyber assessment with UK AISI, its first public output in months.

Reported by pressNextgov/FCW ↗ ·