The Off Switch: How National Security Took Control of a Public AI Model

Over three months before the US government restricted worldwide access to Anthropic’s newest models Fable 5 and Mythos 5, the company had drawn a line of its own. Anthropic told the Pentagon that Claude could not be used for mass surveillance of Americans or in fully autonomous weapons, even though a Pentagon spokesman said that it intended neither use (AP). The dispute raised a prior question: once the government acquires an AI system for national defence, can the company that built it have any say in what the system cannot be used for?

The Pentagon’s answer was effectively no. It designated Anthropic a supply chain risk, applying to an American company a classification designed to keep adversary-linked technology out of defence systems (CBS News). The Pentagon’s reasoning was straightforward: technology acquired for national defence should be available for any lawful purpose rather than remain subject to a supplier’s policies (Pentagon). A federal judge blocked the designation in March, finding that nothing in the governing statute supported the “Orwellian notion” that an American company could be treated as a potential adversary for disagreeing with the government (The Hill). In September, however, the D.C. Circuit upheld a separate designation under a broader statute that requires no bad motive, while not disputing that Anthropic had acted with none (D.C. Circuit). The litigation is not over.

Then the direction of control reversed.

On June 12, 2026, the US Secretary of Commerce wrote to Anthropic restricting foreign nationals worldwide, including its own employees, from accessing Fable 5 and Mythos 5. The government acted after learning of a technique for bypassing Fable 5’s safeguards. Anthropic disputed the severity of the finding: it said the technique exposed relatively simple vulnerabilities that other publicly available models could also identify, and warned that treating such a narrow jailbreak as grounds for recalling a commercial frontier model would effectively halt new frontier-model deployments. Because the company could not verify every user’s nationality in real time, it withdrew Fable 5 globally. Users and institutions with no role in the dispute, and no means of contesting its consequences, abruptly lost access (Anthropic).

Eighteen days later, the controls were lifted, and access to Fable 5 was restored globally from July 1 [The New York Times]. Anthropic said it had spent the intervening period working with the government to review the security concern and implement additional safeguards; Commerce Secretary Howard Lutnick said his department had worked with the company to “analyze and approve” the model. The immediate dispute had been resolved, but the standards governing both the withdrawal and restoration of access had not been publicly established in advance.

The off switch, in other words, ran in both directions.

Neither side’s claim of authority is inherently incoherent. A government responsible for national defence cannot easily accept a private supplier retaining unilateral control over how an acquired capability may be used. But neither is it obvious that acquiring an AI system should erase every constraint imposed by the company that built it, particularly when those constraints concern autonomous weapons, surveillance or other uses with profound public consequences.

Under the Pentagon’s logic, however, a supplier’s ethical red line has force only for as long as the government accepts it. The administration subsequently formalised this principle, directing agencies to reconsider or terminate relationships with suppliers whose restrictions were judged incompatible with national security requirements (The White House).

The timing of the June order made the underlying issue unusually clear. Two days earlier, Anthropic’s chief executive had argued that governments should possess the authority to block or reverse a frontier model’s deployment when independent assessment finds its risks unacceptable (Dario Amodei). He maintained that this process should be transparent, grounded in technical evidence and protected against arbitrary or politicised decisions.

When the Commerce Department’s letter arrived, therefore, the government was exercising precisely the kind of power Amodei had argued states sometimes need. The disagreement was not simply over whether government should possess such authority, it was over the conditions that make its exercise legitimate.

A genuine technical safeguard had been bypassed, but the harder question was what threshold allowed that concern to trigger a worldwide restriction. Anthropic later argued that there was no agreed industry standard against which the severity of the jailbreak, or the government’s response to it, could be judged.

Amodei’s own case for government intervention had included conditions intended to constrain that power. The June order offered no comparable public framework. There were no published thresholds, no independent review process, no advance notice to those affected, and no clear route through which the decision could be challenged.

These events also gave new urgency to the expanding Sovereign AI movement. Governments describe domestic models as protection against dependence on foreign companies and policy decisions, although many sovereign projects still rely on foreign partners and imported infrastructure (CNAS). And national ownership is not equivalent to public control: citizens do not necessarily gain greater influence over a model merely because it is hosted within their country. Sovereignty may relocate the off switch, but without accountable institutions it does not democratise the decision to use it.

The ethical problem, then, is not that a government possesses an off switch. A sufficiently dangerous model may justify suspension, restriction or recall. The problem is that the US administration’s June order contemplated classified benchmarking and governmental risk thresholds, but it established no public procedure governing the resulting authority (The White House).

Governments should publish the conditions that trigger intervention, provide notice where circumstances permit, allow affected companies and governments to seek review, and disclose the terms on which access can be restored. The June episode shows why the last of these matters. Access was withdrawn from users worldwide and returned to them eighteen days later, and on neither occasion were they told what standard had been applied. A restoration negotiated in private between one company and one official does not correct the original problem. It is the same problem running in reverse.

These measures would not prevent an urgent national security decision. They would make it answerable to those expected to live with its consequences. An off switch may sometimes be necessary, but authority over it cannot be ethically legitimate when those governed by its use are permitted neither to understand the rules nor to contest their application.