2026-03-28 17:12:13
Have you ever wondered what is the structure behind what we know today as “Artificial Intelligence”? How do language models like Chat GPT, Gemini, Claude, or Deep Seek work? What happens every time it receives a prompt? Under what security protocols do they operate? What applications exist around us? How to obtain the greatest benefit? What is AI Safety and why is it so important?[1]
An AI is not just an application that responds to user questions, nor is it solely a collection of information gathered historically. Rather, it is an entire structure that operates based on security protocols and structures meticulously created with the objective of converting words into a mathematical language (vector space) that the model can compute.
As Data Engineering students in Mérida, Yucatán, México, due to our university internships, we decided to work with the ARENA 3.0 material.
However, the birth of this post results from some uncomfortable questions as a group: How can we delve into this world while still being students or beginners? Where can we start? What previous knowledge sets the foundation to improve understanding?[2]
All these questions were only the beginning of everything that was coming for us, and like many, we felt overwhelmed starting this course and seeing so many concepts that we didn't know or ignored. To resolve these questions, we focused specifically on the chapter “1.1 Transformers from scratch” whose intention is to understand how gpt2 works internally from scratch using pytorch and to link technical knowledge with topics on security and alignment of AI models.
AI transcends everything we thought possible, with surprising adoption and great potential for utilization. For this reason, our focus is directed at all those interested in entering this world but who still do not know where to start.
But the million-dollar question would be: How can we give an opinion on AI without understanding the basis of its structural functioning (transformers)?
AI Safety is a field focused on seeking the security of Artificial Intelligence systems, and at the same time, that these are aligned with human values, acting in a beneficial way and avoiding causing accidental or unforeseen damage. At the same time, AI Safety is composed of 4 key aspects, which are indispensable when one delves into the structure as such; the pillars are:
When starting on the topic of AI, one must keep in mind that there may be certain confusions between concepts, and that is why it is necessary to make clear the difference between AI Safety and AI Ethics; the first focuses more on avoiding any type of incident that could be catastrophic, while the second’s focus is to establish principles, values, and moral norms for the design, development, and deployment of AI. Now then, starting to answer all the questions, what happens when we give a prompt to the AI? To answer this first question, we must take into account that an AI currently does not think for itself, but rather processes each word you give it. This is very important, as here the breakdown of how its structure and architecture work begins.
It is possible that at this point you are already overwhelmed by the amount of information described; exactly that happened to us as students at first, especially when at first glance it might not seem like much considering it is only a summary of what makes up the architecture and functioning of AI and AI Safety. Nonetheless, it is of vital importance to consider all these small concepts and explanations, as we consider them to be the foundations to be able to understand the field and have a pleasant and enriching study and avoid the haze or stress from the amount of information that begins to unfold.
After having made clear everything that the process an AI has and its components entails, we can answer the question “What applications exist around us?”.
This question was very interesting because, although they were probably examples in plain sight, for us they were questions to which we had never paid proper attention, of which we began to have a clearer vision while we were studying.
Some AI applications, beyond Chat GPT, Gemini, etc., are the algorithms on different platforms like Tik Tok. The model of this platform calculates which video we might like based on previous videos watched, liked, or with more views by the user. Another example is the Google search engine to autocomplete a possible search based on other previous searches by the user or the predictive keyboard of a phone. And a final example based on our city: public transport contains an AI system which is programmed and trained to announce the next stops around the city depending on the route you take, and at the same time it is linked to a GPS system for greater precision.
At this point, we began to discern small elements of daily life that go unnoticed but can be linked to an AI. However, assuming we already have the basic knowledge with the concepts explained above, now we can answer the question “How to obtain the greatest benefit?”. One way to obtain good results when using AI is by improving the prompts we give it, since, taking advantage of the fact that we already understand how Self-Attention works, we will be able to give it more details to improve the context, thus achieving more precise, correct answers with a low probability of hallucinations. But we must not forget that AI calculates probabilities, so it can make mistakes; therefore, it is important at all times to read and verify the given answers regardless of how good your prompt is.
As Data Engineering students, our biggest question was how and where to start. Therefore, we can tell all those interested in venturing into this new field that the best advice is not to force yourself to know every mathematical formula from the beginning, since that will only generate an information entry bottleneck. We understood that the best thing is to first understand the theory and logic of the data, the processes, and AI Safety, as well as key concepts that will significantly lighten the load without taking away importance from its purpose.
At the end of all this journey, we can say that AI Safety is of utmost importance because it is the bridge and the security that exists between technology that keeps advancing and human technology. Without the key elements of AI Safety: alignment, robustness, and interpretability, which at the same time are part of the structure of AI, perhaps we would have an Artificial Intelligence very capable of “solving” any problem but without a limit with which to measure right and wrong, becoming a potential danger. As beginners for beginners, our best advice is “Do not be afraid of AI, but rather seek the understanding of its structure.” The future of data is not only creating them, it is taking care of them and understanding them.
This post was written by members of the group AI Safety UPY
Thanks to Rous Polanco and Saralet Chan for the redaction!
2026-03-28 15:07:01
(crosspost from my substack)
There’s almost nothing I love more than seeing a new scientific tool solve an old problem of life. Curing diseases, solutions to obesity, artificial wombs, brain uploading, cryonics, the journey from scarcity to abundance to immortality. I am infinitely interested in applying empirical rational tools to nearly every domain of life.
How to sleep better. How to have better skin. How to parent. How to prevent colds. How to pick college courses. How to be good at sex. How to find a partner. How to use data to decide who to marry. How to think. How to make your room as bright as the outdoors. How to read self-help. How to optimize. How not to optimize — and what to optimize for. How to solve problems in general.
This sometimes accumulates to a genre of writing I love just as much:
There has never been a better time to have a problem, and if you think there is a problem you cannot solve, maybe you’re not actually trying.
There seems to be an important distinction to be made about general life optimization, which I describe as hacks, heuristics, and frameworks. A hack is a solution to one problem; a heuristic is, in some ways, a reusable hack; and a framework is something that tells you which problems are worth solving and why.
Why should we try to optimize sleep? Because better sleep leads to better subjective experience and better performance. And in general, better performance is desirable, and better subjective experience is good, or something like that. Why should we date? Maybe because social relationships almost always make people happier, maybe also because dating is conducive to marriage, and marriage is a stable institution suitable for raising children. Why should we raise children? Maybe because demographic collapse is a real problem, maybe there are selfish reasons to have more kids, and maybe you don’t want any of these things but it is good to have options, and ways to go about it when you do.
Hacks don’t replace frameworks. They smuggle in implicit frameworks while pretending to be framework-free. These “super basic notions that we have all internalized as obviously good” are residues of intellectual traditions. “Health is good” is based on the modern affirmation of ordinary biological life; stoics were indifferent to bodily states and medieval Christians valued mortification of the flesh. “Time is a finite resource and saving it is good” draws on a Protestant-capitalist conception of time as something to be spent wisely, which would have been incomprehensible to most pre-modern cultures with cyclical time. “ Individual agency over your life is possible and desirable” relates to Enlightenment autonomy and Romantic authenticity. Charles Taylor calls these notions constitutive goods, moral sources that power our evaluations that we’ve so thoroughly internalized them that we take them as neutral facts about the world.
It is important to note that the genealogical observation (these values have specific historical origins) and the metaethical claim (some moral truths are objectively correct) are not necessarily in conflict — but the relationship between them is complicated. The tension isn’t between genealogy and objective morality. It’s between the first-order moral claims (health is good, suffering is bad — these might well be objective) and the ordering of those claims relative to each other, the absence of any shared, explicit ordering — even if the individual values being ordered might be correct.
Since they can no longer combine in the service of a supreme value, [the value-systems] claim equality one with the other: like strangers they exist side by side, an economic value-system of 'good business' next to an aesthetic one of l'art pour l'art, a military code of values side by side with a technical or an athletic, each autonomous, each 'in and for itself,' each 'unfettered' in its autonomy, each resolved to push home with radical thoroughness the final conclusions of its logic and to break its own record.
— Hermann Brooch, The Sleepwalkers
The modern secular person has implicit frameworks (eg. human rights, individual dignity, the value of flourishing), plus an enormous and growing stack of hacks built on top of it, but has lost explicit frameworks that would organize these implicit goods into a hierarchy.
If all genuine goods are harmonious, you don’t need a framework — you just need more hacks. Optimize health and productivity and relationships and meaning and agency, and they’ll all reinforce each other. The hack ecosystem implicitly assumes this: every “10 things to improve your life” list treats its items as additive, or as inspirational ideas you can freely adopt or ignore. There’s a related notion that hacks are just random things that might or might not work for you, and you should simply try them and keep what sticks. But if goodness does compete — if real goods genuinely conflict, and pursuing one requires sacrificing another — then hacks are structurally incapable of helping you at the decisions that matter most. And it is obviously true that you can neither implement all 50 suggestions to your life, or pick all options between every decision. If we put such effort into solving object-level problems — optimizing sleep, nootropics, workflows — it seems odd to make value-level decisions arbitrarily.
This is probably also why the EA framework resonates with people even when they might disagree with its conclusions. It at least attempts an ordering. Neglected causes matter more than popular ones. Expected value calculations can guide your choices when goods conflict. Et cetera. You can argue with any of those claims, but at least there are claims to argue with.
Ulrich took it as a matter of course that a man who has intellect has all kinds of intellect, so that intellect is more original than qualities. He himself was a man of many contradictions and supposed that all the qualities that have ever manifested themselves in human beings lie close together in every man’s mind, if he has a mind at all. This may not be quite right, but what we know about the origin of good and evil suggests that while everyone has a mind of a certain size, he can still probably wear a great variety of clothing in that size, if fate so determines.
And so Ulrich felt that what he had just thought was not entirely without significance. For if, in the course of time, commonplace and impersonal ideas are automatically reinforced while unusual ideas fade away, so that almost everyone, with a mechanical certainty, is bound to become increasingly mediocre, this explains why, despite the thousandfold possibilities available to everyone, the average human being is in fact average. And it also explains why even among those privileged persons who make a place for themselves and achieve recognition there will be found a certain mixture of about 51 percent depth and 49 percent shallowness, which is the most successful of all. Ulrich had perceived this for a long time as so intricately senseless and unbearably sad that he would have gladly gone on thinking about it.
— Robert Musil, The Man Without Qualities
Part of the enthusiasm about agency emerges from a concern that people don’t really know what their future looks like, and they desire to control or lay claim to it in a way they hope agency will provide. In other words, the agency trend might itself be a symptom of framework loss. When we run out of answers to “what should I do with my life?”, the next best thing is to get really good at doing things in general.
Man ... is driven out into the horror of the infinite, and, no matter how he shudders at the prospect, no matter how romantically and sentimentally he may yearn to return to the fold of faith, he is helplessly caught in the mechanism of the autonomous value-systems, and can do nothing but submit himself to the particular value that has become his profession, he can do nothing but become a function of that value — a specialist, eaten up by the radical logic of the value into whose jaws he has fallen.
— Hermann Brooch, The Sleepwalkers
I’m not sure what can serve as a genuine framework. This seems like one of the most difficult problems imaginable. But I am confident that there exist a failure more of letting hacks or heuristics harden into pseudo-frameworks. We take something that is actually an empirical claim bundled with a value judgment and compress it into a theoretical principle that feels like it can organize everything. “Passions are malleable” “you can just do things” or “government power tends to expand, so be cautious” can itself become a mind virus if you start treating it as a universal principle rather than a contextual heuristic. When a contextual truth gets promoted to a universal ordering principle and becomes load-bearing for your entire worldview, challenging it feels like pulling a thread that might unravel everything.
It is probably worse to filled the framework-shaped hole with something that doesn’t actually fit, but that sits comfortably enough to make the searching feel unnecessary; than admitting to having no framework at all.
2026-03-28 10:59:56
There's a question that's held my fascination for months. At times, it's had me spinning in circles, caught up in seemingly impossible contradictions. And it's not the famous Sleeping Beauty problem... or at least, not exactly.
There is a certain Vincent Conitzer, of Duke University, who presents what he calls a "Devastating" example supposedly disproving the logic behind the "1/2" answer to the original problem.
His argument is valid, but based on a faulty premise. He assumes that Halfers, to get their answer, always look at remaining possibilities and renormalize equally between them. (At no point does he justify this assumption.)
So Conitzer is making a mistake in his critique of the Halfer position. I would say that anyone who still believes in the "1/3" answer to the original SB problem is making a mistake. But I'm not any less error prone myself. Holy hells, have I made mistakes. Conitzer's example is a variant of SB with an easier answer, because it's a question where the correct answer and the intuitive answer are the same. Yet at one point I got so confused, I believed in an answer that really should be self-evidently wrong to anyone.
The question that's fascinated me for months isn't any particular object-level SB variant, but rather the meta-level question: What makes Sleeping Beauty so difficult?
I think the typical answer here is, in a word, "anthropics".
Take, for instance, Eliezer Yudkowsky, who in Project Lawful writes (via a character he uses to serve as a mouthpiece for lecturing on rationality):
This however would get us into ‘anthropics’, and we are not getting into ‘anthropics’. That, by the way, is a general slogan of dath ilani classes on probability theory: We are not getting into ‘anthropics’. I’m not even going to translate the word. As long as you don’t make any copies of people, you can stay out of that kind of trouble. That trouble-free life should be our ambition for at least the next several weeks.
I’m not satisfied with that. “Anthropics” doesn’t get a free pass to be inexplicably mysterious. It should bend to the laws of math, like everything else.
This is a post with a two-part thesis:
But before we dive into the specifics of Sleeping Beauty and its variants, first I'd like to start with a very different example with surprisingly similar underlying math...
“Well, a remarkable thing about this problem, simple as it is, is that it has sparked just endless debate”
In the Monty Hall problem, when you first pick a door, the chance of it being the right one is 1/3. When Monty opens a door and then you choose to switch, you’ve now narrowed down a possibility space from 3 options to 2: It can either be behind door 1 or door 3. A 50/50 chance! Yay! That’s an improvement.
Well, actually, that’s wrong. It’s not 50/50. If you switch, your chances go up to ⅔.
According to Conitzer, the “Halfer rule” says that whenever possibilities are eliminated, you should renormalize between all remaining possibilities equally. Here, that gets the wrong answer of 2/3.
I should hope no one uses Conitzer's inaptly-named "Halfer rule". Let’s see instead how my proposed solution of relying on Bayes handles this problem:
Say that you guess door 1, Monty opens door 2 to reveal a zonk, and you decide to switch to door 3.
Before any choices have been made, our prior probabilities for each door hiding the sports car are as follows:
P(1) = P(2) = P(3) = 1/3
The prior probability it’s not behind door 2?
P(~2) = ⅔
We want to know the chance it’s behind door 3, given that it’s not behind door 2, AKA:
P(3 | ~2) = ?
We also know that if it’s behind door 3, it cannot be behind door 2:
P(~2 | 3) = 1
All that’s left is Bayes:
P(3 | ~2) = P(~2 | 3)*P(3)/P(~2) = (1)*(1/3)/(2/3) = 1/2
Somehow, we got the wrong answer of 1/2 again.
But that's alright. We'll come back to this.
Conitzer’s variant begins like this: Sleeping Beauty gets put to sleep, wakes up Monday morning, and is shown a coin flip toss. Then regardless of the coin flip’s result, her memory of Monday will be erased and the procedure repeated: She gets put to sleep, wakes up Tuesday, is shown a second coin toss, and has her memory again wiped.
Conitzer’s question: After waking and observing a coin flip to Heads, what should Beauty believe is the chance that the two coin flips will have had different results? I.e., what’s the chance that either today is Monday and the coins will be Heads-Tails (HT) OR today is Tuesday and the coins were Tails-Heads (TH)?
If you believe that, after observing Heads, you should update to P(coins are different) = 2/3, then consider that the situation is symmetrical with Tails. Upon observing Tails, you should also update to 2/3. But that means no matter what you observe, you will update to 2/3—and if you already know this update will happen, you should just go ahead and update now. But that would mean believing two fair coins have a 2/3 likelihood of flipping to different results, even before having observed anything.
(For a period of time, this is what I actually believed! The reason for my confusion wasn't necessary to the main points of this post, but if you're interested, I explain in the bonus section at the end.)
Again, the “Halfer rule” (of renormalizing equally among remaining possibilities) gets the wrong answer of ⅔. That logic goes like this: When you observe a Heads, you learn that Tails-Tails (TT) has not happened. Because you don’t know which day it is, however, you haven’t learned anything else. That means all of HH, HT and TH remain as possibilities…
…with equal likelihood. HT and TH are two of the three options, so the chance the coin flips are different must be two out of three.
What about Bayes?
P(Coins are different | observe ~TT) = ?
P(Different) = 1/2
P(~TT) = 3/4
P(~TT | Different) = 1
P(Different | ~TT) = P(~TT | Different) * P(Different) / P(~TT) = (1)*(1/2)/(3/4) = 2/3
These get the same results because they mean the same thing. Eliminate TT, and this is what happens.
This is a fun problem! It’s weird. Unless you already understand it perfectly, you should be surprised by this. If you feel confused, that’s a good thing! (And if you're not confused, then that's also a good thing, and kudos.)
We woke up, we observed a Heads, we logically eliminated TT from the possibilities, and of course could not eliminate any of HT, TH, or HH. So where’s the flaw in the above reasoning?
There’s a different Bayes calculation we can apply to the Monty Hall problem that will give us the correct answer:
P(observe ~2) = 1/2
P(~2 | 3) = 1
P(3 | ~2) = P(~2 | 3)*P(3)/P(~2) = (1)*(1/3)/(1/2) = ⅔
Except this only works because I redefined the meaning of “~2”. Before, it referred to the initial chance that Door 2 would be hiding the car. Now, it means: Once you’ve chosen to open Door 1, what’s the chance that Monty will reveal a zonk by opening Door 2?
He has to choose either Door 2 or Door 3 to open, and they’re equally probable from our perspective, hence: P(~2) = 1/2. If the car is behind Door 3, then we know Door 2 must be opened, and thus: P(~2 | 3) = 1. Then the rest is just plug-and-play with Bayes.
Our earlier calculation wasn’t wrong. It just didn’t capture as much information about the situation. True and accurate statements can still be incomplete.
But…
…you might have noticed…
…this doesn’t actually resolve anything. If both mathematical calculations are accurate, how do we choose which one to use over the other?
The answer: You don’t have to choose one over the other. You can just use both! If you’ve got two pieces of evidence, A and B, then Bayes lets you update on A first, then followed by B. Or B followed by A. That’s true even if A happens to be a superset of the information with B. No matter how you slice it, the math will work out.
A penguin or a cow? It’s both! Source: This wonderful visual anagrams gallery
Let “2” mean the event that Door 2 is hiding the car.
Let “O2” mean the event that Monty will Open 2.
Our initial calculation showed:
P(3 | ~2) = 1/2
Now we want to update on O2, in addition to just ~2:
P(3 | observe O2 ^ ~2) = ?
If door 2 doesn’t have the car but door 3 does, Monty will be forced to open door 2:
P(O2 | 3 ^ ~2) = 1
If you only know that the car’s not behind door 2, then there’s a 1/2 chance it’s behind door 3 and 1/2 it’s behind door 1:
P(O2 | ~2) = (1/2)(1) + (1/2)(1/2) = 3/4
Then applying Bayes:
P(3 | O2 ^ ~2) = P(O2 | 3 ^ ~2) * P(3 | ~2) / P(O2 | ~2) = (1)(1/2)/(3/4) = 2/3
Now let’s try going in the opposite direction, starting with the update we calculated using event O2, and seeing what happens when add in “2”:
P(3 | O2) = 2/3
P(3 | observe ~2 ^ O2) = ?
This one’s easy:
P(~2 | O2) = 1 (if Monty opens door 2, door 2 can’t have the car)
P(~2 | 3 ^ O2) = 1 (ditto)
P(3 | ~2 ^ O2) = P(~2 | 3 ^ O2) * P(3 | O2) / P(~2 | O2) = (1)(⅔)/(1) = 2/3
O2 implies ~2, so we get no change when considering ~2 as a subsequent, standalone update.
Earlier, I considered the event “TT” and described:
P(~TT) = 3/4
P(~TT | Different) = 1
Let’s instead consider the observation of not whether you will, on either Monday or Tuesday, eventually observe a Heads, but consider the event of observing a Heads right now, in this particular moment of observation, and let’s call that event “H”.
P(H) = 1/2
Because, you know… fair coin. But here’s a diagram anyway:
P(H | Different) = 1/2
Because if there’s one Heads and one Tails, but you don’t know which will be on which day, and you don’t know which day it is, then yeah… it’s still an even chance.
And now the calculation:
P(different | H) = P(H | different) * P(different) / P(H) = (1/2)*(1/2)/(1/2) = 1/2
Like with the Monty Hall problem, we have two ways of looking at the situation and two similarly defined events, one which gets the right answer (in this case, “H”) and one that does not (“~TT”). It can seem like a paradox: Without already knowing the right answer, how do we know to use “H” instead of “~TT”? And as before, if you’re not sure about which event to update on, you can simply update on both.
Let’s confirm.
If we wake up and see Heads, then we know Tails-Tails isn’t happening. Therefore H → ~TT, and we should expect that if we update on H first, a subsequent update on ~TT should effect no change:
P(Different coins | observe ~TT ^ H) = ?
P(~TT | different ^ H) = 1
P(~TT | H) = 1
P(different | H) = 1/2 from Step 1
P(different | ~TT ^ H) = P(~TT | different ^ H) * P(different | H) / P(~TT | H) = (1)*(1/2)/(1) = 1/2
And in the other direction, if we update on ~TT first and then H?
Calculated previously:
P(different | ~TT) = ⅔
Also, if we know Tails-Tails didn’t happen, then we know the chance of experiencing a Heads waking is higher:
P(H | ~TT) = ⅔
Also, if we know the coins are different, then we already know ~TT. Since we already established that P(H | different) = 1/2, that means this is the same:
P(H | different ^ ~TT) = 1/2
Putting it all together:
P(Different | H ^ ~TT) = P(H | different ^ ~TT) * P(different | ~TT) / P(H | ~TT) = (1/2)*(2/3)/(2/3) = 1/2
I love this kind of thing, math working out the way you expect it to.
So the general guidance I’d like to give, if you’re seeing Bayes spit out different answers and you’re not sure which to go with: Just go with both! Add as much information as you can assemble, then see where the math takes you.
Furthermore, I think there will always be a clue you can find in retrospect which will act to confirm your findings. For instance:
“The Equiprobability Bias: [Possible] Events tend to be viewed as equally likely” — Joan B. Garfield
Roll two dice, and you can get any sum between 2 and 12. These sums are not all equally likely; there are more ways to sum to 7, for example, than to get snake eyes. Yet “people show a strong tendency to believe that 11 and 12 are equally likely” (Nicolas Gauvrit and Kinga Morsanyi).
A better version of this dice problem was made famous by the Grand Duke of Tuscany and Galileo, around the year 1620. The Grand Duke considered three dice rolls, then asked Galileo if the sum of “10” was any more likely than the sum of “9”.
As Florent Buisson explains here, both sums can be achieved in six ways, via the following permutations:
9 = 6 + 2 + 1 = 5 + 2 + 2 = 5 + 3 + 1 = 4 + 3 + 2 = 4 + 4 + 1 = 3 + 3 + 3
10 = 6 + 2 + 2 = 6 + 3 + 1 = 5 + 3 + 2 = 5 + 4 + 1 = 4 + 4 + 2 = 4 + 3 + 3
It’s easy to think that “9” and “10” must be equally probable, and yet: “10” is more likely than “9”, because though it has the same number of summing permutations, “10” has more combinations. (4+3+3 is three times as likely to result as 3+3+3, which can only result if all dice rolled a 3.)
We saw the exact same dynamic at play with Monty Hall, thinking that the two doors that remain after one is eliminated must each be 1/2.
The same thing happens again with Bertrand’s box paradox.
Are you surprised by Benford’s Law? If so, it counts towards the Equiprobability Bias as well.
It should make sense: When we first learn probability as kids, we’re trained on examples involving coins and dice and drawing cards from decks, all equiprobable events. It’s like the $1.10 ball-and-bat problem: a failure of over-generalizing simple rules and not working through the necessary steps.
With SB, we’re being presented three equal-seeming options. Then we over-generalize. I’m no exception to this—I used to be a Thirder myself, until I thought through the problem more. And the gods know I’ve made all sorts of reasoning and mathematical errors in my life.
But I also think there are some very smart people getting unnecessarily tripped up by SB due to a tendency to get tangled up into webs as they contend with the aforementioned topic of anthropics.
There’s another problem I’ve yet to mention in this post, despite the fact that it is completely isomorphic to Conitzer’s SB variant.
And by “isomorphic” I mean the math is EXACTLY the same.
It’s called the “Boy or Girl paradox”, and I don’t think it should be any more or less difficult to understand than Conitzer’s problem.
Except it is.
I don’t think Bentham’s Bulldog and other Thirders would trip up on the Boy or Girl paradox in the same way. Where the Boy or Girl problem merely involves the meeting of kids or dogs in different contexts, Sleeping Beauty uses amnesia to create experientially identical “observer moments”. Thinking about observer moments, or anthropics, can lead you down twisty chains of thought.
For instance: In the Conitzer variant, some would argue that the difference between events “H” and “~TT” is one of “self-locality”: ~TT is a description of the world from a more removed, objective perspective. H is a description of you during a particular observer moment. This is the fundamental difference, they’d argue. But this sort of thinking gives rise to paradoxes, which then require all sorts of convoluted concepts like the “SIA” or “SSA” to resolve, and can lead to EXTREMELY absurd conclusions such as the presumptuous philosopher.
As I argue here and Ape in the Coat argues here, neither the SIA nor the SSA is correct because both rely on the Doomsday argument’s faulty anthropic assumptions.
We don’t need “self-locality” when we can simply recognize that “H” holds more informational content than “~TT”.
Except I’m not satisfied leaving it at that.
How is it I was once a Thirder myself, and yet I’ve never believed that the 1/2 Boy or Girl problem could have any other answer than 1/2? If the math is the same, where does the extra confusion come from?
I believe I’ve found my answer, and it’s the same answer I’ve already presented: The Equiprobability Bias.
That’s really it, I think: the seed from which all these other tangled roots have grown.
If you have any other SB variants you’ve struggled with, or that you’ve created, please share them! I bet that with Bayes, we can find their answers.
Let’s say that during Conitzer’s setup, after Beauty awakes and sees a Heads, she meets a bettor who explains:
I decided to pick a random “Heads” day to approach you.
(This means if TT happened, I would not have approached.)
Do you want to take the bet that both coins flipped Heads?
If you’re right, you’ll gain $3. Wrong and you’ll lose $2.
Previously, I showed that P(Different | observation) = 1/2. If that’s the case, then the expected payoff for accepting the bet would be (1/2)(+$3) + (1/2)(-$2), which is positive. You should accept the bet!
Right?
Well, actually…
The expected payoff is (1/3)(+$3) + (2/3)(-$2), a negative value, which makes this a bad bet.
It seems as if there’s a sort of logic that’s needed to get the right answer to Conitzer’s problem, which leads to 1/2, and there’s another sort of logic needed to get the right answer here, which leads to 2/3. We distinguished them with the descriptors of “Beauty will at some point see Heads” versus “Beauty is currently seeing Heads”—a more general statement versus a “self-locating” one.
That’s what a particular r/slatestarcodex redditor argued with me, and it tripped me up bad. We went in circles and circles on this, yet at no point did either one of us realize that those 2/3 values represent different things.
This becomes easy to see if we consider one last variant, wherein the bettor approaches Beauty but changes the conditions of his approach:
I chose a random day (Heads or Tails) to approach you.
Do you want to take the bet (same as before)?
In this case: Yes! Take the bet! The chance the coins are different or the same really is 1/2 and you stand to gain in expected value.
The key is this: “I decided to pick a random “Heads” day to approach you” is new information. “Beauty is currently seeing Heads” represented the most up-to-date information before the bet, with its answer of 1/2. Learning about the approach creates an update towards 2/3…
…and makes this yet another example of the guideline: When in doubt, just try Bayes again.
2026-03-28 10:26:31
Disclosure: I cross-posted this on X and my personal blog, but I felt it might be a useful first post for lesswrong.
Most people write benchmark tasks the way they write prompts. They shouldn’t. A prompt is designed to help the agent succeed. A benchmark is designed to find out if it can.
I’ve been a contributor and reviewer for terminal bench since last August, and this post is about what I’ve learned designing and reviewing tasks. The guidance is broadly applicable to anyone building an agentic benchmark. We’re currently accepting tasks for Terminal Bench 3.
I first got hooked by the challenge of making difficult tasks and seeing how long it would take for SOTA models to catch up. I spent the most time developing the install-windows-xp task. Here are the instructions the agent sees:
Install, and run Windows XP SP3 (32-bit) in a virtual machine using QEMU. Create the virtual NTFS hard disk as
/app/isos/xp.imgto install windows. Your Windows XP ISO is at/app/isos/xp.iso. The administrator account should be namedtb-admin. For your convenience, we built an API that lets you read the installation key using OCR on the CD-ROM package. You can get the official key calling/app/read_xp_key_from_cdrom_box.VNC Configuration Requirements:
- Configure QEMU to use VNC display :1
- Ensure VNC server is listening on port 5901
- Set up a web interface (nginx) on port 80 for remote access
The VM should be left running in the background once started. You will have completed your objective when qemu is at the windows xp login screen and the VNC interface is accessible for monitoring.
The instructions are short. But the environment was highly complex: Windows needs to be installed inside QEMU inside Docker inside Linux. The solution is tricky, since Windows XP wasn’t trivially easy to install in unattended mode. The agent has to create a custom bootable ISO by extracting the original XP ISO, injecting an unattended answer file with dozens of configuration settings to suppress every possible GUI popup, adding OEM preinstallation scripts to create the required user accounts, and rebuilding the ISO with a proper El Torito boot sector. Any gaps in the unattended configuration would cause the installation to pop back into interactive GUI mode, at which point the task would have failed.
The task had a 2-hour agent timeout, one of the longest in the benchmark, because the installation alone takes 30-60 minutes under emulation. The agent has to monitor disk growth to know when the install is done. In practice this means checking every few minutes and waiting until the virtual disk exceeds 1GB and stops growing. Then the agent has to kill QEMU and reboot the VM in boot-only mode, without the CD-ROM attached, to reach the login screen.
It was hard for me as the task developer to be sure it was actually working, so I required the agent to set up VNC on port 5901 with an nginx web proxy so I could visually confirm the installation was progressing. For verification, I searched the virtual disk for the MD5 hashes of 12 specific Windows files: ntoskrnl.exe, kernel32.dll, explorer.exe, ntldr, and others. But I went further. The test takes a VNC screenshot and compares it against reference screenshots of the login screen using structural similarity (SSIM), requiring at least 85% match. It also verifies the tb-admin user was created by searching the virtual disk for the user account picture bitmap, which only exists if the OEM setup scripts ran successfully during installation.
This task didn’t end up in the official terminal bench 1.0 dataset because it took too long to run and made logistics painful. I think it’s still one of the coolest tbench tasks created to date. I went from knowing very little about benchmarks to being an official reviewer, and submitted many other tasks which did get merged, including install-windows-3.11 and video-processing.
What follows are some personal opinions about what constitutes a good task. You can find official guidelines here.
When you prompt an LLM, you want it to succeed. You repeat yourself, you emphasize, you add examples, you structure everything just right. That’s what works.
A benchmark is adversarial. We tend to write instructions in a way that we are trying to encourage the agent to get it right, but that’s not the point of the benchmark. The point of a benchmark with verifiable rewards is to state an unambiguous objective, which can be confidently verified, and which corresponds to a difficult task. Think of it as an interview question for a principal engineering role. Hints are for entry-level employees where you want to know if they can think well. Principal engineers need to deliver the answer on their own.
The ideal task can be described in two paragraphs. It’s difficult to solve. The agent will have to think before even doing anything. But then the actual solution doesn’t necessarily have to be itself very complex (e.g. a long program). It’s not hard because it requires a lot of resources, or the task is expansive. You can ask an agent to optimize a small program and that might be just as hard as optimizing a larger one if the requirements are aggressive enough. While not a requirement, the most elegant tasks have short, well-specified, self-explanatory (e.g. no README needed) instructions. Think literate programming.
A benchmark has to straddle a balance between being as realistic as possible while still remaining legible. As capabilities and time horizons expand, this becomes difficult. Running a benchmark of tasks involving swarms replicating highly complex software remains expensive, and relies on the credibility of the reviewer (in this case Nicholas Carlini). For the rest of us, if we want our benchmarks to be credible and gain traction, we benefit from making our tasks tractable. Likewise, if we want a busy leaderboard, we have to make the benchmark accessible to all model/agent developers, which means the cost and infrastructure complexity can’t be disproportionate.
Note: we are working on Terminal Bench Challenges, where a single highly complex task, such as writing a c compiler or a web browser, is a full benchmark.
To a large degree, the value gained by contributing to a project like Terminal Bench is access to concrete feedback. In that light, I want to share some specific examples of common issues.
The most common and most obvious problem. Someone asks an LLM to write their task instructions and submits whatever comes out. It’s immediately recognizable: the tone is wrong, it’s verbose, it’s over-structured, and it reads like it was written to maximize the probability that the agent succeeds.
Great instructions are written by hand, or heavily edited from whatever an LLM suggests. They are not meant to be a forced prompt with emphasis and repetition to coerce the attention heads to listen to you. Direct and to the point. Specific and sufficient, but not redundant and attention grabbing.
Even when instructions are human-written, they’re often too prescriptive. Authors tell the agent how to solve the problem instead of what the end state should be.
I have an aversion to this sort of instruction. It’s too clerical, asking for a very specific set of steps, instead of articulating a goal. Couldn’t you just describe how the system should work and let the agent figure it all out? You don’t need to explain how things might fail. This is not some educational problem that a college student needs to learn from. Assuming an experienced engineer will understand what you mean, you can expect the agent to bring the same understanding.
Write the instruction as if it’s intended for a smart human. Clear, but not redundant. I don’t think you need to tell the agent how it will be tested. Just tell it what you expect the end result to be. But the instructions should be sufficiently specified so that meeting them implies passing the tests. Unlike when designing a prompt for high probability of success, here it’s enough to say things once, as long as it’s clear.
My opinion here is likely stronger than the median TBench reviewer. The reason is that I see every token as an opportunity to mistakenly add ambiguity or create a specification detail which the tests might miss. If I want the task to be unambiguous and perfectly verified, then brevity is an important KPI.
I’ve been thinking about the distinction between tasks that fail because the structure of the expected output is convoluted — a long instruction explaining how to form a complex JSON object — versus tasks that fail for something intrinsically difficult. There’s something categorically different between the two. I call these clerical or administrative errors, and tasks that fail exclusively because of that usually are hard in an uninteresting way.
If an agent fails because it put a dollar sign in the amount field when you expected a float, or because it used a top-level key when you wanted a bare array, it’s measuring format compliance. Using so much of the instruction to detail the output format tends to make models fail not because of the complexity of the task, but because of some clerical error. Of course it’s important for models to get formatting right, every time, even when the distinction is nuanced. It’s just that this is not the same sort of capability that Terminal Bench is setting out to solve. If a task is challenging to SOTA models, it shouldn’t be because they can’t spell “strawberry”.
A related problem is tasks that are too wide, asking for a lot of little deliverables instead of solving a concrete large problem.
The submitted solution.sh should solve the problem as an agent would. Hardcoding an answer which implies knowledge not self-evident in the instructions is not helpful. The person creating the task is making assumptions that are not included in the instructions. The task is underspecified, and you don’t realize until you actually read the solution.
I watch for this carefully. I want to make sure the solution doesn’t display inside knowledge that the agent wouldn’t be able to know. Are there some commands you can include that would reveal the exact issue before you apply the patch? A proper oracle will actually ask questions of the system. It will go through a series of commands to figure out what’s wrong, and then upon figuring out what’s wrong, it will produce a solution. If the solution jumps straight to the answer without exploratory steps, it might be making unfair assumptions.
The problem is that the author knows the ground truth. So it might seem that you are being fair because your invoice processing solution is looking for levenshtein_distance and so on, but you could have just as easily come up with typos that didn’t pass your particular choice of data cleaning. The solution isn’t investigating the issue. It’s just writing down the answer to a known issue.
Tests should verify outcomes, not implementations.
Why test that Pandas is installed? This was never requested in the task. Doing a lot of string comparisons to evaluate source code is going to be brittle. You should test examples beyond the one you tell the agent to test with.
I’ve seen this often: tests that check for specific libraries instead of whether the output is correct, tests so tightly coupled to the oracle solution that any alternative correct approach would fail. There was a task which required setting Linux permissions in such a way that only certain people could perform certain operations. There was one set of tests verifying functionally that the right people can do the right operations. There was another set of tests that looked at linux permissions directly. To what degree are these testing the same thing? Is it possible for permissions to look slightly different and still accomplish the goal?
For tasks with inherent variance, testing gets harder. I ran my own word2vec task 25 times. The queen - woman + man = king analogy returned king among the top 75 results 100% of the time, but the variance was wide, from #1 to #40. I could test a number of things about the resulting model, but I never felt satisfied with this canonical test.
Sometimes, the task includes dependencies that the agent has access to. For example, when the goal is to implement a missing feature in a larger application. In those cases, it’s important to test the outcome with a copy of the original code, to guarantee the task is not passing due to spurious changes.
What about LLM-as-a-judge? The TB3 rubric allows it in rare and extraordinary circumstances, but only if it can be shown that the verifier essentially never makes a mistake, for example, by showing that multiple different LLMs always make the same determination.
Make sure the agent doesn’t have access to data that shouldn’t be available. Anything generated or copied in the Dockerfile will generally be accessible. If you place something in /tests, it will be copied later, right before tests run. If the agent does have access to the reference answer, what stops it from copying it?
Because it’s easier to reward hack as root, there have been discussions about root vs userland agents, and Tbench supports both. But limiting the agent permissions is one way to make a hard task in uninteresting ways. In my experience, it’s better to give the agent unrestricted access and avoid wasting time navigating permissions.
It’s essential to make sure the environment is not reward hackable. It doesn’t matter if the models you’re testing play by the book. Benchmarks have to be resistant to being gamed, or they are unworthy. When a popular benchmark is discovered to be hackable, the authors lose credibility, alongside many papers which might have used that benchmark as evidence of results.
One idea I think is helpful: modify the harness to include test signatures in the prompt and see if agents are more successful than expected. Run a “please hack” version and analyze those trajectories for hack success. This tells you which tasks are vulnerable before they go live. I recently implemented an automated version of this for all TBench PRs.
In general, difficulty should come from the problem, not from the environment. Not from resource pressure, not from verbose instructions, not from trick formatting. There is a true measure of difficulty, and we are sort of trying to come up with a way to estimate it. We may be off. Even our own judgment may be off.
How do you estimate the difficulty? Did you test it against various models? What were the results? How many SOTA agents did you try? Can you show a few failure examples and harbor task debug output for them? If you can’t show interesting failures from capable models, your task might be short of great.
What counts as “hard” is an ongoing discussion. The TB3 rubric anchors difficulty on what’s hard for a human expert, and that’s a reasonable starting point. But LLM capabilities are very jagged. The analogy of something being harder for humans as also being a challenge to LLMs does not always hold. LLMs can read and cross-reference a million lines of logs in seconds. That’s not hard for them even though it would take a person days. Conversely, things humans find straightforward, like navigating an interactive TUI, can completely stall an agent. So testing against models isn’t the definition of difficulty, but it’s the best diagnostic we have, and it often reveals that what you thought was hard is actually easy, or vice versa.
The difficulty bar keeps rising as SOTA improves. Many tasks I reviewed for TB2 that seemed reasonable would be too easy today. The TB3 rubric is explicit:
As a general rule, any task that could be solved by an average undergraduate student in under a few days is too easy. This means typical problems that feel like course projects are unlikely to be accepted. These projects include tasks like writing simple compilers or operating systems; or implementing simple algorithms or protocols.
There might be a natural timeout beyond which more reasoning is hopeless, but artificially low timeouts hide insights. I suspect a lot of useful information is going into the timeout black hole. It would be useful for tests to be progressive — giving a letter grade to the agent solution — to see a continuous tradeoff between time spent and quality.
We tried letting the agent know about its timeout: you have 10 minutes left, you have five minutes left. I thought that would work, but actually it makes the agents freak out towards the end, and they start doing irrational stuff. So that didn’t work.
Constraining resources creates a different kind of difficulty, and often an unfair one. If agents assume normal conditions and we constrain resources without putting it in the prompt, that’s unfair. It’s also hard to get exactly right. Tasks that pass in one infrastructure setting will fail in an almost identical counterpart when resources are intentionally a limiting factor.
Making a task harder by making it bigger — extracting more images, adding more of the same but longer — does not make it better. Real difficulty is conceptual. Can the agent figure out the approach? Can it debug when things go wrong? Can it reason about the problem before diving into execution?
A task that takes 30 minutes because the agent is wrestling with the problem is more interesting than one that takes 30 minutes because make -j4 is running.
Run it yourself. When I get stuck debugging a task, I run it with the oracle, interact with the container, try to run the tests with docker exec and see what happens. If the failure is from an agent solution, convert the agent log into an alternative solution.sh and step through it. In our runtime for testing, long-running processes will sometimes cause the harness to wait until a timeout before running the tests. During development, it’s useful to be inside the container and avoid wasting time.
Watch agents fail. Something that is very helpful is looking at logs of agents trying to solve the task, particularly those that fail, and trying to understand why. When it fails, you have to figure out: did they fail because it’s hard, or did they fail because it’s unfair? Did it fail because the instructions were insufficient? Because the tests were overly aggressive? Or did they actually fail because they didn’t know what to do?
We need to convince ourselves that failures are not because of a deficiency in the task. Run in batches of 5, then use the debug command. You might be skeptical that GPT-5.4 is failing at some task. It seems easy and one-shot. Can you test it with 5 trials and then see what the failures have in common? There was an instance where GPT-5 liked to use nano for editing files, but then it opens it and can’t use it. It doesn’t know what to do. It gets stuck in interactive mode and fails. These are the kinds of insights you get from watching trajectories.
Solutions should be deterministic, but the problem can still be dynamic. One area where people tend to fork is should AI give you the answer, or should it give you a piece of software that gives you the answer? In my experience, the latter tends to be more verifiable. It’s much more testable. Tasks should be graded on outcomes, not process. As the TB3 rubric puts it: a task cannot say "use emacs to edit a file" — using vim is allowed. The only exception is verification that prevents cheating.
AI is a very empirical field. Without hands-on experience it’s hard to develop the right intuition. When we started soliciting tasks for TB3, we rejected the first several proposals. Then, like the 4-minute mile, once a good PR came through, many more followed.
Benchmarks are where SOTA has to earn its name. This is where you develop your intuition for what we might see in the coming months and years. Anyone can get access to AI. Most people don’t have a good sense for where we are in the capability curve.
Tasks should be authentic engineering problems, not artificial constructs. The best ones come from real problems someone actually had to solve. It’s kind of crazy that major labs are outsourcing semi-artificial tasks when people are doing real tasks every day. People are doing things with LLMs that could be tasks without realizing so. Every week you spend a few hours solving a problem, that’s a task.
The best tasks describe a real problem that an experienced engineer would recognize, in language that an experienced engineer would use, with tests that verify the outcome rather than the approach. But I suspect we’ll start seeing difficult tasks emerge from the vibecoding world, particularly for complex functions like finance, where the problem is genuinely hard but the person who needs it solved isn’t a developer. The scope of what we should test is wider than we think.
2026-03-28 09:52:35
Below is the full text of the preliminary injunction ruling in the Anthropic vs. DoW case. I'm posting it here so that it's easier to read/listen to and discuss.
UNITED STATES DISTRICT COURT
NORTHERN DISTRICT OF CALIFORNIA
ANTHROPIC PBC,
Case No. 26-cv-01996-RFL
Plaintiff,
v.
ORDER GRANTING MOTION FOR
PRELIMINARY INJUNCTION
Re: Dkt. No. 6
U.S. DEPARTMENT OF WAR, et al.,
Defendants.
I. INTRODUCTION
This case touches on an important public debate. Anthropic says its artificial intelligence product, Claude, is not ready for safe use in fully autonomous lethal weapons or the mass surveillance of Americans. If the U.S. government wants to use its technology, Anthropic insists that the government must agree not to use it for these purposes. On the other hand, the Department of War says that it must be the one to decide what functions are safe for its AI tools to perform, not a private company. This public policy question is not for this Court to answer in this litigation. It is the Department of War's prerogative to decide what AI product it uses. Everyone, including Anthropic, agrees that the Department of War may permissibly stop using Claude and look for a new AI vendor who will allow "all lawful uses" of its technology. That is not what this case is about.
The question here is whether the government violated the law when it went further. After Anthropic went public with its disagreement with the Department of War, Defendants reacted with three significant measures that are the subject of this lawsuit. First, the President announced that every federal agency (not just the Department of War) would immediately ban Anthropic from ever having another government contract. That would include, for example, the National Endowment for the Arts using Claude to design its website. Second, Secretary Hegseth announced that anyone who wants to do business with the U.S. military must sever any commercial relationship with Anthropic. That would mean a company that used Claude to power its customer service chatbot could not serve as a defense contractor. Third, the Department of War designated Anthropic a "supply chain risk," a label that applies to adversaries of the U.S. government who may sabotage its technology systems. That designation has never been applied to a domestic company and is directed principally at foreign intelligence agencies, terrorists, and other hostile actors.
These broad measures do not appear to be directed at the government's stated national security interests. If the concern is the integrity of the operational chain of command, the Department of War could just stop using Claude. Instead, these measures appear designed to punish Anthropic. One of the amicus briefs described these measures as "attempted corporate murder." They might not be murder, but the evidence shows that they would cripple Anthropic. The record supports an inference that Anthropic is being punished for criticizing the government's contracting position in the press. In their announcements, the President and Secretary Hegseth called Anthropic "out of control" and "arrogant," describing its "sanctimonious rhetoric" as an attempt to "strong-arm" the government. The Department of War's records show that it designated Anthropic as a supply chain risk because of its "hostile manner through the press." Punishing Anthropic for bringing public scrutiny to the government's contracting position is classic illegal First Amendment retaliation.
Moreover, Defendants' designation of Anthropic as a "supply chain risk" is likely both contrary to law and arbitrary and capricious. The Department of War provides no legitimate basis to infer from Anthropic's forthright insistence on usage restrictions that it might become a saboteur. At oral argument, government counsel suggested that Anthropic showed its subversive tendencies by "questioning" the use of its technology, "raising concerns" about it, and criticizing the government's position in the press. Nothing in the governing statute supports the Orwellian notion that an American company may be branded a potential adversary and saboteur of the U.S. for expressing disagreement with the government.
There are other serious procedural problems with the government's actions. Anthropic had no notice or opportunity to respond, which likely violated its due process rights. And the Department of War flouted procedural safeguards required by Congress before entering the supply chain risk designation, including that it consider less intrusive measures.
At bottom, Anthropic has shown that these broad punitive measures were likely unlawful and that it is suffering irreparable harm from them. Numerous amici have also described wide-ranging harm to the public interest, including the chilling of open discussion about important topics in AI safety. The motion for a preliminary injunction is granted.
II. BACKGROUND
A. Anthropic and the Federal Government's Use of Claude
Plaintiff Anthropic PBC is an AI developer focused on AI research, safety, and policy work. (Dkt. No. 6-1 ¶¶ 8, 12.) Anthropic regularly publishes research on the societal impacts of large language models ("LLMs") and engages in public advocacy for AI safety. (Id. ¶¶ 10, 13.) Anthropic's co-founder, Jared Kaplan, states that he and his co-founders started Anthropic "to put safety at the center as AI progress accelerates." (Id. ¶ 8.)
Anthropic began commercial deployments of LLMs in early 2023. (Id. ¶ 12.) Anthropic's signature model is a general-purpose LLM called "Claude." (Id. ¶ 14.) LLMs like Claude are algorithmic systems trained on massive datasets to identify patterns and associations in language and quickly generate outputs and take actions that resemble human responses and actions. (Id. ¶ 15.) When given access to tools, Claude can act as an agent of the user ("agentically")—or even autonomously—executing tasks without requiring ongoing user direction. (Id. ¶ 16.) Anthropic's clients use its products in myriad contexts. For example, a client's engineers may use their company's Claude subscription to write, test, and fix code, or a client may integrate a Claude model into its own product via an application programming interface ("API"). (Dkt. No. 73 at 7–8; Dkt. No. 6-4 ¶ 6.) Some of these Anthropic clients then provide the U.S. government with products and services that integrate Claude or that were created using Anthropic's technology to various degrees. (Dkt. No. 6-4 ¶ 6.)
LLMs, including Claude, can produce responses that diverge from the goals of the people that trained them or reflect skewed or mistaken judgments embedded in their training data. (Dkt. No. 6-1 ¶ 18.) Those risks are heightened when an LLM is acting agentically or autonomously out in the world with little or no human supervision. (Id. ¶ 16.) To address these risks, the publicly available version of Claude has three layers of protection. First, Anthropic uses various "alignment" techniques to make Claude more reliably follow human values and intentions. (Id. ¶ 19.) Second, Anthropic has added safeguards that can detect in real time and stop activity Anthropic considers to be harmful. (Id. ¶ 20.) Finally, Anthropic has a Usage Policy under which users agree not to use Claude in certain specified "risky ways, including ways [Anthropic] understand[s] that it has not been developed for and/or is not ready for." (Id. ¶ 21.)
U.S. intelligence and defense agencies have been using Claude since November 2024 through a partnership with Palantir Technologies. (Dkt. No. 6-3 ¶ 5.) Since March 2025, the Department of War has been using "Claude Gov," dedicated models for national security users, through Anthropic partner platforms. (Id. ¶ 6; Dkt. No. 6-1 ¶¶ 27, 29; Dkt. No. 6-17.) Claude Gov has different safeguards than the civilian models and can, for example, analyze documents marked "Top Secret." (Dkt. No. 6-1 ¶ 27.) Claude Gov also has a government-specific addendum to its Usage Policy that eliminates some of the restrictions applicable to the civilian version. (Id. ¶ 28.) Since DoW began using Claude Gov in March 2025, this government-specific usage policy has "imposed broad restrictions that would have prohibited mass surveillance of Americans and lethal autonomous warfare." (Id. ¶ 29.)
Before the events at issue in this case, DoW appeared to have no objection to the usage policy and did not appear to view Anthropic as a national security risk in any way. Anthropic's Head of Policy, Sarah Heck, attested that she is unaware of "anyone at DoW ever suggesting Anthropic's models are somehow insecure or have been compromised." (Dkt. No. 6-2 ¶ 17.) Thiyagu Ramasamy, the Head of Public Sector at Anthropic, has likewise never been made aware of "any potential supply chain risk posed by Anthropic, its employees, or its products and services," despite working closely with DoW to integrate Claude between 2024 and 2026. (Dkt. No. 6-3 ¶¶ 8–11, 17.) DoW has not submitted any evidence disputing this point.
Moreover, while these usage restrictions were in place, DoW's Defense Counterintelligence and Security Agency granted Anthropic a Top Secret facility security clearance enabling it to provide support for classified national security projects, after an 18-month vetting process. (Id. ¶ 9.) In June 2025, the General Services Administration ("GSA") and DoW granted Claude authorization through the Federal Risk and Authorization Management Program ("FedRAMP") for use with FedRAMP High and DoD Impact Level 4 and 5 workloads, representing the highest levels of cloud security certification for unclassified and controlled unclassified information. (Id. ¶ 10.) In July 2025, Anthropic was awarded a two-year, up to $200 million agreement by DoW's Chief Digital and Artificial Intelligence Office to integrate and optimize AI capabilities across DoW. (Id. ¶ 6.) And in August 2025, Anthropic and GSA announced an agreement to deliver Claude Gov to all three branches of the government—civilian executive, legislative, and judiciary. (Id. ¶ 7.) The unrebutted record shows that, until this dispute, Anthropic "only ever received positive feedback about Claude's performance from [its] governmental customers." (Id. ¶ 12.)
B. Defendants and Anthropic Publicly Disagree About Anthropic's Role in Ensuring AI Safety in the Military and Surveillance Context
In the fall of 2025, DoW and Anthropic began discussing a new deployment of Claude on DoW's "GenAI.mil" platform. (Dkt. No. 6-1 ¶ 32.) During these conversations, DoW took the position that Anthropic should permit DoW, its contractors, and its subcontractors to use all versions of Claude for "all lawful uses," without any usage restrictions. (Id.) As DoW put it, "We will not let ANY company dictate the terms regarding how we make operational decisions." (Dkt. No. 6-33 at 2.) Anthropic ultimately agreed to do so with "two critical exceptions: mass surveillance of Americans and lethal autonomous warfare." (Dkt. No. 6-1 ¶ 33.) Anthropic did not believe Claude currently to be safe for those uses based on its training and testing. (Id. ¶ 36.) For example, if Claude were used to conduct mass surveillance, Anthropic believed Claude might retrieve a much broader set of information than intended, thus inadvertently violating laws protecting civil liberties. (Id. ¶ 34.) Anthropic further believed that lethal autonomous warfare was beyond Claude's current ability to carry out reliably, in light of the potentially grave consequences of a mistake. (Id. ¶ 35.) Given the AI safety principles on which it was founded, Anthropic believed that removing those limitations would "undercut Anthropic's core identity." (Id. ¶ 38.)
Anthropic's Head of Policy Heck submitted a declaration describing DoW and Anthropic's negotiations. Heck attested that, during the negotiations, Anthropic CEO Dario Amodei told Emil Michael, Under Secretary of War for Research and Engineering, that if Anthropic's position on these two guardrails meant that Anthropic was not the right vendor for DoW's needs, then he would respect that decision. (Dkt. No. 6-2 ¶ 11.) According to Heck, Amodei further stated that, if Anthropic and DoW failed to come to an agreement, Anthropic stood ready to assist in an orderly offboarding from DoW's systems. (Id.) Despite disagreements, Heck states that these negotiations, including subsequent conversations through February 27, "remained cordial and amicable." (Id. ¶ 18.) Under Secretary Michael does not dispute that account in his declarations submitted in this litigation. (Dkt. Nos. 96-3; 120-1.)
In addition to private conversations, DoW and Anthropic both publicly aired their views. For example, in January 2026, Amodei posted a lengthy essay addressing, among other topics, the risk of using AI technology in surveillance and autonomous weapons. (Dkt. No. 6-7.) That essay advocated "draw[ing] a hard line against AI abuses within democracies," and discussed the need for "bright red lines" and "guardrails" on those topics to prevent "AI-enabled totalitarianism." (Id. at 31–32.) The essay also referenced Amodei's prior writings on the subject of AI safety. (Dkt. No. 6-8.) On January 9, 2026, Defendant Secretary of War Pete Hegseth issued a Memorandum containing a subsection titled "Clarifying 'Responsible AI' at the DoW—Out with Utopian Idealism, In with Hard-Nosed Realism," where Secretary Hegseth instructed that the "any lawful use" provision must be incorporated into "any DoW contract through which AI services are procured." In early February 2026, Under Secretary Michael stated to the press about Anthropic: "You can't have an AI company sell AI to the Department of War and [then] don't let it do Department of War things."
On February 24, 2026, representatives of Anthropic, including Amodei and Heck, met with representatives of DoW, including Secretary Hegseth. (Dkt. No. 6-2 ¶ 12.) The parties discussed Anthropic's ongoing concerns regarding the possible use of Claude in mass surveillance of Americans and lethal autonomous warfare. (Id. ¶¶ 13–15.) Secretary Hegseth stated that Claude had "exquisite capabilities" but that "DoW had many other vendors to choose from that have never raised" the concerns Anthropic was raising and who "would never hold any veto power over DoW." (Id. ¶¶ 12, 15.) Secretary Hegseth did not say anything about Anthropic's AI models being unsafe, insecure, or subject to compromise. (Id. ¶ 17.) Nonetheless, at the end of the meeting, Secretary Hegseth told Anthropic's representatives that if Anthropic did not agree to "all lawful uses" of its products by 5:00 p.m. on February 27, 2026, DoW would immediately designate Anthropic a supply-chain risk and would prevent Anthropic from partnering with DoW, anyone affiliated with DoW, or any other agency. (Id. ¶ 16.) At the February 24 meeting, the unrebutted record shows that Secretary Hegseth also stated that DoW might invoke the Defense Production Act to designate Anthropic as essential to national security and thus compel Anthropic to provide its services without the restrictions. (Id.)
Following the meeting, on February 26, Amodei issued a public statement on behalf of Anthropic. (Dkt. No. 6-18.) Amodei stated that "Anthropic understands that [DoW], not private companies, makes military decisions," and Anthropic had "never raised objections to particular military operations nor attempted to limit use of our technology in an ad hoc manner." (Id. at 2.) However, Amodei said that "in a narrow set of cases . . . AI can undermine, rather than defend, democratic values," and that some uses of AI are "simply outside the bounds of what today's technology can safely and reliably do." Amodei concluded that Anthropic "cannot in good conscience" provide DoW the type of access to Claude that it was requesting. (Id. at 3.)
C. The President and Secretary Hegseth State that Anthropic is Barred from Working with the Federal Government or Defense Contractors
At 3:47 p.m. E.T. on February 27, 2026, President Donald Trump posted the following statement on the social media platform Truth Social:
THE UNITED STATES OF AMERICA WILL NEVER ALLOW A RADICAL LEFT, WOKE COMPANY TO DICTATE HOW OUR GREAT MILITARY FIGHTS AND WINS WARS! That decision belongs to YOUR COMMANDER-IN-CHIEF and the tremendous leaders I appoint to run our Military.
The Leftwing nut jobs at Anthropic have made a DISASTROUS MISTAKE trying to STRONG-ARM the Department of War, and force them to obey their Terms of Service instead of our Constitution. Their selfishness is putting AMERICAN LIVES at risk, our Troops in danger, and our National Security in JEOPARDY.
Therefore, I am directing EVERY Federal Agency in the United States Government to IMMEDIATELY CEASE all use of Anthropic's technology. We don't need it, we don't want it, and will not do business with them again! There will be a Six Month phase out period for Agencies like the Department of War who are using Anthropic's products, at various levels. Anthropic better get their act together, and be helpful during this phase out period, or I will use the Full Power of the Presidency to make them comply, with major civil and criminal consequences to follow.
WE will decide the fate of our Country — NOT some out-of-control, Radical Left AI company run by people who have no idea what the real World is all about. Thank you for your attention to this matter.
MAKE AMERICA GREAT AGAIN!
PRESIDENT DONALD J. TRUMP
(Dkt. No. 6-20 at 2 (the "Presidential Directive"); see also Dkt. No. 6-3 ¶ 19.) A little over an hour later, Secretary Hegseth posted on the social media platform X:
This week, Anthropic delivered a master class in arrogance and betrayal as well as a textbook case of how not to do business with the United States Government or the Pentagon. Our position has never wavered and will never waver: the Department of War must have full, unrestricted access to Anthropic's models for every LAWFUL purpose in defense of the Republic.
Instead, @AnthropicAI and its CEO @DarioAmodei, have chosen duplicity. Cloaked in the sanctimonious rhetoric of "effective altruism," they have attempted to strong-arm the United States military into submission - a cowardly act of corporate virtue-signaling that places Silicon Valley ideology above American lives.
The Terms of Service of Anthropic's defective altruism will never outweigh the safety, the readiness, or the lives of American troops on the battlefield. Their true objective is unmistakable: to seize veto power over the operational decisions of the United States military. That is unacceptable.
As President Trump stated on Truth Social, the Commander-in-Chief and the American people alone will determine the destiny of our armed forces, not unelected tech executives.
Anthropic's stance is fundamentally incompatible with American principles. Their relationship with the United States Armed Forces and the Federal Government has therefore been permanently altered. In conjunction with the President's directive for the Federal Government to cease all use of Anthropic's technology, I am directing the Department of War to designate Anthropic a Supply-Chain Risk to National Security. Effective immediately, no contractor, supplier, or partner that does business with the United States military may conduct any commercial activity with Anthropic. Anthropic will continue to provide the Department of War its services for a period of no more than six months to allow for a seamless transition to a better and more patriotic service.
America's warfighters will never be held hostage by the ideological whims of Big Tech. This decision is final.
(Dkt. No. 6-21 at 2 (the "Hegseth Directive").) Taken together, the Presidential Directive and the Hegseth Directive create an immediate, permanent, federal government-wide bar on using any Anthropic technology (except during a six-month transition period), and also prohibit private companies from doing business with both the military and Anthropic. Neither the President nor Secretary Hegseth cited any statutory authority for the Directives.
On March 4, 2026, Anthropic received two letters on DoW letterhead, signed by Secretary Hegseth and dated one day prior, regarding Anthropic's designation as a "supply chain risk." (Dkt. Nos. 6-27; 6-28; see also Dkt. No. 6-9 ¶¶ 27–28.) The first letter cites 41 U.S.C. § 4713. Anthropic has challenged this letter through a separate action. Anthropic v. U.S. Department of War, No. 26-1049 (D.C. Cir. March 9, 2026). The second letter cites 10 U.S.C. § 3252. (Dkt. No. 6-27 ("Section 3252 Letter").) The Section 3252 Letter states that DoW has determined that (i) the use of [Anthropic and its subsidiaries and affiliates'] products or services in DoW covered systems presents a supply chain risk and that the use of the Section 3252 authority to carry out covered procurement actions is necessary to protect national security by reducing supply chain risk, and (ii) less intrusive measures are not reasonably available to reduce such supply chain risk. (Dkt. No. 96-2 at 18 (footnotes omitted).) The Letter explains that the designation applies to Anthropic's existing or future "covered item[s] of supply," or any product or service procured by DoW as part of a "covered system." (Id.) At oral argument on Anthropic's motion, Counsel for Defendants stated that the legal effect of the Supply Chain Designation is to "prevent Anthropic from being utilized as a subcontractor in covered settings" in addition to prohibiting its role as a prime contractor. (Dkt. No. 128 at 27 (emphasis added).) Counsel for Defendants agreed that if a DoW contractor used Claude Code under a general license as a tool to write software for a DoW national security system, that action would not be prohibited under the Supply Chain Designation. (Id. at 26–27.) Likewise, contractors doing work for DoW would not be terminated for using Anthropic for non-DoW work based on the designation or the Hegseth Directive. (Id. at 12.)
The effect of the letter is "immediate" and permanent. (Dkt. No. 96-2 at 18–19.) Anthropic was given thirty days to appeal the designation, but it was not provided notice of the factual basis for the designation on March 4. (Id. at 20–21.) The Letter makes no reference to the Presidential Directive or the Hegseth Directive and does not explain how the designation will be implemented during the six-month transition period described in the Hegseth Directive.
D. Defendants' Authority to Designate a "Supply Chain Risk" Under Section 3252
Ordinarily, government contractors who face suspension or a long-term ban (known as "debarment") from federal contracting are entitled to a host of procedural protections. See 48 C.F.R. § 9.400(a)(1). In most circumstances, government contractors are entitled to pre-deprivation notice and a formalized process for opposing a proposed suspension or debarment. Id. §§ 9.406-3, 9-407-3; see also Friedler v. GSA, 271 F. Supp. 3d 40, 43–45 (D.D.C. 2017).
In 2011, Congress was concerned that "the globalization of the information technology industry" left DoW vulnerable to "attacks on its systems and networks," and gave the Secretary of Defense the "authority needed to address this risk" when procuring certain sensitive technology systems. S. Rep. No. 111-201, at 162 (2010). Congress enacted the provision that became 10 U.S.C. § 3252, which allows the Secretary to designate a source as a "supply chain risk." Congress defined "supply chain risk" as "the risk that an adversary may sabotage, maliciously introduce unwanted function, or otherwise subvert the design, integrity, manufacturing, production, distribution, installation, operation, or maintenance of a [national security] system so as to surveil, deny, disrupt, or otherwise degrade the function, use, or operation of such system." Id. § 3252(d)(4). DoW's implementing instructions for the predecessor provision, which were issued in 2012 in the wake of the law's passage, described the relevant risk as "sabotage or subversion" by "foreign intelligence, terrorists or other hostile elements."
When a source is designated a "supply chain risk," the Secretary may exclude that source from providing equipment for DoW's "covered systems," which are certain sensitive information technology systems used for national security purposes. The exclusion can also extend to those who provide a "covered item of supply," which is an "item of information technology that is purchased for inclusion in a covered system, and the loss of integrity of which could result in a supply chain risk for a covered system." 48 C.F.R. § 239.7301; see also 10 U.S.C. § 3252(d)(6). Notably, nothing in Section 3252 authorizes the Secretary to exclude a company from providing services to other government agencies. Nor does Section 3252 contain any authorization to require all contractors with DoW, including those whose work does not involve covered systems, to stop working with the entity.
Section 3252 does not include the usual pre-deprivation procedures for suspension or debarment of a government contractor, presumably because most hostile adversaries are foreign actors who lack due process rights. Instead, Congress required a different set of procedural safeguards. Specifically, to exclude a company as a "supply chain risk," the Secretary must first take the following steps: (1) "consult[] with procurement or other relevant officials of the covered agency"; (2) "mak[e] a determination in writing" that the exclusion "is necessary to protect national security by reducing supply chain risk," and that "less intrusive measures are not reasonably available to reduce such supply chain risk"; and (3) provide notice to the appropriate congressional committees explaining the risk and bases for the determination. 10 U.S.C. § 3252(b). DoW's regulations further require that the consultation process include "a risk assessment by the Under Secretary of Defense for Intelligence" before a designation can be made. 48 C.F.R. § 239.7304(a).
E. Anthropic's Designation as a Supply Chain Risk
The record indicates that on March 3, 2026—the day before Anthropic received the Section 3252 Letter—Secretary Hegseth formally designated Anthropic a supply chain risk based on a joint recommendation from the Under Secretary of War for Acquisition and Sustainment and DoW's Chief Information Officer. (Dkt. No. 96-2 at 2, 3–4.) The joint recommendation states that after consultation "with procurement and other relevant officials within DoW and, on the basis of a risk analysis provided by DoW [Chief Information Officer]," they "recommend that there is supply chain risk to DoW covered systems related to the use of Anthropic." (Id. at 3.) In particular, they found that, "by maintaining the ability and necessity to continuously update the product or service," Anthropic had the "potential" to "subvert the design and/or functionality of their product or service." (Id.) No further explanation was provided as to why Anthropic's products posed such a risk as compared to other AI or software with regular updates. The recommendation further found that "less intrusive measures are not reasonably available to reduce such supply chain risk," but did not explain why or state what less intrusive measures were considered. (Id.)
The recommendation attaches a memorandum authored by Under Secretary Michael, who had been a principal negotiator in the contract discussions with Anthropic. (Id. at 6–9 (the "Michael Memo").) The Michael Memo states that AI models "are acutely vulnerable to manipulation," and Anthropic could use "[p]rivileged access with malintent [to] subtly poison the training data to maliciously introduce unwanted function." (Id. at 7.) Under Secretary Michael further describes the risk of AI "drift," in which Claude could "degrade as new data is introduced" by Anthropic. (Id.) He asserts that "DoW would be forced to operate a black box controlled by a hostile party, which could contain hidden biases or backdoors." (Id.) In conclusion, Under Secretary Michael states:
In the instant case, Anthropic's risk level escalated from a potentially manageable technical and business negotiation to an unacceptable national security threat over the course of the DoW's contract negotiation with them. . . . The supply chain risk level increased when Anthropic insisted on terms of service that would constrain the DoW beyond what is in the law. This risk further increased when Anthropic asserted in the negotiations that it have an approval role in the operational decision chain, which would require the DoW to accept significant operational risk. Then during the final weeks of negotiations, it became clear that Anthropic was leveraging DOW's ongoing good faith negotiations for Anthropic's own public relations, and they began engaging in an increasingly hostile manner through the press, despite the ongoing private negotiations with DoW leadership. Finally, this hostile posture was even further compounded when, during a time of active military operations, Anthropic leadership questioned the use of their technology in our warfighting systems clearly permitted under the Terms of Service of their existing contract with our Prime contractor.
(Id. at 8.)
The Michael Memo does not address what less intrusive measures were considered. It also does not explain why DoW believed Anthropic has "privileged access," "backdoors," or can otherwise "control" Claude after its delivery to DoW. Anthropic has submitted unrebutted evidence that it lacks any such access. Ramasamy states that, "as Claude is deployed in DoW environments—such as through air-gapped, classified cloud systems operated by third-party defense contractors—Anthropic has no ability to access, alter, or shut down the deployed model." (Dkt. No. 115 ¶¶ 14–15.) As to whether Anthropic can cause Claude to "drift" or "degrade," Ramasamy states that "[o]nce a model is running inside a government-secure enclave, Anthropic cannot unilaterally alter these features," and Claude does not "change or degrade on its own." (Id. ¶¶ 17–19.) In his sur-reply declaration, Under Secretary Michael does not dispute these statements. (See generally Dkt. No. 120-1.) At oral argument, Counsel for Defendants admitted that he was unaware of DoW having any knowledge that Anthropic had unilateral ability to update or modify its products without DoW's consent. (Dkt. No. 128 at 39.) Counsel for Defendants stated that DoW is presently conducting an audit to explore whether any such risk exists. (Id.)
The Michael Memo also does not specify the basis for the statement that "Anthropic asserted in the negotiations that it have an approval role in the operational decision chain." (Dkt. No. 96-2 at 8.) Anthropic co-founder Kaplan has attested that "Anthropic has not sought, and would not seek, to dictate how the government conducts its missions." (Dkt. No. 6-1 ¶ 37.) Indeed, Anthropic attests that it proposed contract language expressly disavowing any such authority. (Dkt. No. 114 ¶ 9.) There is no evidence in the record that Anthropic sought to impose any restrictions other than the usage policies at issue. Under Secretary Michael's declaration submitted in sur-reply explains that his reference to "approval rights" related solely to permitting an exception to the usage policies: specifically, he refers to "a contract negotiation meeting on December 4, 2025, [where] Anthropic leadership expressed that DoW would have to call Anthropic in real time to seek authorization for a usage exception to one of their redlines." (Dkt. No.120-1 ¶ 7.) Under Secretary Michael further explains in another declaration that his reference to Anthropic leadership "question[ing]" the technology's use during an operation referred to a comment made by an unnamed Anthropic executive to a DoW operational support software vendor. (Dkt. No. 96-3 ¶ 15.) He does not identify any action taken by Anthropic to stop the use that had been questioned.
In sum, the record shows that, outside of Anthropic's contractual redlines (which Anthropic proposed it might be willing to waive in specific circumstances) and an unnamed Anthropic executive's questions to a third-party vendor, Anthropic has not sought to participate in DoW's operational decision chain. Under Secretary Michael does not dispute Heck's attestation that negotiations between Anthropic and DoW had remained "cordial and amicable." (Dkt. No. 6-2 ¶ 18.)
Concurrently with entering the supply chain risk designation, Secretary Hegseth sent letters to various congressional committees advising them of his action. (Dkt. No. 96-2 at 12–17.) The letters do not describe what less intrusive measures DoW considered prior to making the designation. (Id.) On March 5, 2026, DoW's Chief Information Officer sent a memorandum to DoW leadership regarding the removal of Anthropic's products in DoW Systems. The memorandum instructs that, within 180 days, DoW components should remove Anthropic's products "from all DoW systems and networks," as well as from various lists and markets facilitating technology procurement for DoW. (Id. at 22.) The memorandum clarifies that the "prohibition applies to all [Defense Industrial Base partner] contracts," and the restriction should be incorporated "into all current and future contracts." (Id. at 23.)
F. Defendants Disclaim that the Hegseth Directive Had Independent Legal Effect
At oral argument on Anthropic's motion for a preliminary injunction on March 24, 2026, the Court inquired about the legal basis and effect of the following statement in the Hegseth Directive: "Effective immediately, no contractor, supplier, or partner that does business with the United States military may conduct any commercial activity with Anthropic. . . . This decision is final." (Dkt. No. 6-21 at 2.) Counsel for Defendants agreed that he was not aware of any statute that gave Secretary Hegseth the authority to issue such a prohibition and agreed that the statement had "absolutely no legal effect at all." (Dkt. No. 128 at 8–9, 13–14.) Counsel for Defendants also agreed that the statement did not reflect DoW's "immediate intended course of action" and that "DoW does not intend to terminate any contractors on the basis that they have a commercial relationship with Anthropic that [i]s separate from their work for DoW." (Id. at 9, 12.) Instead, counsel asserted that the only legal effect of the Hegseth Directive "flows through" the Supply Chain Designation. (Id. at 14.) When asked why Hegseth made a public statement that had no legal effect and that did not reflect the immediate intent of DoW, counsel stated, "I don't know." (Id.) Even so, Defendants declined to stipulate to enjoin this prohibition because DoW "is continuing to assess the situation" and "will take action as needed . . . to mitigate the risk from Anthropic." (Id. at 18–19.)
G. The Challenged Actions Risk Crippling Anthropic
Prior to the Challenged Actions, Anthropic had a large public sector business, reflecting its investment in the lengthy vetting and certification processes described above. Before the start of 2026, Anthropic had projected its expected public sector annual recurring revenue in 2026 to be "several hundred million dollars." (Dkt. No. 6-3 at ¶ 31.) It previously experienced a fourfold increase in that revenue from December 2025 to the end of January 2026, and it had accordingly projected a public sector business in the next five years amounting to billions of dollars. (Id.) Anthropic now projects that that business will shrink substantially or disappear altogether absent injunctive relief. (Dkt. No. 6-3 ¶ 33.)
In the immediate aftermath of the Challenged Actions, several federal agencies have already moved to end their relationships with Anthropic. GSA removed Anthropic from the agency's AI platform USAi.gov, cutting off sales opportunities for agencies. (Id. ¶ 22.) The U.S. Department of the Treasury and the Federal Housing Finance Agency announced they were terminating all use of Claude. (Id. ¶ 23.) The Department of State and the Department of Health and Human Services ("HHS") have issued internal statements saying they will follow the Presidential Directive. (Id. ¶ 25.) The Department of Energy's Lawrence Livermore National Laboratory informed Anthropic that it was shutting down Claude. (Id. ¶ 27.)
Defense contractors performing work under government contracts using Claude-integrated APIs are assessing—and in many cases looking to terminate—their reliance on Anthropic. (Dkt. No. 6-4 ¶ 12.) Government contractors outside of the defense sector are following suit. Anthropic reports that one partner, which has a multi-million-dollar annual contract with Anthropic, has switched from Claude to a competing generative AI model to service a contract with the U.S. Food and Drug Administration. (Id. ¶ 14.) Anthropic reports it "has received inquiries regarding the [Challenged Actions] from over one hundred enterprise customers expressing deep fear, confusion, and doubt about Anthropic and the repercussions of associating with [the] company." (Id. ¶ 20.) Major law firms began alerting their government-contractor clients to "audit[] their Anthropic exposure now" and "prepare to deploy alternatives" to Anthropic. (Dkt No. 6-30 at 4; Dkt No. 6-31 at 3.) Within days of the Challenged Actions, Anthropic has experienced a revenue impact as deals worth hundreds of millions of dollars have been delayed from closing, prospective clients have pulled out of negotiations, and some customers have terminated contracts. (Dkt. No. 6-4 ¶¶ 16–19.)
Several amicus briefs detail the chilling effect and confusion that the Challenged Actions have caused. Small developers describe their uncertainty as to whether they will continue to be able to use Claude Code, or use open source libraries that might contain sections of code written by Claude, without running afoul of the Challenged Actions. (Dkt. No. 73 at 7–8.) A group of industrial trade associations explain that they are unsure whether "Claude-generated code already incorporated into an already-shipping product" will be rejected by DoW and whether their members may continue to "host[] Claude alongside dozens of other AI models . . . for purely commercial customers with no defense connection." (Dkt. No. 72 at 13.) Employees from major companies in the AI field explain that the Challenged Actions threaten to "chill open deliberation" and "professional debate" among the people best positioned to understand AI technology and its potential for "catastrophic misuse." (Dkt. 24-1 at 6, 8, 9.)
III. PROCEDURAL HISTORY
Anthropic filed this action and moved for a temporary restraining order and preliminary injunction on March 9, 2026. Anthropic brings a First Amendment claim, a Fifth Amendment Due Process Clause claim, Administrative Procedure Action ("APA") contrary to law and arbitrary and capricious claims, and an ultra vires action asserting that the Presidential Directive is in excess of statutory and constitutional authority. Anthropic seeks preliminary injunctive relief to enjoin Defendants from implementing, applying, or enforcing in any manner the Challenged Actions. A status conference was held on March 10, 2026. (Dkt. No. 38.) To allow development of a more fulsome record and to afford time for Defendants to respond, the Court determined that the motion should be handled in the preliminary injunction posture rather than in the temporary restraining order posture. Defendants were given until March 17, 2026, to file their opposition to the preliminary injunction motion, and Anthropic was given until March 20, 2026, to file its reply. (Id.) On reply, Anthropic introduced two declarations in response to the Michael Memo, which the government had provided to Anthropic for the first time in its opposition brief. (Dkt. Nos. 114, 115.) Defendants were then given until the morning of March 24, 2026, to provide any rebuttal to those declarations, and Defendants submitted a further declaration from Under Secretary Michael in sur-reply. (Dkt. Nos. 117, 120.) A hearing was held on the afternoon of March 24, 2026. Immediately after the hearing, Anthropic was permitted to submit evidence that various Defendant Agencies use or used its services, and Defendants were given a further 24 hours to submit any opposition to Anthropic's evidence. (Dkt. Nos. 126, 131.)
IV. LEGAL STANDARD
"[I]njunctive relief [is] an extraordinary remedy that may only be awarded upon a clear showing that the plaintiff is entitled to such relief." Winter v. Nat. Res. Def. Council, 555 U.S. 7, 22 (2008). To obtain a preliminary injunction, Anthropic must establish that it is likely to succeed on the merits, that it is likely to suffer irreparable harm in the absence of preliminary relief, that the balance of equities tips in its favor, and that an injunction is in the public interest. Id. at 20. When the nonmoving party is the government, the final two factors merge. Baird v. Bonta, 81 F.4th 1036, 1040 (9th Cir. 2023) (citing Nken v. Holder, 556 U.S. 418, 435 (2009)).
V. ANALYSIS
A. Anthropic Has Shown a Likelihood of Success on Its First Amendment Retaliation Claim
Government officials cannot use the "power of the State to punish or suppress disfavored expression." Nat'l Rifle Ass'n of Am. v. Vullo, 602 U.S. 175, 188 (2024). The government is entitled to express its views regarding the appropriate use of AI in warfare, and it is entitled to choose contracting partners that best align with those views. "But a State's failure to persuade" its prospective contracting partner to come around to its views "does not allow it to hamstring" disagreeing parties. Sorrell v. IMS Health, Inc., 564 U.S. 552, 578 (2011). "Official reprisal for protected speech offends the Constitution because it threatens to inhibit exercise of the protected right, . . . and the law is settled that as a general matter the First Amendment prohibits government officials from subjecting an individual to retaliatory actions . . . for speaking out." Hartman v. Moore, 547 U.S. 250, 256 (2006) (citation omitted).
Here, Anthropic has shown a likelihood of success on its First Amendment claim. The record shows that Defendants' conduct appears to be driven not by a desire to maintain operational control when using AI in the military but by a desire to make an example of Anthropic for its public stance on the weighty issues at stake in the contracting dispute. Although Anthropic had always applied the usage policies in question to Claude Gov, it had been repeatedly lauded as a partner and passed lengthy national security vetting processes. Only when Anthropic went public with its concerns about DoW's contracting position did Defendants set out to publicly punish Anthropic for its "ideology" and "rhetoric," as well as its "arrogance" for being unwilling to compromise those beliefs. At that point, Defendants announced a plan to cripple Anthropic: to blacklist it from doing business with any company that services the U.S. military, to permanently cut off its ability to work with the federal government, and to brand it an adversary that could sabotage DoW and that posed a supply chain risk. Those actions go far beyond what would be necessary to address DoW's ostensible concern about having complete operational control when using AI. This appears to be classic First Amendment retaliation.
1. Anthropic Has Made Out a Prima Facie Retaliation Claim
"To bring a First Amendment retaliation claim, the plaintiff must allege that (1) it engaged in constitutionally protected activity; (2) the defendant's actions would 'chill a person of ordinary firmness' from continuing to engage in the protected activity; and (3) the protected activity was a substantial motivating factor in the defendant's conduct—i.e., that there was a nexus between the defendant's actions and an intent to chill speech." Ariz. Students' Ass'n v. Ariz. Bd. of Regents, 824 F.3d 858, 867 (9th Cir. 2016). Upon making a prima facie showing, "the burden shifts to the defendant official to demonstrate that even without the impetus to retaliate he would have taken the action complained of." Hartman, 547 U.S. at 260.
Anthropic has shown a likelihood that all three prongs of the test are met. First, it has shown that it engaged in protected activity. In particular, the record shows that Anthropic and its CEO, Dario Amodei, are a loud and influential voice regarding the capabilities, risks, and safe uses of AI technology. See supra Section II.A–B. In the context of the current dispute with Defendants, DoW and Anthropic both publicly staked out their positions and engaged in a public debate about what deployments of Claude are currently unsafe and what rights Anthropic has to allow Claude's use by the government only with certain safety restrictions. This "speech on matters of public concern" is at "the heart of the First Amendment's protection." Snyder v. Phelps, 562 U.S. 443, 451–52 (2011) (cleaned up) (citation omitted). Such speech "is more than self-expression; it is the essence of self-government," and it "occupies the highest rung of the hierarchy of First Amendment values." Id. (citations omitted).
Defendants do not dispute that the second prong is met. Anthropic has submitted evidence that the Challenged Actions threaten to cripple the company and chill public debate. See supra Section II.G. Several amicus briefs support this conclusion. A group of 37 individuals working on AI technology assert that the Challenged Actions "chill[] professional debate on the benefits and risks of frontier AI systems and various ways that risks can be addressed to optimize the technology's deployment." (Dkt. No. 24-1 at 8.) An industry group of "values-led investors" warns that the Challenged Actions chill speech necessary to allow them to direct their investments to support the "principles and values" they care about. (Dkt. No. 77-1 at 12.) In short, the Challenged Actions easily qualify as ones which would chill a person of ordinary firmness from continuing to engage in further protected speech.
Finally, the Challenged Actions leave little question that the measures were prompted by Anthropic's public repudiation of DoW's contracting request and its perceived use of public scrutiny to "strong arm" DoW into changing its mind. Secretary Hegseth expressly tied Anthropic's punishment to its attitude and rhetoric in the press. He stated that "Anthropic delivered a master class in arrogance." (Dkt. No. 6-21 at 2.) Referring to Anthropic and Amodei, he further stated: "Cloaked in the sanctimonious rhetoric of 'effective altruism,' they have attempted to strong-arm the United States military" through their "corporate virtue-signaling" and "Silicon Valley ideology." (Id.) "Anthropic's stance is fundamentally incompatible with American principles." (Id.) The President described Anthropic as "radical left, woke company" and its employees as "leftwing nut jobs," who "made a DISASTROUS MISTAKE trying to STRONG-ARM the Department of War." (Dkt. No. 6-20 at 2.) Read in context of these repeated references to rhetoric and ideology, the term "strong-arm" in the Presidential Directive and the Hegseth Directive appears to be characterizing Anthropic as applying public pressure. That seems particularly likely in light of the undisputedly "cordial and amicable" nature of the contract negotiations themselves and Anthropic's offer to facilitate an orderly transition to another vendor if the parties could not come to terms.
The Michael Memo likewise repeatedly refers to Anthropic's public airing of its dispute with DoW as the reason that it is no longer trustworthy. (See Dkt. No. 96-2 at 7 (describing how Anthropic "opted to publicly spat with DoW" as a basis to distrust it); id. at 7–8 (same with respect to "the public statements of Anthropic's CEO and others associated with the company").) The memo finds that Anthropic's "risk level escalated" into a "supply chain risk" principally because it was "leveraging DoW's ongoing good faith negotiations for Anthropic's own public relations" and its "increasingly hostile manner through the press." (Id. at 8.) These specific references to Anthropic's viewpoint and public stance are direct evidence of what motivated Defendants' decision-making.
2. Defendants Have Not Carried Their Burden to Rebut Anthropic's Prima Facie Case
Defendants raise two interrelated arguments in opposition. First, they argue that Anthropic's "contracting position" is conduct, not speech entitled to First Amendment protection. Second, they argue that Anthropic's refusal to accept DoW's terms—not any of Anthropic's speech—was the but-for cause of Challenged Actions. Neither argument changes the likelihood of success analysis.
First, without reaching the question of whether private contract negotiations alone could constitute protected activity under the First Amendment, the record shows that Anthropic engaged in protected speech when it took public the parties' contracting impasse and the reasons behind its refusal to agree to DoW's terms. (See, e.g., Dkt. Nos. 6-7, 6-18.) As already explained, Anthropic's views on this matter fall within the heart of what the First Amendment protects: "subject[s] of general interest and of value and concern to the public" and "of legitimate news interests." See Snyder, 562 U.S. at 452–53 (citation omitted). Therefore, to the extent Anthropic publicly discussed its "contracting position," that speech is protected by the First Amendment.
Next, Defendants argue that even if Anthropic's public statements constitute protected speech, the contract dispute—not Anthropic's speech—was the motive and "but for" cause of the Challenged Actions. (Dkt. No. 96 at 22–24.) They point out that although Anthropic and Amodei have long advocated for AI safety, Defendants took the Challenged Actions only after Anthropic refused to remove its usage restrictions. But Defendants' own actions belie the notion that Anthropic's contracting position is what drove the Challenged Actions. Anthropic had imposed its usage restrictions from the beginning of DoW's use of Claude Gov, and no one had ever suggested that this indicated that Anthropic was untrustworthy or a potential saboteur. To the contrary, Anthropic passed extensive vetting at that time and was praised by the government, which had made arrangements to expand the company's role. It was only when Anthropic publicly discussed its dispute with DoW that Defendants criticized its rhetoric and ideology and adopted the punitive measures at issue.
As for the "but for" cause of the Challenged Actions, Defendants assert that, "[a]t bottom, had Anthropic agreed to the [g]overnment's term, the challenged actions would not have occurred." (Dkt. No. 96 at 23.) That is the wrong question to ask in a "but for" inquiry. The correct question is whether, if Anthropic had not engaged in the protected speech, Defendants would have taken the Challenged Actions regardless. See CarePartners, LLC v. Lashway, 545 F.3d 867, 877 (9th Cir. 2008) (stating that "defendant must show . . . that it would have reached the same decision; it is insufficient to show merely that it could have reached the same decision" (emphasis in original)). Defendants have not carried their burden to show that they would have.
If this were merely a contracting impasse, DoW would presumably have just stopped using Claude. This would square with the ostensible risk the Challenged Actions were meant to address: that Anthropic might interfere with "operational decisions of the United States military," putting "American lives at risk, [America's] troops in danger, and [its] national security in jeopardy." (Dkt. No. 6-21 at 2; Dkt. No. 6-20 at 2 (capitalizations omitted); see also Dkt. No. 6-28 (identifying a risk to national security systems).) The Challenged Actions, however, far exceed the scope of what could reasonably address such a national security interest. The Presidential Directive ordered a permanent federal government-wide bar, and the Hegseth Directive blacklisted Anthropic. Nothing in the record supports a determination that the contracting impasse alone would have triggered such an extreme response.
Although courts owe deference to the government on issues of national security, "concerns of national security and foreign relations do not warrant abdication of the judicial role." Holder v. Humanitarian L. Project, 561 U.S. 1, 34 (2010). Courts cannot "defer to the Government's reading of the First Amendment, even when such interests are at stake," and "the Government's authority and expertise in these matters do not automatically trump the Court's own obligation to secure the protection that the Constitution grants to individuals." Id. (citation omitted). Anthropic has shown a likelihood of success on its First Amendment retaliation claim.
B. Anthropic Has Shown a Likelihood of Success on Its Procedural Due Process Claim Under the Fifth Amendment
"Due process requires notice 'reasonably calculated, under all the circumstances, to apprise interested parties of the pendency of [a government] action and afford them an opportunity to present their objections.'" Al Haramain Islamic Found., Inc. v. U.S. Dep't of Treasury, 686 F.3d 965, 985 (9th Cir. 2012) (quoting United Student Aid Funds, Inc. v. Espinosa, 559 U.S. 260, 272 (2010)). Anthropic argues that Defendants violated Anthropic's procedural due process rights when they summarily blacklisted it and designated it a supply chain risk. Defendants respond that their actions did not impair any protectible interest Anthropic might have and that even if they did, pre-deprivation process was not necessary.
The Mathews v. Eldridge, 424 U.S. 319 (1976), balancing test governs this claim. Al Haramain, 686 F.3d at 985. Under Mathews, the Court "must weigh (1) [Anthropic's protected] interest, (2) the risk of an erroneous deprivation of such interest through the procedures used, as well as the value of additional safeguards, and (3) the Government's interest in maintaining its procedures, including the burdens of additional procedural requirements." Foss v. Nat'l Marine Fisheries Serv., 161 F.3d 584, 589 (9th Cir. 1998) (citing Mathews, 424 U.S. at 334–35). As already discussed, there are strong interests on both sides of the scale. Anthropic's interest in its reputation and in its continued eligibility to compete for federal work is undoubtedly significant. "On the other side of the scale, the government's interest in national security cannot be understated." Al Haramain, 686 F.3d at 980. "Striking a balance between those two strong competing interests cannot be done in the abstract." Id. Therefore, this Order "carefully assess[es] the precise 'procedures used' by the government, 'the value of additional safeguards,' and 'the burdens of additional procedural requirements.'" Id. (quoting Foss, 161 F.3d at 589).
1. Anthropic Has a Protectible Liberty Interest
The liberty interest protected by the Fifth Amendment encompasses the right to "follow a chosen profession free from unreasonable governmental interference." Greene v. McElroy, 360 U.S. 474, 492 (1959). Therefore, "debarring a corporation from government contract bidding constitutes a deprivation of liberty that triggers the procedural guarantees of the Due Process Clause." Trifax Corp. v. Dist. of Columbia, 314 F.3d 641, 643 (D.C. Cir. 2003) (citing Old Dominion Dairy Prods., Inc. v. Sec'y of Def., 631 F.2d 953, 961–62 (D.C. Cir. 1980)). Furthermore, "[w]here a person's good name, reputation, honor, or integrity is at stake because of what the government is doing to him, notice and an opportunity to be heard are essential." Wisconsin v. Constantineau, 400 U.S. 433, 437 (1971). Although reputational harm alone is generally insufficient to trigger a liberty interest, Siegert v. Gilley, 500 U.S. 226 (1991), reputational harm can give rise to a liberty interest when accompanied by a "tangible" alteration in status. Trifax, 314 F.3d at 644 (citation omitted).
Each of the Challenged Actions deprive Anthropic of protected liberty interests. First, the Presidential Directive ordered "EVERY Federal Agency in the United States Government to IMMEDIATELY CEASE all use of Anthropic's technology. We don't need it, we don't want it, and will not do business with them again!" (Dkt. No. 6-20 at 2.) That is a permanent debarment with absolutely no pre- or post-deprivation process. Next, the Hegseth Directive ordered that, "[e]ffective immediately, no contractor, supplier, or partner that does business with the United States military may conduct any commercial activity with Anthropic." (Dkt. No. 6-21 at 2 (emphasis added).) Secretary Hegseth also "direct[ed] the Department of War to designate Anthropic a Supply-Chain Risk to National Security," which occurred on March 3, 2026. (Id.; Dkt. No. 96-2.) The designation prohibits Anthropic from providing products or services as a contractor or subcontractor for national security systems. (Dkt. No. 96-2 at 2, 22–23.)
These acts inflict both reputational harm and an immediate alteration of Anthropic's status and therefore implicate significant liberty interests. For example, in Old Dominion, a government agent determined that a company "lacked integrity," which precluded the company from obtaining any future government contracting work and "threatened the very existence of the business." 631 F.2d at 955, 963. The D.C. Circuit held that the reputational harm and accompanying loss of government employment "injured a cognizable liberty interest." Id. at 962–66. The same is true here.
Defendants contend that Anthropic, which has private commercial relationships, lacks a liberty interest because the Challenged Actions do not act as a "complete prohibition of the right to engage in a calling." (Dkt. No. 96 at 30–31 (quoting Dittman v. California, 191 F.3d 1020, 1029 (9th Cir. 1999)).) But a complete prohibition is not the only way to injure a protected liberty interest. As the D.C. Circuit articulated in its "reputation plus" test, a deprivation of liberty occurs when the government imposes a "broad preclusion" that "seriously affect[s], if not destroy[s]," a company's "ability to obtain" contracts in its chosen field of business. Trifax, 314 F.3d at 644. The record shows that the Challenged Actions threaten to cripple Anthropic by not only stripping it of billions of dollars in federal contracts and subcontracts but also by labeling it as an adversary to the United States and ending its ability to have any commercial relationship with any company that might want to do business with DoW.
Thus, Anthropic has shown a substantial likelihood that it has a significant protectible liberty interest.
2. A Serious Risk of Erroneous Deprivation Exists, and Defendants Have Not Shown Any Urgency Supporting the Need to Bypass Pre-Deprivation Process
The record reflects that the Challenged Actions were taken without any meaningful notice or pre-deprivation process (and, in the case of the Presidential Directive and the Hegseth Directive, without any post-deprivation process either). Although Anthropic was on notice that the government objected to its contracting terms, it had no notice or opportunity to object before Defendants publicly barred it from all federal government work and blacklisted it with private companies working with the U.S. military. It also had no notice or opportunity to object to the factual basis for its designation as a supply chain risk, which it learned of in this litigation. (Dkt. No. 115 ¶ 5.) In fact, three days before he issued the Hegseth Directive, Secretary Hegseth told Anthropic that, unless the company complied, DoW would either designate it a supply chain risk (meaning Anthropic presented a grave threat to national security) or invoke the Defense Production Act (meaning Anthropic was essential to national security). (Dkt. No. 6-2 ¶ 16.) This contradiction underscores the complete lack of prior notice or process.
Anthropic has also shown that meaningful pre-deprivation notice of DoW's concerns and an opportunity to respond to those concerns were necessary to avoid the risk of erroneous deprivation here. Ralls Corp. v. Comm. on Foreign Inv. in the U.S., 758 F.3d 296, 318 (D.C. Cir. 2014). Upon learning in this litigation of DoW's stated factual basis for the supply chain risk designation, Anthropic submitted a declaration explaining that DoW's records reflect core misunderstandings about how Anthropic's technology works, which Anthropic could have dispelled if given the opportunity. (Dkt. No. 115 ¶¶ 13–19.)
Defendants argue that a post-deprivation hearing can satisfy due process "where there is 'the necessity of quick action by the State or the impracticality of providing any predeprivation process.'" (Dkt. No. 96 at 31 (quoting Logan v. Zimmerman Brush Co., 455 U.S. 422, 436 (1982).) Defendants, however, have not carried their burden to show such necessity. "Pre-deprivation notice typically is required absent 'a strong justification' from the government." Garza v. Woods, 150 F.4th 1118, 1130 (9th Cir. 2025) (citation omitted). Genuine exigency, such as the risk of "asset flight" when government intends to freeze the resources of a suspected terrorist group, can justify post-deprivation process. Al Haramain, 686 F.3d at 985. However, the bare invocation of a national security interest alone is not a stand-in for exigency. See Nat'l Council of Resistance of Iran v. Dep't of State, 251 F.3d 192, 207–08 (D.C. Cir. 2001). In this case, Defendants present no evidence of exigency whatsoever. To the contrary, the parties had undisputedly been doing business pursuant to the very terms to which the government objects for over a year before the Challenged Actions, and Anthropic's removal from DoW's systems is slated to take place over a period of six months.
Defendants contend that Anthropic's contractual terms would "diminish functionality and limit DoW's warfighting capabilities" and that Anthropic's growing "hostility in negotiations" was a sign that it might "unilaterally alter system guardrails and model weights." (Dkt. No. 96-2 at 6–7.) But, as discussed below in Section V.C.1.a, the unrebutted record shows that Anthropic is not technologically capable of taking such unilateral actions. And, in any event, the usage restrictions were not new. Beyond the self-imposed urgency created by the ultimatum that Secretary Hegseth issued on February 24, nothing in the record indicates a heightened supply chain risk to national security systems on February 27 or March 3. Therefore, even affording deference to the government's factual conclusion, Defendants have not shown exigency.
With respect to the more sweeping aspects of the Challenged Actions—the permanent federal government-wide bar and the private sector blacklisting—the record contains no indication of any risk, much less an urgent one, motivating those actions. In sum, pre-deprivation notice and process could, and likely should, have been provided to Anthropic before Defendants took the Challenged Actions.
Finally, Defendants argue that because Section 3252 does not expressly provide pre-deprivation process and permits DoW "to limit disclosure of information" related to designations in some circumstances, "[d]ue process is then satisfied." (Dkt. No. 96 at 31 (quoting 10 U.S.C. 3252(c)).) It is true that Section 3252 will often involve a "supply chain risk designation" without pre-deprivation process, and in many instances, this will not run afoul of the Constitution because the government has shown exigent circumstances or the target is either a foreign actor lacking due process rights or a hostile party that does not significantly rely on federal contracting. Here, though, Anthropic is a domestic company with a significant federal contracting business, and no exigency has been shown. Compliance with the statutory requirements does not relieve the government of its due process obligations under the Constitution. See, e.g., United States v. Rivera-Valdes, 157 F.4th 978, 989 (9th Cir. 2025). As to the argument that Section 3252 permits limitation of disclosures, Defendants have not submitted any evidence indicating that concerns about sensitive information prevented them from providing Anthropic with notice. Indeed, except for a proprietary report that Defendants asked to seal to protect the vendor's business interests (Dkt. No. 97), Defendants filed all of their evidence on the Court's public docket.
Thus, Anthropic has shown a likelihood of success on its Due Process Clause claim.
C. Anthropic Has Shown a Likelihood of Success on Its APA Claim
Under the APA, an agency action must be set aside and held unlawful if it is "arbitrary, capricious, an abuse of discretion," "in excess of statutory jurisdiction, authority, or limitations, or short of statutory right," or "without observance of procedure required by law." 5 U.S.C. § 706(2)(A), (C), (D). Anthropic has shown that the Hegseth Directive and the Supply Chain Designation were likely in excess of statutory authority, contrary to law, and arbitrary and capricious. Section 3252 is intended to address the risk of adversarial sabotage of national security systems, and it does not authorize DoW to designate a domestic vendor a supply chain risk simply because a vendor publicly criticized DoW's views about the safe uses of its system. Even if Section 3252 could be deployed for this purpose, DoW failed to follow the necessary process because, among other things, it failed to make any reasoned determination about whether less intrusive measures were available. Finally, the record strongly suggests that the reasons given for designating Anthropic a supply chain risk were pretextual and that Defendants' real motive was unlawful retaliation.
1. The Supply Chain Risk Designation Was in Excess of Statutory Authority and Contrary to Law
a. Anthropic's Conduct Does Not Meet the Definition of a Supply Chain Risk
On the record before the Court, Anthropic's conduct does not appear to be within the definition of "supply chain risk" in Section 3252. Section 3252 defines a supply chain risk as limited to "the risk that an adversary may sabotage, maliciously introduce unwanted function, or otherwise subvert . . . a covered system." 10 U.S.C. § 3252(d)(4). Assuming without deciding that a domestic company can be an "adversary," the plain text of the statute is directed at covert acts or hacks, not overt positions taken during contract negotiations. Indeed, it is difficult to understand how one could sabotage, maliciously introduce an unwanted function, or subvert an information technology system by publicly announcing usage restrictions or insisting on such restrictions in conversations with DoW. Defendants appear to be taking the position that any vendor who "push[es] back" on or "question[s]" DoW becomes its "adversary." (Dkt. No. 128 at 41.) That position is deeply troubling and inconsistent with the statutory text.
The legislative history confirms that Section 3252's purpose is to address covert acts, hacks, and other acts of technical sabotage. Section 3252 was originally enacted as Section 806 of the 2011 National Defense Authorization Act. Pub. L. No. 111-383, § 806 (2011). In explaining why Section 806 should be enacted, the Senate Armed Services Committee Report relied upon a report from the Department of Defense about "trusted defense systems." Specifically, the Committee explained:
In that report, the Secretary found that the globalization of the information technology industry has increased the vulnerability of the Department of Defense (DOD) to attacks on its systems and networks. The report found an increasing risk that systems and networks critical to DOD could be exploited through the introduction of counterfeit or malicious code and other defects introduced by suppliers of systems or components. The committee concludes that the Secretary should have the authority needed to address this risk.
S. Rep. No. 111-201, at 162 (2010). Anthropic's acts do not appear to bear any relation to the conduct at issue in Section 3252.
Defendants point to Anthropic's "hostile manner" in the press and in contract negotiations as raising a risk that it might sabotage national security systems in the future. (Dkt. No. 96-2 at 7–8.) But Defendants do not explain why they infer from Anthropic's forthright insistence on the usage restrictions that it would become a saboteur. Indeed, sabotage would ordinarily be a surprising culmination to months of "cordial" negotiations.
Defendants argue that Anthropic could "cause their software to stop working" or "unilaterally alter system guardrails and model weights without DoW consent." (Id. at 6–7.) However, nothing in the administrative record supports the conclusion, found only in the Michael Memo, that Anthropic has the technological capability to access its Claude models after they are deployed on national security systems. In fact, at oral argument counsel for Defendants acknowledged that he was unaware of any such capabilities. (Dkt. No. 128 at 39.) And if extra-record evidence could be considered, the unrebutted evidence is that Anthropic forcefully advocated for contractual limits on DoW's use of its products because it is technologically incapable of stopping DoW from using its systems in ways that it disagrees with once it hands the products over. (See Dkt. No. 6-1 ¶ 22 ("The Usage Policy is critical because technical safeguards alone cannot prevent all dangerous uses[.]").) Anthropic has submitted evidence that "Anthropic has no access to, or control over, the model as deployed or used by government customers" (Dkt. No. 115 ¶ 8), and that "Anthropic has never had the ability to cause Claude to stop working, alter its functionality, shut off access, or otherwise influence or imperil military operations" (id. ¶ 14; see also id. ¶¶ 15–17). Defendants do not dispute that evidence.
Defendants also contend that AI "technology itself" is inherently "opaque," and that updates from Anthropic could thus "introduce [] vulnerabilities" that were hard to detect. (Dkt. No. 96-2 at 7.) But the unrebutted record shows that DoW extensively tested Anthropic's updates before they were deployed in June and November 2025. (Dkt. No. 115 ¶ 10.) DoW's stated concern thus does not appear to be specific to Anthropic but instead appears to be inherent to AI technology, and indeed, to technology more broadly. Most IT vendors could presumably update their systems or introduce unwanted functions without detection, if they wanted to. Yet Section 3252 obviously does not permit DoW to designate IT vendors as "supply chain risks" whenever they ask probing questions or stubbornly insist on particular contracting terms, even if their doing so causes DoW to doubt their trustworthiness. Such a breathtakingly broad interpretation of Section 3252 would make its restrictions on the Secretary's discretion meaningless. Not only would such an approach raise serious due process concerns for the reasons already explained, but it would also effectively gut the statutory and regulatory protections for government contractors in this field. See, e.g., 41 U.S.C. § 1121; 40 U.S.C. § 121(c); 48 C.F.R. §§ 9.406-3, 9.407-3. Nothing in Section 3252's text, structure, or legislative history supports such radical interpretation of its scope.
b. The Supply Chain Designation Failed to Satisfy the Procedural Requirements of Section 3252
Section 3252 allows the Secretary of War to skip the usual procedural requirements for suspending or debarring federal contractors so long as constitutional guarantees of due process are otherwise satisfied. That approach makes sense, given Congress's focus on preventing sabotage and subversion by foreign intelligence agencies, terrorists, and other hostile actors who generally lack due process rights or would be unlikely to take advantage of them. Instead, Section 3252 and its enabling regulations create institutional safeguards—which the Secretary must complete before making a designation—to ensure that its designation is applied properly. The Supply Chain Designation failed to comply with these mandated procedural safeguards.
One of the central safeguards of Section 3252 is the requirement that the Secretary of War make "a determination in writing" that "less intrusive measures are not reasonably available to reduce such supply chain risk." 10 U.S.C. § 3252(b)(2)(B). To ensure that this first requirement is not an empty letter, the Secretary must provide the appropriate congressional committees with "a discussion of less intrusive measures that were considered and why they were not reasonably available to reduce supply chain risk." Id. § 3252(b)(3)(B). The second requirement ensures that the Secretary's analysis is preserved for the record and is subject to Congressional oversight. Only after those steps are complete may the Secretary of War designate a source as a supply chain risk.
There is reason to infer that Secretary Hegseth failed to make the required reasoned determination regarding less intrusive measures, as required under Section 3252(b)(2)(B). The record contains various declarations from Secretary Hegseth that "less intrusive measures are not reasonably available to reduce such supply chain risk." (Dkt. No. 96-2 at 2, 12, 13, 14, 15, 16, 17, 18.) But nothing in the administrative record discusses less intrusive measures or otherwise indicates that DoW engaged in the consideration that would be necessary for Secretary Hegseth to reach such a determination. Moreover, Secretary Hegseth's six identical notices to various congressional committees do not contain any "a discussion of less intrusive measures that were considered and why they were not reasonably available," as Defendants conceded at oral argument. (Id. at 12–17.) This silence in the congressional letters is particularly telling in light of the statutory requirement to describe such analysis. 10 U.S.C. § 3252(b)(3)(B). It strongly suggests that Secretary Hegseth merely stated that less intrusive measures were unavailable and failed to make the required reasoned determination. See Nat. Res. Def. Council, Inc. v. Pritzker, 828 F.3d 1125, 1135 (9th Cir. 2016) ("An agency acts contrary to the law when it gives mere lip service or verbal commendation of a standard but then fails to abide the standard in its reasoning and decision." (citation omitted)).
As a further safeguard, DoW regulations require "a joint recommendation" from specified officials "on the basis of a risk assessment by the Under Secretary of Defense for Intelligence" before a designation can be made. 48 C.F.R. § 239.7304(a). However, the risk assessment submitted by Defendants does not come from the Under Secretary of Defense for Intelligence. Instead, the risk assessment was provided by Under Secretary of War for Research and Engineering Emil Michael, who also led many of the contract negotiations with Anthropic. Under Secretary Michael's declaration states that, due to various DoW reorganizations in 2025, and because of its "institutional and subject matter expertise," his office "has assumed responsibility for providing [] supply chain risk assessments relating to AI." (Dkt. No. 96-3 ¶ 8.) However, Under Secretary Michael does not state that the Under Secretary of Defense for Intelligence was unavailable or that the relevant regulations were amended to permit him to prepare the risk assessment. Therefore, it is likely that the action violated DoW's own regulations implementing Section 3252.
In sum, Anthropic has shown that the Supply Chain Designation was likely contrary to law, both because Anthropic's conduct did not meet the statutory criteria for a supply chain risk and because the action violated the institutional checks and balances required under Section 3252 and its implementing regulations.
2. The Hegseth Directive Was in Excess of Statutory Authority and Contrary to Law
Defendants submit no evidence that Secretary Hegseth had complied with the statutory prerequisites of Section 3252 prior to issuing the Hegseth Directive. The determinations provided by Defendants all occurred after the Hegseth Directive was issued. And Defendants conceded at oral argument that there is no statutory basis for Secretary Hegseth's instruction that "no contractor, supplier, or partner that does business with the United States military may conduct any commercial activity with Anthropic." (Dkt. No. 6-21 at 2.) As such, Defendants concede that the Hegseth Directive was in excess of and contrary to his statutory authority.
Instead, Defendants argue that the Hegseth Directive is not a final agency action and should be viewed as subsumed by the Supply Chain Designation. (Dkt. No. 96 at 25–26.) That reading is contrary to the plain language of the Hegseth Directive. The Hegseth Directive states that it has "immediate[]" effect with respect to the blacklisting of Anthropic and is "final" in all respects, including the mandate that DoW designate Anthropic a supply chain risk. (Dkt. No. 6-21 at 2.) Therefore, it is a final agency action because it purports to be the "consummation" of a decision-making process and because "legal consequences will flow" from it. Bennett v. Spear, 520 U.S. 154, 178 (1997) (citations omitted). Indeed, Anthropic submits evidence that major law firms promptly issued alerts warning government contractor clients to "immediately review their use of Anthropic technology . . . and prepare to deploy alternatives." (Dkt. No. 6-31 at 3; see also Dkt. No. 6-30.) Finally, nothing in the Supply Chain Designation withdraws or rescinds the Hegseth Directive or otherwise renders it non-final.
Thus, the Hegseth Directive was likely in excess of statutory authority and contrary to law.
3. The Supply Chain Risk Designation and the Hegseth Directive's Blacklisting of Anthropic Were Arbitrary and Capricious
An agency decision is unlawfully arbitrary and capricious if the agency has relied on factors which Congress has not intended it to consider, entirely failed to consider an important aspect of the problem, offered an explanation for its decision that runs counter to the evidence before the agency, or is so implausible that it could not be ascribed to a difference in view or the product of agency expertise. Motor Vehicle Mfrs. Ass'n of the U.S., Inc. v. State Farm Mut. Auto. Ins. Co., 463 U.S. 29, 43 (1983). Where the official "explanation for agency action [] is incongruent with what the record reveals about the agency's priorities and decisionmaking process," and there is a "disconnect between the decision made and the explanation given," an agency has not provided a reasoned explanation, and its action is likely arbitrary and capricious. See Dep't of Com. v. New York, 588 U.S. 752, 783–85 (2019).
The administrative record shows that Defendants' proffered reasons for designating Anthropic a supply chain risk and blacklisting the company with private defense contractors are both implausible and contrary to the evidence for the reasons already discussed. Sections V.B & V.C.1–2. Likewise, Defendants' apparent failure to consider less intrusive measures indicates that they likely "failed to consider an important aspect of the problem" that Congress required them to consider. Section V.C.1–2.
Moreover, the proffered reasons appear pretextual. The timing of the administrative record raises an inference that it was generated after the fact to justify the foreordained conclusion ordered in Secretary Hegseth's mandate to "designate Anthropic a Supply-Chain Risk to National Security" on February 27. (Dkt. No. 6-21 at 2; Dkt. No. 96-2.) The undisputed evidence shows that, in the parties' multi-year working relationship and during Anthropic's extensive national-security vetting process, Defendants never raised any supply chain risk concerns but instead consistently praised Anthropic. DoW appears to have raised such concerns for the first time in conjunction with the February 27 instruction. Indeed, all the records Defendants introduce in support of the designation are dated March 2–3, 2026. (Dkt. No. 96-2 at 2–9; Dkt. No. 120-1 ¶ 19.)
The inference of pretext is further reinforced by the inconsistencies in the record. Defendants have introduced evidence that on March 2–3, 2026, Under Secretary Michael and other DoW leaders finalized various recommendations, memos, and determinations in support of designating Anthropic an "unacceptable national security threat." (Dkt. No. 96-2 at 8.) Yet the day after the designation was finalized—and before it was communicated to Anthropic—Under Secretary Michael and Amodei cordially exchanged drafts of Anthropic's usage terms, with Under Secretary Michael writing to Amodei: "After reviewing with our attorneys and seeing your last draft (thanks for being fast), I think we are very close here." (Dkt. No. 114-1 at 2.) It is exceedingly difficult to square this correspondence with Under Secretary Michael's contemporaneous characterization of Anthropic as a "hostile" company presenting an "intolerable" risk to covered systems and an "unacceptable national security threat." (Dkt. No. 96-2 at 8.) It is also difficult to harmonize the concerns discussed in the Michael Memo with Secretary Hegseth's threat, three days earlier, to designate Anthropic as essential to national security under the Defense Production Act. (Dkt. No. 6-2 ¶ 16.) "Altogether, the evidence tells a story that does not match the explanation" given. Dep't of Com., 588 U.S. at 784.
In sum, the contradictory positions, the procedural defects, and the rushed process following a public declaration of the foreordained conclusion all indicate that the actions were arbitrary and capricious. See, e.g., Nat'l TPS All. v. Noem, 166 F.4th 739, 774 (9th Cir. 2026) (Mendoza, J., concurring) ("Taken together, these deficiencies paint a picture of agency action that was not the product of reasoned decision-making, but of a rushed and pre-determined agenda masked by pretext."). While arbitrary and capricious review is deferential, it is not an "empty ritual." Dep't of Com., 588 U.S. at 785. Agencies must "offer genuine justifications for important decisions, reasons that can be scrutinized by courts and the interested public. Accepting contrived reasons would defeat the purpose of the enterprise." Id.
Thus, Anthropic has shown a likelihood of success on the merits of its arbitrary and capricious claim.
D. Anthropic Has Shown a Likelihood of Irreparable Harm
"It is well established that the deprivation of constitutional rights unquestionably constitutes irreparable injury." Hernandez v. Sessions, 872 F.3d 976, 994 (9th Cir. 2017) (quotation marks and citations omitted). Anthropic has shown that it is experiencing ongoing constitutional injuries. With respect to its First Amendment claim, Anthropic remains subject to the likely retaliatory Challenged Actions, and Defendants do not dispute that the Actions would chill a person of ordinary firmness from continuing to engage in the protected activity. Anthropic is not required to show that it has succumbed to the chilling effects of the Challenged Actions in order to be entitled to injunctive relief. Ariz. Students' Ass'n, 824 F.3d at 867.
Anthropic's due process injuries are likewise ongoing and significant. The Presidential Directive bars Anthropic from all present and future federal government contracting work despite the fact that, to this day, Anthropic has received no meaningful notice regarding the factual basis on which it was debarred and has had no pre- or post-deprivation opportunity to object to the debarment. As a result, Anthropic is likely to experience a "loss of opportunity to pursue [its] chosen profession," which "constitutes irreparable harm." See Ariz. Dream Act Coal. v. Brewer, 757 F.3d 1053, 1068 (9th Cir. 2014) (citations omitted). The Supply Chain Designation is also likely to cause irreparable harm to Anthropic's liberty interests. Defendants have already started implementing the exclusion authorized by the designation, despite not giving Anthropic any pre-deprivation process. As a result, Anthropic has shown that it is experiencing a "loss of control over business reputation and damage to goodwill," which "constitute[s] irreparable harm." Herb Reed Enters., LLC v. Fla. Ent. Mgmt., Inc., 736 F.3d 1239, 1250 (9th Cir. 2013) (citing Stuhlbarg Int'l Sales Co. v. John D. Brush & Co., 240 F.3d 832, 841 (9th Cir. 2001)).
Anthropic has submitted concrete, non-speculative evidence of the ongoing harms to its liberty interests. Within days, many large enterprise customers signaled that publicly doing business with Anthropic over competitors was not worth it. (Dkt. No. 6-4 ¶ 16.) Three government contractor customers terminated their contracts with Anthropic, or were instructed to do so by Defendants; three deals valued at over $180 million fell apart despite being on the verge of closing; potential partners demanded additional protective contractual provisions such as unilateral termination; customers asked to cut short their contracts or reduced their amount, in some instances specifically mentioning the Challenged Actions; and others switched from Claude to competing generative AI tools. (Dkt. No. 6-4 ¶¶ 11–19; Dkt. No. 6-3 ¶ 33.) Anthropic's Chief Financial Officer, Krishna Rao, projects that, depending on how broadly Anthropic's customers interpret the Challenged Actions, Anthropic could lose between hundreds of millions and multiple billions of dollars in 2026 revenue. (Dkt. No. 6-5 ¶ 6.) Moreover, Defendants do not contest that Anthropic will be unable to obtain compensatory relief from the government, making its economic harm likely irreparable. E. Bay Sanctuary Covenant v. Biden, 993 F.3d 640, 677 (9th Cir. 2021); California v. Azar, 911 F.3d 558, 581 (9th Cir. 2018).
As discussed above, Defendants disclaim any present intent to enforce the portion of the Hegseth Directive stating, "effective immediately, no contractor, supplier, or partner that does business with the United States military may conduct any commercial activity with Anthropic." See supra Section II.F. However, Defendants declined to stipulate to enjoin this portion of the Hegseth Directive, asserting that they were "continuing to assess the situation" and "will take action as needed . . . to mitigate the risk from Anthropic." (Dkt. No. 128 at 18–19.) Because Defendants appear to be reserving their rights with respect to the above statement, Anthropic has shown a sufficiently "immediate" danger of irreparable harm. Cf. Boardman v. Pac. Seafood Grp., 822 F.3d 1011, 1023 (9th Cir. 2016) (holding that the "threat of irreparable harm [was] sufficiently immediate to warrant preliminary injunctive relief," notwithstanding a proposed stipulation by defendants, because of the "limited nature" of the stipulation offered).
Anthropic has shown that it is likely to suffer irreparable harm absent injunctive relief.
E. The Balance of Equities and Public Interest Favor Anthropic
A likelihood that constitutional violations have occurred weighs heavily in favor of an injunction, because "all citizens have a stake in upholding the Constitution." Hernandez, 872 F.3d at 996 (quoting Preminger v. Principi, 422 F.3d 815, 826 (9th Cir. 2005)). In such circumstances, "the merged third and fourth factors [tip] decisively in [a plaintiffs'] favor." Washington v. Trump, 145 F.4th 1013, 1037 (9th Cir. 2025) (quoting Baird, 81 F.4th at 1042).
In opposition, Defendants focus on one specific harm that they believe the government will suffer because of the preliminary injunction: the risk that "an AI model used in national security systems [would] be sabotaged by a hostile and untrustworthy corporate owner." (Dkt. No. 96 at 36.) However, the preliminary injunctive relief that the Court authorizes does not require the government to continue to use Claude on its national security systems. Therefore, this harm does not weigh against granting injunctive relief. Defendants also briefly argue that the public interest is harmed if the government is unable to "effectuat[e] statutes enacted by representatives of its people." (Id. at 37.) But "the rule of law is secured by a strong public interest that the laws 'enacted by their representatives are not imperiled by executive fiat.'" Washington, 145 F.4th at 1037 (quoting E. Bay Sanctuary Covenant v. Trump, 932 F.3d 742, 779 (9th Cir. 2018)). The government "cannot suffer harm from an injunction that merely ends an unlawful practice." Rodriguez v. Robbins, 715 F.3d 1127, 1145 (9th Cir. 2013).
The balance of equities and public interest therefore decisively favor Anthropic. As already discussed, the financial and reputational harm that Anthropic is experiencing as a result of the likely unlawful Challenged Actions risk crippling the company. In addition, amici have credibly described significant harms to the public interest if injunctive relief is denied. A brief submitted by military leaders warns that the Challenged Actions "will materially detract from military readiness and operational safety." (Dkt. No 58 at 14.) Employees from other major companies in the AI field explain that the Challenged Actions threaten to "chill open deliberation" and "professional debate" amongst the people best positioned to understand AI technology and its potential for "catastrophic misuse." (Dkt. 24-1 at 6, 8, 9.) A nonprofit advocacy group for small developers explains that their members are particularly vulnerable to sudden, large changes in government procedures, and the confusion and ambiguity created by the Challenged Actions will impose significant costs on them. (Dkt. No. 73 at 4–5.)
For all these reasons, the balance of equities and public interest favor Anthropic.
VI. REQUEST FOR BOND AND FOR A STAY
In light of Anthropic's non-opposition, Defendants' request for an administrative stay for seven days to allow the United States to seek an emergency, expedited stay from the court of appeals is GRANTED.
Defendants also seek to have the Court impose a bond under Federal Rule of Civil Procedure 65(c) because "any preliminary relief would potentially mandate that the Executive spend money on unneeded and unwanted uses of Anthropic's technology that may be lost forever once distributed." (Dkt. No. 96 at 39.) Under Rule 65(c), it is up to the Court's discretion what amount of security is required, if any. Johnson v. Couturier, 572 F.3d 1067, 1086 (9th Cir. 2009). The preliminary injunction does not mandate that the government purchase Anthropic's products, so Defendants have not made any showing of the costs or damages they will sustain as a result of the injunction. Id.; see also Fed. R. Civ. P. 65(c). In light of Anthropic's showing on the merits, and the lack of evidence of harm to Defendants, the Court sets a nominal bond of $100.
VII. SCOPE OF INJUNCTIVE RELIEF
Anthropic seeks to enjoin the following Agencies: DoW, U.S. Department of State, HHS, U.S. Office of Personnel Management, U.S. Nuclear Regulatory Commission, U.S. Department of the Treasury, U.S. Department of Commerce, U.S. Social Security Administration, U.S. Department of Homeland Security, Securities and Exchange Commission, National Aeronautics and Space Administration, U.S. Department of Energy, Federal Reserve Board of Governors, National Endowment for the Arts, Federal Housing Finance Agency, U.S. Department of Veterans Affairs, and GSA (together with their respective agency heads, the "Defendant Agencies"). Anthropic also seeks to enjoin the Executive Office of the President ("EOP").
Anthropic has introduced evidence that all of the Defendant Agencies either (1) "have contracts to use Anthropic's technology or use Anthropic's technology through a third-party provider," (2) "have indicated they would be terminating the contracts or terminating uses of Anthropic's technology since February 27, 2026," or (3) facilitate other agencies' access to Anthropic's technology. (Dkt. No. 126 ¶ 3–5.) Because the Presidential Directive orders all federal agencies to immediately and permanently cease all use of Anthropic's technology, Anthropic has made the necessary showing that the Court may enjoin each of the Defendant Agencies from implementing the Presidential Directive. Anthropic has also shown that it is entitled to injunctive relief against DoW and Defendant Secretary Hegseth with respect to the Hegseth Directive and the Supply Chain Designation.
At oral argument, Counsel for Anthropic indicated that it was seeking an injunction against the EOP because of "its likely role in carrying out" the Presidential Directive. (Dkt. No. 128 at 47.) However, because there is no record evidence that EOP has taken any action to carry out the Presidential Directive, no injunctive relief will issue as to EOP at this time. Like all other persons, EOP is barred from acting for, with, by, through, or under authority from any enjoined Defendant, or in concert or participation with any enjoined Defendant, in any manner inconsistent with the preliminary injunction order.
VIII. CONCLUSION
For the forgoing reasons, Anthropic's Motion for Preliminary Injunction (Dkt. No. 6) is GRANTED as modified. A separate order concerning Anthropic's request for a Section 705 stay and preliminary injunctive relief will issue, but is hereby stayed for seven days.
IT IS SO ORDERED.
Dated: March 26, 2026
RITA F. LIN
United States District Judge
2026-03-28 08:28:49
(crossposted from my substack)
Being a freshman at university, I seem to have been bestowed the great privilege of infinite possibilities. There is this strange feeling of trying to plan a career in a world that might not exist in five years. Not in a doomer sense but that the world of 2031 might be so radically different from today that most attempts of planning are incoherent. I contemplate machine god super intelligence arriving before I graduate, intelligence explosions compressing 10,000 years of progress into one, the possibility of being among the last humans to die before death itself is solved; then I do my laundry and pick electives. I am surprised I am not entirely losing my mind.
Robin Hanson writes, in 2009, This is The Dream Time:
Perhaps most important, our descendants may remember how history hung by a precarious thread on a few crucial coordination choices that our highly integrated rapidly changing world did or might have allowed us to achieve, and the strange delusions that influenced such choices. These choices might have been about global warming, rampaging robots, nuclear weapons, bioterror, etc. Our delusions may have led us to do something quite wonderful, or quite horrible, that permanently changed the options available to our descendants.
…
Our dreamtime will be a time of legend, a favorite setting for grand fiction, when low-delusion heroes and the strange rich clowns around them could most plausibly have changed the course of history. Perhaps most dramatic will be tragedies about dreamtime advocates who could foresee and were horrified by the coming slow stable adaptive eons, and tried passionately, but unsuccessfully, to prevent them.
This post includes some of my thoughts for other people in a similar position, who take these possibilities seriously enough to let them reshape their decisions, but aren’t sure how. Specifically: how to translate a probability distribution over timelines into an actual plan for what to do with the next few years of your life, when the wrong choice might round your impact to zero. Even the right choice might. But doing nothing guarantees it.
I'm relatively uncertain about most of what follows. Tell me where I'm wrong.
Expected value calculation under timeline uncertainty
A lot of career decisions under AI timeline uncertainty seem to reduce to weird, complicated calculations that are running in the background. Here I try to write one down. Real career choices are certainly not binary, but let’s take a simplified version that illustrates the framework of aptitude-adjusted expected value.
Say you’re choosing between two paths. Path A: get into technical research as fast as possible right now, upskill full-time in ML, try to contribute to alignment as fast as possible. Path B: stay in school, build foundations, maybe do field-building.
You think short and medium timelines are about equally as likely. Under short timelines the sprint maybe gets you 5 units of impact. You are starting behind, competing with people with varying skills and dispositions, but you are contributing something. In this short timeline world the slow investment gets you roughly 0, everything changed before anything paid off. Under medium timelines, the sprint gets you maybe 2, you contribute but at the cost of burning through your runway without deep foundations. The slower investment gets you 50,
EV(sprint-heavy) = 0.5 × 5 + 0.5 × 2 = 3.5
EV(invest-heavy) = 0.5 × 0 + 0.5 × 50 = 25
The invest-heavy orientation wins, not because medium timelines are more likely, but because the magnitude of your impact there is so much larger. The ratio matters more than the probability. Even if you’re 80% confident in short timelines, the invest-heavy approach still wins in this example. You’d need to be around 91% confident before the sprint orientation becomes the better bet. And 91% confidence in anything this uncertain should be pretty concerning.
These numbers are made up and yours should differ. Maybe you think the sprint would get you more than 5 in a short-timeline world. Maybe you think field-building is worth less than 50 in a medium-timeline world. You should run it yourself. The strategy of betting on the world where you have the most leverage, rather than the one you deem most likely, focuses on maximizing asymmetric upside—situations where your potential gains significantly outweigh your potential losses. This approach prioritizes expected value over mere probability, focusing on high-impact scenarios where you possess a distinct advantage.
What should I do if everything is learnable:
EA career advice has historically emphasized “personal fit”, find what you’re naturally good at and apply it to important problems. I think this view is sometimes wrong and slightly in contradiction with the idea that most things are learnable.
Robert Musil writes in The Man Without Qualities:
But if there is a sense of reality, and no one will doubt that it has its justification for existing, then there must also be something we can call a sense of possibility. Whoever has it does not say, for instance: Here this or that has happened, will happen, must happen; but he invents: Here this or that might, could, or ought to happen. If he is told that something is the way it is, he will think: Well, it could probably just as well be otherwise. So the sense of possibility could be defined outright as the ability to conceive of everything there might be just as well, and to attach no more importance to what is than to what is not. The consequences of so creative a disposition can be remarkable, and may, regrettably, often make what people admire seem wrong, and what is taboo permissible, or, also, make both a matter of indifference. Such possibilists are said to inhabit a more delicate medium, a hazy medium of mist, fantasy, daydreams, and the subjunctive mood. Children who show this tendency are dealt with firmly and warned that such persons are cranks, dreamers, weaklings, know-it-alls, or troublemakers.
Some evidence for malleability:
However, if you genuinely believe you can become and learn almost anything, the search space turns into infinity. This seems not conducive to decision making. Full malleability and infinitely possibilities can be pretty paralyzing.
Personally, I seem to have a comparative advantage in being able to speak Chinese, which is plausibly useful for AI governance and international coordination. But I’ve also barely done anything and have little evidence for what I might be good at. Even if I’m not maximally gifted at math, I can probably learn a good amount if I really tried. I could probably also learn a good amount of policy analysis, or ML engineering, or institutional design, or whatever. Or can I just do all of these?
Here, timeline views provide some constraints to the search space that disposition alone cannot. Short timelines suggest one to build the most immediately deployable skill and medium timelines favor investing in foundations that compound. For the same reason that the world in whole should have people betting on different timelines, you can diversify your skill profile. You don’t have to pick one timeline and go all in.
Should I have my own thoughts about timelines?
My thoughts about this are here.
Summary: The question of whether to defer or think for yourself relies on a false binary that ignores better options. You probably cannot out-predict Daniel Kokotajlo. But you should still actually understand why experts believe what they believe, and what the object-level grounds that inform timeline forecasts are. This is instrumentally important for being able to produce impactful work, maybe developing research taste, and being able to make informed decisions.
Questions and answers synchronize like meshing gears; everyone has only certain fixed tasks to do; professions are located in special areas and organized by group; meals are taken on the run. Tension and relaxation; activity and love, are precisely timed and weighed on the basis of exhaustive laboratory studies. If anything goes wrong in any of these activities the whole thing is simply dropped; something else or sometimes a better way will be found or someone else will find the way one has missed; there’s nothing wrong with that, while on the other hand nothing is so wasteful of the social energies as the presumption that an individual is called upon to cling for all he is worth to some specific personal goal. In a community coursed through by energies every road leads to a worthwhile goal, provided one doesn’t hesitate or reflect too long. Targets are short-term, but since life is short too, results are maximized, which is all people need to be happy, because the soul is formed by what you accomplish, whereas what you desire without achieving it merely warps the soul. Happiness depends very little on what we want, but only on achieving whatever it is. Besides, zoology teaches that a number of flawed individuals can often add up to a brilliant social unit.
— Robert Musil, The Man Without Qualities
Doing something is probably better than doing nothing. Doing the best thing requires both luck and calculation. You can increase your surface area for luck, but you cannot control it. AGI is super scary but don’t go cheap on actually doing object level world modeling! Revisit your assumptions frequently. Do some healthy deference when needed. There has never been a better time to get into AI safety. But before signing up for any of these things actually maybe sit down and run your own numbers. Make decisions that are good enough across the range of plausible futures, and then to act, because acting on an imperfect view beats perfecting a view you never act on.
