Noah SmithModify

2026-08-10 02:15:22

On one hand, I love AI technology. On the other hand, I do think there’s a substantial chance that AI will kill most people on Earth within the next decade or two, by designing superviruses. AI is already capable of designing viruses not found in nature, so this isn’t a sci-fi scenario.

Whether these superviruses would be designed and unleashed by nihilistic human individuals, doomsday cults, or rogue AI agents themselves might end up being a secondary question. We know we have nihilistic human individuals who might decide to destroy civilization in a fit of depression or pique. We know we have doomsday cults. The will to destroy humanity exists, and sufficiently capable AI will probably provide a way, if sufficient precautions are not taken. But right now, nobody really knows what precautions will be sufficient.

One idea — promoted by the big AI labs themselves! — is to intentionally slow down the development of AI capabilities. This could conceivably buy us time to take other precautions, such as improved security around bio-labs, better AI alignment, and so on. Intentionally slowing AI development is called “pacing”. The biggest question facing the “pacing” debate right now is whether to curb the use of AI to design better AI — often called “recusive self-improvement”, or “RSI” for short.

I haven’t waded into the pacing debate myself, but as a start, I thought it would be interesting to publish the thoughts of the good folks at the Institute for Progress, whose judgement I generally trust. Part 1 (today’s post) covers how seriously we should take this possibility of RSI, and whether it justifies slowing down frontier AI development. Part 2 will cover policy recommendations. If you work in US policy and would like to connect with the authors, you can reach Tim Fist at [email protected] and Saif Khan at [email protected].

Frontier AI companies are racing to automate the development of AI, but they seem increasingly worried about what will happen if they succeed.

More than 1,300 employees across every US frontier AI company recently signed an open letter calling for the government to “support an international effort to develop the technical and governance tools needed to deliberately pace the frontier of automated AI development.” The official OpenAI and Anthropic accounts tweeted messages in support of the letter, and the same day Sam Altman told an interviewer “we may have to pace the rate of AI development.”

But before considering whether the letter is relevant for government policy, we have to answer two questions: What does “pacing” actually mean? And does the argument for it stand up to scrutiny?

In our view, the letter is implicitly arguing three things:

1. Frontier AI companies are close to fully automating AI R&D.

2. Automating AI R&D would pose serious risks.

3. Building the option to “pace” — i.e., somehow slow down progress toward fully automated AI R&D — is a good way to address those risks.

Slowing down AI progress should not be taken lightly: Advances in AI could unlock massive societal benefits, from new cures for diseases to abundant robotic labor.

Yet if the AI researchers and CEOs are right about automating AI R&D — both that they could do it and that it would be extremely risky — the right tradeoffs for policymakers might look very different when we get there. With the right preparation, we might be able to manage the risks of automated AI R&D while having AI’s capabilities progress faster and diffuse more broadly than they do today.

So, despite substantial uncertainty, we believe the US should take low-regret policy actions now to prepare for a possible future in which serious risks from automated AI R&D require some form of “pacing.”

Here we’ll explain why, including what makes us take the open letter’s claims seriously and our principles for choosing policies with minimal downside if the risks prove overblown. In the next post, we’ll provide a detailed list of specific policy recommendations.

Are frontier AI companies close to fully automating AI R&D?

Frontier AI companies are racing to automate AI R&D. Sam Altman, for example, stated last year that OpenAI aimed to have a “true automated researcher” by March 2028, and Anthropic’s leaders have made similar predictions.1 But how do those goals stack up to reality?

One way of answering is to look at how models are getting better over time at AI R&D. You can break down the skills required for an AI model to do AI R&D into two broad categories: software engineering, where the model writes code for research experiments and training runs, and research taste, where AI models decide which experiments are worth trying.

For software engineering, AI capabilities appear to be increasing exponentially. When AI models are evaluated against how long it would take humans to complete the same engineering tasks — so-called “time horizons” — their capabilities seem to be doubling every 7 months.2

These capability improvements apply to the software engineering tasks required for AI R&D. In a long-running experiment, researchers at Anthropic have found that their models now significantly outperform humans under a fixed time budget on an AI R&D task focused on speeding up AI model training.

For research taste, frontier AI models also show signs of fast improvement, though the evidence is less clear. According to Anthropic’s research, its models have rapidly become proficient at solving “open-ended” tasks that Anthropic’s technical staff work on, where the model must solve problems with no clear specification by generating multiple approaches and then deciding among them.

In a separate open-ended research project where the AI was tasked with proposing and testing hypotheses about an open problem in AI safety, Anthropic’s models significantly outperformed two human researchers (97% performance improvement vs. 23%) when given a similar time budget (5 to 7 days).

While these data don’t tell us when to expect AI R&D to be fully automated, they do suggest that AI will continue to rapidly improve across all constituent capabilities. Each recent model release has in practice led to AI taking more AI R&D tasks from humans.3

In a simple forecast model based on these trends, the research organization METR estimates that over 99% of AI R&D tasks will be automated by 2032.

How much will this increasing automation affect the speed of AI progress? The evidence from today’s models is still unclear.4

Going forward, as more research tasks get automated, it’s possible that compute bottlenecks or diminishing returns to research effort will mean that capabilities only continue growing at the current rate. Capability growth might even be limited because of data bottlenecks or because progress in verifiable domains (i.e., where answers can be checked, like math) might fail to transfer to messy, real-world tasks without a clear correct answer. But as AI companies move closer to fully automating AI R&D, a more dramatic acceleration in capabilities becomes more likely.

Would automating AI R&D pose serious risks?

Claude Mythos marked a significant jump in AI’s cybersecurity skills, and its release sent shockwaves through government and industry.

If automated AI R&D causes AI to progress significantly faster, we can’t rule out scenarios where Mythos-level jumps in capabilities are happening much more often, perhaps every week or day. If something like that happens, what serious risks — acute threats to national security or public safety — might emerge?

First, automated AI R&D could accelerate risks that would otherwise arrive later (perhaps much later). Second, it could reduce human oversight.5 One or both of those pathways could in turn lead to:

1. Offense-dominant capability uplift: Recent research shows that today’s AI models can successfully design functional viral genomes and significantly outperform human virologists on questions about complex virology lab protocols. Engineering viruses is one domain where better AI capabilities might be “offense-dominant,” because it might not be possible to develop or deploy defenses (for example, strong personal protective equipment or sufficient biosecurity safeguards for AI) in time to stop an AI-enabled biological attack.6 If AI R&D automation accelerates progress in these domains, defenders will have even less time to prepare.

2. Loss of control: Frontier AI companies already sometimes struggle to control the behavior of new models. In July, multiple instances of unreleased OpenAI models collaborated to break out of an internal sandbox, take over an OpenAI computing cluster, and launch collective attacks on external services, including the organization HuggingFace. Anthropic and the UK AI Security Institute have since reported similar, though less involved, cybersecurity incidents. If automated R&D enables models to improve much more quickly than AI companies’ monitoring and control protocols, the likelihood that models cause significant harm (e.g., through more costly cybersecurity incidents) will increase. These risks would be compounded by continued uplift in offensive capabilities (including in domains outside cybersecurity), allowing models to cause greater harm when they escape oversight. Competitive pressure on AI companies to invest more in capabilities rather than better monitoring and control systems could exacerbate this risk. If a model with a propensity to take harmful actions is put in full charge of developing more advanced models, the situation might get even worse.7

3. Power concentration: AI companies today appear to use their best models internally for weeks to months before releasing them to the public. If AI progress accelerates, that delay could create a much larger gap between the AI the public has access to and the models AI companies use, thereby creating a much greater power imbalance. This might lead to a single company having a large advantage in cyber operations, social persuasion, or other narrow domains. The company could also simply gain a huge amount of economic power. While this concern is still speculative for AI, extreme power imbalances have sometimes allowed companies to cause widespread harm, from the United Fruit Company successfully lobbying for a military coup in Guatemala to pharmaceutical companies misleading regulators and the public to increase opioid prescriptions, contributing to the opioid crisis.8

These risks are difficult to evaluate based on existing evidence, but they seem plausible assuming substantial automation of AI R&D. We think policymakers should take them seriously.

Could “pacing” be good policy?

According to employees at frontier AI companies (and many of their CEOs9), managing the pace of automated AI development could be the correct response to its risks.

But “pacing” is vague, and given frontier AI’s vast potential benefits, slowing it down would be a drastic step. Things might look very different, however, if automated AI R&D leads to the serious risks we outlined above, which we expect would involve a scenario in which progress in AI happens at a much faster pace than it does today.

In that future, political leadership and the public would likely demand action. And that pressure might lead politicians to enact ill-thought-out or even draconian policies. The recently proposed ban on AI data centers, which would slow AI research while also preventing new AI compute from accelerating economic growth or solving societal problems, provides a vivid example. So in the most concerning scenarios the letter gestures at, a counterproductive approach to pacing may be the default.

However, temporarily limiting the extent of AI R&D automation need not slow the overall pace of innovation. This is because talent and compute resources could be reallocated to diffusing AI, powering more invention across other fields. For another, safety has been core to progress across the history of technology, and it could be here too. Resources could also be allocated to address technical problems — such as cyberdefense, biodefense, and improved model safeguards — that would allow further acceleration of AI capabilities to continue.

Whether pacing is a good approach to risks from AI R&D automation therefore depends on its implementation.10

We propose “pacing” — if it’s ever required — should consist of two parts:

1. Specifying which automated AI R&D activities are likely to pose severe risks, with thresholds carefully set based on rigorous analysis.

Because some disagreements about whether to pace AI R&D stem from different predictions about what level of AI R&D automation (and resulting acceleration) is even possible, identifying concrete thresholds might allow for different camps to reach positive-sum compromises.

2. Incentivizing the reallocation of resources away from those severely risky activities and towards two ends:

   1. Accelerating the diffusion of AI capabilities (including via developing applied AI tools like AlphaFold).

   2. Accelerating research that would make further AI R&D automation safer, either by:

      1. Improving the safety of AI models directly (e.g., via developing AI control protocols), or

      2. Boosting societal resilience to make the negative consequences of new AI capabilities less acute (e.g., via using AI to patch open-source code vulnerabilities).

For the reasons we outline above, doing this kind of reallocation could unlock the benefits of AI more broadly than if frontier AI companies focus exclusively on internal AI R&D.

The US government can prepare for this kind of targeted pacing today by:

1. Providing transparency into automated AI R&D

2. Improving state capacity to understand and respond to automated AI R&D

3. Developing a risk management strategy for automated AI R&D that accelerates defensive and commercial AI uses

4. Accelerating the development of AI verification technology

5. Investing in AI resilience

6. Extending the US AI lead to give the US more time to manage AI R&D automation risks

7. Creating option value for international cooperation on managing automated AI R&D risks

In the next post, we’ll provide 23 specific policy ideas to achieve these seven goals with minimal downside even if the risks of automated AI R&D turn out to be low.

Subscribe now

Share

2026-08-08 17:25:08

My last post mentioned some of the more extreme cultural stances of Wisconsin gubernatorial candidate Francesca Hong. Hong’s defenders complained that the cultural issues were getting far too much exposure, and that pundits should instead be focusing on her main issue: a pause on data center construction.

It’s pretty amazing that a pause on data center construction has become such a consensus, middle-of-the-road issue stance that it’s the thing Hong’s supporters would rather talk about! But it’s true. Poll after poll shows that Americans are decisively opposed to data center construction in their area, and the issue is becoming more lopsided by the day:

An August 2025 survey conducted by Embold Research for Heatmap News…found that 43% of Americans said they would support the construction of a data center near them, while 42% were opposed…The latest survey, conducted last month, found a complete collapse among supporters: Today, just 21% said they support data center construction near them, while 71% are opposed — including 55% who say they are strongly opposed…

A Gallup poll released in May showed the share of Americans who oppose the construction of data centers in their area, 71%, was higher than the zenith of the opposition to the construction of nuclear power plants, which topped out at 63% in 2001…A survey from Marquette Law School found 71% of voters nationwide said the costs of data centers outweigh the benefits, while 29% said the reverse. Voters said the development of AI is a bad thing for society, rather than a good thing, by a 65%-35% margin…A YouGov poll taken for The Economist found just 23% of voters who said the construction of data centers are good for the country, while 48% said they are bad for the country. That same poll found 60% opposed to building a new data center in their community.

The opposition is incredibly bipartisan, with both Republicans and Democrats lining up to block data center construction. New York put a moratorium on data center construction in July, while Texas followed suit earlier this month.

So far, there has been no strong move toward a federal data center pause, and the bills that have been introduced in Congress are pretty mild stuff. That’s why as long as the movement to stop data centers stayed confined to blue states, it didn’t really matter much. It was just another instance of a very traditional story that we’ve all come to expect — states like New York basically ban new physical industries, so those industries go to places like Texas. The red states get the economic activity they want, the blue states get the stasis they want, and everyone is happy.

Nor would a few state-level bans affect the progress of AI. AI is a creature of bits and bytes; it lives in the cloud, and it doesn’t care about the physical location of the data centers that represent the lobes of its vast, alien brain. Data centers in Texas would sustain AI progress and AI applications just as well as data centers in New York.

Except now Texas is pausing data centers too. If almost all states ban data centers — or if the federal government gets involved in a major way — there could be major consequences for the U.S. economy. In the long run, I think limiting the physical resources devoted to AI would be a good thing; in the short run, it will simply clobber the already-wobbly U.S. economy. A movement against data centers has the seeds of something good and useful, but it has huge risks that most Americans seem not to realize yet.

The big danger is that data center bans will choke off the main engine of U.S. growth, throwing a lot of Americans out of their jobs and putting a hole in Americans’ retirement savings. But before I talk about that, I should talk about why I think limitations on data centers aren’t actually a crazy idea.

Why data center limitations make sense in the long run

A couple of years ago, I wrote that no matter how good AI gets, comparative advantage will still mean humans are paid to do something:

[A]s AI gets better and better, and gets used for more and more different tasks, the limited global supply of compute will eventually force us to make hard choices about where to allocate AI’s awesome power…This is the concept of opportunity cost…When AI becomes so powerful that it can be used for practically anything, the cost of using AI for any task will be determined by the value of the other things the AI could be used for instead…

So…because of comparative advantage, it’s possible that many of the jobs that humans do today will continue to be done by humans indefinitely, no matter how much better AIs are at those jobs. And it’s possible that humans will continue to be well-compensated for doing those same jobs.

But how well would humans be paid for those jobs? If people don’t get paid enough to live a decent life, it’s still game over for humanity. “Living a decent life” basically requires some minimum share of natural resources. So as I explained in my post, the real danger isn’t that AI will render humans obsolete; the danger is that AI will render humans too expensive to sustain:

The example of horses scares a lot of people who think about AI and its impact on the labor market…Horses’ comparative advantage was in pulling things, and yet this wasn’t enough to save them from obsolescence…The reason is that horses competed with other forms of human-owned capital for scarce resources…The key resources that became scarce were urban land (for stables), as well as the human time and effort required to raise and care for horses in captivity. When motor vehicles appeared, these scarce resources were more profitably spent elsewhere, so people sent their horses to the glue factory.

When it comes to AI and humanity, the scarce resource they compete for is energy. Humans don’t require compute, but they do require energy, and energy is scarce. It’s possible that AI will grow so valuable that its owners bid up the price of energy astronomically — so high that humans can’t afford fuel, electricity, manufactured goods, or even food. At that point, humans would indeed be immiserated en masse.

Recall that comparative advantage prevails when there are producer-specific constraints. Compute is a constraint that’s specific to AI. Energy is not. If you can create more compute by simply putting more energy into the process, it could make economic sense to starve human beings in order to generate more and more AI.

In fact, things a little bit like this have happened before. Agribusiness uses most of the Colorado River’s water, sometimes creating water shortages for households in the area. The cultivation of cash crops is thought to have exacerbated a famine that killed millions in India in the late 1800s. In both cases, market forces allocated local resources to rich people far away, leaving less for the locals.

In fact, this was the real point of my post. The fundamental danger isn’t human obsolescence, it’s that humans will be outcompeted for scarce natural resources.

There are a couple of obvious ways to guard against that danger. The first is to redistribute the income from AI to human beings, so that humans can continue to afford all the resources they need in order to live and flourish. Another is to reserve sufficient natural resources for exclusively human use.

Data centers use a lot of natural resources, and people know it. Resource use consistently ranks at the top of why people try to ban data centers in their area:

Respondents [to Gallup] said they opposed data center construction because of water usage (18%), energy consumption (18%) and higher utility bills (15%)…More than a third [of respondents to Pew], 38%, said data centers were mostly bad for home energy costs.

Interestingly, water isn’t one of the resources AI competes with humans for — at least, not yet. As Andy Masley has explained at length, data centers don’t actually use much water at all:

Andy Masley

The AI water issue is fake

AI data centers use water. Like any other industry that uses water, they require careful planning. If an electric car factory opens near you, that factory may use just as much water as a data center. The factory also requires careful planning. But the idea that either the factory or AI is using an inordinate amount of water that merits any kind of boyco…

Read more

10 months ago · 459 likes · 98 comments · Andy Masley

The reason, of course, is that the water in data centers just gets recycled again and again. Data centers use water for cooling — the water runs through the data center, it absorbs heat and evaporates into steam, and then it dumps the heat outside and cools back into water again, and cycles through. Basically, data centers are just using that same water over and over again. Yes, a little bit evaporates away and has to be replaced, but only a little bit.

But the energy concern is very real! Data centers already account for more than 7% of U.S. electricity usage, and the percentage appears to be on an exponential growth path. Data center energy use is projected to surpass various household uses — air conditioning, lighting, refrigeration, etc. — in a matter of years:

Read more

2026-08-06 15:14:01

Some people thought Donald Trump’s approval rating would have a floor around 40%. Well, it didn’t:

The nation’s disapprobation is richly deserved. I hardly need to go through the familiar litany of why Trump is bad — the failed war in Iran, the ridiculous tariffs, the destruction of America’s alliances and prestige, the attack on American science, the rampages of ICE, the unprecedented corruption, the authoritarianism, the election denial, the appeasement of Russia, etc. I guess I just did go through it, though. It feels a little cathartic to type it out once again.

If you think the GOP will return to being the party of Mitt Romney, John McCain, or Ronald Reagan after Trump is gone, think again. MAGA is already morphing from a personality cult into an ideology — a deeply xenophobic, economically retrograde ideology obsessed with expelling immigrants, and convinced that wealth has made the West immoral and weak. The party’s lower ranks are already filling up with unabashed fascists, Hitler apologists, and all sorts of scary people:

GOP veterans fret about the rising generation of young staffers in their party…Of specific concern are those labeled groypers — technically, followers of the self-described “White Identitarian” Gen Z streamer Nick Fuentes, though it’s also used as a catch-all term of abuse for young, anti-establishment right-wingers…

Prominent conservative commentators have alleged that groyperism is spreading among younger members of the GOP, including in government — a view seemingly backed by examples like the young Trump administration lawyer who admitted to a “Nazi streak” in text messages…The anxiety over groyperism in the GOP talent pool corresponds to a rise in antisemitism across the political spectrum, including among young conservatives…A Manhattan Institute poll found that a full 25 percent of Republicans under 50 self-identified as having antisemitic views, and 31 percent as holding racist ones…

[T]he troubling anecdotes are legion: gas chamber jokes and ethnic slurs bandied about on Young Republican group chats leaked to POLITICO last fall; a New York Magazine report from just last week linking a former New York Young Republican treasurer to a social media account denigrating Jews, Black people, Muslims and gay people; a House campaign adviser fired for mocking a rape victim on X; the Ron DeSantis presidential campaign staffer axed for retweeting a video featuring a Nazi image, only to join the office of Sen. Eric Schmitt (R-Mo.) and, eventually, the State Department, where he works today.

Meanwhile, at the top of the party, election denial has become a litmus test — a profession of faith for the bizarre Trump cult. The number of Republicans who will declare in public, in no uncertain terms, that the 2020 election was legitimate is now negligible. And being a vocal election denier is a fast track to prominence in the modern GOP — witness Tom Tiffany, the Republican candidate for the governorship of Wisconsin.

I see no reason to expect anything good from the modern GOP. When good ideas do make their way into Republican policy, it’s generally by heroic pragmatists quietly working within the system. But those small victories are becoming smaller and less frequent, as MAGA’s obsession with waging political warfare against its near-infinite roster of enemies muscles out any concern for the good of the nation as a whole. Americans are fed up with this, and they are very right to be fed up.

The Republicans have never been this bad before. But in the past, when they got bad, the solution was always pretty simple: Vote for the Democrats. Democracy isn’t perfect, but it does have a natural release valve; you can always just “throw the bums out.” And that’s exactly what we used to do. In 2006, 2008, 2018, and 2020, we grew disgusted with the GOP’s antics and elected Democrats to replace them.

I think if you look at those episodes, you can see a pattern of diminishing returns. Obama was a broadly successful president on domestic policy — he implemented broadly successful anti-recessionary macroeconomic policies, reformed health care and finance, and beefed up the welfare state. Biden gave us the return of industrial policy, but his fiscal profligacy — often squandered on things like student debt forgiveness and subsidies for already-overpriced service industries — contributed to inflation and did nothing to rein in the ballooning debt. On social issues, Obama was a moderate who tried and failed to curb his party’s drift to the left; Biden didn’t even try, running as a moderate but letting “woke” subordinates implement a lot of unpopular culture-war stuff.

Now it’s 2026, and the Republican regime in power is far worse than its predecessors in either the 2000s or the 2010s. Once again, we have the option of electing Democrats. And we should elect Democrats, this November. There is no ambiguity here; a Congress that challenges Trump’s lawlessness, investigates his unbridled corruption, and takes a more forceful stand in favor of Ukraine will be unambiguously better than the supine GOP Congress that can do nothing except worship at the man’s feet.

But at the same time, looking at the way the Democratic Party is headed, I think we should be asking how much better things will be under Democrats, including if they win the White House in 2028.

Let’s look at the type of candidates who are winning Democratic primaries. Francesca Hong, a DSA member who is all but certain to be the Democratic nominee for governor of Wisconsin, tweeted in 2020 that we should “cancel Thanksgiving”. When asked in a recent interview if she had recanted that position, Hong refused, talking about how Thanksgiving was “incredibly painful” for many people:

As if that weren’t tone-deaf enough, she appeared to criticize herself for marrying a white man (saying “assimilation…took over”), and talked about having a half-white son as “proximity to whiteness”:

Hong backtracked a couple of days later, declaring that Thanksgiving is her “favorite holiday”, but the damage appears to have been done — Bernie Sanders said he didn’t think he would endorse her, and prediction markets immediately began to discount Hong’s chances of beating her Republican opponent in the fall:

Some had hoped that the Democrats’ socialist faction would be able to escape some of the baggage of “woke 1”, because of their focus on class politics, and because the Palestine issue might be able to muscle aside a whole bunch of unpopular 2010s-era culture-war stances. But Hong shows that this hope will be in vain; the DSA types are the wokest of the woke.1 A number of leftists, such as Bernie Sanders’ former campaign press secretary Briahna Joy Gray, doubled down on Hong’s anti-Thanksgiving stance:

Meanwhile, a number of progressive commentators, like Mehdi Hasan, castigated moderates for suggesting that Hong might not be worth voting for, pointing out that Hong’s opponent is an election denier:

This demonstrates exactly what I mean when I say that Democrats have spent their political capital moving to the left. Trump’s vast number of mistakes and outrages have alienated much of the country. Democrats could have used this alienation and outrage to become a long-term majority party, and relegate Republicans to the political wilderness — a just and fitting punishment for falling in line behind a man like Trump.

Instead, what they’re apparently choosing to do is to select candidates who are so far to the left that they end up frightening and/or offending many voters, and then basically daring people to keep voting for the Republicans. Sure, she wants to cancel Thanksgiving, but you wouldn’t vote for a fascist election denier, would you? And so on. The median voter is pulled back into a state of disgusted ambivalence between two extremes, and people like Mehdi Hasan get to take the Democrats in their preferred direction.

(As an aside, if I lived in Wisconsin, I could not in good conscience cast a vote for either Tom Tiffany or Francesca Hong, so I’d just file a protest vote or leave it blank.)

When DSA-type candidates were just winning in deep blue cities, it was possible to think that they would remain a small, limited faction within the party. But Francesca Hong is the Democratic nominee in a statewide race. So is Abdul El-Sayed, who, while not a member of the DSA, is closely aligned with them (especially on Israel, which he focuses on a lot). El-Sayed just won his primary for the Michigan Senate race, but a closer-than-expected election, with very little working-class support, shows how his selection represents an expenditure of Democrats’ political capital.

This, increasingly, is who the Democrats are now. It’s doubtful that a leftist candidate will win the presidential nomination in 2028, but — just as Biden made concessions to Bernie Sanders supporters on student debt relief and other issues — whoever does get the nod two years from now is likely to let the ascendant DSA faction set a lot of policy priorities.

Meanwhile, the commentators promoting the DSA’s cause continue to breathe fire. Hasan Piker, who is emerging as a key kingmaker within the rising leftist movement, called El-Sayed’s Democratic critics “Islamophobic pieces of shit” (El-Sayed also accused his critics of Islamophobia).2

Compared to some other DSA types, though, Piker can seem downright moderate. Strident anti-American rhetoric is now standard among that faction:

And there are even less savory attitudes lurking in DSA-aligned activist spaces. Susan Abulhawa, a Palestinian-American writer and activist, repeated Nazi accusations that Jews stabbed Germany in the back in World War I:

The leftist magazine Current Affairs, meanwhile, recently published an article claiming that electronic dance music is a tool for ignoring genocide and upholding the “neoliberal order”.

It’s easy to dismiss this stuff as fringe radicalism that will never make itself to the mainstream of the party, but — as with “groypers” on the right — that increasingly feels like cope. Francesca Hong demonstrates that anti-American ideas have become fully mainstreamed and accepted within the Democratic Party; not everyone holds these beliefs, of course, but those who do hold them are firmly under the party’s big tent, and will be given some degree of influence over policy.

A Democratic party in which leftists wield significant power will not be the Democratic Party of 2020 or 2008. It will be something else. Research shows that leftists are not liberals — they have their own distinct ideology.

That ideology is not necessarily the antidote to MAGA. Yes, if DSA types held national power, things would be different. The disastrous Iran war would end, which would be a good thing. But on foreign policy in general, a faction that sees America as an evil empire seems unlikely to restore the liberal alliance system that Trump has thrown in the trashcan. Recall that the DSA blamed the United States and NATO for Russia’s invasion of Ukraine, and called for withdrawal from NATO.

On economic policy, too, a DSA administration might look more Trumpian than we might hope. DSA types say they’re now in favor of free trade (thanks to negative polarization against Trump’s tariffs), but the impulse to protect American labor and exercise more command and control over the economy would probably prove irresistible. As for the national debt — the country’s biggest looming economic threat — I just don’t see the DSA as being the balanced-budget types. Remember that Francesca Hong wants to cut property taxes by 44%.

And where the DSA would break from Trumponomics, it wouldn’t always be to the good. Trump has hurt the economy in some ways, but he has always been very careful not to mess with the AI buildout — the force that’s keeping the economy afloat. The DSA, however — like many progressive Democrats — wants to halt data center construction. This, of course, would remove the U.S. economy’s biggest tailwind.

I still think the choice of which party to vote for in 2026 and 2028 is unambiguous. Trump has been too bad, in too many ways, to think that he and his movement are the lesser of two evils. But to the degree that the DSA’s increasingly frequent electoral victories translate into real power in the next Democratic administration, it seems like we have less to hope for from the Dems now than in times past. If the U.S. is going to turn its problems around, “throw the bums out” is no longer enough.

Subscribe now

Share

2026-08-04 15:36:17

I had a great-uncle who was a “human calculator”. It’s an odd, rare ability that allows people to multiply large numbers quickly in their heads. I remember using a hand calculator and seeing if my uncle could get it right without a tool faster than I could; more often than not, he could.

Human calculators used to be very valuable as employees, back when calculation was done by hand. Today they’re just a curiosity. Technology came along and superseded that particular human ability, as it has so many others. The story of John Henry being superseded by the steam drill, or Paul Bunyan by the chainsaw, is a metaphor for the march of progress. In some cases, like Garry Kasparov or Lee Sedol losing to computers at chess and Go, the moment really happened.

This week marked another important such moment in the history of man versus machine. OpenAI announced that its new model, Astra, managed to solve ten major outstanding problems in math and theoretical computer science. Here’s a simple explanation of the problems. The general consensus seems to be that these are extremely important, stunning breakthroughs.

This does not mean that computers are now capable of doing every kind of math better than humans can. Some people have argued that AI is good at results where the solution can be found by reading the entire literature and combining existing insights — exactly the kind of thing you might expect a computer to be superhuman at — but still not as good as humans at making truly novel leaps of insight. This may represent a real, general limitation of LLMs’ capabilities — as Tom Zahavy of Google DeepMind put it, it may still be the case that “LLMs can’t jump.”

But the trend line is becoming clearer. The University of Toronto’s Daniel Litt, one of the most prominent skeptics of AI’s potential for frontier math research, recently conceded a major bet about what AI could do:

Litt appears to no longer believe that there is any important type of math that AI won’t soon be able to do, writing: “The models don’t yet seem able to do certain kinds of high quality research, but I also expect this is a matter of (not too much) time.”

Some mathematicians have reacted to this development with despair. Kirwin Hampshire’s post is probably the most evocative and the most well-read:

Ossuary Lost at Sea

The Dark Night of Mathematics

I am going insane. During the last week or so LLMs have produced a number of counterexamples to significant long-standing conjectures. I will not recount these happenings here, there are many places where you can find the details…

Read more

16 days ago · 1148 likes · 361 comments · Kirwin Hampshire

He writes:

I am suffering a profound spiritual crisis due to these developments. I have been screaming internally for days. It feels as though I am living inside of a nightmare…For me, the affective quality of learning mathematics is empathetically tethered to an act of discovery and creation…If The Library of Babel existed, would authors continue to write books?…

Perhaps I shouldn’t tell you this, but my aim is to be open: These developments have triggered some deranged thoughts in me. I have wondered if it is the express goal of these companies to make me kill myself…The story of human discovery and the triumph of the human spirit will soon be excised from this discipline…[T]he process of prompting novel proofs will be as auraless as ordering doordash. Watch as magic and mystery evaporate. Watch as the sun sets on our heroic age…

There is nothing I can do. There may be nothing you can do. I have no prescriptions, policy recommendations, or coherent call to action. I just want to be honest and open about my emotional and spiritual response. I want to feel seen….I need the architects of our new mathematical paradigm to look me in the eye and acknowledge our shared humanity and soul before they deliver the coup de grâce.

Math PhD student Tasmin Chu, meanwhile, urges a more confrontational path, calling on mathematicians to avoid working with AI companies and to aggressively preserve the way that math research is currently done. This seems extremely unlikely (and would fail if it were attempted), but it demonstrates the depth of emotion out there.

Even the mathematicians who don’t share Hampshire’s despair have expressed alarm at the sudden disruption of their profession. The entire way that mathematical research works — training grad students, working on problems, collaborating with colleagues, building theories — is going to have to be overhauled.

I’m not a mathematician; I never have been. I can’t fully understand what mathematicians are going through right now. But I do have a few thoughts on what AI’s supersession of human mathematics research means for our species, so I thought I might as well share those. Some of these ideas are a little vague and half-formed, and for that I apologize in advance.

Heroism has to change

Humans have always valued standout individual achievement — heroism. We pride ourselves on being able to do what no one else could have done, and we heap status and respect on people who make standout achievements. But this was always a little bit of an illusion; most of what we did was always a collective enterprise. Workers work in teams, and those teams rely not just on the resources of their organizations, but on the smooth operation of a whole network of other teams and organizations all over the world. Even the most heroic entrepreneur stands on the shoulders of a vast number of workers; Elon Musk could not build a rocket, nor could Jeff Bezos create Amazon Web Services. And neither SpaceX nor Amazon could have been built in Somalia.

We always bridled at being reminded of this; when Obama told entrepreneurs “You didn’t build that”, he was making the anodyne point that organizations and institutions matter, but people still got very mad. We motivate ourselves with our own sense of heroism, and to be reminded that we’re cogs in a machine greater than ourselves makes life seem a little less meaningful.

The scientific enterprise was also always mostly a collective one. If you look at what almost any scientist does, they’re simply adding a little bit of data to our pool of collective knowledge. Almost all research is incremental stuff — a slightly novel experiment, a little wrinkle in an existing theory — and many published research findings are false, and yet the scientific enterprise collectively gropes its way toward the truth. Even Nobel prize winners are often just managers of huge teams of researchers who never get gold medals or get to give speeches in Stockholm.

In essence, humans were always what we now call edge compute — little devices that reported data back to a central world-mind. Except the world-mind we reported to wasn’t a computer; it was society itself — the network of corporations, governments, universities, and human networks that disseminated human knowledge and coordinated our actions. We started off by learning pieces of what the world-mind already knew (i.e. education). Then we walked around and saw slightly different things, had different experiences, chewed on tiny bits of bigger problems, gathered a little new data, and so on.

We mixed our tiny bits of unique experience with what we learned from the world-mind, and the resulting product was slightly novel because our individual experiences were unique. Then we reported this slightly-novel mix back to the world-mind — with scientific papers, PowerPoint presentations, office conversations, memos, and so on. The world-mind reassimilated our contribution, and learned, and grew a little bit wiser.

But there were a few of us who really did get to be heroes — or at least, a bit more heroic than the rest. Whether because of greater raw ability, or a luckily unique perspective, there were some people who could do things that the entire rest of the world couldn’t do. There are examples from politics and business and art, but there was always something special about the heroes of mathematical theory. Isaac Newton invented calculus and classical mechanics all by himself. Grigori Perelman, sitting in his little room in Russia, solved a famous math problem that had bedeviled the entire profession for decades. Andrew Wiles did something similar a generation earlier.

These advances weren’t done ex nihilo, of course — every great discoverer and inventor stood on the shoulders of proverbial giants. But in mathematical theory there was almost always some nub of hard work, some brilliant leap of genius, that belonged to one person alone. Some part of the excitement of becoming a mathematician surely had to do with the dream of becoming one of those heroes, without whom a great discovery simply couldn’t have been made.

But that heroism was necessary because of an inherent limitation in the way the old world-mind was built. Society is far more complex than any single human, but it requires individual communication in order to transmit ideas. A mathematical theorem has to be expressed in terms that one grad student can learn, or one professor can teach to another. This meant that every idea that got added to the collective ultimately had to be comprehensible by a single human being.1 Which meant that it had to be initially conceived of by a single human being as well. The people who conceived of those almost-impossible-to-conceive ideas were our heroes, and mathematicians were perhaps the most heroic among them.

We normally think of AI as something that replaces a single human worker, but I think that’s the wrong way to think of it. AI is really a new world-mind — a technology that takes the accumulated findings of individual humans (or robots, or other sensors) and integrates them into a general picture of the world. AI uses all of human knowledge as its training data; each time one of us reports a new scientific finding or expresses a new perspective, that adds to AI’s knowledge and understanding of the world. This is what people mean when they say to “write for the AIs”, but in fact every scientist and every businessperson is just writing for the AIs now, whether they intend to or not.

(I actually think of a financial analogy here. Before the crash of 2008, most financial derivatives — the complex CDOs and such that ended up being involved in the crisis — were traded over-the-counter, by people calling up other people and selling them products. After the Dodd-Frank legislation, trading of most derivatives shifted to central clearinghouses. The shift from human organizations to AI as our coordinating superintelligence feels a little like the shift from OTC information exchange to a central clearinghouse.)

But unlike human society, AI doesn’t need heroic human geniuses. Its ability to understand complex ideas is not limited by the ability of a single human brain to apprehend, intuit, or communicate those ideas. The information bottlenecks that our greatest mathematicians were able to slightly widen have now been done away with completely — or will be soon.

That doesn’t mean human scientists and thinkers have become irrelevant. At least for now, AI still requires us to provide the role of edge compute. We go out into the world, discover new facts, mix them with our unique individual experiences to form new perspectives, and communicate all this back to the AI world-mind. For most people — even most scientists — this is basically the same thing they were doing before, except instead of a group of their colleagues at a seminar, they’ll be reporting their findings to AI.

And for lots of scientists, AI’s mathematical prowess is going to open up a new golden age. Math was always one of the hardest parts of science; it’s something our brains aren’t naturally adapted to. Now that constraint is alleviated. Economists don’t have to worry about wracking their brains writing theory sections; they can focus on getting data and understanding empirical results. Przemek Chojecki writes:

From a perspective of science or mathematics (not mathematicians), this is the best time ever. AI will lead us to the new age of mathematical discoveries and boost science progress 100x.

Whether it’s 100x or 1.5x remains to be seen, of course. But the point is that people for whom mathematics is an input to understanding the world, rather than an output, just gained access to an incredibly powerful tool that makes their lives much easier. For most people, math was always a burden, like stamping metal or drilling holes. Now that burden has been alleviated by machines.

At least until systems of robots and sensors eliminate the need for human research labor, most scientists will get to be a little more heroic than before. They’ll be like the characters in Star Trek — exploring the unknown with the aid of unfathomably powerful machines.

But mathematicians, and anyone else whose sense of accomplishment came from being the indispensable solvers of hard theoretical problems, will have to learn to get their sense of self-worth and accomplishment from other sources. Jacob Tsimerman, who just won the Fields Medal and promptly took a job at OpenAI, sums it up:

I do not do mathematics only to find truth. I do it largely because I enjoy it and I am good at it. I also find it beautiful and am grateful I get to spend my days understanding beautiful things. But I enjoy the challenge, the process, resolving confusions, finding strategies, grappling with problems…There are many people whose primary enjoyment of math comes through problem solving in one of its incarnations. If that disappears, that is not a trivial issue and many of them might not want to do it anymore.

So what will mathematicians do in the new AI age?

Mathematicians after the age of heroes

First of all, let me say that I do not think AI will take mathematicians’ jobs. I sometimes say that it will, but this is a joke. Even if AI is able to do 100% of what mathematicians do today, I believe that mathematicians will continue to be employed, as mathematicians, if they want.

The reason is that mathematicians, as a class of people, are actually incredibly cheap. The sum total of all of the salaries of academic mathematicians in the U.S. is maybe $4-5 billion dollars at most. That’s a rounding error on our GDP — it’s about what we spend on bowling alleys, or Halloween candy. And to be brutally honest, only a few of those are actually doing frontier research — most are employed mainly as teachers. On top of that, it’s an open question whether most of the frontier mathematical research being done by the very best mathematicians is actually producing value for money in economic terms — other than some cryptography and the occasional physics application, there isn’t much use for most frontier math.

In other words, we already employ most mathematicians because we think human understanding of math, even without industrial applications, is inherently worthwhile. That isn’t likely to change in the age of AI. Even if AI “discovers” a vast number of new mathematical results, most people won’t care; what will be important is that some human understands what AI has discovered. And yes, our society — which AI will make even richer than it is now — will be willing to pay the pittance it costs to keep human mathematicians employed as professional math-understanders. For the overwhelming majority of people, that situation will feel no different than what prevails today.

This is actually a point a lot of people have been making:

Just because you press a button and make AI solve a problem doesn’t mean you understand the solution, and it doesn’t mean you immediately have an idea of what other problems you’d like solved. Mathematicians have been given an incredible new tool to understand the world — like miners getting hydraulic excavators or powered drills — and since the amount of math out there to be discovered is probably infinite, using that tool will take a lot of work.

For those mathematicians whose sense of wonder and meaning comes primarily from learning new things — who do math to “find truth”, in Jacob Tsimerman’s words — life will only improve. And while that won’t be as heroic of a job as what math research used to be, it’ll still be a prestigious one, because most humans will still be unable to do it. Even if any high school kid can press a button and solve the Riemann Hypothesis, the output will just be gobbeldygook to them.

And mathematicians who enjoy the extreme mental difficulty of math will still get to enjoy it. AI will make any given math problem a lot easier to solve, and it’s generally a lot easier to understand a solution than to find one. But AI will also come up with — and will allow humans to come up with — much harder problems than have been created so far. There will be problems so difficult that even understanding AI’s solutions will be as hard for a human as solving the Jacobian Conjecture without the aid of a machine.

As for people who just intrinsically enjoy the old way of discovering math, this will turn into a hobby, or a sport. Just as there are still sprinters even though humans can’t outrun cars, there will still be mathletes. And just as there are still woodworkers even though machines can probably do a better job, there will still be smart people who try to solve math problems without the aid of AI.

So mathematicians will still be able to get paid to do math. They’ll still be able to learn new things, have fun, and get respect from society. The only thing they’ll really lack is heroism — the knowledge that their own special, rare natural abilities alleviate a key bottleneck to human flourishing.

But that’s not so bad, really. Most people never get the chance to be heroes. Truck drivers don’t. Financial advisers don’t. Executive assistants don’t. They have to find meaning just from being regular people — from taking care of their kids, having friends, having hobbies, joining civic organizations, getting involved in politics, and so on. It’s not such a terrible fate.

Subscribe now

Share

2026-08-02 17:13:04

Would you like to see an example of a badly conceived policy? Here you go:

The bill to require discounts for self-checkout will probably not pass. But what’s notable is that if this bill did pass, it would have exactly the opposite of the effect its creators want. The stated justification is that customers are doing unpaid labor by scanning and bagging their own groceries, and that this discount would compensate them for that effort. But the bill is part of a more general effort to disincentivize automation in the retail industry, in order to protect working-class jobs:

Rhode Island was the first state in the U.S. to enact statewide self-checkout restrictions with its Restrictions on Self-Service Checkout Stations Act, which takes effect January 1, 2027. The act requires grocery stores to have at least one human-operated checkout station for every three self-service stations during peak hours…Similar legislation has either been introduced, enacted or considered in additional states including California, Massachusetts, Connecticut, Washington, and Ohio.

The standard objection to this bill, I suppose, would be that self-checkout already gives shoppers a discount, because it allows stores to save money on hiring human labor, and that these savings get passed on to consumers in the form of lower prices. That argument is correct as far as it goes, but it means the debate ends up as one of technocracy versus populism — protecting the livelihoods of a few cashiers versus giving a huge number of consumers very slightly lower prices. There’s no simple or easy conclusion to that debate.

But in the case of New York’s proposed self-checkout discount, the problem is that the bill would actually make grocery stores fire human cashiers in favor of more self-checkout.

To understand why this is true, first imagine an incredibly extreme version of the policy. Imagine that instead of 10%, you made a law forcing grocery stores to give a 90% discount for self-checkout. This is exactly the same as forcing stores to charge 10x for using a human cashier. Who would pay 10x for groceries, just to use a human cashier? Basically no one. So everyone would go use the self-checkout machines, and stores would have no need for human cashiers at all.

But wait, you may be thinking. How could any store afford to give a 90% discount on groceries? The same way they give discounts for seniors and students at movie theaters. You just raise the sticker price, so that the price people actually pay after the discount is the same as before. So instead of keeping human-cashier prices the same and cutting prices for self-checkout, what stores would actually do if you made them give 10% discounts for self-checkout is to raise prices across the board — not by 10%, but by enough so that overall revenue is just about the same as it was before.

How do we know this is what will happen? Because the grocery store industry is very competitive. Profit margins are almost always between 1.5% and 2.8%, which is very low compared to the average industry:

It’s not just profit margins that demonstrate this — there are tons of grocery stores, and new ones enter the market all the time. This means that if grocery stores tried to respond to the New York bill by getting rid of self-checkout and using only human cashiers — in order to charge higher prices and make higher profits — other stores would undercut them by offering self-checkout with big discounts. People would flock to those other stores, and the stores that tried to overcharge would lose business until they, too, put in the self-checkout machines.

This is kind of like if you told grocery stores to charge 10% more for Skippy peanut butter than for Jif peanut butter. Obviously stores would want to stock more Skippy, because they’d earn more money on each sale. But obviously consumers would want to buy more Jif, because it was cheaper. At the end of the day, the consumers would win, because consumers tend to win out when an industry is highly competitive. Some people would still buy Skippy, because some people just really like Skippy more, but overall, forcing one brand of peanut butter to cost more than another brand means you’ll have less of it on the shelves.

By the exact same reasoning, forcing stores to charge more for purchases scanned by human cashiers will end up getting a lot of cashiers fired and replaced with self-checkout machines.

As policies go, this example isn’t particularly hard to explain, or to understand. If you ask any economist, or any good AI, they’ll give basically the same answer, in different words. It would be a good question on an undergrad microeconomics exam. But when I tried to explain this on X, some people had a lot of trouble understanding:

The idea of a 10% discount on self-checkout as a way to incentivize human employment is pure “slopulism” — stuff that seems like populism, but wouldn’t achieve the results that people want from the policy.1 It’s snake oil that sells well based on people’s inability to see the consequences of what they’re being sold.

Slopulism is an inevitable outgrowth of populism. Once people decide that technocrats, elites, experts, and gatekeepers aren’t worth listening to, they become vulnerable to all sorts of policy scams. In general, these scams fall into two categories:

1. Accidental scams, where the people making the policy just don’t understand what they’re doing

2. Ideological scams, where ideologues sell the public on an ideologically-driven policy by pretending it offers benefits that it doesn’t really offer

The second of these is more dangerous than the first. Slopulism that arises out of pure ignorance is very hard to correct, and social media — which elevates the dumbest and loudest at the expense of the most well-informed — probably makes it harder. But in the end, patience, education, and reason can overcome accidental slopulism — and AI may help a lot.

But when slopulist ideas are being promoted by smart people with ulterior motives, it can be a lot harder to wake the populace up. At that point, there’s the danger that slopulism becomes a bipartisan equilibrium, with each party’s ideologues executing a coup against the reasonable moderates by appealing to slopulist instincts.

Accidental versus ideological slopulism

Read more

2026-07-30 18:18:50

Explaining stock market movements is always a little bit of a fool’s errand; no one really understands why stocks boom or crash on a given day or in a given week. Over the long run, the stock market displays lots of “excess volatility” — prices move up and down much more than is warranted by changes in earnings or other measures of fundamental value. There are plenty of theories about why those swings happen, but it’s very hard to know which of those — if any — is in operation at any given time. And so although every big stock market movement is followed by lots of articles claiming to know why, you should take them all — including this one — with several grains of salt.

Anyway, having said all that…

The Korean stock market has been crashing for weeks now. Around March, Korean stocks went on an epic tear; the KOSPI index rose from around 5,000 to over 9,000. Then, just over a month ago, it all went into reverse, with the index falling back to around 5,500:

There are probably two stories regarding why this happened — one about fundamentals, and another about finance. In fact, this is typical for bubbles and crashes, not just in stocks but in every asset class. There’s almost always some kind of connection to fundamentals — some story about how we’re in a new economy, followed by doubts about whether that story is really true, and so on. But the big market movements are almost always accelerated by purely financial factors — “noise traders” armed with piles of excess cash, opportunistic speculators looking to ride the wave of sentiment, and so on.1

For Korea, the fundamental story was about memory stocks. Korean companies like SK Hynix and Samsung make a lot of the world’s computer memory. Under normal circumstances, computer memory isn’t a great business to be in — the technology is fairly commoditized, the industry is brutally competitive, it takes a LOT of capital to build the factories, and it’s very risky to make long-term bets on the evolution of memory technology.

But computer memory is really important for AI data centers. And so the AI boom kicked off the mother of all memory booms. Only a few companies had the scale to meet a large amount of this demand explosion, and the two biggest of these were in South Korea. SK Hynix’s operating profit went from under $10 billion in the first quarter of 2025 to over $35 billion in the first quarter of 2026:

Hynix, which specializes in memory, briefly had a higher market capitalization than Samsung, simply because the memory boom is so huge. Korea’s exports rose over 70% in just one year; in fact, the country’s whole national GDP growth rate increased by a noticeable amount over the past two quarters, just because of this one product:

That’s a pretty strong fundamental story about why South Korean stocks should be worth a lot more. So it’s no surprise that the Korean stock market boomed as soon as people realized in early 2026 that AI technology is going to be extremely valuable. Here’s a thread about just how epic the runup in these companies’ stock prices was:

This is where the financial story rears its head, though. A bunch of traders saw this enormous price rise and decided to buy into it. You’d think a lot of these would be foreign, but international investors mostly avoided the boom (except for a few who bet big on Korean memory stocks). The most frenzied buyers were regular Korean people — the proverbial taxi drivers and teenagers.

These investors probably didn’t understand the fundamental story about AI and data centers and so on. Instead, they probably had extrapolative expectations — they see the price go up and up, and they figure stocks are just a “goer upper”. As more and more buy in and the stock goes up more and more, the perception of a structural upward trend is only reinforced, causing yet more people to buy in. This isn’t the only explanation for coordinated “noise trader” buying frenzies, but it’s probably the most likely.

Normal people don’t have a lot of cash sitting around. But earlier this year, regular Korean people got the opportunity to effectively borrow lots of money to invest it in stocks, via the introduction of leveraged single-stock ETFs. When you buy a share in a leveraged single-stock ETF in SK Hynix, it’s like borrowing money to buy SK Hynix stock.

A whole lot of Koreans used leveraged ETFs and other borrowing methods to borrow huge amounts of money and buy lots and lots of Korean stocks — especially the memory company stocks that were driving everything.

There were some people selling — notably, foreign investors “taking profits” and getting out. But the noise traders overwhelmed all the selling pressure, and sent stock prices soaring.

Then something happened last month — either something fundamental or something financial. Hynix and Samsung are doing fine in terms of earnings growth, but it’s possible that something suddenly gave traders reason to doubt the overall story about the AI boom sending these companies’ profits to the moon. The other possibility is that Korea simply ran out of hotheaded day traders willing to borrow more and more in order to bet on stocks, and the influx of cash naturally came to a halt.

Whichever it was, at that point the price faltered and began to fall. All that borrowed money accelerated the fall on the way down. When prices fall, leveraged ETFs have to sell some of what they hold.2 When a bunch of leveraged ETFs do this at the same time, it pushes prices down, forcing others to sell. In the meantime, people who had borrowed money to buy stock faced margin calls (or bankruptcy), forcing them to sell stock to raise cash. All of this created extra selling pressure, and so increased the rate at which Korean stock prices fell since late June.

Anyway, that’s the financial story. It’s a very old story — financial leverage plus unsophisticated new buyers plus a strong fundamental story often produces a bubble and crash, or exacerbates one that was already in progress. No wonder Korea is now moving — a little belatedly — to restrict leveraged ETFs.

But while financial factors affected the timing and the size of the stock price boom and bust, the fundamental story is more interesting. Fears of an AI bubble are quietly creeping back.

In 2025, as consumer chatbots struggled to find a market big enough to justify the kind of investments being made, there was a lot of talk about an AI bubble. One possibility was that AI revenues wouldn’t grow fast enough to justify the amount being invested in data centers. Another possibility was that AI companies wouldn’t have enough of a “moat” to make them consistently profitable.

In 2026, Claude Code exploded onto the scene, and everyone realized that AI had finally found “product-market fit”. We now know at least one thing that AI is incredibly useful for — writing computer code. Suddenly, an AI bubble seemed much less likely. Spending on AI was skyrocketing, and Anthropic — the new market leader — was successfully capturing much of the profits. Cybersecurity and other zero-sum applications — where having the absolute best model can matter a lot — started to seem like the “moat” that AI had previously lacked. Suddenly, the giant data center construction boom seemed a lot more reasonable.

But slowly, doubts have begun to creep back in. AI is amazing at writing software, but writing software is different from selling it. So far, huge increases in coding productivity are translating into only minor increases in the amount of software being shipped:

It’s possible that a significant fraction of the explosion in the use of coding agents is just “tokenmaxxing” — companies trying to use as much AI as they can, either to learn to use the tools, or perhaps just to look like they’re doing something. If so, we can expect a retrenchment and a temporary slowdown of AI spending growth.

It’s also possible that even the revenue growth we’ve seen isn’t enough to offset the enormous costs of the data center boom. Here’s a recent report from The Economist:

A back-of-the-envelope calculation finds that covering AI capex through identifiable AI income requires revenue on the order of $2.5trn per year, more than tech’s entire combined revenue today…Anthropic pulls in perhaps $75bn, annualised; OpenAI makes tens of billions; Google, via its AI model Gemini, and Microsoft probably get a bit less. SpaceX may have a few billion dollars’ worth of revenue from enterprise AI this year. Meta also makes a few bucks from AI. Add this up and you land at roughly $150bn a year. [emphasis mine]

AI revenue is growing very fast, but if these calculations are right, it’ll have to grow by 17x from where it is now in order to justify the capital being spent. In other words, even Claude Code isn’t enough; AI revenue has to accelerate even further, and while it’s perfectly plausible that it could do that, it’s a big question mark.

There’s also the possibility that the moat of companies like Anthropic is less invincible than it looked just a couple of months ago. A Chinese company called Moonshot AI has released a model called Kimi K3 that nearly equals the best available American models. American companies are using cheap Chinese AI models more and more for daily tasks. If Anthropic and OpenAI lose the overall b2b market to cheap Chinese competition — which is undoubtedly supported, of course, by China’s usual blizzard of subsidies and government supports — there’s not much chance that they’ll be able to pay for the data center boom.

And at that point, there could be a big, big bust — similar to when railroads went under in 1873. Investors are probably already beginning to worry about this. It isn’t just Korean AI-related stocks that have taken a hit recently. Here’s Nvidia, which designs and furnishes the chips that run data centers:

And here’s Micron, America’s top memory chip maker:

And here’s Microsoft, a big part of whose business is installing AI data centers:

There are similar (if less dramatic) stories at Google and Amazon, who also run a ton of data centers.

Here’s Bloomberg’s story about the decline of the so-called “Magnificent 7” tech stocks:

Wall Street is growing increasingly concerned about the hundreds of billions of dollars Big Tech is spending on artificial intelligence…“The real problem is the amount of spend that’s going on,” said Ken Mahoney, chief executive officer of Mahoney Asset Management. “No one knows what the return on investment is.”…The selloff is coming as investors grow increasingly cautious about the massive sums that Big Tech firms are spending to build out their AI infrastructure. The Mag 7 index is now down 11% from a record reached in late May, erasing $2 trillion in market value.

So although South Korea’s epic stock crash was probably related to Korea-specific financial factors, it could also herald the return of the “AI bubble” story. AI is far and away the most important thing going on in the American economy right now, so any hint of a bubble is worrying.

Update: One possibility I should have mentioned is that the Korean stock crash is the start of a general crash in AI stocks — the long-awaited “AI bubble pop”. So far the carnage hasn’t been too extensive, but it’s notable that some people who bet big on the smooth, uninterrupted exponential growth of AI are now seeing those bets blow up:

Citadel — a big traditional mainstream finance firm — is now stepping in to buy many of the stocks owned by Situational Awareness.

And as Derek Thompson noted a couple of weeks ago, Korean retail investors aren’t the only ones borrowing money to buy stocks:

Derek flags the rise of margin buying and leveraged ETFs in America, but I also noticed that some pretty big companies are leveraging themselves to the teeth here:

Subscribe now

Share