2026-08-04 15:36:17
I had a great-uncle who was a “human calculator”. It’s an odd, rare ability that allows people to multiply large numbers quickly in their heads. I remember using a hand calculator and seeing if my uncle could get it right without a tool faster than I could; more often than not, he could.
Human calculators used to be very valuable as employees, back when calculation was done by hand. Today they’re just a curiosity. Technology came along and superseded that particular human ability, as it has so many others. The story of John Henry being superseded by the steam drill, or Paul Bunyan by the chainsaw, is a metaphor for the march of progress. In some cases, like Garry Kasparov or Lee Sedol losing to computers at chess and Go, the moment really happened.
This week marked another important such moment in the history of man versus machine. OpenAI announced that its new model, Astra, managed to solve ten major outstanding problems in math and theoretical computer science. Here’s a simple explanation of the problems. The general consensus seems to be that these are extremely important, stunning breakthroughs.
This does not mean that computers are now capable of doing every kind of math better than humans can. Some people have argued that AI is good at results where the solution can be found by reading the entire literature and combining existing insights — exactly the kind of thing you might expect a computer to be superhuman at — but still not as good as humans at making truly novel leaps of insight. This may represent a real, general limitation of LLMs’ capabilities — as Tom Zahavy of Google DeepMind put it, it may still be the case that “LLMs can’t jump.”
But the trend line is becoming clearer. The University of Toronto’s Daniel Litt, one of the most prominent skeptics of AI’s potential for frontier math research, recently conceded a major bet about what AI could do:
Litt appears to no longer believe that there is any important type of math that AI won’t soon be able to do, writing: “The models don’t yet seem able to do certain kinds of high quality research, but I also expect this is a matter of (not too much) time.”
Some mathematicians have reacted to this development with despair. Kirwin Hampshire’s post is probably the most evocative and the most well-read:
He writes:
I am suffering a profound spiritual crisis due to these developments. I have been screaming internally for days. It feels as though I am living inside of a nightmare…For me, the affective quality of learning mathematics is empathetically tethered to an act of discovery and creation…If The Library of Babel existed, would authors continue to write books?…
Perhaps I shouldn’t tell you this, but my aim is to be open: These developments have triggered some deranged thoughts in me. I have wondered if it is the express goal of these companies to make me kill myself…The story of human discovery and the triumph of the human spirit will soon be excised from this discipline…[T]he process of prompting novel proofs will be as auraless as ordering doordash. Watch as magic and mystery evaporate. Watch as the sun sets on our heroic age…
There is nothing I can do. There may be nothing you can do. I have no prescriptions, policy recommendations, or coherent call to action. I just want to be honest and open about my emotional and spiritual response. I want to feel seen….I need the architects of our new mathematical paradigm to look me in the eye and acknowledge our shared humanity and soul before they deliver the coup de grâce.
Math PhD student Tasmin Chu, meanwhile, urges a more confrontational path, calling on mathematicians to avoid working with AI companies and to aggressively preserve the way that math research is currently done. This seems extremely unlikely (and would fail if it were attempted), but it demonstrates the depth of emotion out there.
Even the mathematicians who don’t share Hampshire’s despair have expressed alarm at the sudden disruption of their profession. The entire way that mathematical research works — training grad students, working on problems, collaborating with colleagues, building theories — is going to have to be overhauled.
I’m not a mathematician; I never have been. I can’t fully understand what mathematicians are going through right now. But I do have a few thoughts on what AI’s supersession of human mathematics research means for our species, so I thought I might as well share those. Some of these ideas are a little vague and half-formed, and for that I apologize in advance.
Humans have always valued standout individual achievement — heroism. We pride ourselves on being able to do what no one else could have done, and we heap status and respect on people who make standout achievements. But this was always a little bit of an illusion; most of what we did was always a collective enterprise. Workers work in teams, and those teams rely not just on the resources of their organizations, but on the smooth operation of a whole network of other teams and organizations all over the world. Even the most heroic entrepreneur stands on the shoulders of a vast number of workers; Elon Musk could not build a rocket, nor could Jeff Bezos create Amazon Web Services. And neither SpaceX nor Amazon could have been built in Somalia.
We always bridled at being reminded of this; when Obama told entrepreneurs “You didn’t build that”, he was making the anodyne point that organizations and institutions matter, but people still got very mad. We motivate ourselves with our own sense of heroism, and to be reminded that we’re cogs in a machine greater than ourselves makes life seem a little less meaningful.
The scientific enterprise was also always mostly a collective one. If you look at what almost any scientist does, they’re simply adding a little bit of data to our pool of collective knowledge. Almost all research is incremental stuff — a slightly novel experiment, a little wrinkle in an existing theory — and many published research findings are false, and yet the scientific enterprise collectively gropes its way toward the truth. Even Nobel prize winners are often just managers of huge teams of researchers who never get gold medals or get to give speeches in Stockholm.
In essence, humans were always what we now call edge compute — little devices that reported data back to a central world-mind. Except the world-mind we reported to wasn’t a computer; it was society itself — the network of corporations, governments, universities, and human networks that disseminated human knowledge and coordinated our actions. We started off by learning pieces of what the world-mind already knew (i.e. education). Then we walked around and saw slightly different things, had different experiences, chewed on tiny bits of bigger problems, gathered a little new data, and so on.
We mixed our tiny bits of unique experience with what we learned from the world-mind, and the resulting product was slightly novel because our individual experiences were unique. Then we reported this slightly-novel mix back to the world-mind — with scientific papers, PowerPoint presentations, office conversations, memos, and so on. The world-mind reassimilated our contribution, and learned, and grew a little bit wiser.
But there were a few of us who really did get to be heroes — or at least, a bit more heroic than the rest. Whether because of greater raw ability, or a luckily unique perspective, there were some people who could do things that the entire rest of the world couldn’t do. There are examples from politics and business and art, but there was always something special about the heroes of mathematical theory. Isaac Newton invented calculus and classical mechanics all by himself. Grigori Perelman, sitting in his little room in Russia, solved a famous math problem that had bedeviled the entire profession for decades. Andrew Wiles did something similar a generation earlier.
These advances weren’t done ex nihilo, of course — every great discoverer and inventor stood on the shoulders of proverbial giants. But in mathematical theory there was almost always some nub of hard work, some brilliant leap of genius, that belonged to one person alone. Some part of the excitement of becoming a mathematician surely had to do with the dream of becoming one of those heroes, without whom a great discovery simply couldn’t have been made.
But that heroism was necessary because of an inherent limitation in the way the old world-mind was built. Society is far more complex than any single human, but it requires individual communication in order to transmit ideas. A mathematical theorem has to be expressed in terms that one grad student can learn, or one professor can teach to another. This meant that every idea that got added to the collective ultimately had to be comprehensible by a single human being.1 Which meant that it had to be initially conceived of by a single human being as well. The people who conceived of those almost-impossible-to-conceive ideas were our heroes, and mathematicians were perhaps the most heroic among them.
We normally think of AI as something that replaces a single human worker, but I think that’s the wrong way to think of it. AI is really a new world-mind — a technology that takes the accumulated findings of individual humans (or robots, or other sensors) and integrates them into a general picture of the world. AI uses all of human knowledge as its training data; each time one of us reports a new scientific finding or expresses a new perspective, that adds to AI’s knowledge and understanding of the world. This is what people mean when they say to “write for the AIs”, but in fact every scientist and every businessperson is just writing for the AIs now, whether they intend to or not.
(I actually think of a financial analogy here. Before the crash of 2008, most financial derivatives — the complex CDOs and such that ended up being involved in the crisis — were traded over-the-counter, by people calling up other people and selling them products. After the Dodd-Frank legislation, trading of most derivatives shifted to central clearinghouses. The shift from human organizations to AI as our coordinating superintelligence feels a little like the shift from OTC information exchange to a central clearinghouse.)
But unlike human society, AI doesn’t need heroic human geniuses. Its ability to understand complex ideas is not limited by the ability of a single human brain to apprehend, intuit, or communicate those ideas. The information bottlenecks that our greatest mathematicians were able to slightly widen have now been done away with completely — or will be soon.
That doesn’t mean human scientists and thinkers have become irrelevant. At least for now, AI still requires us to provide the role of edge compute. We go out into the world, discover new facts, mix them with our unique individual experiences to form new perspectives, and communicate all this back to the AI world-mind. For most people — even most scientists — this is basically the same thing they were doing before, except instead of a group of their colleagues at a seminar, they’ll be reporting their findings to AI.
And for lots of scientists, AI’s mathematical prowess is going to open up a new golden age. Math was always one of the hardest parts of science; it’s something our brains aren’t naturally adapted to. Now that constraint is alleviated. Economists don’t have to worry about wracking their brains writing theory sections; they can focus on getting data and understanding empirical results. Przemek Chojecki writes:
From a perspective of science or mathematics (not mathematicians), this is the best time ever. AI will lead us to the new age of mathematical discoveries and boost science progress 100x.
Whether it’s 100x or 1.5x remains to be seen, of course. But the point is that people for whom mathematics is an input to understanding the world, rather than an output, just gained access to an incredibly powerful tool that makes their lives much easier. For most people, math was always a burden, like stamping metal or drilling holes. Now that burden has been alleviated by machines.
At least until systems of robots and sensors eliminate the need for human research labor, most scientists will get to be a little more heroic than before. They’ll be like the characters in Star Trek — exploring the unknown with the aid of unfathomably powerful machines.
But mathematicians, and anyone else whose sense of accomplishment came from being the indispensable solvers of hard theoretical problems, will have to learn to get their sense of self-worth and accomplishment from other sources. Jacob Tsimerman, who just won the Fields Medal and promptly took a job at OpenAI, sums it up:
I do not do mathematics only to find truth. I do it largely because I enjoy it and I am good at it. I also find it beautiful and am grateful I get to spend my days understanding beautiful things. But I enjoy the challenge, the process, resolving confusions, finding strategies, grappling with problems…There are many people whose primary enjoyment of math comes through problem solving in one of its incarnations. If that disappears, that is not a trivial issue and many of them might not want to do it anymore.
So what will mathematicians do in the new AI age?
First of all, let me say that I do not think AI will take mathematicians’ jobs. I sometimes say that it will, but this is a joke. Even if AI is able to do 100% of what mathematicians do today, I believe that mathematicians will continue to be employed, as mathematicians, if they want.
The reason is that mathematicians, as a class of people, are actually incredibly cheap. The sum total of all of the salaries of academic mathematicians in the U.S. is maybe $4-5 billion dollars at most. That’s a rounding error on our GDP — it’s about what we spend on bowling alleys, or Halloween candy. And to be brutally honest, only a few of those are actually doing frontier research — most are employed mainly as teachers. On top of that, it’s an open question whether most of the frontier mathematical research being done by the very best mathematicians is actually producing value for money in economic terms — other than some cryptography and the occasional physics application, there isn’t much use for most frontier math.
In other words, we already employ most mathematicians because we think human understanding of math, even without industrial applications, is inherently worthwhile. That isn’t likely to change in the age of AI. Even if AI “discovers” a vast number of new mathematical results, most people won’t care; what will be important is that some human understands what AI has discovered. And yes, our society — which AI will make even richer than it is now — will be willing to pay the pittance it costs do keep human mathematicians employed as professional math-understanders. For the overwhelming majority of people, that situation will feel no different than what prevails today.
This is actually a point a lot of people have been making:
Just because you press a button and make AI solve a problem doesn’t mean you understand the solution, and it doesn’t mean you immediately have an idea of what other problems you’d like solved. Mathematicians have been given an incredible new tool to understand the world — like miners getting hydraulic excavators or powered drills — and since the amount of math out there to be discovered is probably infinite, using that tool will take a lot of work.
For those mathematicians whose sense of wonder and meaning comes primarily from learning new things — who do math to “find truth”, in Jacob Tsimerman’s words — life will only improve. And while that won’t be as heroic of a job as what math research used to be, it’ll still be a prestigious one, because most humans will still be unable to do it. Even if any high school kid can press a button and solve the Riemann Hypothesis, the output will just be gobbeldygook to them.
And mathematicians who enjoy the extreme mental difficulty of math will still get to enjoy it. AI will make any given math problem a lot easier to solve, and it’s generally a lot easier to understand a solution than to find one. But AI will also come up with — and will allow humans to come up with — much harder problems than have been created so far. There will be problems so difficult that even understanding AI’s solutions will be as hard for a human as solving the Jacobian Conjecture without the aid of a machine.
As for people who just intrinsically enjoy the old way of discovering math, this will turn into a hobby, or a sport. Just as there are still sprinters even though humans can’t outrun cars, there will still be mathletes. And just as there are still woodworkers even though machines can probably do a better job, there will still be smart people who try to solve math problems without the aid of AI.
So mathematicians will still be able to get paid to do math. They’ll still be able to learn new things, have fun, and get respect from society. The only thing they’ll really lack is heroism — the knowledge that their own special, rare natural abilities alleviate a key bottleneck to human flourishing.
But that’s not so bad, really. Most people never get the chance to be heroes. Truck drivers don’t. Financial advisers don’t. Executive assistants don’t. They have to find meaning just from being regular people — from taking care of their kids, having friends, having hobbies, joining civic organizations, getting involved in politics, and so on. It’s not such a terrible fate.
Well, maybe not. Society can handle ideas more complex than a single human can understand, but those ideas have to be able to be broken down into pieces that a single human brain can comprehend.
2026-08-02 17:13:04
Would you like to see an example of a badly conceived policy? Here you go:
The bill to require discounts for self-checkout will probably not pass. But what’s notable is that if this bill did pass, it would have exactly the opposite of the effect its creators want. The stated justification is that customers are doing unpaid labor by scanning and bagging their own groceries, and that this discount would compensate them for that effort. But the bill is part of a more general effort to disincentivize automation in the retail industry, in order to protect working-class jobs:
Rhode Island was the first state in the U.S. to enact statewide self-checkout restrictions with its Restrictions on Self-Service Checkout Stations Act, which takes effect January 1, 2027. The act requires grocery stores to have at least one human-operated checkout station for every three self-service stations during peak hours…Similar legislation has either been introduced, enacted or considered in additional states including California, Massachusetts, Connecticut, Washington, and Ohio.
The standard objection to this bill, I suppose, would be that self-checkout already gives shoppers a discount, because it allows stores to save money on hiring human labor, and that these savings get passed on to consumers in the form of lower prices. That argument is correct as far as it goes, but it means the debate ends up as one of technocracy versus populism — protecting the livelihoods of a few cashiers versus giving a huge number of consumers very slightly lower prices. There’s no simple or easy conclusion to that debate.
But in the case of New York’s proposed self-checkout discount, the problem is that the bill would actually make grocery stores fire human cashiers in favor of more self-checkout.
To understand why this is true, first imagine an incredibly extreme version of the policy. Imagine that instead of 10%, you made a law forcing grocery stores to give a 90% discount for self-checkout. This is exactly the same as forcing stores to charge 10x for using a human cashier. Who would pay 10x for groceries, just to use a human cashier? Basically no one. So everyone would go use the self-checkout machines, and stores would have no need for human cashiers at all.
But wait, you may be thinking. How could any store afford to give a 90% discount on groceries? The same way they give discounts for seniors and students at movie theaters. You just raise the sticker price, so that the price people actually pay after the discount is the same as before. So instead of keeping human-cashier prices the same and cutting prices for self-checkout, what stores would actually do if you made them give 10% discounts for self-checkout is to raise prices across the board — not by 10%, but by enough so that overall revenue is just about the same as it was before.
How do we know this is what will happen? Because the grocery store industry is very competitive. Profit margins are almost always between 1.5% and 2.8%, which is very low compared to the average industry:
It’s not just profit margins that demonstrate this — there are tons of grocery stores, and new ones enter the market all the time. This means that if grocery stores tried to respond to the New York bill by getting rid of self-checkout and using only human cashiers — in order to charge higher prices and make higher profits — other stores would undercut them by offering self-checkout with big discounts. People would flock to those other stores, and the stores that tried to overcharge would lose business until they, too, put in the self-checkout machines.
This is kind of like if you told grocery stores to charge 10% more for Skippy peanut butter than for Jif peanut butter. Obviously stores would want to stock more Skippy, because they’d earn more money on each sale. But obviously consumers would want to buy more Jif, because it was cheaper. At the end of the day, the consumers would win, because consumers tend to win out when an industry is highly competitive. Some people would still buy Skippy, because some people just really like Skippy more, but overall, forcing one brand of peanut butter to cost more than another brand means you’ll have less of it on the shelves.
By the exact same reasoning, forcing stores to charge more for purchases scanned by human cashiers will end up getting a lot of cashiers fired and replaced with self-checkout machines.
As policies go, this example isn’t particularly hard to explain, or to understand. If you ask any economist, or any good AI, they’ll give basically the same answer, in different words. It would be a good question on an undergrad microeconomics exam. But when I tried to explain this on X, some people had a lot of trouble understanding:
The idea of a 10% discount on self-checkout as a way to incentivize human employment is pure “slopulism” — stuff that seems like populism, but wouldn’t achieve the results that people want from the policy.1 It’s snake oil that sells well based on people’s inability to see the consequences of what they’re being sold.
Slopulism is an inevitable outgrowth of populism. Once people decide that technocrats, elites, experts, and gatekeepers aren’t worth listening to, they become vulnerable to all sorts of policy scams. In general, these scams fall into two categories:
Accidental scams, where the people making the policy just don’t understand what they’re doing
Ideological scams, where ideologues sell the public on an ideologically-driven policy by pretending it offers benefits that it doesn’t really offer
The second of these is more dangerous than the first. Slopulism that arises out of pure ignorance is very hard to correct, and social media — which elevates the dumbest and loudest at the expense of the most well-informed — probably makes it harder. But in the end, patience, education, and reason can overcome accidental slopulism — and AI may help a lot.
But when slopulist ideas are being promoted by smart people with ulterior motives, it can be a lot harder to wake the populace up. At that point, there’s the danger that slopulism becomes a bipartisan equilibrium, with each party’s ideologues executing a coup against the reasonable moderates by appealing to slopulist instincts.
2026-07-30 18:18:50
Explaining stock market movements is always a little bit of a fool’s errand; no one really understands why stocks boom or crash on a given day or in a given week. Over the long run, the stock market displays lots of “excess volatility” — prices move up and down much more than is warranted by changes in earnings or other measures of fundamental value. There are plenty of theories about why those swings happen, but it’s very hard to know which of those — if any — is in operation at any given time. And so although every big stock market movement is followed by lots of articles claiming to know why, you should take them all — including this one — with several grains of salt.
Anyway, having said all that…
The Korean stock market has been crashing for weeks now. Around March, Korean stocks went on an epic tear; the KOSPI index rose from around 5,000 to over 9,000. Then, just over a month ago, it all went into reverse, with the index falling back to around 5,500:
There are probably two stories regarding why this happened — one about fundamentals, and another about finance. In fact, this is typical for bubbles and crashes, not just in stocks but in every asset class. There’s almost always some kind of connection to fundamentals — some story about how we’re in a new economy, followed by doubts about whether that story is really true, and so on. But the big market movements are almost always accelerated by purely financial factors — “noise traders” armed with piles of excess cash, opportunistic speculators looking to ride the wave of sentiment, and so on.1
For Korea, the fundamental story was about memory stocks. Korean companies like SK Hynix and Samsung make a lot of the world’s computer memory. Under normal circumstances, computer memory isn’t a great business to be in — the technology is fairly commoditized, the industry is brutally competitive, it takes a LOT of capital to build the factories, and it’s very risky to make long-term bets on the evolution of memory technology.
But computer memory is really important for AI data centers. And so the AI boom kicked off the mother of all memory booms. Only a few companies had the scale to meet a large amount of this demand explosion, and the two biggest of these were in South Korea. SK Hynix’s operating profit went from under $10 billion in the first quarter of 2025 to over $35 billion in the first quarter of 2026:
Hynix, which specializes in memory, briefly had a higher market capitalization than Samsung, simply because the memory boom is so huge. Korea’s exports rose over 70% in just one year; in fact, the country’s whole national GDP growth rate increased by a noticeable amount over the past two quarters, just because of this one product:
That’s a pretty strong fundamental story about why South Korean stocks should be worth a lot more. So it’s no surprise that the Korean stock market boomed as soon as people realized in early 2026 that AI technology is going to be extremely valuable. Here’s a thread about just how epic the runup in these companies’ stock prices was:
This is where the financial story rears its head, though. A bunch of traders saw this enormous price rise and decided to buy into it. You’d think a lot of these would be foreign, but international investors mostly avoided the boom (except for a few who bet big on Korean memory stocks). The most frenzied buyers were regular Korean people — the proverbial taxi drivers and teenagers.
These investors probably didn’t understand the fundamental story about AI and data centers and so on. Instead, they probably had extrapolative expectations — they see the price go up and up, and they figure stocks are just a “goer upper”. As more and more buy in and the stock goes up more and more, the perception of a structural upward trend is only reinforced, causing yet more people to buy in. This isn’t the only explanation for coordinated “noise trader” buying frenzies, but it’s probably the most likely.
Normal people don’t have a lot of cash sitting around. But earlier this year, regular Korean people got the opportunity to effectively borrow lots of money to invest it in stocks, via the introduction of leveraged single-stock ETFs. When you buy a share in a leveraged single-stock ETF in SK Hynix, it’s like borrowing money to buy SK Hynix stock.
A whole lot of Koreans used leveraged ETFs and other borrowing methods to borrow huge amounts of money and buy lots and lots of Korean stocks — especially the memory company stocks that were driving everything.
There were some people selling — notably, foreign investors “taking profits” and getting out. But the noise traders overwhelmed all the selling pressure, and sent stock prices soaring.
Then something happened last month — either something fundamental or something financial. Hynix and Samsung are doing fine in terms of earnings growth, but it’s possible that something suddenly gave traders reason to doubt the overall story about the AI boom sending these companies’ profits to the moon. The other possibility is that Korea simply ran out of hotheaded day traders willing to borrow more and more in order to bet on stocks, and the influx of cash naturally came to a halt.
Whichever it was, at that point the price faltered and began to fall. All that borrowed money accelerated the fall on the way down. When prices fall, leveraged ETFs have to sell some of what they hold.2 When a bunch of leveraged ETFs do this at the same time, it pushes prices down, forcing others to sell. In the meantime, people who had borrowed money to buy stock faced margin calls (or bankruptcy), forcing them to sell stock to raise cash. All of this created extra selling pressure, and so increased the rate at which Korean stock prices fell since late June.
Anyway, that’s the financial story. It’s a very old story — financial leverage plus unsophisticated new buyers plus a strong fundamental story often produces a bubble and crash, or exacerbates one that was already in progress. No wonder Korea is now moving — a little belatedly — to restrict leveraged ETFs.
But while financial factors affected the timing and the size of the stock price boom and bust, the fundamental story is more interesting. Fears of an AI bubble are quietly creeping back.
In 2025, as consumer chatbots struggled to find a market big enough to justify the kind of investments being made, there was a lot of talk about an AI bubble. One possibility was that AI revenues wouldn’t grow fast enough to justify the amount being invested in data centers. Another possibility was that AI companies wouldn’t have enough of a “moat” to make them consistently profitable.
In 2026, Claude Code exploded onto the scene, and everyone realized that AI had finally found “product-market fit”. We now know at least one thing that AI is incredibly useful for — writing computer code. Suddenly, an AI bubble seemed much less likely. Spending on AI was skyrocketing, and Anthropic — the new market leader — was successfully capturing much of the profits. Cybersecurity and other zero-sum applications — where having the absolute best model can matter a lot — started to seem like the “moat” that AI had previously lacked. Suddenly, the giant data center construction boom seemed a lot more reasonable.
But slowly, doubts have begun to creep back in. AI is amazing at writing software, but writing software is different from selling it. So far, huge increases in coding productivity are translating into only minor increases in the amount of software being shipped:
It’s possible that a significant fraction of the explosion in the use of coding agents is just “tokenmaxxing” — companies trying to use as much AI as they can, either to learn to use the tools, or perhaps just to look like they’re doing something. If so, we can expect a retrenchment and a temporary slowdown of AI spending growth.
It’s also possible that even the revenue growth we’ve seen isn’t enough to offset the enormous costs of the data center boom. Here’s a recent report from The Economist:
A back-of-the-envelope calculation finds that covering AI capex through identifiable AI income requires revenue on the order of $2.5trn per year, more than tech’s entire combined revenue today…Anthropic pulls in perhaps $75bn, annualised; OpenAI makes tens of billions; Google, via its AI model Gemini, and Microsoft probably get a bit less. SpaceX may have a few billion dollars’ worth of revenue from enterprise AI this year. Meta also makes a few bucks from AI. Add this up and you land at roughly $150bn a year. [emphasis mine]
AI revenue is growing very fast, but if these calculations are right, it’ll have to grow by 17x from where it is now in order to justify the capital being spent. In other words, even Claude Code isn’t enough; AI revenue has to accelerate even further, and while it’s perfectly plausible that it could do that, it’s a big question mark.
There’s also the possibility that the moat of companies like Anthropic is less invincible than it looked just a couple of months ago. A Chinese company called Moonshot AI has released a model called Kimi K3 that nearly equals the best available American models. American companies are using cheap Chinese AI models more and more for daily tasks. If Anthropic and OpenAI lose the overall b2b market to cheap Chinese competition — which is undoubtedly supported, of course, by China’s usual blizzard of subsidies and government supports — there’s not much chance that they’ll be able to pay for the data center boom.
And at that point, there could be a big, big bust — similar to when railroads went under in 1873. Investors are probably already beginning to worry about this. It isn’t just Korean AI-related stocks that have taken a hit recently. Here’s Nvidia, which designs and furnishes the chips that run data centers:
And here’s Micron, America’s top memory chip maker:
And here’s Microsoft, a big part of whose business is installing AI data centers:
There are similar (if less dramatic) stories at Google and Amazon, who also run a ton of data centers.
Here’s Bloomberg’s story about the decline of the so-called “Magnificent 7” tech stocks:
Wall Street is growing increasingly concerned about the hundreds of billions of dollars Big Tech is spending on artificial intelligence…“The real problem is the amount of spend that’s going on,” said Ken Mahoney, chief executive officer of Mahoney Asset Management. “No one knows what the return on investment is.”…The selloff is coming as investors grow increasingly cautious about the massive sums that Big Tech firms are spending to build out their AI infrastructure. The Mag 7 index is now down 11% from a record reached in late May, erasing $2 trillion in market value.
So although South Korea’s epic stock crash was probably related to Korea-specific financial factors, it could also herald the return of the “AI bubble” story. AI is far and away the most important thing going on in the American economy right now, so any hint of a bubble is worrying.
Update: One possibility I should have mentioned is that the Korean stock crash is the start of a general crash in AI stocks — the long-awaited “AI bubble pop”. So far the carnage hasn’t been too extensive, but it’s notable that some people who bet big on the smooth, uninterrupted exponential growth of AI are now seeing those bets blow up:
Citadel — a big traditional mainstream finance firm — is now stepping in to buy many of the stocks owned by Situational Awareness.
And as Derek Thompson noted a couple of weeks ago, Korean retail investors aren’t the only ones borrowing money to buy stocks:
Derek flags the rise of margin buying and leveraged ETFs in America, but I also noticed that some pretty big companies are leveraging themselves to the teeth here:
For the best simple explanation of how financial markets can go haywire, I recommend the famous paper by DeLong et al. (1990). If you don’t feel like reading through a mathematical model, just ask AI to explain it to you in simple terms.
Usually, futures or options or some other derivatives tied to the value of the underlying stock.
2026-07-28 08:33:43
Hallo, awesome readers! It’s time for a roundup. I actually never think of a theme for these in advance. I just dump everything in my list of “stuff I don’t have time to write a full post about" into the Substack editor, then try to figure out some kind of thread that ties the items together. Today’s list is, roughly speaking, about industrial policy — deliberate policies that governments can take in order to encourage specific sectors or pieces of their economies.
But first, podcasts! I went on Scott Galloway’s Prof G Markets podcast, to talk about the macroeconomic implications of AI:
You can also watch a video if you prefer! Not sure why, but it looks like I haven’t shaved in a long time even though it’s only 12 hours of beard shadow.
I also went on Jayden Clark’s podcast, Members of Technical Staff, with Sam D’Amico (founder of Impulse Labs). We talked about San Francisco’s local politics, while cooking steak on an Impulse stove:1
Anyway, on to this week’s list of interesting stuff!
The Trump administration, generally speaking, has been the worst on science policy in the country’s entire history. As part of its effort to root out wokeness from American institutions, it has slashed science budgets. Thanks to its conviction that immigrants are foreign invaders intent on overthrowing Western civilization, it has blocked or scared away lots of the foreign researchers that American science depends on. And due to its bizarre political alliance with pseudo-leftist conspiracy theorists, it has elevated quasi-mystical antivaxxers to positions of authority over biomedical policy.
But at the same time, there are some people within the Trump administration who are trying to take things in a more positive direction. The Office of Science and Technology Policy recently released a report called “Science: A New Golden Age” that contains a number of interesting and promising ideas. Alec Stapp of the Institute for Progress wrote a quick summary of what he likes about the plan:
The most important proposal is probably the idea to shift funding from organizations to individual scientists:
Refocus on the Individual Scientist: Put the working researcher back at the center of America’s scientific enterprise. Free them from the growing administrative burdens that now weigh them down for nearly half their working hours. Bet on people, not just projects, by expanding portable graduate fellowships like the National Science Foundation (NSF) Graduate Research Fellowship Program (GRFP), backing early-career independence, and scaling long-horizon grants for the best and brightest modeled on National Institutes of Health (NIH) Director’s Pioneer Award. Open alternative pathways beyond standard academia, and ensure that selection rests purely on merit, not the political fashions of the day.
This is probably a very good idea on the merits. And you can also see how the OSTP folks are selling it to a skeptical Trump administration. Bypassing universities and funding researchers directly seems like it would naturally appeal to administration people who think universities are engines of woke ideology.
Another good idea is to diversify federal science funding mechanisms:
Move beyond consensus-driven peer review by adopting a broader menu of selection mechanisms suited to different kinds of science. Examples include “golden tickets” that empower individual reviewers to champion ambitious proposals, fast grants that deliver rapid funding decisions, prize challenges and advanced market commitments that pay for results, and regranting models that delegate funding authority to scientists to draw on distributed expertise…
Many of today’s most important problems are too large for an academic lab, too cross-disciplinary for a single department, and too hard to commercialize for a private corporation. Federal funding should support a wider range of performers. The recently launched X-Labs can assemble agile, time-bound teams of professional scientists and engineers to break specific bottlenecks. Advanced Research Projects Agencies (ARPAs) can empower individual program managers to make bold bets and curate researchers to execute them. Curiosity-driven institutes can give our best minds the stability needed to pursue fundamental questions over long time horizons.
Science is changing fast in the age of AI, and old funding models may not always be the best for every discipline. Also, notice that there’s a bit of industrial policy here — the mention of advanced market commitments suggests that Kratsios and the OSTP understand the intimate relationship between lab results and industrial scale-up in fields like chemistry and materials science.
Finally, OSTP wants to do metascience — they want to evaluate new funding methods as they’re implemented, to see what’s working and what isn’t:
Stand up an empowered metascience unit in federal science agencies, reporting directly to the director, with authority to run controlled experiments on review and funding mechanisms and to drive change across the organization.
I am in favor of all of these proposals. They’re very good, and most are long overdue. The problem is that they’re going to cost a lot of money. OSTP’s report isn’t a piece of legislation, or even a funding request; it’s just a plan, by one little piece of the government, that may or may not ever be implemented. Trump’s budgets still call for huge cuts to science funding, which would basically throw the OSTP plan right in the trashcan or (at best) turn it into a tiny symbolic demonstration program.
But these ideas are good enough that they need to be bipartisan. If Democrats take back the White House in 2028, they need to pick up this plan, slap a little token partisan gloss on it, maybe have Claude rewrite the language a little, and fully fund it.
I’ve been shouting about the national debt for years now (including when Biden was President), so it’s good to see other people who aren’t fiscal perma-hawks shouting about it too. One of these people is Martha Gimbel, who worked for Biden’s Council of Economic Advisors. She writes that America’s national debt is already being reflected in higher long-term interest rates:
What should matter is that the consequences of this debt are not off in the future, but already here. The government’s deficits have saddled many American families with higher costs, largely from rising interest rates. The Budget Lab, the policy research center at Yale where I am the executive director, recently estimated that congressional-spending decisions since 2015 have raised Treasury yields by almost a full percentage point, which affects what American households pay to borrow. For someone taking out a 30-year mortgage at last year’s median home price, this rise in long-term interest rates has increased their borrowing costs by about $2,500 a year, or roughly $76,000 over the life of the loan.
The bloated government budgets and waning federal revenues of the past decade are driving up costs across the board. Compared with a world in which these fiscal-policy changes did not take place, the annual borrowing costs on a typical auto loan are now up by about $120, and by about $770 on a typical small-business loan. Credit-card borrowing rates are also hovering near record highs. [emphasis mine]
Gimbel is no one’s idea of a fiscal perma-hawk. But actual perma-hawks, like the people at the Committee for a Responsible Federal Budget, are naturally freaking out even more. Here is their illustration of how government debt can spiral out of control, when higher interest rates force the government to spend more even as they depress growth and reduce tax revenues, leading to even higher interest rates:
Relentlessly rising federal interest payments show why we’re in danger of this spiral right now.
Once this kind of spiral takes off, the only thing you can really do is print money, which can easily lead to horribly damaging inflation. A few macroprogressives on the left and die-hard supply-siders on the right still discount that possibility, but the experience of 2021-22 shows why Democrats and Republicans need to ignore those folks.
So what do we do? In fact, we have lots of levers to fix the debt. Here’s Gimbel again:
The main remedies for these problems—higher taxes and spending cuts—are generally politically unpopular. Every budget fix will have its critics, but some options are more palatable than others. Better funding for the IRS, for example, could help close the “tax gap”—the amount of taxes legally owed that are not paid in a timely way—which the IRS estimated at about $700 billion a year in 2022. Other levers include raising the retirement age and reducing Social Security benefits for high earners, who also tend to live longer; reforming Medicare Advantage, a program that has been shown to allow private insurers to overcharge the federal government; and removing the tax exemption on employer-provided health insurance, so that these benefits can be taxed as income. The Congressional Budget Office regularly publishes policies that could help close the deficit, and Americans need to decide what we’re willing to pay for and what we’re not.
It’ll take political will, but we managed to muster that political will back in the early 1990s. What we need, most urgently, is to destroy all vestiges of the bipartisan “deficits don’t matter” consensus that emerged in the 2010s and early 2020s. Yes, deficits DO matter.
Over the past decade or two, Europe has become increasingly hostile to the software industry. Although Europe does have a few successful software companies (SAP, Spotify), these are few and becoming fewer. Europe has generally chosen to regulate software rather than build it, enacting the free world’s most onerous software regulation — GDPR — in 2016. That regulation, which is why you have those useless, annoying “Accept cookies” popups on every website you visit, convinced European leaders that they can control the direction of software development by regulation alone — trading access to the European market in exchange for the ability of European leaders to have a say in how global software is made and run.
Why exactly Europe decided to regulate software instead of building it is a topic that deserves a deeper and longer discussion. It’s possible that European leaders just concluded, like Xi Jinping did in 2021, that software is a consumer toy that isn’t “real” technology — that it absorbs resources that are better used for manufacturing, that it doesn’t strengthen the nation or create real value, etc. It’s also possible that, like progressives in America, European leaders got negatively polarized against the “techbro” class — the industrialists who amassed vast personal fortunes figuring out new uses for software — and who used software regulation as a way to deprecate the social status of those industrialists.
But whatever it was, the European bias against software is looking more and more dangerous and self-defeating in the age of AI. AI is definitely not a toy, and not just a consumer good — it will be a crucial input into every manufacturing process on Earth, and a critical tool of national security. Countries that don’t have access to the best models may be at the mercy of those that do. They will be more vulnerable to cyberattacks, less capable of controlling drone armies (which now dominate the battlefield), and far less competitive in any export industry.
In other words, software matters now in a way it didn’t matter as much back when Europe decided it didn’t need the industry.
A group of European think-tankers and researchers have written a document called Europe 2031, illustrating the danger that their region is facing. It’s a bit long and dramatic, but it gets the point across. The report paints a vivid picture of what it would look like for Europe to become a de facto economic satellite state of the U.S. and China.
Avoiding this will require Europe to adopt a different approach toward AI than it did toward the consumer internet. Overregulation needs to be avoided, but that’s not enough. As the authors of Europe 2031 note, simple NIMBYism — similar to the kind that paralyzes the U.S. — will be sufficient to stop Europe from building data centers. European leaders aren’t just going to have to allow the AI industry to develop — they’re going to have to overcome their ambivalence toward software technology enough to fight entrenched NIMBY interests to get data centers built.
Fortunately, the Europeans appear to understand the gravity of the situation. But modern Europe has a long and storied history of issuing rhetoric and proclamations about the need for policy change, but not making anything happen. Basically, Europe needs to re-learn the political art of making things happen, because the era when it could coast on the legacy of post-WW2 industrialization has already come to an end.
The U.S. notoriously has one of the worst rail transit networks in the world. This is partly because building trains is so incredibly expensive in America. The Transit Costs Project did a deep dive into figuring out why, and concluded that there are a bunch of different factors — bad procurement procedures, lack of standardization, and so on. Basically, my interpretation of this report is that American cities and states decided that trains weren’t that important, which prompted them to A) gut their train-related state capacity, B) allow a bunch of essentially parasitic consulting firms, contractors, and unions to dominate what rail budgets did exist, and C) ignore the need to streamline regulations that prevented rail from getting built. If we started caring about trains more, we could start fixing those problems.
But it’s also notable that trains aren’t equally expensive everywhere in America. The Boyd Institute produced a good chart of the Transit Costs Project’s data, showing the dispersion of costs per mile of tunneled train in various countries:
This is a logarithmic chart, so the cost differences are very big. The U.S. cost per mile is 4x that of the median rich country. But note that in median terms, we’re barely more expensive than Singapore, and we’re actually cheaper than Hong Kong — both of which have great rail networks. If we wanted to, we could build great trains too, like Singapore and Hong Kong, despite the costs.
But we don’t. To me, this drives home the fact that our cost problem has a feedback loop with our unwillingness to build. Yes, high costs make us less willing to build trains, but not building trains keeps costs high. If we’re going to get better rail in America, we can’t only focus on pure cost factors like station standardization and procurement processes (though these are important). We’ve got to want to build more trains! That probably means things like denser development patterns and less crime and disorder on trains.
But the chart also shows something else — a few train systems, like NYC’s, cost an absolutely enormous amount. NYC is already train-dependent, so their problem isn’t a lack of trains — it’s the inefficiency, corruption, and incompetence they’ve allowed to become endemic in their system. So in NYC, the problem really is just fixing the system. The Transit Costs Project does have a new report on how to do exactly that, but it’ll require New Yorkers to get very mad at their government and demand change.
“Critical minerals” are metals that are very important for modern technology — traditional industrial metals like aluminum, nickel, and cobalt, rare earths like neodymium and yttrium, and others like gallium and germanium. For years now, American leaders have understood that our country is in grave danger from China cutting off our access to many of these critical minerals.
But a lot of Americans seem to still misunderstand why that danger exists. A lot of people seem to think that it’s all about mining the minerals, and that if we just A) discovered more deposits, and B) loosened regulation enough to allow mines to be constructed, America would have ample supply of these metals. This is wrong and misinformed. The reason China controls our supply of critical minerals is not because we don’t mine these metals, it’s because we don’t refine them.
When you dig metal up out of the ground, it’s not in a form you can use in industrial applications. To turn it from ore into something usable takes a bunch of steps — mostly, dunking it in various baths of chemicals. This is called refining.
Most critical mineral refining is done in China. Every year the IEA comes out with a report on critical minerals. Here’s a chart from this year’s report that shows how dominant China is in refining, even for things like cobalt that China mines very little of:
In fact, China is becoming even more dominant in minerals like lithium and manganese. This is despite the fact that other than graphite and gallium, China doesn’t dominate the mining of any of these metals.
Why does China dominate refining of critical minerals? A few reasons:
China dominates the downstream industries in which some of these minerals are used. For example, China makes most of the world’s lithium-ion batteries, so it makes economic sense to locate lithium refining near to the Chinese factories.
China’s financial system gives a huge amount of cheap bank loans to companies that refine critical minerals, making it very cheap to finance these expensive plants.
China has somewhat looser environmental restrictions than Western countries.
China has been investing heavily in metal refining for decades, meaning that they’ve built up a tremendous amount of distributed tacit knowledge in difficult refining processes (such as rare earth refining).
If the U.S. and its allies want to get out from under the Chinese yoke, they will need to address all of these shortcomings at once. They’ll need to promote downstream industries like batteries (which Trump has discouraged and attacked) and electronics. They’ll need to revamp their financial systems to allow big loans for refineries. They’ll need to loosen environmental restrictions in targeted, judicious ways. And they’ll probably need to use AI to quickly reproduce the distributed tacit knowledge that makes Chinese refineries so efficient.
One big barrier to Indian industrialization is that women in India haven’t gone to work en masse. In most countries, industrialization starts when a ton of women move off of farms into cities and work in factories doing “light industry” — clothing, fabrics, toys, electronics assembly, and so on. This creates manufacturing companies with the capital and know-how to move up the value chain into more valuable and capital-intensive parts of manufacturing. Eventually the wealth created by high-value manufacturing creates the demand for high-value services industries.
That’s the typical story, but in India it isn’t working yet. This isn’t because factory automation has made labor-intensive “light industry” irrelevant — witness Bangladesh, which has grown a bit faster than India since 2004, largely on the back of its garment industry. Bangladesh has a relatively low female labor force participation rate, but it has increased its rate steadily over the decades. India has not; women in India are much less likely to work outside the home than women in other developing countries throughout the region:
Goldman Sachs analysts believe that getting Indian women out of the house and into the factory represents a huge economic opportunity. But Indian society seems to be resisting this.
A new paper by Alison Andrew and Andrea Smurra helps us shed a little bit of light on what’s going on. Low-income Indian women are increasingly not even leaving their homes:
We use rich time-use data on where, and how, individuals spend their time to quantify and describe women’s seclusion in India. We document extremely high levels of seclusion with the median married woman leaving home for just 0.5 hours/day and 45% not leaving home at all on a given day. Seclusion has increased markedly over the past two decades. While richer and more-educated women were, and still are, the most secluded on average, this gradient has flattened over time….Women are more likely than men to specialize in activities that are more readily done at home but are also more likely to carry out any given activity at home.
Why are Indian women becoming such hikikomori? Why hasn’t the lure of factory jobs been enough to draw them out of the countryside to the bright lights of the big city, as it was in Bangladesh, Vietnam, China, and a bunch of other countries? Andrew and Smurra don’t even hazard a guess. It’s not because they’re busy having lots of kids; Indian fertility rates are low and falling. Part of it might be simple lack of awareness of economic opportunities in factories. But whatever it is, it presents a big economic problem for India.
A lot of people like to have nostalgia for the 1950s. In fact, measuring by the murder rate, America is actually less violent now than it was back in the 50s (after a brief spike in the late 2010s and early 2020s):
People who think of the 1950s as being very safe tend to instinctively resist this conclusion; they think something must be wrong with the data. And so many latch on to the myth that murder rates have only fallen because of improved trauma care in American hospitals.
But it’s just not true. In a recent blog post, Jeff Asher explained how this myth got started, and how we know the narrative is wrong:
Basically, the myth rests on a 2002 paper by Harris et al., claiming to show that Americans are getting shot more and more often, but are surviving the shootings much more, leading to a decline in death but a rise in violence. Here’s their key chart:
So what’s wrong with this chart? The data on “lethality” isn’t official data — it’s just the authors dividing the aggravated assault rate by the murder rate.2 And the aggravated assault data they use is bad. They use the aggravated assault rate as reported by police. But as Asher explains in his post, American police forces got a lot more professional and thorough throughout the late 20th century. If someone hit you with a bat in 1955, there was a good chance you wouldn’t call the cops, or they wouldn’t record the incident. By 1995, there was a much better chance.
In fact, we can just go and ask people if they’ve been the victims of a shooting or an assault. When we do that, we can easily see that even as police-reported shootings rose in the 70s and 80s, shootings reported by victims didn’t rise:
So did aggravated assaults really rise in the 60s, 70s, and 80s? Almost certainly not. So aggravated assaults were probably not becoming less lethal over that time period. Yes, we had better trauma care, but we also had a lot better guns and a lot more dedicated murderers.3
In other words, America today really is about as safe as 1950s America. Sure, violence in the 50s was concentrated in poor and Black neighborhoods, but that’s true today too.4 What really changed in America since the 50s isn’t the lethal sort of violence, it’s urban decay and disorder. That’s what we need to focus on fixing.
Disclosure: I own shares in Impulse Labs.
This rests on the bad assumption that aggravated assaults are just unsuccessful murders. In fact, most aren’t. But anyway, that isn’t the main reason the chart is bad.
My guess is that a lot more murders these days are connected with the drug trade than in the 1950s, meaning that big money is at stake instead of just momentary passion.
Of course as I often point out, one reason America is as safe as it is today is because people moved out of the inner cities. Suburbanization acts as a defense-in-depth against street violence. So some of the 50s nostalgia isn’t misplaced. But nice suburbs in the 50s — the kind you see in all the nostalgic photos and old advertisements and memes — were probably a little more violent than the equivalent suburbs of today.
2026-07-26 15:56:22
“[I]ntelligence, that counterentropic conjoined twin of information, must become the most powerful force in the universe, the energy to which all other physical laws must eventually kneel…Intelligence was destiny, manifest.” — Ian McDonald, “Verthandi’s Ring”
Not a lot of people expected that AI would come for the mathematicians before it came for the truck drivers, but it did. The other day, an AI model disproved the Jacobian Conjecture — an 87-year-old open problem that human mathematicians had struggled to solve. The greatest living human mathematician, Terence Tao, turned to AI to help him understand the solution. Around the same time, AI solved a very important open question in quantum cryptography. Solving Erdos problems has now become almost child’s play for the best AI models. And this is the worst AI will ever be at math. Model capabilities, and the amount of compute available, both continue to increase at rapid rates. (Meanwhile, long-distance trucking employment is slightly higher than it was a decade ago.)
I don’t expect mathematicians to actually lose their jobs en masse, of course.1 But it’s becoming clearer and clearer that humankind has invented machines that are smarter than we are. Intelligence isn’t defined for machines the same way it is for humans — AI’s capabilities are spiky in different ways than ours — but it’s undeniable that the technology is improving rapidly in every domain of cognitive capability. It’s still possible to find some mental tasks that humans are better than machines at, but those final advantages tend to disappear almost as quickly as we can identify them. “AGI”, or “ASI”, or whatever you want to call it, is certainly here.
And yet…the world remains much the same. In lots of sci-fi books, as soon as artificial superintelligence arrives, it bootstraps itself to even more godlike intelligence in an explosive “singularity” that rapidly transforms the entire physical universe. Lots of people, especially “AI safety” and “effective altruist” types, expected things to play out basically the same way in reality. But looking around, not much has changed since we entered the intelligence explosion. There’s a huge data center boom, and most people use AI on a daily basis, but we still live basically the same lives — driving to work or taking the train, sitting in front of a computer, scrolling on our phones, collecting a paycheck. People are staying in their jobs longer, but employment hasn’t been disrupted in a significant way:
Meanwhile, we’ve had decently robust productivity growth, but nothing really amazing:
A lot of people I know are surprised by this. Ruxandra Teslo writes:
Walking around the world today one might notice that it is weirdly unchanged…To many, this is surprising. Just the other day I was at a conference where someone remarked that if he could have seen today’s AI capabilities a few years ago, he would have been astonished — and would have assumed the world by now would look far more transformed, with much higher GDP growth.
And Clifford Sosin writes:2
Superintelligence arrived. You probably didn’t notice, because it turned out to be kind of incremental…Don’t believe me? Run the test. Talk to Fable 5 for an hour, then talk to your ten smartest friends. Which one is smarter? Don’t worry, they won’t be offended…We were told to expect something bigger. Once machines crossed some line, the system’s IQ would climb to heights we couldn’t follow, and we’d be sharing the planet with something that designs warp drives and thinks thoughts as far past us as mine are past my dog…What we got is a tool that writes excellent code, beats people at a startling range of tasks, and will clearly reshape the economy. It’s also, somehow, incremental. No takeoff. No explosion. That’s the strange part.
Teslo blames bottlenecks — governance and other “frictions” — for the slow economic impact. But some others are advancing a more radical hypothesis3 — that intelligence itself is subject to diminishing returns.
One of these is Francois Chollet, an AI researcher who specializes in measuring AI’s capabilities. In a highly controversial series of tweets back in March, he conjectured that intelligence might be subject to diminishing returns:
One of the biggest misconceptions people have about intelligence is seeing it as some kind of unbounded scalar stat, like height. "Future AI will have 10,000 IQ", that sort of thing. Intelligence is a conversion ratio, with an optimality bound. Increasing intelligence is not so much like "making the tower taller", it's more like "making the ball rounder". At some point it's already pretty damn spherical and any improvement is marginal.
Now of course smart humans aren't quite at the optimal bound yet on an individual level, and machines will have many advantages besides intelligence -- mostly the removal of biological bottlenecks: greater processing speed, unlimited working memory, unlimited memory with perfect recall... but these are mostly things humans can also access through externalized cognitive tools.
In fact, this is a possibility I myself had raised in a post a year earlier:
It seems possible that humans are simply incredibly specialized in a few types of cognitive tasks — extracting patterns from sparse data, synthesizing various patterns into “intuition” and “judgement”, and communicating those patterns in language — and that we’ve basically approached the theoretical maximum in those narrow areas…That would explain why AI has gotten much better at things like math and coding and forecasting over the last year, but why the basic chatbot interface doesn’t seem much more “intelligent”. It would also explain why when you talk to Terence Tao about math, it’s like talking to a superhuman, but when you talk to him about where to get lunch or which movies are the best, he’ll just sound like a fairly smart normal dude. AI will eventually get better than Tao at math…but it may never get much better than the most thoughtful, eloquent humans at deciding where to get lunch or recommending movies. It may simply not be mathematically possible to get much better than we already are at that sort of thing.
Why would intelligence top out like this? Well, if we think of intelligence as the ability to extract information from data, then even an infinitely advanced model endowed with infinite compute will be limited by the fact that there’s a limited amount of information that can be extracted from the data.
For one thing, data itself is in limited supply. You can’t transform the world unless you can (in some generalized sense) understand it, and you can’t understand the world unless you can measure it, and our ability to measure the world is inherently limited and finite.
This is basically the hypothesis advanced by Arvind Narayanan and Sayash Kapoor:
We think there are relatively few real-world cognitive tasks in which human limitations are so telling that AI is able to blow past human performance (as AI does in chess). In many other areas, including some that are associated with prominent hopes and fears about AI performance, we think there is a high “irreducible error”—unavoidable error due to the inherent stochasticity of the phenomenon—and human performance is essentially near that limit….We predict that AI will not be able to meaningfully outperform trained humans (particularly teams of humans and especially if augmented with simple automated tools) at forecasting geopolitical events (say elections).[emphasis mine]
Sosin says something similar:
We hold a thin scatter of facts about the world, and intelligence or reasoning is whatever fills the space between them…Where the space between the facts behaves well, this is close to godlike…Coding, math and most administrative work are [like this]. What makes them easy is that they have relatively smooth solution spaces and are tractably verifiable…Most of what matters doesn't behave like that. The universe is mostly the emergent behavior of complex systems…The limit is contact with reality. A smarter reasoner fills the gaps between known facts in simple areas faster and better, but it doesn't produce new facts.
Sosin makes an important point here, which is that the limitations of intelligence aren’t necessarily about limited data. Even if we can keep on collecting infinite data, the cost of extracting additional information from that data might explode to infinity. This is the idea of chaos. Even in a deterministic universe where the present states of all particles are enough to perfectly determine the future, our ability to predict the future can be inherently limited; the tiniest infinitesimal error in our measurement of the present explodes into a huge error when we attempt to extrapolate even a small distance into the future.
So although we don’t know yet, it’s possible that humans were already hitting the point of diminishing returns with regards to individual cognitive capacity, and that superintelligent machines will never be as far beyond us as we are beyond dogs. But even if that’s true, I can think of at least three reasons why machine superintelligence could still deliver huge productivity gains.
The most obvious advantage that machine superintelligence confers is replicability. The number of human intelligences is fixed by the fertility rate, and we don’t know how to substantially boost that rate; in fact, it’s falling inexorably, and the human race is set to shrink.
AI isn’t bound by those limitations. By building more data centers with more compute, you can run more agents in parallel — essentially, you get more cognitive work. It’s not free, but nor is it limited. It’s good old physical capital; unlike human capital, you can just build more of it whenever you like, simply by reinvesting some portion of your economic output. Imagine if we suddenly discovered a way to manufacture more land in any city on the planet; this is similar.4
Of course, lots of tasks are physical ones, and for these you need physical machines — basically, robots. This is probably why so many AI people are now working on robotics, world models, physical AI, and so on. The roboticization of the world has a huge tailwind — the battery revolution, which allows energy to be stored and moved around much more easily. Conveniently, we got the physical tools to turn dumb matter into smart matter just as we also got the digital tools.
(It’s also worth noting that like the energy in batteries, the intelligence in robots is divisible. A robot the size of an ant can be remotely controlled by a data center the size of a football field. This is also a capability that human intelligence lacks.)
This doesn’t mean economic output will explode to infinity. But what it does mean is that humans will be able to use physical capital — GPUs and robots — to do more and more tasks at once, including many cognitive tasks that we used to do the hard way. In the long-run steady state, this should increase the capital-to-labor ratio of our society; each human will essentially leverage an army of intelligent machines. It’s basically another industrial revolution, and it has very little to do with whether artificial intelligence is smarter than human intelligence in any sort of head-to-head matchup.
The German company Zeiss makes the best glass on the planet. If one of the mirrors that Zeiss makes for ASML’s EUV chipmaking machines were the size of Germany, the biggest bump on that mirror would be just one millimeter high. Only a few other companies — and maybe no other company on Earth — can match that. Zeiss’ mirrors also have a number of other amazing properties, like not distorting much due to temperature changes.
How does Zeiss make glass this good? No one knows — not even the people at Zeiss. If the technology were capable of being written down on a blueprint, China would have hacked Zeiss and stolen it, the way Huawei hacked Cisco and Nortel. If the technology were capable of being explained by a former Zeiss employee, or even several former Zeiss employees, China would have paid those people many millions of dollars to spill the beans.
Zeiss’ technology basically can’t be stolen, because it’s tacit and distributed. It consists of a vast number of little tricks and techniques that a huge number of individual employees use on a daily basis. These people don’t always even realize all those little things they’re doing that make the glass come out so good. And each employee knows a different set of tricks and techniques. The knowledge exists at the level of the organization itself, and is thus very hard to steal or recreate.
This is true of lots of corporate technology. A big part of the reason China can cut off the supply of rare earths to the rest of the world any time it wants to is that other countries aren’t very good at refining rare earths. Rare earths are difficult to separate from each other in solutions; it takes a ton of little chemistry tricks to do it cheaply at scale. Chinese refiners have spent four decades building up those little tricks and techniques; American or Japanese refiners won’t simply be able to replicate their efficiency overnight, and so it’ll continue to cost much more to produce rare earths outside China.
Except in the age of AI, this might change. Suppose American rare earth refiners give their employees a bunch of equipment to record everything they do — smart glasses, gloves, and so on — in addition to sensors distributed throughout their plants. AI will be able to synthesize all that information and very rapidly suggest small ways to improve the production process. Many of those little experiments will fail; others will succeed and will quickly be adopted, allowing another round of experimentation and improvement to begin very quickly. Crucially, AI’s ability to do this doesn’t depend on its raw intelligence — only on its ability to handle huge amounts of data very quickly.
In other words, in the age of AI, distributed tacit knowledge might not be nearly as big of a barrier to technological diffusion. This could improve economy-wide productivity, as lagging firms catch up to leading firms much more quickly. A more equal distribution of productivity would also make the economy more competitive, creating more surplus for consumers (though possibly reducing the incentive for firms to innovate, by making technology less excludable).
AI’s ability to quickly produce distributed tacit process knowledge might also supercharge productivity growth at the frontier. Imagine if any company could optimize any production process five times faster than today. The whole economy would speed up, as components got cheaper, turnaround times and product cycles got shorter, and scale-up got much faster.
And as with the previous example, improving the production of distributed tacit knowledge wouldn’t depend on AI’s raw intelligence. It would spring from AI’s ability to act like a computer — to interface directly with sensors, to handle lots of data, to perceive tiny details, and to do everything very very quickly.
For decades, researchers in the field of natural language processing tried to figure out the principles behind human linguistic communication. They made frustratingly little progress; the processes by which humans convey information to each other through words just don’t seem to obey simple laws, like the ones that govern electromagnetism or the circulatory system.
Then along came AI, and suddenly linguistic communication seemed like a solved problem. LLMs can reliably sound like a human being, even if we don’t understand how they manage to do it.
What if there are lots of other aspects of the Universe that work the same way — too complex to understand in terms of simple laws, but not so complex that they just dissolve into unknowable chaos? It’s possible that we can reliably control these complex phenomena with AI, even if we never reduce them to the kind of principles that we can teach a grad student in a textbook. In fact, I wrote an essay called “The Third Magic”, where I suggested that this might be equivalent to a whole new scientific revolution:
Another way of saying this is that there may be laws of the universe that humans can’t understand but AI can. I call these “cloud laws” — causal regularities that can be exploited by technology, but which are too diffuse and complex for an individual human being to either intuit or communicate.
Human language seems to obey cloud laws, so why not other phenomena too? Perhaps social sciences like economics, sociology, and political science obey similarly complex regularities, and AI can help us find them. Perhaps there are physical processes — plasma, or topological materials, or aerial turbulence, etc. — that obey cloud laws instead of chaos?
In other words, thanks to AI, we might be on the precipice of a new age of scientific advancement. And this won’t necessarily depend on how smart AI is in comparison to a single human; it’ll depend on its computer-like ability to hold huge amounts of data in its working memory and extract complex patterns from that data.
If this turns out to be true, it means Francois Chollet is wrong. Chollet hypothesizes that groups of humans, using pre-AI computing tools, can approach AI’s level of scientific competence. But human collaboration is bottlenecked — it’s limited by our ability to intuit patterns at an individual level, and to communicate these patterns from one individual to another. AI, being a computer, just doesn’t have this sort of limitation; it can work with vast, diffuse patterns without having to break them into pieces or simplify them in order to compress them into the tiny pipelines of person-to-person explanation.
So even if AI never gets much better than humans at the kind of science that humans have done heretofore, it might open up whole realms of scientific discovery that have previously been totally inaccessible to even the largest groups of the smartest humans. If much of the Universe turns out to be ruled by cloud laws, we could be on the precipice of a scientific renaissance.
The common thread in all three of these examples is that AI may revolutionize productivity not by being much smarter than a single individual human — not by simply solving harder and harder math problems — but by marrying human-style intelligence to the vast, inhuman capabilities of computers. We could simply be thinking about the benefits of intelligence wrong — arrogantly privileging the kind of mental tasks we humans happen to do especially well, while ignoring the value of the tasks we do poorly.
Why? Several reasons. Tenure still exists. Humans who can do math at a high level will still be needed if we care about understanding the results that AI spits out. Human math teachers will still probably be valuable. And most importantly, humans will be needed to tell AI what kind of math we want it to solve, and why.
Or at least, prompts AI to write.
I think there is a widespread, tacit assumption among many intelligent humans that the quality that had made them stand out among their peers was the fundamental stuff of the Universe, the font of all value. I will write more about this at some point, but I think it helps explain why the idea that intelligence is just one production factor among many seems so unthinkable, heretical, and revolutionary to so many people in the tech industry.
Yes, I know we can reclaim land from the ocean. But the opportunities for this are fairly limited, and the cost is often very high.
2026-07-24 15:46:56
The story of the rise and possible impending fall of the American university system is, in many ways, the story of modern America. It ties together the changes in our culture, our economy, and our politics since the mid-20th century. Understanding why universities came to be our most important and most functional institution, and why that model is now under threat, can help us understand how our country might look different going forward.
Let me try to tell a condensed version of that story.
The United States used to have a bunch of institutions that bound us together and forged us into a unified society. Many American communities were centered around churches; these facilitated networking, helped people find spouses, provided community services like day care and mutual aid, and homogenized values and culture at the local level. During the World Wars (and to a lesser extent, the Vietnam War), the military was very large and threw together Americans from various social classes. In the early postwar decades, corporations were also a unifying institution. Mass media provided us with shared cultural context. Even public transit put people of various backgrounds in close social contact on a daily basis.
In the half-century from around 1970 to 2020, those unifying institutions became much weaker. Church attendance slowly declined and then fell off a cliff in the 2010s. The military shrank into a small, professionalized volunteer force. Corporations ended their brief flirtation with lifetime employment, and outsourced many of their roles. Mass media fragmented in the age of the internet. Public transit dwindled in importance as Americans moved to the suburbs and drove.
As the country’s unifying institutions withered, one new institution attempted to step into the void: the American university. Universities were not new in the late 20th century, of course, but mass attendance certainly was. From 1970 to 2020, the share of Americans age 25-29 with a bachelor’s degree went from 16.4% to 39.6%. Around two out of three have completed some college.
College went from something that only the upper crust did, to something that most people were expected to do if they wanted to be economically successful in life. This expansion inadvertently but inevitably thrust a new role on American universities — that of a broad socially unifying institution. College was where people from a variety of backgrounds mingled and mixed — not just in classes, but in dorms, campus activities, and college towns.
College became the new church, the new military, and the neighborhood bowling alley all rolled into one. Instead of preachers homogenizing Americans’ values from the pulpit, university administrators taught college kids to value things like diversity, consent, and so on, and college students hashed out their own differences in millions of late-night dorm room discussions. Instead of meeting their first love in high school, Americans increasingly delayed sex until college; many of these relationships turned into marriages. Young people went into college as children and came out adults.
This was a heavy burden to bear for an institution that hadn’t been designed for it. But American universities were accustomed to taking on big new duties. Our universities began as essentially a copy of the British model — teaching-focused institutions designed to provide broad education and mentorship to the upper class. In the early 20th century they tacked on the German model — a research-focused lab apprenticeship system by which top scientists taught other top scientists and readied them for research jobs while also producing cutting-edge basic research.
This dual system enabled a remarkable form of cross-subsidization. Undergrad tuition payments — and state support for undergrad education, and undergrad alumni gifts — created a flood of money that paid for grad student stipends, lab facilities, professor salaries, and more. The federal government and companies also funded university research, of course, through grants and sponsorships. But undergrad money was a huge tailwind. And the prestige generated by successful and famous researchers helped universities charge undergrads more.
The hybrid of the old British and German models was naturally symbiotic, and it meant that even as America’s other institutions came under pressure, universities thrived and grew — especially once they used their prestige to attract high-paying and highly skilled foreign students from around the globe. Universities became the lynchpin of America’s research effort:
American universities had a lock on both the nation’s research output and on its production of human capital, and those roles were mutually reinforcing.
I suspect that their success at handling education and research at the same time probably gave American universities a lot of confidence about their ability to handle society and culture as well. Colleges spent more and more on dorms and “student services” and hired administrators (many of whom dealt with undergrad life) at an astonishing rate.1
But replacing churches, the military, and the neighborhood bowling alley proved harder than replacing the corporate lab had been. There was just one basic problem with college as America’s primary unifying institution, which is that not everyone can go to college.
First of all, college is difficult. To complete college courses, you need some degree of raw intelligence, but you also need work ethic and a certain degree of independence. As much as we might like to believe otherwise, not everyone in America has those traits, and we don’t yet know how to instill them in everyone. As a result, there’s a limit to how much you can expand college enrollment — and college completion — without loosening standards.
In fact, loosening standards is exactly what American colleges have done. Universities have necessarily become less and less selective over the years, as they have taken in a larger and larger fraction of the young American population. For a while, this led to lower college completion rates, but in the 1990s, more and more students began finishing their bachelors’ degrees. Why? Because colleges implemented grade inflation, making it easier to finish school without learning the material well. Here’s Denning et al. (2021):
We find that most of the increase in graduation rates can be explained by grade inflation, and that other factors such as changing student characteristics and institutional resources play little or no role. This is because GPA strongly predicts graduation and that GPAs have been rising since the 1990s. This finding holds in national survey data and in records from 9 large public universities. We also find that at a public liberal arts college, grades increased holding performance on identical exams fixed.
At some point, though, this process hits a wall. A large fraction of young Americans just isn’t prepared for college, even with grade inflation, and doesn’t end up going. College enrollment by recent high school graduates plateaued in the early 2000s and actually fell back to early 1990s levels during and after the pandemic:
#/media/File:The_University_of_the_Arts_(UArts)_(53590485989).jpg){kind=link}