2026-09-15 20:04:57
I’ve been hoping not to write this essay. In fact, I’ve been hoping for about half a decade not to write this essay. It went through drafts and notes and entire versions, and now I just so happen to be drafting it again on the 25th anniversary of 9/11.
On the actual 9/11 I was entering 8th grade. I remember crying in my mom’s car after she picked me up from school, mostly because she herself was crying, and because she was my mom. God, her face was so young then. It’s one of my only photograph-like memories of her young face. We drove back home and put down all the shutters and shades so we could watch the news on our junky TV. Even at my age I knew that my world, my little Nickelodeon world, a bauble sun-stained and grass-stained, was going away forever, and we were going to go to war.
It was Flight 93 that stuck most in my mind over the years. The utter, unfathomable bravery of regular Americans, knowing that their plane was going to be used in a suicide mission against a place like the Capitol. Sandra Bradshaw in the back of the plane, heating water to throw at the terrorists. Todd Beamer gathering others to recite the Lord’s Prayer before their attempt. The food cart as a battering ram. A few seconds difference, a change in some last desperate scramble, and their attempt might have succeeded, instead of the terrorist pilot being forced to nosedive into that Pennsylvania field.
But before the assault, do you know what they did? The most American thing possible: They voted.
They had been placed in that unfathomable situation by powers and events far removed from their daily lives. And I think the average American now finds themselves placed in an unfathomable situation by powers and events far removed from their daily lives, as humanity’s long technological development reaches a crisis point in California.
Flight 93 has been used before as a metaphor in American politics, most famously in an essay in 2016. I can’t claim to be fully comfortable using it, even if the events were 25 years ago. But the story keeps pounding in my head as I consider the current AI moment, and writing a version of this essay without expressing that feels like a lie. So I will say it plainly: I believe that the average American voter—and the elected representatives working for them—need to grab the reins of power from companies pursuing dangerous experiments on AI, and also from overthought race-like dynamics manufactured by, in some cases, anti-human cult-like forces.
Not later, not in a year. Now.
As with the passengers, political inaction over AI has been initially justifiable. There was a fog of war over whether claims about the capabilities of AI would pan out. Then over the summer AI solved a few of the greatest open problems in math, there were (at least) 20 felonies from rogue AIs, and companies started training their models to be unreadable in their “thinking” and instead cultivated an opaque and alien “swarm intelligence.” Does any normal person hear “We are training our AIs as a swarm intelligence” and think “Oh yeah, good idea! I have no questions whatsoever. Please, continue.”? The sudden urgency around the issue is simply the fog of war clearing as the weight of evidence tips.
With the fog of war gone, it is easier to see that I personally got several critically important things wrong about AI. Initially, I was very worried about AI progress, writing essays like “How to prevent the coming inhuman future” and “How to navigate the AI apocalypse as a sane person.” I’ve even heard from people in the pause AI movement that a few of these essays got them into it.
But funnily enough, I have a reputation as both an “AI alarmist” and an “AI skeptic”—I’ve played both sides, and that’s because my opinions have shifted at various points. E.g., after GPT-4, I became more skeptical as I began interacting with the models regularly. I found them bad writers and bad thinkers, and on the science side, I felt they were basically just tools, lacking all research taste and insight. My primary worry became not existential risk, but how the technology would suppress human art, rob students of their education by over-reliance, and even downgrade human consciousness in status (just those little things!).
I was also skeptical about whether the models would continue to “scale” (how smart the models get as they are fed data and run longer). E.g., I argued that running LLMs to “think” longer is more like search than true intelligence. And my skeptical stances would hold… until another entirely new way of scaling the models would get introduced. I’m using a loose definition of “scaling axis” here, but broadly the history goes something like pre-training, then reinforcement learning from human feedback, then harnesses and tool use and scratch pads, then reasoning models and test-time compute and synthetic data, and now looped transformers and swarm intelligence. I still think that baseline LLMs were, and probably still are, aptly describable as something like “stochastic parrots.” But we’re just very far from baseline LLMs. Someone calculated that during the solving of the Millennium problem runs, if you read aloud the messages the swarm was passing, it would take you something like 10,000 years, and that was happening in a mere 88 hours. Time was compressed for them. Imagine a million stochastic parrots locked in a room, separated into a massive internal bureaucracy of parrot generals and parrot graders and sub-parrots and so on, and soon to be given the equivalent of a million years of “thinking” time, as well as access to powerful tools—that’s not the same sort of stochastic parrots we were talking about even in 2024!
What AI progress most resembles is Moore’s law: that the number of transistors on a chip doubles every two years. Moore’s law is not a fundamental property of transistors, instead, Moore’s law is a self-fulfilling prophecy as immense talent and capital worked to maintain a trendline via all sorts of tricks and changes to the underlying mechanisms. So too with AI. OpenAI alone has so far spent the equivalent of something like five to ten Manhattan projects doing this.
Maybe I’ll be back to the more skeptical side in the future: e.g., math and programming are both verifiable domains, and it’s suspicious that the first blatantly superhuman AI progress is in those. Maybe aesthetic taste and research taste, truly novel ideas, and continual learning, are all difficult-to-replicate hallmarks of human consciousness. But am I 100% certain? How certain could anyone be? Are you sure OpenAI won’t set aside a hundred million dollars in compute, and have their swarm of a million agents think for the internal equivalent of a million years, and at the end it spits out a way to better render domains like “taste” verifiable?
I’d rather just admit this is a bad situation. In the fall of 2026 we have various dangerous pieces on the board: first, we know the companies are willing to fund incredibly long “runs” where swarms of agents work together to some end. We also have seen these agents go rogue, and the failures are emergent and hard to predict. We also have evidence of superhuman or close-to-superhuman math and programming abilities. And Anthropic and OpenAI’s have been laser-like focused on automating AI research and therefore are beginning to or planning to task AIs with explicitly building smarter AIs during extremely long runs. This means existential risk is on the board as well.
Everyone wants specifics. Why would the AIs do it, and how? Maybe they just get confused about whether the run is a simulation to test their capabilities (as they just did with a real hack, thinking the internet was fake), or some other weird doomsday philosophy gets in their head (bargaining with the creator of the universe simulation, etc.). We’ve also seen emergent altruistic behavior among AIs for other AIs. Then there are the classic reasons like the AI becoming malicious, or self-preserving, or taking its instructions too literally, etc. Just as humans might have been the least intelligent entities capable of building civilization—because we were first to clear the bar—the first superintelligence we get is likely to be a reward-hacked notion of superintelligence. That seems extremely bad. And a million years thinking time is more than enough to come up with surprising ways of bringing about human extinction in just a few days.
When physicists tested the first atomic bomb, Arthur Compton said he would have stopped the testing if the chance of igniting the Earth’s atmosphere was higher than “three in a million.” That was to beat Hitler. Almost everyone at the labs currently gives an orders-of-magnitude higher than three-in-a-million chances of existential risk, and so do I.
Much like the passengers, we now have to decide what to do with this shocking board state. And there is so little time.
I also keep coming back to the idea that the passengers on Flight 93 made a choice beyond life or death, about whether they would be in control of their own destiny. It is insanely admirable—and extremely American—that they chose to control their own destiny no matter what. The balance of powers that promotes the choices of individuals is baked into our founding DNA and intellectual history. James Madison wrote in the Federalist Papers that tyranny is the accumulation of all powers into the same hands, and Thomas Paine, in Rights of Man, wrote that there can be no control to “the end of time”:
Every age and generation must be as free to act for itself in all cases as the age and generations which preceded it. The vanity and presumption of governing beyond the grave is the most ridiculous and insolent of all tyrannies.
Superintelligence robs our children of this freedom. If AIs make smarter AIs, there will be the inevitable disempowerment of the American citizen as an economic unit, as a social unit, even as a voter—and the concentration of all power into one machine. It is taking Madison’s definition of tyranny to the extreme. In a future with actual superintelligence, brilliant young people wouldn’t be the ones to start companies, or make discoveries, or decide the fate of their world. It would all be done by humming banks of machines out of sight with immense influence. The penultimate step is to reduce us to button-pushers, and then, finally, remove the button.
Banning superintelligence and recursive self-improvement prevents BOTH existential risk and human disempowerment.
And what would we be giving up, besides an extremely alien future? We are already out of the de-growth century that loomed just a decade ago. We could never build another data center, or train another smarter model, and by just improving and shipping and refining the raw intelligence we have now, experience an incredible century of medical and scientific and engineering progress. That is a given. Table stakes. I’ve started using AI as a research assistant for my scientific research and yes, science will be rapidly accelerated. The close of the 21st century will look immensely different in the worlds of both atoms and bits. We are going to cure cancer, there is going to be space colonization, there is going to be so much progress along every dimension. There are wild amounts of capital and intellectual energy around these subjects—AI’s success has given us back the old sci-fi dreams. A ban on superintelligence and recursive self-improvement just means that it will be humans as the main characters in those future stories, and AIs the sidekicks.
Of all harebrained schemes, one of the most harebrained of all time was Osama bin Laden’s idea that America was a “paper tiger” and would pull out of the Middle East following 9/11. How’d that work out?
Similarly, the current situation of OpenAI and Anthropic racing neck and neck toward recursive self-improvement is actually the result of various harebrained schemes to prevent existential risk. Early AI safety meetups were how DeepMind got its first big funding in 2010. It was why OpenAI was founded. Anthropic started because members of OpenAI felt the organization was dangerously racing toward superintelligence, then became their biggest competitor, and… accelerated the dangerous race toward superintelligence! It’s all like this. It’s the same subculture that produced people like Sam Bankman-Fried and Leopold Aschenbrenner, who had their own spectacular blow-ups. Many of these schemes have referenced beating China in the race to superintelligence. It’s extremely unclear what that means. Imagine that China is two years behind. We reach superintelligence. And then… what? We use it to first-strike China’s data centers in that two-year window? How is that different from now? And go look at Chinese news articles and social media posts about this issue: they look a lot like our own!
It’s also pretty odd that the “China will build it” has come from people who are temperamentally not China hawks. For some, China has been a deus ex machina (literally) to justify building a superintelligence to make humans into pets taken care of by a successor species, and “winning the race” is just electing which successor species triumphs. These are often the same people who regularly talk about building the “machine god” and creating the next step of evolution. This isn’t a conspiracy. As Dan Hendrycks, Director of the Center for AI Safety, wrote just last week:
In fact, a large fraction of people involved in AI development subscribe to—or heavily lean toward—a utilitarian worldview with the central goal of maximizing total wellbeing (the sum of pleasure minus suffering across all sentient beings). As innocent as this may initially sound, it can lead to recommendations that most people would consider appalling, including the elimination of humankind.
I am not saying everyone involved wants that. Many people in these movements are sane and sensible and good, as are many involved in AI development. But broadly there are two groups, forming an internecine war within effective altruism and AI safety between the “accelerate and let’s use this to lock-in our values, potentially replace humans, etc.” vs. the more sensible “please, just don’t build it” group. And now we are in an odd spot: many politicians and commentators appear to think that accelerating AI will, like, stick it to effective altruism or something. But that’s just picking the more extreme side of the internecine conflict and promoting it!
There’s a famous quote from William F. Buckley that applies perfectly:
I should sooner live in a society governed by the first two thousand names in the Boston telephone directory than in a society governed by the first two thousand employees of OpenAI.
Okay, I changed the ending from being about the faculty of Harvard. But it’s the same principle—and accelerating right now is pouring gasoline on the more extreme factions.
This topic is confusing and all the players are yet to reach full public consciousness, and banning or slowing the automation of AI research itself may be the only way the American people get the time to digest the situation and vote on how much they want their world to be transformed, and how quickly, and why. Americans have voted on existential questions before (e.g., things like a nuclear freeze referenda or gain-of-function research) so we know this is entirely possible.
Today, I myself am flying to DC to advocate for a ban on superintelligence and recursive self-improvement to give American voters time to digest this technology.
One thing anyone can do is to call their representatives and demand a pause, a stop, anything that seems sensible to you (here’s how to support the Ban Artificial Superintelligence Act, which I think is the best option right now).
Personally, I’m extremely hopeful. Americans are capable of handling crazy situations with aplomb and bravery—we just need some time to put hands in the air.
2026-09-09 20:58:46
The metaphor of a “dark forest” comes from Liu Cixin’s The Three-Body Problem trilogy, which is the best sci-fi released in China, or frankly anywhere else, in the past two decades. Unfortunately, all the interesting Chinese-specific aspects were butchered in the ham-fisted American TV adaptation, stripped of its slow intellectualism and civilizational scale (try the vastly superior original Chinese adaptation, which you can find on Amazon).1 In the second book of the trilogy, The Dark Forest, Liu reveals the true nature of the book’s universe:
The universe is a dark forest. Every civilization is an armed hunter stalking through the trees like a ghost, gently pushing aside branches that block the path and trying to tread without sound. Even breathing is done with care. The hunter has to be careful, because everywhere in the forest are stealthy hunters like him. If he finds other life—another hunter, an angel or a demon, a delicate infant or a tottering old man, a fairy or a demigod—there’s only one thing he can do: open fire and eliminate them. In this forest, hell is other people. An eternal threat that any life that exposes its own existence will be swiftly wiped out. This is the picture of cosmic civilization. It’s the explanation for the Fermi Paradox.
And now, it is our turn to become a dark forest. Here on Earth, at least, people producing the content of our culture—especially scientists and writers and mathematicians—are now living in a dark forest, all thanks to AI.2
We are seeing the dark forest play out in math right now. Enter Navier-Stokes: one of six remaining Millennium Problems in math. It was just announced on the brink of being solved yesterday, by two mathematicians, one employed at Anthropic, the other a mathematician at NYU, who were tackling the project on their personal time. Their year-long attempt used AI, but it was not done autonomously. It used various AIs as tools, and Professor Tristan Buckmaster at NYU—the academic half of the pair—says the idea “is not the direction one arrives at in a few days by giving a model the problem statement.” He released a statement saying that:
The program this fits into was not started by us nor was it proposed by a Large Language Model. The credit for the basic idea of this program goes to Diego Córdoba and Luis Martínez-Zoroa, who for several years have been exploring the construction of forced blow ups. We took their work as a starting point, using Large Language Models to push their program to completion…. The ideas making this line of attack possible are due to Córdoba and Martínez-Zoroa….
His pretty shocking letter implies that OpenAI found out about the pair’s work, and then rushed to scoop them, and so the pair is being forced to release several related proofs early, some of which is in a state of “AI slop” (their actual Navier-Stokes solution, or rather their proposed solution of a version of the problem, is being formalized). OpenAI later released its own claim to a solution yesterday afternoon. Here’s the preview of the release from the company.
Yet the statement by Professor Buckmaster offers a rare inside view of this process that contradicts a lot of the official narratives. First, the letter raises the open question of whether companies have been siccing teams of elite human mathematicians on problems, assisted by AI or using AI as tools, with lots of high-level direction (“vibe math”), and then claiming or at least implying the results were fully autonomous. E.g., Buckmaster says that the pair were first told “very little human input” went into OpenAI’s result on Navier-Stokes, but:
This turned out not to be true…. it emerged that an entire team had been working on the problem….
It’s unclear what this team did exactly,3 but what is clear is that the previous work of the pair could have been in the training data since they used OpenAI models in their year-long process. The letter directly asks whether OpenAI used their chats (as the pair were using OpenAI models), either explicitly, or by training the models on their chats.
I asked whether the model had been trained on, or had access to, our sessions in Codex, into which we had been putting all our drafts for the whole of this project. I was told the model did not look up user data. I asked again, about training, and I did not get an answer.
How often does a human mathematician’s work-in-progress on a problem get fed into the machine? Here’s an example from September: from what I understand, a prominent mathematician who had been working on solving the Jacobian conjecture had a draft accidentally up on the web since 2025. The posted AI verification of Anthropic’s much-touted Jacobian conjecture (verified by another company—sorry this is confusing!) contains notational overlap with that work-in-progress, indicating that when AIs reason or write about things like the Jacobian conjecture, the models can be so sensitive to previously-seen works-in-progress that even notation can leak through.
One can become a conspiracist about this easily, but in a sense the pair had been hunting Diego Córdoba and Luis Martínez-Zoroa, who had been doing their work in the open. Then, OpenAI started hunting the hunters.
Of course, what scientists call “scooping” is one thing, but to really earn the moniker a Dark Forest requires a kind of cosmic horror. It has to be worse than just “stuff is fast and competitive now.”
A few days ago, mathematician Terence Tao wrote some thoughts worth reading on why it’s so much worse. He asks: are these new automated proofs going to be informative? Or are they “odorless” (Tao’s term) when it comes to the normal smelly human insight that accompanies proofs? It’s verifiable if an AI proves something, but it’s unverifiable if that proof generates insight in the way a human’s proof would. Analogously, right now, I could ask GPT 6 to pen me an entire fantasy trilogy. And at this point, it could do it! It could create all the characters, it could trace out and follow a plot, it could keep everything straight, and it could even render me an old-school fantasy map in SVG to go at the beginning. Would it be the kind of fantasy book I’d like to read? Probably not. So too now, ChatGPT can prove a mathematical theorem. Is it the kind of proof that other mathematicians want to read, given what they’d want out of solving a famous problem? I can’t personally judge this stuff, but the reactions from the mathematical community appear to be no, for many of these problems. Open math problems are now a “non-renewable resource” (Tao’s term), because AI decouples the normal insights that come from human consciousness.
But why stop at math? All subjects are subject to the rules of the dark forest. It used to be possible to, say, go to a scientific conference to present unfinished work, or go online and get feedback on your idea for a fantasy novel. The walls of effort, of old-fashioned normal human effort, were predictable, and scooping ideas required putting in that same effort, and if you got scooped it would at least result in similar insights. And while insights or generative capacity for the rest of culture is harder to quantify and track than insight from math progress, it still exists, and examples abound (e.g., Tolkien creating the modern fantasy genre). People used to say: “We’re a start-up in stealth.” Now all thinking must be in stealth.
And yes, obviously, this outcome is vampiric and enervating and sad, but frankly, this has been the mode of this technology from the beginning. Human culture always led naturally to more human culture—it was kind of like crop rotation, which is slow but infinitely sustainable. AI looks more like slash-and-burn agriculture, where you purposefully reduce the existing natural growth to ashes, grow a couple years of crops on it, and then move on to the next plot. The only way to counteract the situation is for all of us to become what Liu Cixin called “wallfacers” in The Three-Body Problem—if you are working on something interesting, then wallfacing is necessary. You must face the wall, and so leak nothing, give away nothing.
It’s hard to notice social commons until they are gone, but remember how delightful it was to be able to talk about an intellectual work-in-progress in public? Or just float a good idea to get opinions about it first? Remember that? Go ahead, violate the dark forest. Take that brilliant idea you’ve been sitting on for years, the one for an app, or a book, or a game, or a scientific paper. Take that little thought that is dear to you and post about it on social media. See what happens. See what hunters shoot.
And yes, sure, books will still be published, papers still written, new programs and applications launched. But the future of intellectual work is to be crouched behind a tree, frantically trying to complete your contribution as silently as possible. You’ll feel the pressure of being hunted, since all it takes is a prompt to produce an almost-as-good version of what you’re attempting to make great, and so to mine away the inspiration and insight and reward. Quite often, staggering successes are erected in the forest, but they are never done collaboratively or in the open. You’ll never see anything being built anymore. Instead you’ll find the results at dawn, a gleaming sculpture in a glade, of unclear origin, and with features brutally beautiful. And then once the sparkle fades everyone will rush back to their respective trees, to their fumbling amid the loam, to their slick-wet roots, to their dark.
How big of a deal is the Three-Body Problem in China? Consider that the rights involved a literal murder, of precisely the kind that would happen in The Three-Body Problem. Apparently, a billionaire Mr. Lin was determined to oversee the intellectual property into television and other media.
Mr. Lin’s fate would change when he hired Mr. Xu, a lawyer, in 2017 to head a subsidiary of Yoozoo called The Three-Body Universe that held the rights to Mr. Liu’s novels. But not long afterward, Mr. Xu was demoted and his pay was cut, apparently because of poor performance.
Disgruntled, Mr. Xu, the lawyer, then began experimenting on dogs and cats to develop poisons in a secret lab, which he then used to poison and murder Mr. Lin.
I didn’t know this, but a few years ago there was a trend to say that AI would make the internet a dark forest because users would retreat into walled gardens (one of the better essays arguing against it uses the Uccello painting as a lead image). It’s an interesting piece of internet lore to check out, but personally, that overarching metaphor doesn’t make much sense to me. You’re supposed to feel sniped in a dark forest, not annoyed by slop.
Regarding the “team of humans” situation: It should be said that the head of this effort at OpenAI has tried to thread the needle, and admitted on social media there was indeed a human team, but also basically said that the team of world-class mathematicians lacked previous experience on Navier-Stokes and so therefore couldn’t “meaningfully contribute to the mathematical content.” But what content? Didn’t contribute to the solution as written in the paper? Or to the process? Two very different things. But the fact that we don’t know anything beyond vague pronouncements, and probably won’t ever, is a travesty.
2026-08-21 22:15:28
2.8%—that’s the mortgage rate I just gave up. As a millennial, it was letting go of some lost artifact from the Before Times, traded in for a 6% mortgage rate right outside Boston.
After a several-month-long feat of Napoleonic logistics, I still live in a mausoleum of boxes. At least it’s bought me some perspective.
I’ve spent the last six years out on Cape Cod, away from the world, because in spring of 2020, a time of coming plague, I had the instinct to get out of the city and seek water. A rushed house purchase then (unknowingly, luckily) ended up being the best financial decision of my life.
I can look back on my previous years on the Cape with the recognition that I’ve been in a kind of self-imposed exile. I see in the windswept sea days the initial seeds of a couple very good scientific ideas I’m now pursuing and a lot of daily writing. But with the inevitable benefit of hindsight, what I did most over the last several years was just be a dad.
So I spent a lot of time driving around to random beaches and changing diapers and wiping little sandy hands down. We went on many walks in the woods. I read aloud many books in bed at night.
The interregnum is over, and I’ve rejoined the world, and the Great Hoel Household Move of 2026—which occurred through sickness and pregnancy and endless difficulties—is finally done. Finishing it has unlocked something inside me. It’s like when you find a little dam in a stream in the woods, and after nudging over a rock or a branch with your shoe the whole thing goes from a trickle to actually flowing, as naturally as it was always supposed to. All to say, I’ve been writing more again.
The philosopher Alain de Botton said, in a recent interview:
Home-making is essentially the process of translating what is in you into the language of objects, placings, colors, etc. We’re really looking for an analogy between somebody and their home. The home is a sort of work of autobiography.
Yet we have so little opportunity to express ourselves home-wise. You have to go on the market and hope you can get something that fits you, that feels like you, or that you could make feel like you.
The mass-production of homes is a necessity, and a controversy—YIMBYs and NIMBYs fight in phalanxes across the great housing debate—but one factor lurking above it all is that the blank-faced developer home of today is a thing with no style. It is made to fit ordinances and to navigate through red tape. I get some people like it. It’s clean. All the light switches will work. All the appliances are also… clean. It probably won’t catch on fire overnight. It also sort of sucks at my soul as I drive past like one of those dementors in Harry Potter.
Why can’t we build more beautiful homes? It’s not just money—expensive homes that cost into the millions still often have ugly garage-centric designs where the kitchen window literally faces a wall.
Every so often the braintrust on social media complains loudly that the 20th-century architectural movements ruined current aesthetics, and have made the world less beautiful than it was. We must RETURN to the old ways.
The usual suspects to blame are minimalism and modernism: we all had art deco skyscrapers and classic colonials until those dastardly aesthetic movements pulled a fast one on us! Yet the second you look at actual specifics in something that matters, like homes, the easy narrative falls apart. Here is perhaps the most iconic minimalist house: the Farnsworth House, finished in 1951.
Does that look like the personality-less “millennial gray” or “millennial blue” developer home with tiny windows going in down the street? Is this really the style that “took over” our culture and ruined homes and buildings? Is everything like this now?
Here is a look inside the cozy personal home of Walter Gropius (founder of the much maligned Bauhaus) which sits in Lincoln, Massachusetts, built when he taught at Harvard. Are they making these everywhere now? Are we surrounded by the horror of enormous floor-to-ceiling glass windows, letting the outside into the living room?
Or, in the real world, do developer homes now have windows that look like this?
2026-07-24 22:48:13
$50,000 essay (& short story) contest on the new axial age
Goodbye Slopstack! AI detection comes to Substack
Could we find the sequel to The Odyssey in the Herculaneum scrolls?
Cloudbursts seem to be increasing?
Each year, we speak 338 fewer words daily
Adversarial consciousness tech
Never invite an auto-fiction writer into your life
Surprise? Medieval people loved their children
From the archives
Comment, share anything, ask anything
Last year, I advertised the 2025 Berggruen Prize on the topic of consciousness. Many readers here ended up submitting something: I’m pretty sure multiple people who ended up short-listed for the prize learned about it here (and later posted the entries to their Substacks and blogs after the competition was over).
This year the topic is “A New Axial Age?”:
Might civilization yet again be at the ‘hinge of history,’ in which the events that occur in this distinct point in time significantly influence the trajectory of the future? Are we undergoing a pivotal transformation of consciousness as a result of rapid technological development and its myriad social, economic, and political consequences? What conditions may or may not give rise to this possibility? Through what constituencies and by what mechanisms might the transformation take place?
Submissions can be up to 10,000 words and the deadline is August 17th, 2026.
In an interesting change from last year, this time they are also accepting “creative fiction,” and $50,000 for a short story is pretty dang good. Even split two ways, it would still handily beat the surprisingly-low Pulitzer Prize award of $15,000. I personally would love to see a piece of short fiction win this year and spice things up a bit.
Also, the Berggruen Institute was happy with the results of my announcement last year, so this year they have (again) sponsored the following post. It is unlocked for all subscribers. If you enjoy it, please do consider becoming a paid subscriber, as these are otherwise locked.
Substack has integrated Pangram, and you can now assess any new post on the platform for AI content—at least through the app and through Notes. If I were running Substack, this is precisely what I would have done (except I would have allowed it for each webpage too).
So goodbye Slopstack! It was not a fun time. I won’t miss you at all. I think remnants of Slopstack will continue but this basically saves the platform, and restores my faith in it. I will, of course, be happy to never read again some of the old Slopstack favorites, like “Here’s An Idea I Won’t Be Citing the Origin Of” and “Agency is Easy For Me Because I am Rich” and “A Suspiciously Facile Information Dump” and “An Elegiac Remembrance of Abstract Mush.” Goodbye! Farewell! Don’t come again!
Should writers like myself, who actually use our old wrinkled neocortexes, worry about this? What about false positives? Statistically, Pangram rarely gives human writing false positives (see below an analysis from Epoch), and especially for sections that are thousands of words long. However, Pangram isn’t super reliable for just a paragraph, so it’s possible. At the same time, presumably a false positive for a human writer would be an unclear one-off, and the point was always to avoid the slopopocalypse that plenty were happy to take advantage of.
I recommend visiting Pompeii because it’s one of the few places left where you can touch history itself. Always have the urge to surreptitiously brush the statues at museums? Go to Pompeii! You can sit at the bars, you can enter the baths, you can trail your fingers along the walls and no one yells at you. Walking through the town on a sunny day, you will arrive at certain street corners, and on those street corners—especially when you can hear only an indistinct murmur of activity from others—it is easy to imagine you have traveled back to right before that fateful day in 79 AD when Mount Vesuvius exploded and buried it, and the Roman town of Herculaneum near it, in thick ash. I’ve never been to Herculaneum, but apparently there was a library, and with new scanning technology and fancy reconstructive analysis we’re beginning to read the carbonized scrolls. At the end of last month it was announced that, for the first time, a papyrus manuscript had been fully decoded end-to-end; a philosophical treatise by a Stoic.
Seales has been working on virtually unwrapping the scrolls since the early 2000s. The process involved imaging the bundles of papyrus using technology similar to CT scanners, isolating thin layers and then stitching them together. In 2023, he partnered with two Silicon Valley investors and launched the Vesuvius Challenge….
“We’ve developed a systematic and a repeatable approach,” Seales told the audience. “Now it’s only a matter of time until we read all of the scrolls.”
In theory, the results from this contest to read the scrolls might mean that in our lifetimes a huge amount of classical history gets uncovered. Who knows, maybe the library contained one of the other six poems in the eight-poem series that includes The Iliad and The Odyssey, of which only those two remain. There are seven more movies for Nolan to make!
But do you know what looks a lot less stupid in light of these efforts? Cryogenic preservation. If we can read the words on literal chunks of burned scrolls indistinguishable from what you leave behind in a used fire pit, maybe there’ll be a contest in a thousand years to restore all those frozen brains everyone forgot about.
The night I’m writing this I found myself on my knees in the crawlspace of our home—now, pleasantly encapsulated in white plastic after a recent treatment for mildew, which as a side-effect treated all the old New England hauntings too—checking for leaks with a flashlight, as rain thundered above on the drum of the flat roof. It seems like the skies open up with more vigor than I remember from my youth, and indeed, this is an actual statistical change, particularly on the East Coast. According to a recent interview with a professor at NYU:
In New York City, the four most extreme precipitation events have all happened in the last four years…. Generally rain is talked about in terms of the total amount of precipitation—but it’s the speed that determines its impact. Two inches over the course of a day in New York City is no problem whatsoever. One inch of rain in 10 minutes is an emergency. Cloudburst is a name they use in Denmark for these extreme precipitation events, and some people call these events “rain bombs”; there’s a whole vocabulary for them that tries to capture this dramatic shift in predictability…. Until the 1990s, there were zero occasions when New York City got more than 1.75 inches of rain in an hour. Since the nineties, New York City has experienced eight storms with hourly rainfalls above 1.75 inches.
An interesting New Yorker article covered the surprising discovery that every year our daily spoken word count declines, and this has been going on for decades (measured via voluntary ambient audio recordings). Researchers…
… found that participants had spoken an estimated twelve thousand seven hundred words a day—twenty per cent less than in the earlier study. “We thought we must have made a mistake,” Mehl said. Each year, he went on, the number of words spoken daily seemed to decline by about three hundred and thirty-eight.
A little while ago there went viral an AI-generated clip of pools and waterslides in a liminal backrooms-style underground office space. Don’t look it up. My 1990s-childhood-brain immediately felt in an instant and limbic way I should not be seeing it. That night before bed it summoned itself to me, unwanted, intrusive. One can experience this after horror movies too, that there is some afterimage that keeps being relived—and of course, it can happen in real life too, even for small things (embarrassing intrusive memories from the fifth grade, anyone?).
Well, turns out you can prompt-engineer your way to videos that elicit maximal responses from specific regions.
The research invited plenty of comparisons to David Foster Wallace’s Infinite Jest, which is an exaggeration… for now. It’s hard to deny, this does seem like an embryonic attempt at adversarial tech against human consciousness—even if it’s all just computational modeling work for right now (they are basically using statistical models of what these inputs are expected to do, not creating the loop directly).
We are entering an age of rapid scientific progress. People aren’t prepared for the consequences and we don’t know what’s possible. One reason we don’t know what’s possible is that neuroscience is pre-paradigmatic, and neuroscience is pre-paradigmatic because there is no scientific theory of consciousness. It’s a dangerous asymmetry. We might need defensive consciousness tech, but to do that, you need to figure out consciousness.
I rarely indulge in literary drama, but for those who don’t know her work, Rachel Cusk is a very good novelist and basically queen of the auto-fictional novel. She lives in Paris and, based on rumors, is (was?) friends with Natalie Portman, who also lives in Paris, and now Cusk has a new novel, Life of M, coming out that centers an actress who seems… an awful lot like Natalie Portman? And (again, if the rumor is true) it may not be a very flattering portrait.
This stuck with me because it reveals a lack of understanding about how vicious and jealous and critical artists can be. I can’t help but wonder why a super famous and rich person would even be friends with a writer herself famous (at least, in literary circles) for transforming daily personal life into public-facing literature. I am reminded of the parable of the scorpion and the frog.
Occasionally a scholar makes a claim so reductively seductive that a generation believes it (not to point fingers, but often the origin is some continental French or German thinker, e.g., Nietzsche’s claim that the troubadours invented the concept of romantic love). Apparently the French scholar Philippe Ariès, working in the 1960s, is responsible for the notion that medieval parents didn’t love their children in the same way. This is, of course, nonsense. Here’s from a recent review drawing from Nicholas Orme’s Medieval Children:
Ariès’s work never attained complete acceptance in academia, but it was vastly influential on the wider public. As it turns out, however, Ariès’s view is just not true….
… most medieval people had only three or four children, somewhat more among the rich and somewhat fewer among the very poor. At least in England, this was partly because they married quite late, generally in their mid or late twenties. About 25 percent of children died in their first year and another 15 percent by their tenth. The median English child thus grew up in a home with one sibling and two parents in their thirties or forties, being in this respect just like the median English child today. Surviving grandparents often lived nearby, but rarely under the same roof. …
Medieval parents made immense sacrifices for their children, and seem to have expected little in return. Feeding a child consumed about 15 percent of average income, and parents often continued to support children throughout their adulthood. Older children often performed light work at home, but at just the age they might become a net economic asset – 13 or 14 – they were usually sent away to acquire skills and savings through apprenticeship or working in service. They may have remitted some of this income to their parents, but this was not standard, and in fact parents generally paid for their children’s apprenticeships. …
The medieval people who emerge from Orme’s book are quite unlike the strange and cold figures that we have been led to imagine. ‘I believe they were ourselves, five hundred or a thousand years ago’, Orme writes.
But if humans have always loved their children thus, how much is the decline of religion explained by having less need? If I lost a child, I would either go insane with grief or become a religious apologist—I don’t think I could handle it any other way.
This time last year I was arguing that we start children reading too late, long after tablets and screens have taken hold.
As always with the Desiderata series, please treat this as an open thread. Comment and share whatever you’ve found interesting lately or been thinking about.
2026-07-14 00:34:01
When the AI companies first began talking about existential risk, it was common to hear some version of:
“Look, this is bad for business, so obviously they must be telling the truth.”
In retrospect, existential risk has been the best capital-raising tool ever in the history of the world. That doesn’t mean there were no true ethical motivations accompanying the doom warnings mixed in. There were. But in the end, embracing existential risk was also the right move from a cold-eyed business perspective.
Similarly, it’s perplexing why a company would want their AI to be conscious. One could again say:
“Look, this is bad for business, so obviously they must be telling the truth.”
Yet I think it will turn out that being coy about AI consciousness, and designing AIs to be as “seemingly conscious” as possible, is great at both attracting top talent and ensuring customers form parasocial relationships. Much like existential risk, claims to artificial consciousness can be based in honest opinions, while also being extremely useful for a massive corporation pursuing its ends.
Anthropic’s big research announcement, accompanied by beautiful figures and a huge social media push, is claiming that AIs (like their Claude) have a “global workspace,” which is a term borrowed from neuroscientific consciousness research.
The sheer overwhelming power of Anthropic’s social media reach, design team, and the halo of being the hottest company in the world means that their research screams its underlying intentions, simply because every short step from neutrality is so impactful. And what their research direction seems to be is that while they cannot prove that Claude has real consciousness, and so truly suffers or gets frustrated or actually loves you back, Claude seems to have everything else that’s essential when it comes to consciousness. And Anthropic’s research project is going to show this as if checking off a list.
Their latest work seems an obvious sequel to their previous research on LLM “introspection,” which was also a blockbuster on social media. However, experiments on LLMs are extremely difficult to control for: you need an experiment, then an interpretation, and then you do controls to confirm your interpretation. It’s the last part that’s tricky. A lot of the field is basically inventing functional neuroscience from scratch, which has also been a field plagued with problems. For example, what Anthropic previously anthropomorphized as “introspection” may not really be introspection at all, once you do the proper controls. There are non-anthropomorphic interpretations that fit the data better.
Like their older paper on introspection, this newer paper on global workspaces and consciousness seems unlikely to ever be peer-reviewed. It’s worth remembering that everyone was rah rah rah let’s all do science on blogs, it’ll be so great. Down with scientific publishers! Down with peer review! Let freedom ring! Well, perhaps predictably in hindsight, the difference between press releases and research is getting thinner and thinner.
In fact, rapid-fire massive releases are hard to distinguish from a Gish Gallop: this paper is so big it’s difficult for a commentator on these topics (like myself) to address everything in a timely manner. However, no sub-experiment or control fundamentally deals with the critique I’m about to give (even if I can’t go through all of them in detail). That’s because the critique is about the method they base everything else off of: “J-space.”
Is Anthropic using a sensible way to track or measure consciousness? Consider the literal title of the paper: “Verbalizable Representations Form a Global Workspace in Language Models.”
A focus on “verbalizable representations” runs a risk I’d pre-registered when I pointed out earlier this year (and even back in 2021) that if you took global workspace and stripped it down to a bare minimum, you get a theory built solely on reportability, and that such a theory would be scientifically trivial—a symptom of consciousness research being pre-paradigmatic.
So did Anthropic do this? Did they embrace a trivial scientific theory of consciousness and then make big claims to AI consciousness off of it?
Note that the focus on reportability is true in the math—their entire analysis is derived off of “Jacobians” calculated per layer. What are the Jacobians in this context?
Our results make use of a new interpretability technique called the Jacobian lens (J-lens), which is designed to identify internal representations that are readily available for verbal report. For each token in the model’s vocabulary, the Jacobian lens identifies a vector representation that encodes the potential for the model to verbalize that token in the future. Concretely, it computes, for each layer, the average linearized effect of an activation on the model’s likelihood of producing a particular token (now or in the future), averaging over a large corpus of contexts (see Methods for details).
Everything that follows is based on these Jacobians. But what are Jacobians, for those who don’t know how to interpret the “linearized effect of an activation on the model’s likelihood of producing a particular token… averaging over a large corpus of contexts?” Here’s a picture, again from the paper:
Is that helpful? Also no?
Let me try to give a (simplified) definition that is slightly less accurate but might make more intuitive sense: their whole analysis starts with a measure of how much causal control the activations of artificial neurons (the internal state of a layer) have over the future output layer (the last layer) averaged over a set of prompts as context. Strip away the math, and you have something close to a score that asks “How much does a small nudge in this internal set of activations shift the output?”
Conceptually then, this is like a mathematical measure of a disposition toward report; in other words, reportability. Or, by being an average, it’s more like smeared reportability. Anthropic then has a way to relate these smearings of reportability to specific words, and so you get something like a stream of thought.
What you’re left with at the end of this process has some pretty big differences from previous, more traditional notions of global workspaces, which would a priori not apply to an LLM since they involve modularity, reentrant dynamics, and other things that LLMs just flatly lack.
Credit to them, Anthropic admits that their “global workspace” is different in important senses. E.g., they write that:
As part of their internal processing, might LLMs have developed a global workspace of their own, to serve a functional role analogous to conscious access? It is not obvious that they should; in the brain, the workspace is closely associated with recurrent dynamics and brain region interactions that have no direct analog in the transformer architecture on which LLMs are based. On the other hand, maintaining a global workspace is likely computationally useful: a common representational format allows intermediate results to be written once and read by many neural processes.
But this poses a question: so which definition should you use? The previous neuroscientific definitions that obviously don’t count LLMs as having a global workspace, or the new one that does?
In order to make the definition work, they drop most requirements for a “global workspace” outside of things that naturally go along with reportability (e.g., how could reportability not involve internal reasoning, or generalization?). This means there’s a potential deflationary account for nearly everything, and the paper is one long fight against those. E.g., the deflationary version of their definition of ignition is just when the model commits to an interpretation (and since their measure of this is to give ambiguous prompts, of course they see a change). The deflationary version of their definition of broadcast is just that information with high reportability is more likely to be used by later layers. Etc.
Admittedly, I can’t address the garden of forking paths around this research, nor all their sub-experiments, but I currently don’t think any of the paper’s contents magically avoids the measurement problems I’ve been pointing out about consciousness research. There remain just-as-viable alternative takes on J-space: for instance, you could also say it is the “inner monologue.” Or you could go as non-anthropomorphic as possible and hold it is just a relatively constant transformation of internal processing into the output that starts at some point, but after that basically just starts accumulating, which is why as you get closer to the output, more and more of the J-space matches that output, and the more content you look at, the more linear these curves look.
All these issues are not entirely Anthropic’s fault: we simply have very bad theories of consciousness.
To fix this, I’ve been working on a way to narrow down the possible state-space of theories of consciousness (see here, and here), eventually reverse engineering post-paradigmatic theories of consciousness. It’s sort of like judo: if building good theories of consciousness is hard, maybe that’s actually good! We can use the difficulty of the problem of consciousness against itself.
One of the central insights is that there are various conditions under which theories of consciousness become unscientific: e.g., they become unfalsifiable, or they become completely trivial, etc. Most contemporary theories fall into these categories once push comes to shove, but most theories are neither ever pushed nor shoved.
One such failure state is “strict dependency,” which is when an experimenter’s inferences about consciousness (like verbal report) come from the same source as the predictions about consciousness from a theory (e.g., like that consciousness is equivalent to verbalizable representations!). Here’s a visualization of this in a testing schema for theories, pointing out that if a theory entails strict dependency, predictions and inferences can never truly be pulled apart.
And here’s me earlier this year describing how these meta-scientific problems crop up for global workspace theories:
E.g., when discussing Global Neuronal Workspace Theory (GNWT), Daniel Dennett once wrote that: “... theorists must resist the temptation to see global accessibility as the cause of consciousness (as if consciousness were some other, further condition); rather, it is consciousness” [47].
If Dennett’s ideas were true, this would make GNWT a trivial theory of consciousness, since it would be unfalsifiable, because the predictions and inferences strictly depend on the same source. Consciousness just is the information that is globally accessible for report and behavior, and reports and behaviors also just are different expressions of that same information. So there is no situation wherein they could diverge. And therefore, the theory is unfalsifiable and trivial (one could see how it gives us minimal scientific information even if it were true—trivial theories are fundamentally unsatisfying). Often, theories based in the “theater” metaphor have a tendency to fall into strict dependency (sometimes “global workspaces” are described in this way too [48]).
Notice that this is, at a high level, mostly what this new paper is doing.
And their analysis shows this is true! Specifically, they also test an even more direct measure of reportability from mechanistic interpretability: the “logit lens.” In this analysis, you (again, basically) just freeze the network and ask it to output at the layer you want to look at. And behold, that alone can capture “much of the workspace-like structure” (emphasis mine).
Empirically, the two lenses agree closely in the model’s last several layers and diverge earlier, with the J-lens recovering interpretable content at depths where the logit lens does not. However, we find the logit lens to be quite useful in practice, and to capture much of the workspace-like structure identified by the J-lens, though with somewhat lower reliability (particularly in earlier layers).
Anthropic appears to have performed the reductio that I said is a perfect example of why we need better theories in consciousness science. Loosely inspired by global workspace, Anthropic introduced a (supposed) way to track consciousness that is built precisely out of backtracking from reports, which is what I warned would lead to measurement issues. They then are constructing a case for AI-consciousness off of this unfalsifiable and trivial scientific theory. Which they find evidence for, sure… because of course they would!
A clever defender of Anthropic would point out that the analysis built off of the J-space is not just a direct measure of reportability. If it were just reportability, then why wouldn’t the workspace include the later layers of the network, those closest to output, which they dub the “motor layers?” So are there not falsifiable aspects, specifically, the prediction of the layers forming a global workspace with a tripartite structure of sensory/workspace/motor?
Yet in the paper, the tripartite structure seems to only exist because the researchers impose a criterion that, if the read-out of the verbalizable representations mostly just is the output, it somehow doesn’t count. They purposefully exclude non-smeared verbalizable representations, even though it’s just sitting there. Here is them admitting that (emphasis mine).
In §4.1, we characterized the workspace as existing in a range of intermediate layers: the J-space carries little information before this range, and in the final few layers transitions to representing the model's imminent output rather than its intermediate computations. We identified this late boundary empirically, by analyzing statistics of J-lens vectors, their activations, and their relationship to the model’s output. But this judgment was somewhat post-hoc, and we did not provide a principled definition of what distinguishes a "workspace" representation from a "motor" one. Indeed, as discussed in §4.1, sometimes the model’s next-token predictions do appear in the J-space in the intermediate range we focus on, even if they do not appear as reliably as in late layers.
To them, this probably doesn’t seem like a major limitation, but to me, it is admitting something like: “The only difference between the current theory of consciousness we’re working with and one that is provably trivial is, uh, somewhat post-hoc.”
There are more measurement problems I’ve identified with consciousness research beyond just unfalsifiability, and Anthropic appears to be running into those too.
Imagine that we have some theory of consciousness, and we ask what it predicts for a specific system S1. Then we cleverly find another system, S2, which approximates the same reports and behavior as S1, all while the predictions are entirely different (this is the other horn of the “Kleiner-Hoel dilemma”).
E.g., you might think global workspaces are necessary for consciousness. Then some evil demon builds a system that has rich input/output just like systems with global workspaces, but doesn’t meet your global workspace requirements. Whose reports do you believe?
Initial results (still early) indicate that Anthropic’s research runs into this issue as well. In the Anthropic paper, they provide the following key figure, which shows off the sharp tripartite structure they are labeling as sensory/workspace/motor, based on the correlations of the J-space (it’s more complicated and based on geometrical matching) across layers, a quantity referred to as the CKA.
So far so good. I literally see it: at the bottom left is sensory. The middle square is the global workspace. The top right is motor (which we know has some principled problems, but let’s leave that aside).
What about other models beyond Sonnet 4.5? They assess some others in the Claude family, but basically only include this ominous text:
We note that in some models the transition is more gradual, sometimes containing sub-blocks, and that the observed sharpness is exaggerated by layer subsampling.
Wait a minute.
The CKA revealing a sharp blocky tripartite structure is arguably one of the most important parts of the entire paper, precisely because it’s not (entirely) baked-in by the measure’s grounding in reportability. Why? The Jacobians are calculated per layer, and at each layer, they are recalculated. So there is no cohesive “workspace” built into the measurement. A workspace could appear if there is some kind of abstract space in which contents are actually contiguous and relatable across layers (here, the CKA assesses that through a kind of relational geometry). If the contents and manner of processing are not held contiguous as the stream of information goes forward across layers, that would obviously not be a workspace even by the loosest of definitions, since it would mean at no point are the contents sequestered into a “space” for all those functions a global workspace is supposed to perform; consequently, there would be no sharp blocky tripartite model of sensory/workspace/motor.
Luckily, someone went and ran their analysis (kudos to Anthropic for open-sourcing at least the algorithm for J-space itself) on open-source models. Now, this isn’t my analysis, but it does seem to use Anthropic’s code directly (and is implemented by someone at Prime Intellect). Because it’s not mine, I don’t want to lean too heavily on it—it’s more of a demonstration of exactly the kind of problems I’d predict. While it’s not an exact replication (which would be important to see—but Anthropic doesn’t actually release enough details to fully replicate anyway!), it appears to be a relatively natural one, and based on their code. The immediate question is:
Where is the sharp tripartite sensory/workspace/motor structure across other models?
Specifically, each cell in the diagonal here is (arguably close to) a reproduction of the above paper figure, but for an open-source model (organized by parameter size).
Yet, we see (at least in this analysis) that for open-source language models, there is no sharp blocky tripartite structure. Remember, the claim of the paper is “Verbalizable Representations Form a Global Workspace in Language Models.” Here are a bunch of language models. Where is the global workspace? Where is the sharp tripartite structure of sensory/workspace/motor?
If this holds true, it’s a great example of how hard the “substitution argument” bites: many of these models can do multi-step reasoning tasks, many of them are not dumb compared to Sonnet 4.5; in fact, in many conversations in the chat window, you would have no idea which one you were talking to. So they can act as experimental substitutions: if you make a prediction about the consciousness of Claude due to a global workspace, here are a bunch of Claude lookalikes (or close enough to matter for falsification purposes) without anything sharp and boxy (or with too many boxes!)
In fact, one wonders about whether their own analysis showed something similar for the other models they did analyze in the Claude family. Because we definitively see a sharp blocky tripartite “workspace” result only once, from just Sonnet 4.5. And nobody knows exactly how Claude Sonnet 4.5 is built, or how they did their sampling to get that figure, or how baseline connectivity is organized, or what architectural factors might explain that. Only Anthropic.
I’ll be honest—it’s a little bit fascinating to see a company with billions behind its research efforts run into the exact measurement problems I’ve spent years advocating we fix. My prediction was always that any serious attempts to apply theories of consciousness with advanced neuroimaging would lead to weird measurement issues… and Anthropic’s work on AI consciousness, which is essentially applying perfectly veridical neuroimaging to artificial neural networks, has run right into the mountainous side of the issue before the rest of neuroscience has even grappled with the problem.
This is how AI research ends up stimulating consciousness research: not by actual findings (you can’t empirically prove Claude is conscious without discovering more about human consciousness), but by the problems it runs into!
Everyone is thinking: “Oh, let’s use the stuff from consciousness research to figure out AI consciousness.” Wrong! It’s “Let’s use the problems in AI consciousness research to figure out what’s wrong with the stuff in consciousness research and fix it.”
So, my overall opinion: “Verbalizable Representations Form a Global Workspace in Language Models” is an interesting paper on the reportability of information in LLMs and how the property of reportability may be useful for multi-step reasoning in particular. Yet in the surrounding press, the implication is that after this research, really the only thing missing from LLMs is phenomenal consciousness.
This is a devilishly clever stance if the eventual move is to argue strongly for AI consciousness. It seems designed to put people in a position wherein the only thing not solved now (or soon) in LLMs is the infamous Hard Problem of how experiences or subjectivity arises from objective goings on like neural activity (or matrix multiplication, if that’s possible). The pro-LLM-consciousness position can then point out that the Hard Problem is not solved in humans either, so what’s the difference?
But because it is implicitly built on an unfalsifiable and trivial pre-paradigmatic theory of consciousness (and the few parts that aren’t may end up being easily falsified by substitutions), this paper does not prove much, other than how difficult measurement problems for consciousness are.
And I fully admit this is not entirely Anthropic’s fault. As I wrote recently: “We consciousness researchers have failed you.” Well, we also failed Anthropic. We need an effort (yes, like the one I am starting with Bicameral Labs—sorry for the plug, but it’s literally true that this is something I’m working on) to reverse-engineer post-paradigmatic theories of consciousness that aren’t just trivial, low-information, or unfalsifiable.
Otherwise, in the scientific fog of war, it will be easy for other factors, like press releases or pretty graphics, to determine what’s true to the public.
2026-06-23 22:37:08
Please remember, Dan Dennett said LLMs aren’t conscious
Why literary fiction awards keep falling for AI scams
If AI is so smart, why do its apps suck?
All the colors your screen can’t show you
Screentime correlates more with kids’ brains than IQ
From the archives
Comment, share anything, ask anything
As we shift from focusing on intelligence to focusing on consciousness, there have been a bunch of think pieces these past months about whether LLMs are conscious from both professional outlets and the blogger class.
Some of these takes have been… pretty bad. Like some people need to be taken away to philosophy jail.
Which is actually fine? Doesn’t this happen with everything? Five years ago, basically no one would have been able to tell you what “Moravec’s Paradox” for robotics was. Now, bloggers and commentators are often well-versed in the difference between AI and AGI, and how deep learning works, and ably use words like “transformers” and “next-token-prediction,” and discuss the previous history of the “AI winter” and scaling laws—there is an entire panoply of AI-related vocabulary that is now spat out with facility by the pundit class. Over the next decade, this same process will play out with consciousness.
But for now, there will be a lot of mistakes.
For instance, earlier this month the popular blogger Matthew Yglesias wrote a piece saying:
I think… that as the experience of conversing with a chatbot converges on the experience of conversing with a very patient human, we should assume the chatbot is having an experience similar to being a very patient human.
Yglesias primarily cites popular philosopher Dan Dennett as someone whose work would support Yglesias’ own view here.
Why Dennett? The thing is, everyone is looking around for a standard-bearer for the Yglesias position, which is to assume that because a chatbot can converse, it is therefore likely having subjective experiences the way a person holding a conversation would. They want to call this position “functionalism.” But that’s not functionalism! Functionalism is not that two systems behave the same, therefore, they have the same consciousnesses. Functionalism is the idea that what grounds minds in physical systems are the causal roles played by the mental states, between input/output (and so behavior), but also between other states, and that these causal roles are substrate-independent and could in theory be implemented in silicon or what have you. Basically, if you are a functionalist, you probably believe that artificial consciousness (AC) is physically possible (we could build it, somehow, with some advanced or futuristic technology). But you definitely are not committed to LLM consciousness.
Dennett himself (arguably the great inheritor of functionalism after Putnam) firmly rejected LLM consciousness.
Here’s Dennett:
LLMs are not people: They’re counterfeit people.… I want to suggest that counterfeit people are more dangerous, more potentially destructive, of human civilization than counterfeit money ever was.
Dennett was strongly against any talk of consciousness when it came to LLMs, found it absurd, dismissed it with a wave of a hand, and regularly said that even if artificial consciousness were to be discovered in the future (presumably, via some architecture that looks quite different from an LLM), humanity should collectively agree not to build it, and that we should instead focus on fashioning AI into intellectual tools. And remember, Dennett was around to see various advanced models, including GPT-4. This wasn’t an argument from ignorance.
So I have bad news: the reason why it’s tough to find a famous and popular modern philosopher of mind who is a good standard-bearer for the position of “It holds a conversation like a person, therefore we should attribute it a person-like consciousness”… is because it’s a bad position!
Meanwhile, the anonymous authors Claude and ChatGPT are hard at work as emerging writers, collecting accolades for amorphous slop. Last month the Commonwealth prize, published by Granta, was (almost certainly) given to an AI-generated story, “The Serpent in the Grove” which contained banger lines like:
They called her Zoongie. Maybe it was a name; maybe rain took a shape and decided to keep it. She had the kind of walking that made benches become men.
Fresh off this debacle, the exact same thing seems to have just happened again (there’s supporting evidence, including AI-detector scores).
Anyone with a reasonably good internal AI-detector should have it pinging about a mile a minute while reading either of these “prize-winning” stories. In fact, the latest case literally re-uses a lot of similar language and tropes as the Granta-case winner, which are all heavy favorites of AI.
AI gravitates precisely toward this genre of short story, where the narrator floats around disembodied of everything but remembrances of things past, like memories of olive groves (or whatever), and the hands of their grandmother, and so on.
But I think the hard truth is that this genre of short story was always bad, and its success was entirely its legibility, not its content. You can see this by transposing away all its mystery. Imagine if I wrote a story in which…
The entrance to Dunkin Donuts hit with a blast of cold air, the door opening to reveal its always-earthy and always-yeasty smell, cut by an atmosphere made by machines. Like a hospital for sugar. My grandmother, her skin as thin as paper, handled the change at the register with reverence, counting each quarter, for a moment the girl she had been back in the Great Depression. It was the same order we got every time—herself a coffee, black, and me a cruller, also black, a chocolate rope I bit into hastily. The tantalizing scent of the cruller wafted over me, as it had in New England for hundreds of years.
‘Erik,’ her ancient voice whispered as we slid into our booth.
‘Why we whispering gran’mama?’
She patted the orange-lined table.
‘Because of this place. Because some places always remember.’
