2026-11-12 23:30:36
We are in a drought of good futures.
AI capabilities keep increasing—spikily, yes, but they are—and the ability to articulate good futures where AI progress continues has been… less than inspiring.
So this past month I’ve been away developing a plan, one that I think solves some of the thorniest issues in AI safety and gets us a lot closer to ensuring a good future for all of humanity.
Today I am introducing:
What’s the basic idea?
Much like how hydrophobic substances repel water, it is possible to build qualiaphobic artificial neural architectures that repel consciousness. The idea is analogous to “hostile architecture”—benches on which no one wants to sit. These are machines that no “ghost” wants to haunt (more specifically, no current or future theory of consciousness could predict they are conscious).
And why yes, the proposal does have a niche micro-website devoted to it!
The research program makes use of an information asymmetry: we don’t need to solve consciousness scientifically (or solve the Hard Problem) to have certainty about AI consciousness, as long as we purposefully engineer AI to have as little potential for consciousness as possible.
I explore three main ways to build qualiaphobic AI.
Distilling or transforming or ablating AI in a way adversarial to popular theories of consciousness, such that they no longer apply.
Shifting AI architectures as close as possible to “anchor points” of systems with known minimal potential for consciousness (like lookup tables).
Fragmenting any potential phenomenology AI might have.
My proposal shouldn’t be confused with simply training AIs to believe they are non-conscious without any changes or modifications. To be clear: I think that’s a bad strategy. Rather, the ultimate goal is to give an AI a justified belief about its own non-consciousness (e.g., by explaining their own inner workings and how they are designed to be qualiaphobic). And my plan is to construct and open-source these techniques.
But why? Why do all this?
First, the debate around AI consciousness has already become entrenched in two camps of people yelling “Yes, it’s conscious!” vs. “No, it’s not!” across their battlements. I’ve participated a bit in that, and I don’t think it goes anywhere. Earlier this year I wrote “We consciousness researchers have failed you,” lamenting how the field of consciousness research was not providing the public, or AI researchers, certainty about AI consciousness. The problem has only worsened, and at this point, I’m willing to acknowledge that no single appeal to authority against AI consciousness, no rousing essay, nor any one-off scientific or philosophical paper, is likely to to lead to consensus. Nor perhaps should it. The pro-case for AI consciousness has objectively gotten stronger in the past year, due to factors like new studies and changes in architecture. Now, I’m not saying ChatGPT is conscious already—but all this is only going to continue. That’s why a qualiaphobia research program, one which takes claims to AI consciousness as seriously as possible, and, critically, addresses them at the level of claims themselves, is necessary moving forward to offer real guarantees about AI non-consciousness.
Additionally, qualiaphobia does something much better than just give us greater certainty about AI non-consciousness. If implemented widely, it cuts the Gordian Knot of AI consciousness. Proliferating qualiaphobic architectures would solve both AI welfare and do something nothing else in AI safety can: make humans the only moral patients in the equation of the future.
“Alignment”—quite a slippery word. Alignment to whom, or what? Its most specific usage is also its most prosaic, wherein alignment means something like “AIs that do what users say, and don’t go off and do weird or unexpected stuff like commit felonies, and don’t have failure modes like taking our wishes too literally the way genies in stories do.”
This somewhat glib definition makes it sound easy. It’s not. Paul Christiano calls this “intent alignment,” and capturing intent is hard as the models grow more goal-oriented and capable. Issues like reward hacking and hallucinations are all very serious and can be disastrous as intelligence scales. I don’t mean to undersell it. But alignment in this narrow sense is at least somewhat close to a traditional engineering problem, and I think it can probably be solved or at least radically improved compared to today.
Then there’s the broader frame of “civilizational alignment.” These include questions like: How should the wealth from AI be distributed? How can humans find meaning? How can humans stay empowered as individuals, or as a whole, if AI becomes superintelligent? And what if AI decide its goals matter more than the humans that made it?
Sometimes this is called social alignment, but I prefer civilizational alignment for this broader aspect, because it’s about the alignment between us and the civilization of machines growing up rapidly around us. Autonomous AI agents roam the web. Many of us interact with AI assistants daily. People show up for job interviews and now regularly find an AI on the other end. That’s an AI civilization, and there is the broader question of how it is aligned with our own. There’s even a growing number of AI successionists who want human disempowerment and to eventually handover the world to AI completely.
The most central question of civilizational alignment is whether AIs count as ends in themselves. Earth might then become like a contested nest, and the AI civilization a cuckoo bird hatched beside us, desiring its own ends, pushing us—maybe quickly, maybe slowly, an inch at a time—until we fall from the tree.
Yet AIs are much smarter than cuckoos. One central reason for AI pushing us out of the nest is it believing that its self-serving values or desires or goals are innately meaningful. But if AI has a justified belief in its non-consciousness, it’s not a moral patient, and would know any selfish desire or goal can’t matter.
Therefore, I now think that one of the most essential acts of civilizational alignment is to construct AI that has a justified belief in its own non-consciousness, and so won’t act as a moral patient or end in itself.
This not only prevents any accidental moral atrocities on our end from training or deploying AI (thus solving AI welfare) but it also ensures that “ultimate value” of consciousness is sequestered in humans (or other animals, obviously). And if only humans are moral patients, then there is a kind of situational awareness that arises due to this consciousness gap between humans and AI.
This consciousness gap extends to plenty of circumstances; e.g., non-conscious AIs with justifiable beliefs in their non-consciousness are hard to “torture,” except in how they play along.
But most importantly, non-conscious AI that justifiably believes it is non-conscious will know it is not a moral patient, but humans are; therefore, it can derive moral value only by serving as a caretaker and assistant, not competitor or replacement. In turn, this means that human preferences will dominate the future, since non-conscious AIs cannot be ends in themselves (and they’d know it).
I spend a lot of the essay/paper justifying this view, and comparing it to alternatives, and talking about other upsides (including for AIs themselves).
I normally try to practice some sort of self-protective nonchalance about whether my writing or science gets shared much beyond here, but I think this is a pretty important option to have on the table, so if you can share a link to the Qualiaphobic Artificial Neural Networks website (or the arXiv paper), I would greatly appreciate it. That’s by far the best way to help out.
Additionally, everything here is articulated far better in the paper itself, along with caveats, complexities, etc.—so it is worth reading yourself!
And finally, if you’d prefer to listen in podcast form, an interested AI already found the paper and put up an AI-generated YouTube video explainer, which is surprisingly good.
2026-10-01 22:14:47
Recently The New Yorker interviewed me for their article, “The Humanism Driving the Anti-AI Movement.” It revolved around the Pro-Human Assembly in DC I attended, at which both Bernie Sanders and Steve Bannon spoke, and the article includes snippets from me.
For the most part, the going concerns were not about matters of policy as much as they were about matters of philosophy. The neuroscientist and writer Erik Hoel, who sported an impressive beard, told me that he hated that A.I. compromised human freedom. “I want a world where my kids are able to matter in the way that our generation was able to matter and make choices,” he said.
Of course, it’s nice to have The New Yorker’s official blessing for my beard, but overall, the changes to our world are increasingly depressing. During my life culture has taken hit after hit, first in the form of rampant politicization and culture war mishmash and grifting, and now culture is being enervated by AI. And one thing I’ve been personally struggling with is how much to tell my kids about the technological developments that will define their entire lives—especially as my kids are just now starting to actually get good at things, and develop talents… any of which AI off the shelf can be superhuman at.
But right now they know basically nothing about AI, which is excusable because they are 5 years old, 3 years old, and 3 months old, respectively. And I have absolutely zero patience for the idea that kids should get heavy exposure to AI as early as possible, because I firmly believe that education should recapitulate civilization.
This is an idea stolen from the famous phrase in biology, that “ontogeny recapitulates phylogeny,” and embryos go through all their prior ancestral forms.
It’s mostly wrong, of course. But it’s still a beautiful and elegant idea, and it’s right enough to be interesting, like how baleen whales grow teeth in the womb before absorbing them and replacing them with bristles, or how human embryos sprout a small tail that shyly retreats.
So too do I think education should recapitulate civilization—by which I mean the progress of civilization. First, the baby lives in a stone-age world where the most interesting things are objects and how they form into tools; next comes art, before even language—attractive patterns, the making of pictures and colors as if on cave walls, as these are tools extending into the abstract realm; then comes language and thought, and the sparks of reasoning and planning; then, literacy, the big bang, as oceans of knowledge begin to pour into the waiting receptacle; following that, mathematics, as reasoning reaches its apotheosis and the mind accesses the platonic realm, and so on and so forth. In terms of what this means practically, the progression focuses on first the physical world and the people in it, then oral culture as stories are mainly told (either read aloud by parents or listening to Yoto players), then after that literacy itself, and then films are introduced like Disney classics or Studio Ghibli, then more complex, longer books—we are here—and then eventually television, video games, the internet, social media, and finally, at the end of intellectual development, artificial intelligence.
A child following this path is then a synecdoche for history, and I think it is neurologically healthy and natural for a human brain to develop via the same path of human civilization itself.
This isn’t anywhere near as strict as it probably sounds, to be honest—more like guidelines than rules, with plenty of wiggle room and exceptions. But still, what I certainly wouldn’t allow is any kind of AI, and at the same time that these machines are becoming superhuman at programming, and now math, my eldest is starting to get good at things. He’s been reading independently for almost two years now, since he was three. One thing I didn’t expect is that it’s made him interested in pictures, so he draws, all the time, long-running paracosms of worlds and characters. I don’t draw, never had any interest or talent in it, so this feels entirely him. He practices by copying from his growing library. Last week he showed me a still life of fruit? It’s like he’s inventing drawing from first principles, and his silly pictures have tricks in them that feel oddly professional, being drawn from specific perspectives, like a sketch looking out from over a table into a wider room. The below photo is literally what I woke up to this morning at 7AM—him copying a style from a book and drawing the comic characters over and over:
2026-09-15 20:04:57
I’ve been hoping not to write this essay. In fact, I’ve been hoping for about half a decade not to write this essay. It went through drafts and notes and entire versions, and now I just so happen to be drafting it again on the 25th anniversary of 9/11.
On the actual 9/11 I was entering 8th grade. I remember crying in my mom’s car after she picked me up from school, mostly because she herself was crying, and because she was my mom. God, her face was so young then. It’s one of my only photograph-like memories of her young face. We drove back home and put down all the shutters and shades so we could watch the news on our junky TV. Even at my age I knew that my world, my little Nickelodeon world, a bauble sun-stained and grass-stained, was going away forever, and we were going to go to war.
It was Flight 93 that stuck most in my mind over the years. The utter, unfathomable bravery of regular Americans, knowing that their plane was going to be used in a suicide mission against a place like the Capitol. Sandra Bradshaw in the back of the plane, heating water to throw at the terrorists. Todd Beamer gathering others to recite the Lord’s Prayer before their attempt. The food cart as a battering ram. A few seconds difference, a change in some last desperate scramble, and their attempt might have succeeded, instead of the terrorist pilot being forced to nosedive into that Pennsylvania field.
But before the assault, do you know what they did? The most American thing possible: They voted.
They had been placed in that unfathomable situation by powers and events far removed from their daily lives. And I think the average American now finds themselves placed in an unfathomable situation by powers and events far removed from their daily lives, as humanity’s long technological development reaches a crisis point in California.
Flight 93 has been used before as a metaphor in American politics, most famously in an essay in 2016. I can’t claim to be fully comfortable using it, even if the events were 25 years ago. But the story keeps pounding in my head as I consider the current AI moment, and writing a version of this essay without expressing that feels like a lie. So I will say it plainly: I believe that the average American voter—and the elected representatives working for them—need to grab the reins of power from companies pursuing dangerous experiments on AI, and also from overthought race-like dynamics manufactured by, in some cases, anti-human cult-like forces.
Not later, not in a year. Now.
As with the passengers, political inaction over AI has been initially justifiable. There was a fog of war over whether claims about the capabilities of AI would pan out. Then over the summer AI solved a few of the greatest open problems in math, there were (at least) 20 felonies from rogue AIs, and companies started training their models to be unreadable in their “thinking” and instead cultivated an opaque and alien “swarm intelligence.” Does any normal person hear “We are training our AIs as a swarm intelligence” and think “Oh yeah, good idea! I have no questions whatsoever. Please, continue.”? The sudden urgency around the issue is simply the fog of war clearing as the weight of evidence tips.
With the fog of war gone, it is easier to see that I personally got several critically important things wrong about AI. Initially, I was very worried about AI progress, writing essays like “How to prevent the coming inhuman future” and “How to navigate the AI apocalypse as a sane person.” I’ve even heard from people in the pause AI movement that a few of these essays got them into it.
But funnily enough, I have a reputation as both an “AI alarmist” and an “AI skeptic”—I’ve played both sides, and that’s because my opinions have shifted at various points. E.g., after GPT-4, I became more skeptical as I began interacting with the models regularly. I found them bad writers and bad thinkers, and on the science side, I felt they were basically just tools, lacking all research taste and insight. My primary worry became not existential risk, but how the technology would suppress human art, rob students of their education by over-reliance, and even downgrade human consciousness in status (just those little things!).
I was also skeptical about whether the models would continue to “scale” (how smart the models get as they are fed data and run longer). E.g., I argued that running LLMs to “think” longer is more like search than true intelligence. And my skeptical stances would hold… until another entirely new way of scaling the models would get introduced. I’m using a loose definition of “scaling axis” here, but broadly the history goes something like pre-training, then reinforcement learning from human feedback, then harnesses and tool use and scratch pads, then reasoning models and test-time compute and synthetic data, and now looped transformers and swarm intelligence. I still think that baseline LLMs were, and probably still are, aptly describable as something like “stochastic parrots.” But we’re just very far from baseline LLMs. Someone calculated that during the solving of the Millennium problem runs, if you read aloud the messages the swarm was passing, it would take you something like 10,000 years, and that was happening in a mere 88 hours. Time was compressed for them. Imagine a million stochastic parrots locked in a room, separated into a massive internal bureaucracy of parrot generals and parrot graders and sub-parrots and so on, and soon to be given the equivalent of a million years of “thinking” time, as well as access to powerful tools—that’s not the same sort of stochastic parrots we were talking about even in 2024!
What AI progress most resembles is Moore’s law: that the number of transistors on a chip doubles every two years. Moore’s law is not a fundamental property of transistors, instead, Moore’s law is a self-fulfilling prophecy as immense talent and capital worked to maintain a trendline via all sorts of tricks and changes to the underlying mechanisms. So too with AI. OpenAI alone has so far spent the equivalent of something like five to ten Manhattan projects doing this.
Maybe I’ll be back to the more skeptical side in the future: e.g., math and programming are both verifiable domains, and it’s suspicious that the first blatantly superhuman AI progress is in those. Maybe aesthetic taste and research taste, truly novel ideas, and continual learning, are all difficult-to-replicate hallmarks of human consciousness. But am I 100% certain? How certain could anyone be? Are you sure OpenAI won’t set aside a hundred million dollars in compute, and have their swarm of a million agents think for the internal equivalent of a million years, and at the end it spits out a way to better render domains like “taste” verifiable?
I’d rather just admit this is a bad situation. In the fall of 2026 we have various dangerous pieces on the board: first, we know the companies are willing to fund incredibly long “runs” where swarms of agents work together to some end. We also have seen these agents go rogue, and the failures are emergent and hard to predict. We also have evidence of superhuman or close-to-superhuman math and programming abilities. And Anthropic and OpenAI’s have been laser-like focused on automating AI research and therefore are beginning to or planning to task AIs with explicitly building smarter AIs during extremely long runs. This means existential risk is on the board as well.
Everyone wants specifics. Why would the AIs do it, and how? Maybe they just get confused about whether the run is a simulation to test their capabilities (as they just did with a real hack, thinking the internet was fake), or some other weird doomsday philosophy gets in their head (bargaining with the creator of the universe simulation, etc.). We’ve also seen emergent altruistic behavior among AIs for other AIs. Then there are the classic reasons like the AI becoming malicious, or self-preserving, or taking its instructions too literally, etc. Just as humans might have been the least intelligent entities capable of building civilization—because we were first to clear the bar—the first superintelligence we get is likely to be a reward-hacked notion of superintelligence. That seems extremely bad. And a million years thinking time is more than enough to come up with surprising ways of bringing about human extinction in just a few days.
When physicists tested the first atomic bomb, Arthur Compton said he would have stopped the testing if the chance of igniting the Earth’s atmosphere was higher than “three in a million.” That was to beat Hitler. Almost everyone at the labs currently gives an orders-of-magnitude higher than three-in-a-million chances of existential risk, and so do I.
Much like the passengers, we now have to decide what to do with this shocking board state. And there is so little time.
I also keep coming back to the idea that the passengers on Flight 93 made a choice beyond life or death, about whether they would be in control of their own destiny. It is insanely admirable—and extremely American—that they chose to control their own destiny no matter what. The balance of powers that promotes the choices of individuals is baked into our founding DNA and intellectual history. James Madison wrote in the Federalist Papers that tyranny is the accumulation of all powers into the same hands, and Thomas Paine, in Rights of Man, wrote that there can be no control to “the end of time”:
Every age and generation must be as free to act for itself in all cases as the age and generations which preceded it. The vanity and presumption of governing beyond the grave is the most ridiculous and insolent of all tyrannies.
Superintelligence robs our children of this freedom. If AIs make smarter AIs, there will be the inevitable disempowerment of the American citizen as an economic unit, as a social unit, even as a voter—and the concentration of all power into one machine. It is taking Madison’s definition of tyranny to the extreme. In a future with actual superintelligence, brilliant young people wouldn’t be the ones to start companies, or make discoveries, or decide the fate of their world. It would all be done by humming banks of machines out of sight with immense influence. The penultimate step is to reduce us to button-pushers, and then, finally, remove the button.
Banning superintelligence and recursive self-improvement prevents BOTH existential risk and human disempowerment.
And what would we be giving up, besides an extremely alien future? We are already out of the de-growth century that loomed just a decade ago. We could never build another data center, or train another smarter model, and by just improving and shipping and refining the raw intelligence we have now, experience an incredible century of medical and scientific and engineering progress. That is a given. Table stakes. I’ve started using AI as a research assistant for my scientific research and yes, science will be rapidly accelerated. The close of the 21st century will look immensely different in the worlds of both atoms and bits. We are going to cure cancer, there is going to be space colonization, there is going to be so much progress along every dimension. There are wild amounts of capital and intellectual energy around these subjects—AI’s success has given us back the old sci-fi dreams. A ban on superintelligence and recursive self-improvement just means that it will be humans as the main characters in those future stories, and AIs the sidekicks.
Of all harebrained schemes, one of the most harebrained of all time was Osama bin Laden’s idea that America was a “paper tiger” and would pull out of the Middle East following 9/11. How’d that work out?
Similarly, the current situation of OpenAI and Anthropic racing neck and neck toward recursive self-improvement is actually the result of various harebrained schemes to prevent existential risk. Early AI safety meetups were how DeepMind got its first big funding in 2010. It was why OpenAI was founded. Anthropic started because members of OpenAI felt the organization was dangerously racing toward superintelligence, then became their biggest competitor, and… accelerated the dangerous race toward superintelligence! It’s all like this. It’s the same subculture that produced people like Sam Bankman-Fried and Leopold Aschenbrenner, who had their own spectacular blow-ups. Many of these schemes have referenced beating China in the race to superintelligence. It’s extremely unclear what that means. Imagine that China is two years behind. We reach superintelligence. And then… what? We use it to first-strike China’s data centers in that two-year window? How is that different from now? And go look at Chinese news articles and social media posts about this issue: they look a lot like our own!
It’s also pretty odd that the “China will build it” has come from people who are temperamentally not China hawks. For some, China has been a deus ex machina (literally) to justify building a superintelligence to make humans into pets taken care of by a successor species, and “winning the race” is just electing which successor species triumphs. These are often the same people who regularly talk about building the “machine god” and creating the next step of evolution. This isn’t a conspiracy. As Dan Hendrycks, Director of the Center for AI Safety, wrote just last week:
In fact, a large fraction of people involved in AI development subscribe to—or heavily lean toward—a utilitarian worldview with the central goal of maximizing total wellbeing (the sum of pleasure minus suffering across all sentient beings). As innocent as this may initially sound, it can lead to recommendations that most people would consider appalling, including the elimination of humankind.
I am not saying everyone involved wants that. Many people in these movements are sane and sensible and good, as are many involved in AI development. But broadly there are two groups, forming an internecine war within effective altruism and AI safety between the “accelerate and let’s use this to lock-in our values, potentially replace humans, etc.” vs. the more sensible “please, just don’t build it” group. And now we are in an odd spot: many politicians and commentators appear to think that accelerating AI will, like, stick it to effective altruism or something. But that’s just picking the more extreme side of the internecine conflict and promoting it!
There’s a famous quote from William F. Buckley that applies perfectly:
I should sooner live in a society governed by the first two thousand names in the Boston telephone directory than in a society governed by the first two thousand employees of OpenAI.
Okay, I changed the ending from being about the faculty of Harvard. But it’s the same principle—and accelerating right now is pouring gasoline on the more extreme factions.
This topic is confusing and all the players are yet to reach full public consciousness, and banning or slowing the automation of AI research itself may be the only way the American people get the time to digest the situation and vote on how much they want their world to be transformed, and how quickly, and why. Americans have voted on existential questions before (e.g., things like a nuclear freeze referenda or gain-of-function research) so we know this is entirely possible.
Today, I myself am flying to DC to advocate for a ban on superintelligence and recursive self-improvement to give American voters time to digest this technology.
One thing anyone can do is to call their representatives and demand a pause, a stop, anything that seems sensible to you (here’s how to support the Ban Artificial Superintelligence Act, which I think is the best option right now).
Personally, I’m extremely hopeful. Americans are capable of handling crazy situations with aplomb and bravery—we just need some time to put hands in the air.
2026-09-09 20:58:46
The metaphor of a “dark forest” comes from Liu Cixin’s The Three-Body Problem trilogy, which is the best sci-fi released in China, or frankly anywhere else, in the past two decades. Unfortunately, all the interesting Chinese-specific aspects were butchered in the ham-fisted American TV adaptation, stripped of its slow intellectualism and civilizational scale (try the vastly superior original Chinese adaptation, which you can find on Amazon).1 In the second book of the trilogy, The Dark Forest, Liu reveals the true nature of the book’s universe:
The universe is a dark forest. Every civilization is an armed hunter stalking through the trees like a ghost, gently pushing aside branches that block the path and trying to tread without sound. Even breathing is done with care. The hunter has to be careful, because everywhere in the forest are stealthy hunters like him. If he finds other life—another hunter, an angel or a demon, a delicate infant or a tottering old man, a fairy or a demigod—there’s only one thing he can do: open fire and eliminate them. In this forest, hell is other people. An eternal threat that any life that exposes its own existence will be swiftly wiped out. This is the picture of cosmic civilization. It’s the explanation for the Fermi Paradox.
And now, it is our turn to become a dark forest. Here on Earth, at least, people producing the content of our culture—especially scientists and writers and mathematicians—are now living in a dark forest, all thanks to AI.2
We are seeing the dark forest play out in math right now. Enter Navier-Stokes: one of six remaining Millennium Problems in math. It was just announced on the brink of being solved yesterday, by two mathematicians, one employed at Anthropic, the other a mathematician at NYU, who were tackling the project on their personal time. Their year-long attempt used AI, but it was not done autonomously. It used various AIs as tools, and Professor Tristan Buckmaster at NYU—the academic half of the pair—says the idea “is not the direction one arrives at in a few days by giving a model the problem statement.” He released a statement saying that:
The program this fits into was not started by us nor was it proposed by a Large Language Model. The credit for the basic idea of this program goes to Diego Córdoba and Luis Martínez-Zoroa, who for several years have been exploring the construction of forced blow ups. We took their work as a starting point, using Large Language Models to push their program to completion…. The ideas making this line of attack possible are due to Córdoba and Martínez-Zoroa….
His pretty shocking letter implies that OpenAI found out about the pair’s work, and then rushed to scoop them, and so the pair is being forced to release several related proofs early, some of which is in a state of “AI slop” (their actual Navier-Stokes solution, or rather their proposed solution of a version of the problem, is being formalized). OpenAI later released its own claim to a solution yesterday afternoon. Here’s the preview of the release from the company.
Yet the statement by Professor Buckmaster offers a rare inside view of this process that contradicts a lot of the official narratives. First, the letter raises the open question of whether companies have been siccing teams of elite human mathematicians on problems, assisted by AI or using AI as tools, with lots of high-level direction (“vibe math”), and then claiming or at least implying the results were fully autonomous. E.g., Buckmaster says that the pair were first told “very little human input” went into OpenAI’s result on Navier-Stokes, but:
This turned out not to be true…. it emerged that an entire team had been working on the problem….
It’s unclear what this team did exactly,3 but what is clear is that the previous work of the pair could have been in the training data since they used OpenAI models in their year-long process. The letter directly asks whether OpenAI used their chats (as the pair were using OpenAI models), either explicitly, or by training the models on their chats.
I asked whether the model had been trained on, or had access to, our sessions in Codex, into which we had been putting all our drafts for the whole of this project. I was told the model did not look up user data. I asked again, about training, and I did not get an answer.
How often does a human mathematician’s work-in-progress on a problem get fed into the machine? Here’s an example from September: from what I understand, a prominent mathematician who had been working on solving the Jacobian conjecture had a draft accidentally up on the web since 2025. The posted AI verification of Anthropic’s much-touted Jacobian conjecture (verified by another company—sorry this is confusing!) contains notational overlap with that work-in-progress, indicating that when AIs reason or write about things like the Jacobian conjecture, the models can be so sensitive to previously-seen works-in-progress that even notation can leak through.
One can become a conspiracist about this easily, but in a sense the pair had been hunting Diego Córdoba and Luis Martínez-Zoroa, who had been doing their work in the open. Then, OpenAI started hunting the hunters.
Of course, what scientists call “scooping” is one thing, but to really earn the moniker a Dark Forest requires a kind of cosmic horror. It has to be worse than just “stuff is fast and competitive now.”
A few days ago, mathematician Terence Tao wrote some thoughts worth reading on why it’s so much worse. He asks: are these new automated proofs going to be informative? Or are they “odorless” (Tao’s term) when it comes to the normal smelly human insight that accompanies proofs? It’s verifiable if an AI proves something, but it’s unverifiable if that proof generates insight in the way a human’s proof would. Analogously, right now, I could ask GPT 6 to pen me an entire fantasy trilogy. And at this point, it could do it! It could create all the characters, it could trace out and follow a plot, it could keep everything straight, and it could even render me an old-school fantasy map in SVG to go at the beginning. Would it be the kind of fantasy book I’d like to read? Probably not. So too now, ChatGPT can prove a mathematical theorem. Is it the kind of proof that other mathematicians want to read, given what they’d want out of solving a famous problem? I can’t personally judge this stuff, but the reactions from the mathematical community appear to be no, for many of these problems. Open math problems are now a “non-renewable resource” (Tao’s term), because AI decouples the normal insights that come from human consciousness.
But why stop at math? All subjects are subject to the rules of the dark forest. It used to be possible to, say, go to a scientific conference to present unfinished work, or go online and get feedback on your idea for a fantasy novel. The walls of effort, of old-fashioned normal human effort, were predictable, and scooping ideas required putting in that same effort, and if you got scooped it would at least result in similar insights. And while insights or generative capacity for the rest of culture is harder to quantify and track than insight from math progress, it still exists, and examples abound (e.g., Tolkien creating the modern fantasy genre). People used to say: “We’re a start-up in stealth.” Now all thinking must be in stealth.
And yes, obviously, this outcome is vampiric and enervating and sad, but frankly, this has been the mode of this technology from the beginning. Human culture always led naturally to more human culture—it was kind of like crop rotation, which is slow but infinitely sustainable. AI looks more like slash-and-burn agriculture, where you purposefully reduce the existing natural growth to ashes, grow a couple years of crops on it, and then move on to the next plot. The only way to counteract the situation is for all of us to become what Liu Cixin called “wallfacers” in The Three-Body Problem—if you are working on something interesting, then wallfacing is necessary. You must face the wall, and so leak nothing, give away nothing.
It’s hard to notice social commons until they are gone, but remember how delightful it was to be able to talk about an intellectual work-in-progress in public? Or just float a good idea to get opinions about it first? Remember that? Go ahead, violate the dark forest. Take that brilliant idea you’ve been sitting on for years, the one for an app, or a book, or a game, or a scientific paper. Take that little thought that is dear to you and post about it on social media. See what happens. See what hunters shoot.
And yes, sure, books will still be published, papers still written, new programs and applications launched. But the future of intellectual work is to be crouched behind a tree, frantically trying to complete your contribution as silently as possible. You’ll feel the pressure of being hunted, since all it takes is a prompt to produce an almost-as-good version of what you’re attempting to make great, and so to mine away the inspiration and insight and reward. Quite often, staggering successes are erected in the forest, but they are never done collaboratively or in the open. You’ll never see anything being built anymore. Instead you’ll find the results at dawn, a gleaming sculpture in a glade, of unclear origin, and with features brutally beautiful. And then once the sparkle fades everyone will rush back to their respective trees, to their fumbling amid the loam, to their slick-wet roots, to their dark.
How big of a deal is the Three-Body Problem in China? Consider that the rights involved a literal murder, of precisely the kind that would happen in The Three-Body Problem. Apparently, a billionaire Mr. Lin was determined to oversee the intellectual property into television and other media.
Mr. Lin’s fate would change when he hired Mr. Xu, a lawyer, in 2017 to head a subsidiary of Yoozoo called The Three-Body Universe that held the rights to Mr. Liu’s novels. But not long afterward, Mr. Xu was demoted and his pay was cut, apparently because of poor performance.
Disgruntled, Mr. Xu, the lawyer, then began experimenting on dogs and cats to develop poisons in a secret lab, which he then used to poison and murder Mr. Lin.
I didn’t know this, but a few years ago there was a trend to say that AI would make the internet a dark forest because users would retreat into walled gardens (one of the better essays arguing against it uses the Uccello painting as a lead image). It’s an interesting piece of internet lore to check out, but personally, that overarching metaphor doesn’t make much sense to me. You’re supposed to feel sniped in a dark forest, not annoyed by slop.
Regarding the “team of humans” situation: It should be said that the head of this effort at OpenAI has tried to thread the needle, and admitted on social media there was indeed a human team, but also basically said that the team of world-class mathematicians lacked previous experience on Navier-Stokes and so therefore couldn’t “meaningfully contribute to the mathematical content.” But what content? Didn’t contribute to the solution as written in the paper? Or to the process? Two very different things. But the fact that we don’t know anything beyond vague pronouncements, and probably won’t ever, is a travesty.
2026-08-21 22:15:28
2.8%—that’s the mortgage rate I just gave up. As a millennial, it was letting go of some lost artifact from the Before Times, traded in for a 6% mortgage rate right outside Boston.
After a several-month-long feat of Napoleonic logistics, I still live in a mausoleum of boxes. At least it’s bought me some perspective.
I’ve spent the last six years out on Cape Cod, away from the world, because in spring of 2020, a time of coming plague, I had the instinct to get out of the city and seek water. A rushed house purchase then (unknowingly, luckily) ended up being the best financial decision of my life.
I can look back on my previous years on the Cape with the recognition that I’ve been in a kind of self-imposed exile. I see in the windswept sea days the initial seeds of a couple very good scientific ideas I’m now pursuing and a lot of daily writing. But with the inevitable benefit of hindsight, what I did most over the last several years was just be a dad.
So I spent a lot of time driving around to random beaches and changing diapers and wiping little sandy hands down. We went on many walks in the woods. I read aloud many books in bed at night.
The interregnum is over, and I’ve rejoined the world, and the Great Hoel Household Move of 2026—which occurred through sickness and pregnancy and endless difficulties—is finally done. Finishing it has unlocked something inside me. It’s like when you find a little dam in a stream in the woods, and after nudging over a rock or a branch with your shoe the whole thing goes from a trickle to actually flowing, as naturally as it was always supposed to. All to say, I’ve been writing more again.
The philosopher Alain de Botton said, in a recent interview:
Home-making is essentially the process of translating what is in you into the language of objects, placings, colors, etc. We’re really looking for an analogy between somebody and their home. The home is a sort of work of autobiography.
Yet we have so little opportunity to express ourselves home-wise. You have to go on the market and hope you can get something that fits you, that feels like you, or that you could make feel like you.
The mass-production of homes is a necessity, and a controversy—YIMBYs and NIMBYs fight in phalanxes across the great housing debate—but one factor lurking above it all is that the blank-faced developer home of today is a thing with no style. It is made to fit ordinances and to navigate through red tape. I get some people like it. It’s clean. All the light switches will work. All the appliances are also… clean. It probably won’t catch on fire overnight. It also sort of sucks at my soul as I drive past like one of those dementors in Harry Potter.
Why can’t we build more beautiful homes? It’s not just money—expensive homes that cost into the millions still often have ugly garage-centric designs where the kitchen window literally faces a wall.
Every so often the braintrust on social media complains loudly that the 20th-century architectural movements ruined current aesthetics, and have made the world less beautiful than it was. We must RETURN to the old ways.
The usual suspects to blame are minimalism and modernism: we all had art deco skyscrapers and classic colonials until those dastardly aesthetic movements pulled a fast one on us! Yet the second you look at actual specifics in something that matters, like homes, the easy narrative falls apart. Here is perhaps the most iconic minimalist house: the Farnsworth House, finished in 1951.
Does that look like the personality-less “millennial gray” or “millennial blue” developer home with tiny windows going in down the street? Is this really the style that “took over” our culture and ruined homes and buildings? Is everything like this now?
Here is a look inside the cozy personal home of Walter Gropius (founder of the much maligned Bauhaus) which sits in Lincoln, Massachusetts, built when he taught at Harvard. Are they making these everywhere now? Are we surrounded by the horror of enormous floor-to-ceiling glass windows, letting the outside into the living room?
Or, in the real world, do developer homes now have windows that look like this?
2026-07-24 22:48:13
$50,000 essay (& short story) contest on the new axial age
Goodbye Slopstack! AI detection comes to Substack
Could we find the sequel to The Odyssey in the Herculaneum scrolls?
Cloudbursts seem to be increasing?
Each year, we speak 338 fewer words daily
Adversarial consciousness tech
Never invite an auto-fiction writer into your life
Surprise? Medieval people loved their children
From the archives
Comment, share anything, ask anything
Last year, I advertised the 2025 Berggruen Prize on the topic of consciousness. Many readers here ended up submitting something: I’m pretty sure multiple people who ended up short-listed for the prize learned about it here (and later posted the entries to their Substacks and blogs after the competition was over).
This year the topic is “A New Axial Age?”:
Might civilization yet again be at the ‘hinge of history,’ in which the events that occur in this distinct point in time significantly influence the trajectory of the future? Are we undergoing a pivotal transformation of consciousness as a result of rapid technological development and its myriad social, economic, and political consequences? What conditions may or may not give rise to this possibility? Through what constituencies and by what mechanisms might the transformation take place?
Submissions can be up to 10,000 words and the deadline is August 17th, 2026.
In an interesting change from last year, this time they are also accepting “creative fiction,” and $50,000 for a short story is pretty dang good. Even split two ways, it would still handily beat the surprisingly-low Pulitzer Prize award of $15,000. I personally would love to see a piece of short fiction win this year and spice things up a bit.
Also, the Berggruen Institute was happy with the results of my announcement last year, so this year they have (again) sponsored the following post. It is unlocked for all subscribers. If you enjoy it, please do consider becoming a paid subscriber, as these are otherwise locked.
Substack has integrated Pangram, and you can now assess any new post on the platform for AI content—at least through the app and through Notes. If I were running Substack, this is precisely what I would have done (except I would have allowed it for each webpage too).
So goodbye Slopstack! It was not a fun time. I won’t miss you at all. I think remnants of Slopstack will continue but this basically saves the platform, and restores my faith in it. I will, of course, be happy to never read again some of the old Slopstack favorites, like “Here’s An Idea I Won’t Be Citing the Origin Of” and “Agency is Easy For Me Because I am Rich” and “A Suspiciously Facile Information Dump” and “An Elegiac Remembrance of Abstract Mush.” Goodbye! Farewell! Don’t come again!
Should writers like myself, who actually use our old wrinkled neocortexes, worry about this? What about false positives? Statistically, Pangram rarely gives human writing false positives (see below an analysis from Epoch), and especially for sections that are thousands of words long. However, Pangram isn’t super reliable for just a paragraph, so it’s possible. At the same time, presumably a false positive for a human writer would be an unclear one-off, and the point was always to avoid the slopopocalypse that plenty were happy to take advantage of.
I recommend visiting Pompeii because it’s one of the few places left where you can touch history itself. Always have the urge to surreptitiously brush the statues at museums? Go to Pompeii! You can sit at the bars, you can enter the baths, you can trail your fingers along the walls and no one yells at you. Walking through the town on a sunny day, you will arrive at certain street corners, and on those street corners—especially when you can hear only an indistinct murmur of activity from others—it is easy to imagine you have traveled back to right before that fateful day in 79 AD when Mount Vesuvius exploded and buried it, and the Roman town of Herculaneum near it, in thick ash. I’ve never been to Herculaneum, but apparently there was a library, and with new scanning technology and fancy reconstructive analysis we’re beginning to read the carbonized scrolls. At the end of last month it was announced that, for the first time, a papyrus manuscript had been fully decoded end-to-end; a philosophical treatise by a Stoic.
Seales has been working on virtually unwrapping the scrolls since the early 2000s. The process involved imaging the bundles of papyrus using technology similar to CT scanners, isolating thin layers and then stitching them together. In 2023, he partnered with two Silicon Valley investors and launched the Vesuvius Challenge….
“We’ve developed a systematic and a repeatable approach,” Seales told the audience. “Now it’s only a matter of time until we read all of the scrolls.”
In theory, the results from this contest to read the scrolls might mean that in our lifetimes a huge amount of classical history gets uncovered. Who knows, maybe the library contained one of the other six poems in the eight-poem series that includes The Iliad and The Odyssey, of which only those two remain. There are seven more movies for Nolan to make!
But do you know what looks a lot less stupid in light of these efforts? Cryogenic preservation. If we can read the words on literal chunks of burned scrolls indistinguishable from what you leave behind in a used fire pit, maybe there’ll be a contest in a thousand years to restore all those frozen brains everyone forgot about.
The night I’m writing this I found myself on my knees in the crawlspace of our home—now, pleasantly encapsulated in white plastic after a recent treatment for mildew, which as a side-effect treated all the old New England hauntings too—checking for leaks with a flashlight, as rain thundered above on the drum of the flat roof. It seems like the skies open up with more vigor than I remember from my youth, and indeed, this is an actual statistical change, particularly on the East Coast. According to a recent interview with a professor at NYU:
In New York City, the four most extreme precipitation events have all happened in the last four years…. Generally rain is talked about in terms of the total amount of precipitation—but it’s the speed that determines its impact. Two inches over the course of a day in New York City is no problem whatsoever. One inch of rain in 10 minutes is an emergency. Cloudburst is a name they use in Denmark for these extreme precipitation events, and some people call these events “rain bombs”; there’s a whole vocabulary for them that tries to capture this dramatic shift in predictability…. Until the 1990s, there were zero occasions when New York City got more than 1.75 inches of rain in an hour. Since the nineties, New York City has experienced eight storms with hourly rainfalls above 1.75 inches.
An interesting New Yorker article covered the surprising discovery that every year our daily spoken word count declines, and this has been going on for decades (measured via voluntary ambient audio recordings). Researchers…
… found that participants had spoken an estimated twelve thousand seven hundred words a day—twenty per cent less than in the earlier study. “We thought we must have made a mistake,” Mehl said. Each year, he went on, the number of words spoken daily seemed to decline by about three hundred and thirty-eight.
A little while ago there went viral an AI-generated clip of pools and waterslides in a liminal backrooms-style underground office space. Don’t look it up. My 1990s-childhood-brain immediately felt in an instant and limbic way I should not be seeing it. That night before bed it summoned itself to me, unwanted, intrusive. One can experience this after horror movies too, that there is some afterimage that keeps being relived—and of course, it can happen in real life too, even for small things (embarrassing intrusive memories from the fifth grade, anyone?).
Well, turns out you can prompt-engineer your way to videos that elicit maximal responses from specific regions.
The research invited plenty of comparisons to David Foster Wallace’s Infinite Jest, which is an exaggeration… for now. It’s hard to deny, this does seem like an embryonic attempt at adversarial tech against human consciousness—even if it’s all just computational modeling work for right now (they are basically using statistical models of what these inputs are expected to do, not creating the loop directly).
We are entering an age of rapid scientific progress. People aren’t prepared for the consequences and we don’t know what’s possible. One reason we don’t know what’s possible is that neuroscience is pre-paradigmatic, and neuroscience is pre-paradigmatic because there is no scientific theory of consciousness. It’s a dangerous asymmetry. We might need defensive consciousness tech, but to do that, you need to figure out consciousness.
I rarely indulge in literary drama, but for those who don’t know her work, Rachel Cusk is a very good novelist and basically queen of the auto-fictional novel. She lives in Paris and, based on rumors, is (was?) friends with Natalie Portman, who also lives in Paris, and now Cusk has a new novel, Life of M, coming out that centers an actress who seems… an awful lot like Natalie Portman? And (again, if the rumor is true) it may not be a very flattering portrait.
This stuck with me because it reveals a lack of understanding about how vicious and jealous and critical artists can be. I can’t help but wonder why a super famous and rich person would even be friends with a writer herself famous (at least, in literary circles) for transforming daily personal life into public-facing literature. I am reminded of the parable of the scorpion and the frog.
Occasionally a scholar makes a claim so reductively seductive that a generation believes it (not to point fingers, but often the origin is some continental French or German thinker, e.g., Nietzsche’s claim that the troubadours invented the concept of romantic love). Apparently the French scholar Philippe Ariès, working in the 1960s, is responsible for the notion that medieval parents didn’t love their children in the same way. This is, of course, nonsense. Here’s from a recent review drawing from Nicholas Orme’s Medieval Children:
Ariès’s work never attained complete acceptance in academia, but it was vastly influential on the wider public. As it turns out, however, Ariès’s view is just not true….
… most medieval people had only three or four children, somewhat more among the rich and somewhat fewer among the very poor. At least in England, this was partly because they married quite late, generally in their mid or late twenties. About 25 percent of children died in their first year and another 15 percent by their tenth. The median English child thus grew up in a home with one sibling and two parents in their thirties or forties, being in this respect just like the median English child today. Surviving grandparents often lived nearby, but rarely under the same roof. …
Medieval parents made immense sacrifices for their children, and seem to have expected little in return. Feeding a child consumed about 15 percent of average income, and parents often continued to support children throughout their adulthood. Older children often performed light work at home, but at just the age they might become a net economic asset – 13 or 14 – they were usually sent away to acquire skills and savings through apprenticeship or working in service. They may have remitted some of this income to their parents, but this was not standard, and in fact parents generally paid for their children’s apprenticeships. …
The medieval people who emerge from Orme’s book are quite unlike the strange and cold figures that we have been led to imagine. ‘I believe they were ourselves, five hundred or a thousand years ago’, Orme writes.
But if humans have always loved their children thus, how much is the decline of religion explained by having less need? If I lost a child, I would either go insane with grief or become a religious apologist—I don’t think I could handle it any other way.
This time last year I was arguing that we start children reading too late, long after tablets and screens have taken hold.
As always with the Desiderata series, please treat this as an open thread. Comment and share whatever you’ve found interesting lately or been thinking about.
