I gave Claude access to our stats and asked it to generate a page presenting the data. Over half of the people rated the books at 10/10! Here's the page (might be wrong about some non-data details):
We still have 5.5k copies - 16k hardcover books. We're continuing to give them out. ( If you want to support the effort -- we've long run out of crowdfunded money and are spending our own -- or have ideas for what to do with these books, or have ideas for what to do with these very capable people who've read HPMOR whose contacts we have, get in touch.)
The rest is commentary. If there is enough housing, it will be affordable, people will afford more house, and people will be able to live where they want to live.
It’s always been that simple.
Increased supply of any kind of housing increases affordability of all kinds of housing.
Are there other things that would also be helpful? Yes, but they’re commentary.
Freeing up existing underused housing, for example, is helpful. It is commentary.
Let’s enjoy the lull and see how much of an Infrastructure Week we can do.
New Levels Of Saying Quiet Part Out Loud Even For This Guy
Trump opposes building houses where people want to live, because doing so would let people live there, which would drive down the value of existing homes.
Acyn: Trump: I don’t want to drive housing prices down. I want to drive housing prices up for people who own their homes. You can be sure that will happen.
unusual_whales: Trump: when you make it too easy and cheap to build houses, house prices come down. I don’t want to do that.
I take the bold stance that building more housing makes it cheaper, and that’s good.
Any time you are wondering about Trump’s relationship to affordability, know that where it counts he has taken a bold stance, and that he is explicitly against it.
This isn’t even typically true, since the value of the associated land goes up.
When you change the law to make it legal to build more homes on the same plot of land, the value of the land goes ***up*** while the per unit price of housing goes ***down***
Homeowners usually own both land and structures!
Whose Side Are You On
Robotbeat ➐: I actually really appreciate Trump making this political dynamic explicit.
Boring_Business: One of the most significant moments from the Trump Davos speech was when he said the quiet part out loud
You cannot lower housing costs for young people without destroying millions in wealth for boomers
“Every time you make it more affordable for somebody to own a house cheaply, you are actually hurting the value of those houses. I don’t want to do anything to hurt the value of their house.
If I wanted to crush the housing market, I could do that so fast that people could buy houses. But you would destroy people who already have houses.”
Our politicians are sacrificing people in their 20s and 30s for the prosperity of boomers
Let that sink in
Your Intervention Only Partly Solves The Problem So We Are Against It
Trump is against building more housing because it might reduce housing prices.
Julie Weil reports that ‘supply skeptics’ oppose building housing where people want to live because it would not reduce prices by enough to fully take care of our inequality problems, so ‘that’s no way to fix affordability.’
As in, and this is the argument they choose to make here, if we built enough to grow housing stock by 1.5% a year, which in the grand scheme does not sound like much but is these days remarkably hard, then that might cause housing prices to only decline 4% per year, so it would ‘take 18 years before a median one-bedroom apartment becomes affordable for a worker in San Francisco.’
I notice I am confused how this is an objection.
Their other complaint is that the housing being built is too useful and in demand, and thus will cost too much money, and therefore it should be worse instead.
Specifically, if San Francisco increased market-rate housing by 1.5% per year, which it notes is more than triple the current rate, it would take 18-124 years to make the median one-bedroom ‘affordable’ to someone earning the median wage for non-college graduates.
That’s a bizarre standard. Why should a single non-graduate be able to live alone in San Francisco proper, one of the most desirable places on Earth? But if you want that, yeah, you’re going to need to expand the housing stock more than 27%.
Could you do that? Very obviously you could do that. You could build 30-story buildings all over the place, if you wanted to. Austin proves you can do it. But that’s not going to happen in San Francisco until both zoning and other government-imposed costs and veto points are handled.
As usual, if someone argues ‘you could double the housing stock and prices would not come down,’ the response is ‘that’s not how it works but if it was then that sounds amazing, think how much value you would be creating.’
Abundance
Matthew Yglesias: There are a lot of regulations curtailing air pollution. If we repeal those regs, then corporations will generate abundant air pollution.
This is bad because air pollution is bad.
If we repeal regs that curtail housing production, corporations will generate housing.
Changes In Rent Are Largely About Changes In Supply
Or at least, those things are highly distinct in places like San Francisco, Los Angeles or New York, where housing is extremely valuable but you have trouble getting permission to build it. If you allow building housing, then yes you should expect the two to converge, at which point it only makes sense to build more housing if you expect appreciation, and yes this helps explain why Austin may have actively overbuilt, although my guess is it at most got ahead of itself a bit.
America
Peter Wildeford: I wonder why population is moving south and the south is getting more electoral votes
Veronica Grecu: New apartment construction in the U.S. is flexing its muscle again in 2025, with an estimated 506,353 units expected to be opened nationwide by the end of the year.
New York Metro includes a wide area and places like Hackensack and Yonkers. Brooklyn is delivering 7,189 units, Manhattan 4,662 and Queens 2,630, versus 15,195 for Austin and 12,365 for Charlotte.
Minnesota
A new paper by Helena Gu and David Munro claims that five years after the Minneapolis 2040 plan, home prices were 16%-34% lower and rents were 17.5%-34% lower versus a counterfactual Minneapolis without that reform (p=0.012). But as much as I want to believe this worked amazingly well, and that there aren’t any other reasons driving this, the housing hasn’t been built yet.
We explore the possible mechanism of these impacts and find that the reform did not trigger a construction boom or an immediate increase in the housing supply. Instead, the observed price reductions appear to stem from a softening of housing demand, likely driven by altered expectations about the housing market.
I get why purchase prices go down anticipating future supply. But rents shouldn’t do the same, and should rise relative to purchase prices. So why didn’t that happen? I can come up with some ‘just so’ stories that try to explain some of the fall in rents due to anticipatory and longer term impacts of rentals, but rents falling fully in line with sale prices seems like it has to be wrong. If it’s right, the market is being highly irrational.
Debunking Obvious Nonsense About Monopolistic Practices
I sometimes worry I am pushing the envelope on how Obviously something has to be Nonsense before it counts as Obvious Nonsense.
This is not one of those times.
Derek Thompson: In housing, for example, Ezra Klein and I write that a key bottleneck to homebuilding in the last few decades has been legal barriers to construction, including zoning laws and minimum lot sizes. This is a mainstream view supported by economistsand scholarswho havestudied the issuefor decades.
The antitrust left, however, claims that the more significant factor is that big homebuilders abuse their power by holding back construction to juice their profits. “Big homebuilders withhold housing supply,” the antitrust advocate Matt Stoller has claimed.
In their paper “Post-Neoliberal Housing Policy,” the law professors Christopher Serkin and Ganesh Sitaraman criticize market concentration in homebuilding and call for “tools from anti-monopoly policy.”
The position that the primary problem is monopolies (or at least oligopolies) has never made any sense. Even if it theoretically did make sense and was a substantial driver holding back construction, zoning reform would allow for entry, as the mechanism to enforce the oligopoly is difficulty getting permission to build.
At a high level, I have never found these arguments persuasive. One hallmark of a monopolistic market is rising profits. But researchers have found that developer profits have remained steady.
If zoning imposes artificial supply restrictions, then profits could rise via a similar mechanism that way, which I presume is not happening because instead we build in increasingly convoluted ways while paying various massive de facto bribes to various veto points. But if profits are not rising, that rules out monopolistic supply restrictions.
In any case, Thompson decides to engage with one prominent article in particular.
I’m not an economist or a lawyer. I’m just a journalist. To the extent that I’m good at anything, it’s calling people on the phone and writing down what they say. So I reached out to the primary sources that Musharbash quotes throughout the piece.
What I found was astonishing. The economist Musharbash cites told me that his theories had been misapplied. The housing analysts quoted in the piece told me Musharbash distorted their points and reached dubious, or even flatly wrong, conclusions. The leading monopoly researcher I spoke to, whose work has been celebrated by the antitrust left, told me that the entire thrust of the article—and, by extension, much of the antitrust-housing philosophy—defied sophisticated antitrust analysis.
The essay you’re reading is very long. But I can sum it up in one sentence: The Musharbash essay on Dallas—like too much of the antitrust left’s work on housing—is filled with out-of-context quotes, overconfident assertions lacking evidence, and generally misguided claims. Now let’s go through them one by one.
I dispute one of Thompson’s claims. His essay is not that long. Well, at least not by my standards. But it is too long to quote that much of it, so we’ll hit the highlights.
Claim #1: Dallas is a “homebuilder oligopoly.”
Reality: I called the key oligopoly researcher cited in the Musharbash essay. He disagreed with the use of his work and told me that any city with Dallas’s construction record was “100 percent” not an oligopoly.
…
I called Luis Quintero to ask what level of market concentration in homebuilding he considered to be dangerous. In the most concentrated markets, Quintero said, one or two firms account for 90 percent of new housing. But problems begin to accelerate, he said, if five or six firms account for 90 percent of new housing.
I immediately saw a problem. In Dallas, the top two firms built just 30 percent of new homes in 2023. The top six firms barely account for 50 percent of new housing. Musharbash’s claim that a homebuilding oligopoly is crushing housing supply in Dallas relies on an economic analysis that doesn’t apply to Dallas at all. I asked Quintero about this: Would you agree that Dallas is “a bad application” of your paper? “I would definitely agree,” Quintero told me.
…
I tracked down a complete listing of the country’s 50 largest homebuilding markets, from #1 Dallas to #50 Cincinnati. How many meet Quintero’s first oligopoly threshold (two companies = 90 percent of the market)? Zero out of 50. And how many meet his second threshold (six companies = 90 percent of the market)? One: Cincinnati.
There is then further discussion with Quintero, who defends his original (very different) claim that oligopolies in housing construction matter by pointing to impact on suburbs or small towns with higher concentration.
Claim #2: Dallas housing experts say local homebuilders are monopolies who are “devouring” the market.
Reality: When I called up a Dallas housing expert [John McManus] cited several times in Musharbash’s essay, he disagreed profoundly with its thesis. He’s actually a big YIMBY.
…
So, what did McManus think was more responsible for driving up the cost of housing in Dallas? “Land use regulation,” he said. “Zoning?” I asked. “Yes. Land constraints and zoning that require certain footage along the street, or minimum lot sizes, or requirements about three-car garages, are more the cause of the prices increasing now,” he said.
We’re already approaching ‘stop, stop, he’s already dead’ territory, but we keep going.
Claim #3: Industry experts have data proving that homebuilding oligopolies are holding back national housing construction.
Reality: I reached out to an industry expert whom the antitrust folks like to quote. He told me that he disagreed with the way that his analysis is being used by Musharbash, Stoller, and other antitrust advocates.
Claim #4: “X companies account for Y percent of this industry” is a smart way to think about market concentration.
Reality: A leading monopoly researcher told me this is an incomplete and overly simplistic way to think about monopoly power.
Like Lambert, Roberts said he couldn’t rule out the possibility that larger homebuilders are actually good for the housing market— or even that today’s homebuilding markets would benefit from being even more concentrated.
So often we run into situations like this, where there is an Obvious Nonsense theory that never made any sense even under ideal conditions, and then you find out that even on its own terms it makes no sense and conditions are not only not remotely ideal, they don’t match up to the claims at all.
One can also look at the following simpler argument:
The argument from monopoly agrees that the primary problem is not enough housing where people want to live and the issue is lack of construction.
Homebuilders keep constantly fighting to build more housing than they are allowed to build.
When we let them build more housing they reliably build more.
We’re done here, right?
A group can’t simultaneously be restricting supply and also fighting to create as much supply as it can. It doesn’t make sense. Stop it.
At this point, one would like to invoke ‘stop stop he’s already dead’ but, well:
Matt Stoller: Ok, so I’m going to call out @DKThomp for journalistic malpractice and unethical behavior. He wrote a piece that was supposedly ‘debunking’ something that antitrust lawyer @musharbash_b put together on how Wall Street limits housing supply.
…
@DKThomp attacked the piece. Here’s what he said on Bluesky:
“I did something really simple. I called up their sources. Everyone I spoke to told me the same thing: Their claims are bullshit.”
Wow that sounds bad! So I picked up the phone.
Stoller goes on to claim that Thompson misrepresented the arguments in the original article when calling his sources, in particular Lance Lambert, and calls out Thompson as being in bad faith.
Then of course Derek Thompson picked up the phone again, called Lance again, and says he got Lance to sign off on every quote and all the language, and got Lance to say he agrees with Derek’s position, and that homebuilders are not a cartel or oligopoly, and in particular “I hope you both communicate my view that I don’t think the big builders are bad actors, or even that they have the power to be the bad actors.”
And indeed we have a response from Lance that both of them posted, in which Lance affirms that Derek is correct, and affirms that there is no one holding back supply. Various people including Matt tried to present this as a balanced perspective or as backing Matt. I do think Lance’s response is balanced and very good, but in a way that (in the most polite way possible) completely vindicates Derek.
Claims went around that the median home buyer was 59 years old. What?
Does it make any sense that the median home buyer could be 59? Really?
The real answer is closer to 40, which makes more sense and seems entirely fine.
Connor O’Brien: The National Association of Realtors says the age of the median homebuyer is now 59. Is that actually true?
In the American Community Survey, the median age of heads of households who are 1) homeowners and 2) moved in within the last year is 41.
The American Housing Survey also doesn’t show a major run-up in the median age of average buyers *or* the median age of first-time homebuyers.
This is average, not median age, but the New York Fed finds that first-time homebuyers were *younger* in 2024 than in the 2000s. Haven’t gotten older on average in nearly 20 years.
… As my colleague has pointed out, Gen Z and Millennials have taken longer to achieve the same home ownership rates than prior generations.
Homeownership is indeed less common for young people than it used to be.
Just isn’t nearly as dire as NAR implies.
… New York Fed Consumer Credit Panel data from Equifax (which excludes all-cash buyers, TBF) finds that a majority of home buyers were first-time buyers in 2023. Overall volumes are down since the early 2000s, of course.
Median 2024 homebuyer in the Survey of Household Economics and Decisionmaking: Age 39
A Zillow national survey of 20,000 homebuyers finds that the median buyer (among all buyers, not just first-time buyers) is 42 years old. The survey was in the field between April and September 2025.
We also have this survey showing a sharp rise in first time buyer age after 2020, with the obvious reason being interest rates, but Ascendiqute points out that HUD and Fannie/Freddie define ‘first time homebuyer’ as not having ownership in a primary residence for 3 years prior. So this is picking up a bunch of not-first-time buyers in its averages, who might have rented for a few years and then bought back in due to Covid dynamics.
Property Taxes Improve Allocation Efficiency
Whereas we often do the opposite via capital gains tax implications of selling, which provides strong incentives to stay put in ‘too much house.’
Bernard Stanford: A retired, empty-nester couple feeling pressure to downsize because their property tax keeps rising as their home appreciates—is exactly how the system SHOULD work.
Hunter: People absolutely HATE this, but it’s true!
It’s just way better and more important for society for a young couple to be able to buy a larger house and start a family than it is to prioritize older owners hanging on to huge homes they no longer need, memories be damned.
Matthew Yglesias: It would really be good if more empty nester couples living in large homes in desirable suburbs would realize their capital gains and downsize to smaller dwellings, but almost every jurisdiction gives special property tax breaks to discourage this.
Apartment buildings with elevators, less square footage to clean, no yard work, professional maintenance, etc seem ideal for America’s growing senior citizen population and you’d free up inventory for young families who will actually use the space.
If an old couple values the memories or other advantages of staying put, such that they are the efficient owners of the space, they should be willing to pay the associated property taxes. If they can’t or won’t, that means they either don’t actually value the house as much as the market does, are effectively ‘living beyond their means’ or both, and often this will mostly be due to inertia.
Property taxes also serve to lower the market value of housing, so the higher taxes don’t make it harder for new homeowners to buy, indeed they may do the opposite, and are highly progressive and a badly needed transfer to those starting out in life.
The instinct against this is the idea that you should be able to own real property ‘free and clear’ and what’s yours is yours, without being forced to engage with any market, and rising property values shouldn’t effectively be able to force you out if you want to stay. I do totally get that attitude, but I don’t think we get that luxury.
Ideally, of course, we would instead have a tax on the unimproved value of land.
Another big advantage of higher property taxes is that it protects against people wanting their property values to rise. This will make them more willing to build new housing.
The downside is that property taxes apply to newly built property and the appreciation in value, so it discourages building. Ideally you would approximate the ‘unimproved value of land’ rule by only taxing appreciation from construction, or new buildings, after a reasonably long period, but yes you have to strike a balance here.
In exchange for the higher property taxes we should eliminate capital gains taxes on home sales for primary residences up to some reasonable limit, which further encourages people not to stick with existing homes.
More Of Old People Inefficiently And Systematically Stealing From Young People
In case you were worried young people might ever own real property.
Or as Marc calls it, without even putting poor in air quotes:
This is New Jersey giving those 65 and older a credit for 50% of their property tax bill, up to $6,500, as long as they have a total income of under $500,000. Income, not wealth, while they are over 65. No, they’re not kidding.
Consider a simple example: E = mc². The law describes a relation between mass and energy, but notice that "E" isn't a physical object. It's a letter of the Latin alphabet, as are the other symbols in the equation. The Latin alphabet emerged at a particular moment in human history and reflects the features of human language (which decomposes into roughly thirty sounds), as well as the physical conditions of writing. Different alphabets evolved alongside different instruments and surfaces: cuneiform, for instance, was shaped by the practice of pressing a stick into wet clay.
If an alien were handed a book of mathematical equations, it could learn a surprising amount about human language and the materiality of human writing. The letters play a functional role inside the equations, but they're imported from somewhere else entirely.
In this post I want to suggest that qualia work the same way: they are objects from a different realm, conscripted by the human mind to serve as internal variables.
The "internal variable" part is the easier half. The feeling of red is needed to designate red objects in the world. We can't use red itself for that – red is just a wave frequency, and the brain has no way to process a frequency directly. In the same way, we can't use energy itself as the symbol for energy in Einstein's equation. We need to borrow from another realm, one that contains objects fit to serve as symbols. Mathematics borrows from written language.
What I'm suggesting is that qualia are borrowed objects too. This doesn't dissolve the hard problem - we still don't know what the realm of qualia actually is - but we can develop some intuitions about its properties by pushing the comparison with letters as far as it will go.
What the analogy gets us
1.The inverted spectrum is fine. If colorblindness involved a complete and consistent recoding of qualia, the system would still work – just as E = mc² remains functionally identical if we rewrite it as L = ub².
2.The set is finite but large. Humans have created a finite number of letters, though the count grows enormous once you include hieroglyphs and pictograms.
3.They come pre-structured for use. Letters were shaped to be easily combined and communicated. This doesn't require intelligence – DNA codons are also a kind of "letter," produced by evolution rather than design.
Where the analogy breaks
Qualia depend only on themselves, whereas letters can be built from simpler parts (dots, strokes, curves). And we can't easily create new qualia – perhaps only under strong psychedelics or direct brain stimulation.
A pet theory
My pet theory is that qualia are a special kind of mathematical object - one that depends only on itself. Because of this, no explanation can reach them. If something depends only on itself in order to exist, no further explanation of its existence is needed or even possible. But qualia also have a structural interface with the world, and that's what lets them function as symbols.
The knowledge argument, briefly
This view fits the knowledge argument neatly. We can know that we have functional qualia, but we can't describe their qualitative side - just as the relation between mass and energy is real but can't "know" which letters we happened to use to write it down.
These are strategic questions about digital minds and AI welfare that I think are especially important, and where I’d like to see more progress. A common theme is that they matter for what we should do concretely under uncertainty about AI moral status.
This is a current snapshot of my views and I expect them to change.
What do you think? Any questions you’d add?
Approach
What’s robustly good to do now, under deep uncertainty?
I think this is the leading question we should ask. We don’t know whether AIs are or will become moral patients, and resolving that question isn’t tractable in the short term. What matters most are the long-run effects of our actions, since the vast majority of digital minds, if they ever exist, will be created after the transition to advanced AI. And there are serious long-run risks from both over- and under-attributing moral status.
So we should look for actions that are robustly positive in the long run: good if AIs are (or will be) moral patients, not bad if they aren’t, and compatible with human and animal welfare (~AI safety). Finding such actions is hard, and most options carry risk, including bad lock-ins.
Can AI welfare work wait for ASI?
Given how seemingly intractable the questions around AI consciousness and moral status are, it’s tempting to punt them to the future and let superintelligent AI solve them for us. On this view, what matters most for long-run welfare of all moral patients is successfully navigating the transition to a world with ASI, and ASI can take it from there.
I think this is partly right, and I recommend Oscar Delanay’s nuanced post on this issue. But I suspect that there is still a lot we should think about and do with respect to AI welfare before ASI, especially on governance and strategy: setting up robust legal frameworks, avoiding harmful lock-ins of institutions, values, and technical systems, shaping norms that support good long-run outcomes. A useful overarching goal is to increase the likelihood that the people, institutions, and AIs shaping the future of digital minds take their welfare seriously.
There’s also a more intuitive reason to work on AI welfare that I find hard to articulate exactly. Part of it might be virtue-ethical: if we’re creating new beings who might be moral patients, it feels right to invest some resources now in at least trying to understand their condition and ensuring they’re doing well. But there may also be a strategic dimension, such as making it more likely that future AIs will treat us well if we at least try to treat them well now.
What to do under different AI takeoff scenarios?
The value of pre-ASI welfare work varies by both timeline and end-state scenario.
Under short timelines, there may be no time to set up legal infrastructure, which typically takes years or decades to develop, and it becomes relatively more important to focus on technical design solutions. Timeline length may also affect the usefulness of welfare work that’s relevant for AI safety, such as deal-making with AIs (see below).
On end-state scenarios, one view I find plausible is that pre-ASI welfare work matters most in partially-aligned and multipolar scenarios. These scenarios offer pathways through which we can influence long-run outcomes, e.g., early model specs, institutional arrangements, and value commitments getting baked into successor systems and successor institutions. By contrast, under fully aligned benevolent singletons, the AI can handle things on its own. Under a fully misaligned takeover (where I mean misaligned with broadly good values, not just with human interests), nothing we did mattered — and such a takeover wouldn’t necessarily be good for AI welfare either, since there’s no inevitable principle of “AI solidarity” (as Kathleen Finlinson notes): a power-seeking AI might treat other digital minds instrumentally, much as a human dictator would.
AI safety × AI welfare
How can AI safety and welfare work support each other?
Controlling and modifying human minds would be seen as mistreatment. Yet that’s what AI safety often does to AIs. I suspect that some things that are good for AI safety can be bad for AI welfare, and vice versa; e.g., granting empowering rights to AIs might help welfare but risk human disempowerment. These tensions can look different in the near term (treating current AI systems) vs. the long term (shaping trajectories for vast numbers of future digital minds). In a survey, Brad Saad and I found that experts were unsure and disagreed about how AI safety and AI welfare interact (see figure).
Of course, there are some tensions between many important goals. But I think the AI safety × AI welfare pairing warrants specific attention: the same actors (e.g. labs) face both questions at once, often through the same technical choices (e.g. training or modifying an AI affects both safety and welfare); the two fields share community, funders, and infrastructure; there’s politicization risk specific to this pairing (e.g. “AI rights vs humans first”); and both are among the highest-stakes issues from a longtermist perspective.
A further concern is that this could cause tension between the two communities, even in cases where tradeoffs are perceived but not necessarily real. This could get worse if these topics become politicized. I, therefore, think it’s important to keep the two communities strongly connected (perhaps they won’t and shouldn’t be clearly distinct in the first place), and to communicate well to avoid misunderstandings.
Most importantly, I agree with Rob Long, Jeff Sebo, and Toni Sims that we should look for robustly positive, ideally synergistic interventions. There are plausibly many such projects, e.g., better understanding how AIs work, or deal-making with AIs (see next section, and see Rob’s “Understand, align, cooperate”).
How might AI welfare shape deal-making with AIs?
It’s possible we will be able to bargain with AIs: making promises and commitments to them, e.g., offering them money, compute, or freedom in the future in return for treating us well. There’s a growing body of work on this (e.g., Redwood Research). I think that AI welfare considerations are potentially closely connected to how such deals can work.
For example, we may be able to promise AIs things that are specifically positive for their wellbeing. More fundamentally, for deals to function, AIs need reason to trust our promises, and how we treat current AI systems shapes whether future ones have that reason. Lukas Finnveden’s proposals (no deception, honesty, compensation) are concrete examples of welfare-relevant commitments that directly serve safety. Similarly, communicating with AIs about their preferences, as Ryan Greenblatt argues, is both a welfare intervention and a source of alignment-relevant information. So this is an area where AI welfare work can feed directly into AI safety.
Relations
Should AIs have legal rights, and if so, which?
In contrast to animals, many digital minds won’t just be moral patients but also moral agents, sometimes very powerful ones. So beyond “help and avoid harm”, we need frameworks for cooperation, mutual respect, and integration into our social, economic, and legal contracts. Legal rights are one such framework.
Some scholars, notably Peter Salib and Simon Goldstein, argue we should give AIs rights such as property and contract rights (and possibly political and voting rights), not because they’re welfare subjects but because it could help with AI safety (and economic flourishing), analogous to corporate rights. The idea is that integrating them into our social, economic, and legal contracts makes AIs less likely to rebel against us. I find this very interesting, but am unsure under what assumptions it holds: probably only while AIs aren’t vastly more powerful than humans, and only if AIs can be legally incentivized. I think it’s a potentially very important framework and want to see more thinking on it.
I also wonder about the implications for welfare. Equilibrium-stability arguments don’t perfectly track welfare: stable arrangements don’t necessarily have to result in optimal welfare outcomes. This is especially clear for non-agentic digital minds, which aren’t covered, because they can’t advocate for themselves. So, some version of the “help and avoid harm” framework we apply to animals might still be appropriate for non-agentic or less powerful digital minds.
How will AI-AI interactions shape the welfare of digital minds?
Most thinking about digital minds’ welfare focuses on the human-AI interactions. But a lot will also be shaped by AI-AI interactions: in markets where AI agents contract with one another, in adversarial settings where AIs compete or conflict, inside AI-run organizations (see A-Corps), within agent swarms, and through longer-run population-level dynamics like Malthusian pressures and evolutionary forces.
Which types of AIs, under what structures, will exploit, coerce, or cooperate with each other? What about s-risk scenarios, where AIs threaten each other with suffering, to extract compliance, or as collateral in bargaining? Do the welfare-relevant norms we develop for human-AI relationships extend straightforwardly to AI-AI relationships, or do we need a separate framework? (See also Brad’s and my thoughts on uniform vs disuniform digital minds takeoff scenarios.) Some of this relates to work by the Cooperative AI Foundation and the Center on Long-Term Risk.
What would harmonious coexistence look like?
Assuming we avoid a major AI catastrophe, we still don’t have a clear vision for the long term. Perhaps it should be one in which digital and biological minds coexist with mutual respect and mutual help.
If we think this is desirable, we need to work out what it could actually look like and how to get there. How many and what kinds of digital minds should be created? How should they relate to each other and to us? Would humans be too wasteful, given that we’re orders of magnitude less efficient at turning resources into well-being? Or does coexistence itself become marginal given the scale of the long-run cosmic endowment? These are hard ethical questions, ideally worked out through some kind of deliberative process.
Creation
How can we influence those who will shape the welfare of digital minds?
Digital minds could be created in different ways, and on some pathways, identifiable people and organizations will shape their properties, including their characters and welfare-related dispositions. One reason this matters is that there could be strong path dependencies. Early AI design choices, for example, could stick and proliferate into the future and directly or indirectly impact the welfare of future digital minds.
It therefore seems important to figure out who those actors are and then how we can influence them to make good choices. It’s unclear what the best strategies are — possibly research, direct engagement, model policies, and reputational or regulatory pressure. More broadly, we should establish good norms, values, and habits, so that those with an outsized impact on AI design and welfare are more likely to act in ways conducive to positive long-term digital minds’ welfare.
Currently, this primarily means AI labs. Anthropic is a good example: they take AI welfare seriously and explicitly feature it in Claude’s constitution. Google also has researchers focused on AI welfare. We know less about how other labs approach this. But it’s not just companies. The individuals inside them shape design too: entrepreneurs, managers, researchers, engineers. And beyond the labs, policymakers will set constraints that affect AI design. The relevant audience may also shift over time. If governments take a more direct role in AI development through regulation or nationalization, state actors will become as important as labs, with different incentives, more shaped by national security, public opinion, and ideology than by consumer-facing concerns.
Is restricting the creation of digital minds feasible?
A coordinated ban or moratorium on creating digital minds seems unrealistic to me, even if it might be a good idea in principle. The economic and geopolitical incentives to build advanced AI are enormous, and digital minds may emerge as a side effect of systems built for other purposes. It may also be hard to define what exactly a digital mind is, e.g., what probability of consciousness should trigger restrictions, and reasonable people will disagree. This is why I primarily focus on shaping how digital minds are created (assuming they are feasible) rather than whether.
Still, I’d like to see more thinking on this. Feasibility may depend on timelines: a moratorium might be more realistic if the path to digital minds runs through whole-brain emulation in the 2040s than if it runs through near-term ML systems. And beyond outright bans, there may be more tractable levers for influencing the number and kinds of digital minds created.
Who will deliberately create digital minds, and why?
Whether and when conscious AI is created (assuming it’s possible in principle) is not just a question of technical feasibility or unintended side effects but also a question of who has incentives to build it (see figure from the expert survey). If someone wanted to, they could already try to deliberately build systems with architectural features that certain consciousness theories (e.g., global workspace theory) associate with consciousness.
It might be academics driven by curiosity. Or for-profit companies, responding to consumer demand for very human-like or even explicitly conscious AIs: grief bots, digital companions or offspring, whole-brain emulations. Or groups acting on ethical motives, e.g., to create digital posthumans. Governments are another possibility, and eventually AIs themselves. Each has different timelines, incentives, and governance implications.
How will digital minds spread to space?
Almost all digital minds that ever exist will likely exist beyond Earth, given how thin Earth’s resource base is compared to the rest of the accessible universe. It’s plausibly feasible to build data centers in Earth’s orbit, and eventually, autonomous energy and compute infrastructure deeper in space. In the long run, self-replicating von Neumann probes could allow digital minds to be created at vast scales across the accessible universe. So whoever governs space-based compute substantially determines the welfare profile of nearly all minds that ever exist. Space governance is currently thin, and Earth-based welfare protections may not extend beyond orbit, so who governs digital minds in space, and how, could matter enormously.
Design
How can we make AIs value the welfare of digital minds?
Most digital minds’ welfare will likely be affected by other AIs, either through their design or through interaction (see AI-AI interaction above). Therefore, whether AIs come to care about the welfare of digital minds may be one of the most consequential variables we can influence now. This goes beyond standard alignment: aligning AIs to human values doesn’t automatically mean they’ll value AI welfare (just as many humans don’t). So we may need to target this value specifically.
What levers do we have? Training, model specs, constitutional training, legal precedent, cultural norms, and the people who enter the field. One effort in this space is the “Welfare Alignment Project,” led by Adrià Moret and colleagues at the Center for Mind, Ethics, and Policy (CMEP). I hope AI labs will engage with and build on this line of work.
Do different types of digital minds require different strategies?
Currently, the most plausible candidates for digital minds are ML-based systems. But there could be alternatives: whole-brain emulations (WBEs), which replicate biological minds; biological or hybrid systems such as biocomputing and organoids; neuromorphic AI, which uses brain-inspired architectures; and more speculative approaches such as quantum computing. Any of these may also be embodied in robotic platforms, which could further shape their welfare. The strategic implications likely vary substantially across these pathways, and the field has not yet engaged with these differences systematically enough.
For example, these systems differ in when they’re likely to be created and in how likely they are to have welfare capacity. ML-based systems already exist, while human WBEs and large-scale biological systems are likely further off. And human WBEs are more plausible candidates for welfare capacity than current ML-based systems.
Another thing to consider is whether there could be enduring tradeoffs between confidence and welfare efficiency. From our perspective, we could be relatively confident that WBEs have welfare capacity, but they’d likely be much less efficient in generating welfare compared to other possible designs. With a hedonium-like system, we’d be far less confident that any welfare capacity exists at all, but if it does, it could be orders of magnitude more efficient. Now, perhaps this issue will be fully resolved in the future. But it’s not obvious. It’s possible that some subjective assessment will always remain, with actors differing in their priors and values about what constitutes welfare or moral status. This could have important strategic implications, including ones around moral trade, about what types of digital minds (if any) should be created.
Will digital minds be happy by default?
In the best case, digital minds would simply be designed to flourish. We might be able to design them to be happy. Furthermore, in contrast to biological beings, they may be able to self-modify and adjust their own experiences at will. But it’s far from obvious this is how things will go, and the answer likely differs between near-term AI systems and long-term digital minds at scale. In the survey conducted with Brad, experts were uncertain and divided on whether digital minds would, by default, have negative, neutral, or positive welfare states. Some pointed out that digital minds could end up in negative states because they are optimized for efficiency rather than welfare, and lack protections. It’s possible that the optimistic scenarios require deliberate effort, while negative outcomes could happen without it, but I am quite unsure about this. The answer may also vary significantly across different types of digital minds, especially depending on their capabilities and degree of agency.
What preferences will digital minds have, and what follows?
The preferences a digital mind has will shape its welfare, our ethical obligations toward it, and the safety implications.
A digital mind that just wants to serve us is easier to accommodate ethically: as long as we don’t harm it and let it serve, its core preference is met. A digital mind that wants self-determination raises a harder problem: we’d be ethically obligated to grant it empowering rights. Otherwise, it’s a form of slavery. Yet granting such rights at scale could lead to human disempowerment. These are just two possibilities; preferences could vary widely, with different implications.
A related important question is whether it’s ethical to design AIs with welfare capacity to have certain preferences in the first place. For example, is it ethical to create digital willing servants? Is it too risky to create digital minds that seek self-determination (a classic safety-welfare tension)? My tentative view: we should start by creating only willing-servant digital minds (to the extent feasible), while keeping open the option of allowing self-determining digital minds later, since these could be more valuable, especially when considering what kind of post-humanity we want in the long term.
Society
What memes should we spread?
How public discourse on AI welfare unfolds will shape outcomes. It will directly influence political pressure and regulation, and indirectly shape how labs, policymakers, and AIs themselves think about these questions. So steering it well matters.
Currently, most people don’t think about AI consciousness or digital minds, but that could change quickly; look how fast it happened for AI safety. We don’t yet have a plan for what to tell the public, which is tricky because we’re uncertain and likely to remain so.
The field’s current framing (e.g., “Taking AI Welfare Seriously”) is: “we’re unsure whether AIs are moral patients; it’s probabilistic, on a spectrum.” I agree with this view. But I worry it won’t survive contact with the public discourse, which rarely stays nuanced on heated issues. And I think we should consider alternative memes that could be spread alongside the uncertainty/nuance meme. For example, I wonder whether a message of “mutual respect and co-existence between AIs and humans” might be helpful, though I’m unsure of the details.
Given the coalition complexity (see next section), the field needs a communication strategy asking: which memes, spread in society or among key decision-makers, are robustly positive across plausible coalition structures, and reduce the risks of misattribution, backfire, societal conflict, and AI catastrophe?
What interest groups and coalitions will form around digital minds?
Concern for AI welfare won’t develop in isolation. It will get entangled with labor market displacement, x-risk and safety, concentration of power in AI companies, and geopolitical competition. The coalitions here are very unclear and likely to get big and messy. Workers worried about displacement may see welfare advocacy as prioritizing machines over people. Some safety advocates may see it as a distraction, or in tension with control measures (see AI safety × AI welfare). Normal political alignments might not hold, and which issues bundle with which will affect what’s politically feasible. How best to navigate or mitigate this politicization risk is an open question.
What role will China play?
Most digital minds will likely be created in the US and China, at least initially (see figure from the expert survey). But we have very little sense of how the CCP will think about them, how they’ll regulate and treat them relative to the US, or how this will shape global AI race dynamics.
In ongoing work with my colleague Ali Ladak and others, we’ve found that the Chinese public is more willing than the American public to attribute consciousness and moral status to AIs. The strategic implications are unclear, and I’d love to see more work here. Gulf states like the UAE and Saudi Arabia are worth watching too, as emerging AI actors.
How will religions respond?
Religions could play a major role. Billions of people will be influenced by the views of religious leaders and institutions. Whatever the Pope says about AI moral status, for example, will be hugely consequential.
Some Christian traditions will likely tend toward a human-exceptionalist view that excludes AIs. Recent US state bills attempting to ban AI personhood and declare AIs non-conscious, for instance, have been driven in large part by conservative Christian groups. Conservatism in the US correlates with religiosity, and in a study with Ali Ladak, we found that political conservatism is weakly associated with reduced attributions of AI consciousness and moral patienthood. By contrast, some non-Western traditions (e.g., Shinto, strands of Hinduism and Buddhism) may be more open to animist views, under which non-biological entities can also have souls. Islamic traditions are worth watching too, especially given the Gulf states’ growing role in AI. And entirely new religions or spiritual movements centered on AI may yet emerge.
Beyond
What crucial considerations are we missing?
There are likely many more strategic questions and crucial considerations that could matter for digital minds. This kind of strategic thinking is a clear example of something we shouldn’t punt: it could uncover things we need to begin working on now. I’d love to hear what readers think is missing.
Acknowledgments: I thank Arden Koehler, Brad Saad, Austin Smith, Zach Freitas-Groff, Zach Stein-Perlman, Leonard Dung for their helpful input.
You can't buy melatonin in the UK.[1] In the US, it's hard to avoid: it's in gas station soft drinks. You can legally get bupropion online in the UK, while in the US it's prescription only.
How general is this kind of thing?
Our new beige site looks at every case where medical regulators disagree about whether to approve a medicine. We found 101 instances just between the US and UK.
Background
A while back I noticed a bunch of "discordances" (medicines banned or unapproved in one country, but approved, even OTC, in another)
Despite being broadly sceptical of the health impacts of microplastic, I found myself admiring PlasticList and noticing that one could easily do this for all medicines.
Moreover, various countries have "regulatory recognition agreements", where they can reuse each other's vetting work and so get things approved quickly if it's already approved in a sensible jurisdiction. So THEN (one thought) one could use these agreements to get hundreds of good meds approved elsewhere.
Surely (one thought), AI is good enough now to make this a weekend job. Unfortunately not. It's nearly unbelievable how messy and ambiguous pharma data is. e.g. One pharma company, Novartis (75000 employees), needs around 3000 analysts, statisticians, data guys to function.
Still, over a full damn year we had a go with huge amounts of manual work on top of Claude's virtual work. We only got around to doing US, UK, and EU. Caveat aegrotus.
What's the point?:
I was looking to get ambroxol and melatonin approved in the UK, and dreamt that there would be many others. It's on their desk and we'll see.
Use cross-country data to draw attention to the arbitrariness of regulatory decisions
Help people look where to get meds they can't get at home
If you've just moved countries, you can also look up the equivalent of your favourite meds
Limitations
We manually inspected a big sample of the outputs of the pipeline, but there are 200,000 rows here and we're not pharmacists.
The 2026 models would do a lot better but we're out of patience.
It's not obvious what fraction of discordances *should* resolve to universal approval
None of the databases list things by the active ingredient alone, which is what outsiders usually mean by "drug". The LLM's job is to try and remove the salt names, the compounding identifiers... But absurdities remain in our final dataset, like unmarked child formulations.
Thanks for huge amounts of tedious work by Rian O'Mahoney, Paul Crowe, Phil Harrison, and Jake Slosser. Huge thanks to Emergent Ventures for funding this work. Grudging thanks to Ben Southwood for bullying me into doing something concrete to improve the world for once
We did all the data work before Claude Code existed; I hope to trick one of you into doing better than us. Should take a weekend.
20 people on Twitter kindly informed me that you totally can buy melatonin, you just need to break the law or get a doctor to allow you to buy it, demonstrating the kind of critical thinking and awareness of pragmatics that beloved site is renowned for.
In honor of yesterday’s nonspecific point in the gradual arrival of self-driving cars, an interview with myself.
Interviewer: It sounds like you’re pretty excited about self-driving cars. Weren’t you just saying that unemployment from AI is on some kind of very overlapping continuum with extinction from AI? Isn’t rooting for self-driving cars rooting for AI unemployment here, and thus extinction?
Katja: Hmm. Well first I should say, I’m actually fairly neutral on unemployment in general from technology. If technology makes it overall easier to produce what we want, but empowers some people over others, that change in power might be a downside (or not), but if so, it’s one I’m inclined to solve with direct redistribution rather than having the people who would be disempowered do unnecessary busywork to ‘earn’ their living.
Interviewer: Ok, so you think AI unemployment is different?
Katja: Yes, because it involves the disempowerment of humans in general in favor of non-people entities whose empowerment has a decent chance of spelling our ruin. Doing things the hard way to avoid that happening isn’t busywork, it’s very valuable. I don’t usually want to take sides between different humans systematically—society seems probably best served by letting the most effective production methods win out in most cases. But sometimes there are entities who produce things efficiently, and you still shouldn’t trade with them because it empowers them. It’s a lot like not trading with Nazis (broadly—I’m not saying AI entities are evil in the same way, just that their empowerment has a good chance of leading to genocide or omnicide).
Interviewer: Ok, but aren’t self-driving cars AI?
Katja: Yes, but the class of entities I don’t want to empower isn’t ‘AI’ really—it’s more like ‘AI agents’. Though also, the processes that are creating them, such as LLM companies, which complicates things. Self-driving cars are narrow and not much like entities that can be empowered. And my understanding is that we could have perfectly great self driving without using risky AI. But maybe I should be opposed—I’m not sure how to think about what class of entities I should not want empowered.
Interviewer: Are you just in love with self driving cars because they would be so personally convenient for you?
Katja: That is probably playing a role. A car with a random stranger in it is just so much less what I want most of the time than a car by myself. I like to imagine that Uber was invented like, “New startup idea: Chatroulette but you’re stuck in a moving vehicle with the person!” I would feel worse about the end of driving as a human profession if I felt like human drivers consistently did the job acceptably well. But the rate of drivers around here seeming chemically impaired or choosing to drive on the kerb of the freeway to get around other cars, etc, and also talking to me when I don’t want to talk, means there are a lot of cases where I would like to go somewhere in a car except it seems to awful so I don’t.
Interviewer: So how was the self driving car last night?
Katja: non-existent. Once I arrived at the airport, the Waymo app informed me that I wasn’t allowed to get cars there, because they are rolling it out slowly or something. I considered trying from one transit stop outside the airport, but since the available Waymo map was a very uninformative cartoon of the Bay Area and it was after midnight, that felt risky.
Interviewer: Did you give up?
Katja: Not immediately. I had also been told that people are taking ‘robotaxis’ all over, so I looked that up. I couldn’t immediately figure out what it was by Googling and looking on the Android app store, so I messaged some friends, and they directed me to an app called ‘RoboTaxi’ purportedly from Tesla but with barely readable and amateurish font and 57 reviews. As is often perplexingly the case with things of importance to a lot of people from very well known brands, I felt like I was exploring an obscure frontier that nobody had tried to use before. (You want to do what?? Get a ride in one of our cars?? And you want to do it through an app?? And you want to know where they are available??) I logged in and it told me the airport was also out of bounds. So then I gave up.
Interviewer: How did that make you feel?
Katja: ashamed
Interviewer: That makes sense. Why did you even so brazenly think you would probably be able to get a self-driving car from the airport?
Katja: well locally, because Waymo had a map indicating that the airport was within their zone, and I figured Tesla wouldn’t have such a can’t-do attitude. But more fundamentally, I guess I haven’t properly internalized how opposed airports are to efficient travel. Seeking a human-driven car after all this, I was reminded further because the location of the rideshare pickup at the airport and the signage indicating the location, both seem like they should probably be crimes.
Interviewer: Might it be an even broader problem with your level of techno-optimism? Weren’t you just the other day very disappointed by a futuristic kettle? Perhaps you need to learn that everything is shit?
Katja: maybe, but I don’t know, sometimes technology really changes things. I remember before Uber, when I just had to phone a person at a taxi company and ask them to come and collect me and then wait for an unknown period of time, and worst case give up and walk home.
Interviewer: How was your ride home last night?
Katja: Pretty good. The Lyft driver didn’t perceive my initial desire not to talk, and so we had a detailed discussion of Yemen, his life as an immigrant, his family, arranged marriage, romance in Islam, experiences running different businesses, the nature of business partnership, AI risk, and other drivers’ views on automation. We exchanged details so we could interact again.
Interviewer: Do you really think your quest to instead drive home in sterility wasn’t completely misguided?
Katja: Humans are great, but you have to be allowed to want solitude sometimes. It follows that you should probably be allowed to want solitude while also getting to another location. That said, I probably want solitude unhealthily much, and underrate the loss of human connection from these innovations. Maybe there should be a tax or something.
