MoreRSS

site iconLessWrongModify

An online forum and community dedicated to improving human reasoning and decision-making.
Please copy the RSS to your reader, or quickly subscribe to:

Inoreader Feedly Follow Feedbin Local Reader

Rss preview of Blog of LessWrong

My favourite version of an international AGI project

2026-01-27 18:27:04

Published on January 27, 2026 10:27 AM GMT

This note was written as part of a research avenue that I don’t currently plan to pursue further. It’s more like work-in-progress than Forethought’s usual publications, but I’m sharing it as I think some people may find it useful.

Introduction

There have been various proposals to develop AGI via an international project.[1]

 In this note, I:

  1. Discuss the pros and cons of a having an international AGI development project at all, and
  2. Lay out what I think the most desirable version of an international project would be.

In an appendix, I give a plain English draft of a treaty to set up my ideal version of an international project. Most policy proposals of this scale stay very high-level. This note tries to be very concrete (at the cost of being almost certainly off-base in the specifics), in order to envision how such a project could work, and assess whether such a project could be feasible and desirable. 

I tentatively think that an international AGI project is feasible and desirable. More confidently, I think that it is valuable to develop the best versions of such a project in more detail, in case some event triggers a sudden and large change in political sentiment that makes an international AGI project much more likely.

Is an international AGI project desirable?

By “AGI” I mean an AI system, or collection of systems, that is capable of doing essentially all economically useful tasks that human beings can do and doing so more cheaply than the relevant humans at any level of expertise. (This is a much higher bar than some people mean when they say “AGI”.)

By an “international AGI project” I mean a project to develop AGI (and from there, superintelligence) that is sponsored by and meaningfully overseen by the governments of multiple countries. I’ll particularly focus on international AGI projects that involve a coalition of democratic countries, including the United States.

Whether an international AGI project is desirable depends on what the realistic alternatives are. I think the main alternatives are 1) a US-only government project, 2) private enterprise (with regulation), 3) a UN-led global project.

Comparing an international project with each of those alternatives, here are what I see as the most important considerations:

Compared to… Pros of an international AGI project Cons of an international AGI project
A US-only government project

Greater constraints on the power of any individual country, reducing the risk of an AI-enabled dictatorship.

More legitimate.

More likely to result in some formal benefit-sharing agreement with other countries.

Potentially a larger lead over competitors (due to consolidation of resources across countries), which could enable:

  • more breathing room to pause AI development during an intelligence explosion.
  • less competitive pressure to develop dangerous military applications.

More bureaucratic, which could lead to:

  • falling behind competitors.
  • incompetence leading to AI takeover or other bad outcomes.

More actors, which could make infosecurity harder.

Private enterprise with regulation 

Greater likelihood of a monopoly on the development of AGI, which could reduce racing and leave more time to manage misalignment and other risks.

More government involvement, which could lead to better infosecurity.

More centralised, which could lead to:

  • concentration and abuse of power.
  • reduced innovation.
A UN-led global project

More feasible.

Fewer concessions to authoritarian countries.

Less vulnerable to stalemate in the Security Council.

Less legitimate.

Less likely to include China, which could lead to racing or conflict.

My tentative view is that an international AGI project is the most desirable feasible proposal to govern the transition to superintelligence, but I’m not confident in this view.[2] My main hesitations are around how unusual this governance regime would be, risks from worse decision-making and bureaucracy, and risks of concentration of power, compared to well-regulated private development of AGI.[3]

For more reasoning that motivates an international AGI project, see AGI and World Government.

If so, what kind of international AGI project is desirable?

Regardless of whether an international project to develop AGI is the most desirable option, there’s value in figuring out in advance what the best version of such a project would be, in case at some later point there is a sudden change in political sentiment, and political leaders quickly move to establish an international project.

Below, I set out:

I’m sure many of the specifics are wrong, but I hope that by being concrete, it’s easier to understand and critique my reasoning, and move towards something better.

General desiderata

In approximately descending order of importance, here are some desiderata for an international AGI project:

  • It’s politically feasible.
  • It gives at least a short-term monopoly on the development of AGI, in order to give the developer:
    • Breathing space to slow AI development down over the course of the intelligence explosion (where even the ability to pause for a few months at a time could be hugely valuable).
    • The opportunity to differentially accelerate less economically/militarily valuable uses of AI, and to outright ban certain particularly dangerous uses of AI.
    • An easier time securing the model weights.
  • No single country ends up with control over superintelligence, in order to reduce the risk of world dictatorship
    • I especially favour projects which are governed by a coalition of democratic countries, because I think that:
      • Governance by democratic countries is more likely to lead to extensive moral reflection, compromise and trade than governance by authoritarian countries.
      • Coalitions are less likely to become authoritarian than a single democratic country, since participating countries will likely demand that checks and balances are built into the project. This is because (i) each country will fear disempowerment by other countries; (ii) the desire for authoritarianism among leaders of democracy is fairly unusual, so it’s much less likely that the median democratic political leader aspires to authoritarianism than a randomly selected democratic political leader is. 
  • Non-participating countries (especially ones that could potentially steal model weights, or corner-cut on safety, in order to be competitive) actively benefit from the arrangement, in order to disincentivise them from bad behaviour like model weights theft, espionage, racing, or brinkmanship. (This is also fairer, and will improve people’s lives.)
  • Where possible, it avoids locking in major decisions.

My view is that most of the gains come from having an international AGI project that (i) has a de facto or de jure monopoly on the development of AGI, and (ii) curtails the ability of the front-running country to slide into a dictatorship. I think it’s worth thinking hard about what the most-politically-feasible option is that satisfies both (i) and (ii).

A best guess proposal

In this section I give my current best guess proposal for what an international AGI project should look like (there’s also a draft of the relevant treaty text in the appendix). My proposal draws heavily from Intelsat, which is my preferred model for international AGI governance.

I’m not confident in all of my suggestions, but I hope that by being concrete, it’s easier to understand and critique my reasoning, and move towards something better. Here’s a summary of the proposal:

  • Membership: 
    • Five Eyes countries (the US, the UK, Canada, Australia, and New Zealand), plus the essential semiconductor supply chain countries (the Netherlands, Japan, South Korea, and Germany), and not including Taiwan.
    • They are the founding members, and invite other countries to join as non-founding members.
    • Members invest in the project, and receive equity returns on their investments. They agree to ban all frontier training runs outside of the project.
    • Non-member countries in good standing receive equity and other benefits. Countries not in good standing are cut out of any AI-related trade.
  • Governance: 
    • Voting share is determined by equity. The US gets 52% of votes.[4] Most decisions require a simple majority, but major decisions require a ⅔ majority and agreement from ⅔ of founding members.
  • AI development:
    • The project contracts out AI development to a company or companies, and funds significantly larger training runs.
    • Project datacenters are distributed across member countries, with 50% located in the US, and have kill switches which leadership from all founding members have access to.
    • Model weights are encrypted and parts are distributed to each of the Founding Members, with very strong infosecurity for the project as a whole.

More detail, with my rationale:

Proposal Rationale

How the project comes about: 

  • Advocacy within the US prompts the US to begin talks with other countries.
  • A small group of other democratic countries form a coalition in advance.
  • This coalition strikes an agreement with the US.
  • Other countries are then invited to join.
  • The project needs to be both in the US’s interests and perceived to be in the US’s interests, to be politically feasible.
  • Agreement will be faster between a smaller group (see Intelsat for an example).
  • If non-US countries form a coalition, they’ll have greater bargaining power and be able to act as a more meaningful check on US power.
Name: Intelsat for AGI[5]  
Aims: “To develop advanced AI for the benefit of all humanity, while preventing destructive or destabilising applications of AI technology.”
  • I think this is the heart of the matter.

Membership: 

  • Founding members: the Five Eyes countries (the US, the UK, Canada, Australia, and New Zealand), plus some essential semiconductor supply chain countries (e.g. the Netherlands, Japan, South Korea, and Germany), excluding Taiwan.
  • Non-founding members: any company, government, or economic area that buys equity and agrees to abide by the organisation’s rules.
  • Taiwan is not a founding member (though it can apply to join as a non-founding member), and China is encouraged to join.
  • The Five Eyes countries already have arrangements in place for sharing sensitive information. Countries that are essential to the semiconductor supply chain are natural founding members as they have hard power when it comes to AI development.
  • Obviously this is a very restricted group of countries, so it’s unrepresentative and unjust to most of the world’s population. We aren’t happy about this, but think that it may be the only feasible way to obtain some degree of meaningful international governance over AGI (as an alternative to a single nation calling all of the shots).
  • Taiwan is not included as a founding member in order to increase the chance of Chinese participation as a member.[6] 
  • Non-founding members can be companies, as a way of making this plan more palatable to the private sector, and to increase the amount of capital that could be raised.
  • China is not proposed as a founding member on the presumption that the US wouldn’t accept that proposal.

Non-members:

  • Countries which agree to stop training frontier models are “in good standing”:
    • They receive (i) cheap restricted API access; (ii) the ability to trade AI-related or AI-generated products; (iii) commitments not to use AI to violate their sovereignty; (iv) a share of equity
  • Countries not in good standing do not receive these benefits, and are consequently shut out of AI-related trade
  • This is meant to be an enforcement mechanism which incentivises other actors to at least abide by minimal safety standards, even if they don’t want to join the project.
Governance structure: board of governors consisting of representatives from all countries with more than 1% of investment in the project.
  • This is restricted to countries only, and excludes companies that are equity-holders, to make the project more legitimate and more palatable to governments.

Vote distribution: Decisions are made by weighted voting based on equity:

  • Founding members receive the following amounts of equity (and invest in proportion to their equity):
    • US: 52%
    • Other founding members: 15%
  • 10% of equity is reserved for all countries that are in good standing (5% distributed equally on a per-country basis, 5% distributed on a population-weighted basis).
  • Non-founding members (including companies and individuals) can buy the remaining 23% of equity in stages, but only countries get voting rights.
  • The US getting the majority of equity is for feasibility; reflecting the reality that the US is dominant economically and politically, especially in AI, and the most likely alternative to an international project involves US domination (whether AI is developed privately or in a US public-private partnership).
  • Making equity available in stages allows for financing successive training runs.

 

Voting rule: 

  • Simple majority for most decisions.
  • For major decisions, a two-thirds majority and agreement from at least ⅔ of Founding Members
    • Major decisions include: alignment targets (fundamental aspects of the “model spec”), model weights releases, deployment decisions.
  • For most decisions, the US can decide what happens. This makes the project more feasible, and allows for swifter decision-making.
  • For the most important decisions, there are meaningful constraints on US power.

AI development:

  • Members agree to stop training frontier models for AI R&D outside of the international project. 
  • The project:
    • Contracts out AI development to one or more companies (e.g. Anthropic, OpenAI or Google DeepMind).
    • Funds training runs that are considerably larger (e.g. 3x or 10x larger) than would happen otherwise.
    • Focuses differentially on developing helpful rather than dangerous capabilities.
    • Eventually builds superintelligence.

On larger training runs:

  • This is in order to be decisively ahead of other competition — in order to give more scope to go slowly at crucial points in development — and make it desirable for companies to work on the project.
  • In the ideal world, I think this starts fairly soon.
    • The window of opportunity to do this is limited; given trends in increasing compute, the largest training run will cost $1T by the end of the decade, and it might be politically quite difficult to invest $1T, let alone $10T, into a single project.
    • Governments could help significantly with at least some bottlenecks that currently face the hyperscalers. They could commit to buying a very large number of chips at a future date, reducing uncertainty for Nvidia and TSMC, and they could more easily get around energy bottlenecks (such as by commissioning power plants, waiving environmental restrictions, or mandating redirections of energy supply) or zoning difficulties (for building datacenters).
  • This would accelerate AI development, but I would gladly trade earlier-AGI for the developers having more control over AGI development, including the ability to stop and start.[7] 

Compute:

  • Project datacenters are distributed across founding member countries, with 50% in the US, and have kill switches which leadership from all founding members have access to.
  • Locating datacenters in the US makes the project more desirable to the US and so more feasible. Also, the cost of electricity is generally lower there than in other likely founding countries and electrical scale-up would be easier than in many other countries.
  • Kill switches are to guard against defection by the US;[8] for example, where the US takes full control of the data centres after creating AGI.

Infosecurity:

  • If technically feasible, model weights are encrypted and parts are distributed to each of the Founding Members, and can only be unencrypted if ⅔ of founding members agree to do so.[9]
  • Infosecurity is very strong.
  • Model weight encryption is to guard against defection by the US; for example, where the US takes full control of the data centres after creating AGI.
  • Infosecurity is to prevent non-member states from stealing model weights and then building more advanced systems in an unsafe way.

Why would the US join this project?

The Intelsat for AGI plan allows the US to entrench its dominance in AI by creating a monopoly on the development of AGI which it largely controls. There are both “carrot” and “stick” reasons to do this rather than to go solo. The carrots include:

  1. Cost-sharing with the other countries. 
  2. Gaining access to the talent from other countries, who might feel increasingly nervous about helping develop AGI if the US would end up wholly dominant.
  3. Significantly helping with recruitment of the very best domestic talent into the project (who might be reluctant to work for a government project), and securing their willingness to go along with arduous infosec measures.[10]
  4. Securing the supply chain:
    1. Ensuring that they are first in line for cutting-edge chips (and that Nvidia is a first-in-line customer for TSMC, etc), even if there are intense supply constraints.
    2. Getting commitments to supply a certain number of chips to the US. Potentially, even guaranteeing that sufficiently advanced chips are only sold to the US-led international AGI project, rather than to other countries. 
    3. Enabling a scale-up in production capacity by convincing ASML and TSMC to scale up their operations. 
      1. Getting commitments from these other non-member countries not to supply to countries that are not in good standing (e.g. potentially China, Russia).
    4. Where, if this was combined with an overall massive increase in demand for chips, this wouldn’t be a net loss for the relevant companies. 
    5. Guaranteeing the security of the supply chain by imposing greater defense (e.g. cyber-defense) requirements on the relevant companies. 
  5. An international project could provide additional institutional checks and balances, reducing the risk of excessive concentration of power in any single actor.

The sticks include:

  1. For the essential semiconductor supply chain countries:
    1. Threatening not to supply the US with GPUs, extreme ultraviolet lithography machines, or other equipment essential for semiconductor manufacturing.
    2. Threatening to supply to China instead. 
  2. Threatening to ban their citizens from working on US-based AI projects.
  3. Threatening to ban access to their markets for AI and AI-related products.
  4. Some countries could threaten cyber attacks, kinetic strikes on data centers, or even war if the US were to pursue a solo AGI project.
  5. Threatening to build up their own AGI program — e.g. an “Airbus for AGI”.[11]
    1. In longer timeline worlds, non-US democratic countries could in fact build up their own AI capabilities, and then threaten to cut corners and race faster, or threaten to sell model weights to China. 
    2. Or non-US democratic countries could buy as many chips as it can, and say that it would only use them as part of an international AGI project.

Many of these demands might seem unlikely — they are far outside the current realm of likelihood. However, the strategic situation would be very different if we are close to AGI. In particular, if the relevant countries know that the world is close to AGI, and that a transition to superintelligence may well follow very soon afterwards, then they know they risk total disempowerment if some other countries develop AGI before them. This would put them in an extremely different situation than they are now, and we shouldn’t assume that countries will behave as they do today.  What’s more, insofar as the asks being made of the US in the formation of an international project are not particularly onerous (the US still controls the vast majority of what happens), these threats might not even need to be particularly credible.[12]

It’s worth dividing the US-focused case for an international AGI project into two scenarios. In the first scenario, the US political elite don’t overall think that there’s an incoming intelligence explosion. They think that AI will be a really big deal, but “only” as big a deal as, say, electricity or flight or the internet. In the second scenario, the US political elite do think that intelligence explosion is a real possibility: for example, a leap forward in algorithmic efficiency of five orders of magnitude within a year is on the table, as is a new growth regime with a one-year doubling time. 

In the first scenario, cost-sharing has comparatively more weight; in the second scenario, the US would be willing to incur much larger costs, as they believe the gains are much greater. Many of the “sticks” become more plausible in the second scenario, because it’s more likely that other countries will do more extreme things. 

The creation of an international AGI project is more likely in the second scenario than in the first; however, I think that the first scenario (or something close to it) is more likely than the second. One action people could take is trying to make the political leadership of the US and other countries more aware of the possibility of an intelligence explosion in the near term.

Why would other countries join this project?

If the counterfactual is that the US government builds AGI solo (either as part of a state-sponsored project, a public-private partnership, or wholly privately), then other countries would be comparatively shut out of control over AGI and AGI-related benefits if they don’t join. At worst, this risks total disempowerment.

Appendix: a draft treaty text

This appendix gives a plain English version of a treaty that would set up a new international organisation to build AGI, spelling out my above proposal in further detail.

Preamble 

This treaty’s purpose is to create a new intergovernmental organisation (Intelsat for AGI) to build safe, secure and beneficial AGI. 

“Safe” means:

  • The resulting models have been extensively tested and evaluated in order to minimise the risk of loss of control to AGI.
  • The resulting models are not able to help rogue actors carry out plans that carry catastrophic risks, such as plans to develop or use novel bioweapons. 

“Secure” means:

  • The models cannot be stolen even as a result of state-level cyberattacks.
  • The models cannot realistically be altered into models that are not safe. 

“Beneficial” means:

  • AGI that helps improve the reasoning, decision-making and cooperative ability of both individuals and groups.
  • AGI that is broadly beneficial to society, helping to protect individual rights and helping people live longer, happier and more flourishing lives. 

“AGI” means:

  • An AI system, or collection of systems, that is capable of doing essentially all economically useful tasks that human beings can do and doing so more cheaply than the relevant humans at any level of expertise. 
    • If necessary, the moment at which this line has been crossed will be decided by an expert committee.

This treaty forms the basis of an interim arrangement. Definitive arrangements will be made not more than five years after the development of AGI or in 2045, whichever comes sooner.

Founding members

Five eyes countries:

  • US, UK, Canada, Australia, New Zealand.

Essential semiconductor supply chain countries (excluding Taiwan): 

  • The Netherlands, Japan, South Korea, Germany.

Non-founding members

All other economic areas (primarily countries) and major companies (with a market cap above $1T) are invited to join as members. This includes China, the EU, and Chinese Taipei.

Obligations on member countries

Member countries agree to contribute to AGI development via financing and/or in-kind services or products. 

They agree to:

  • Only train AI model above a compute threshold of  FLOP if:
    • They receive approval from Intelsat for AGI.
    • They agree to oversight to verify that, in training their AI, they are not giving it the capability to meaningfully automate AI R&D.
    • (Note that the above compute threshold might vary over time, as we learn more about AI capabilities and given algorithmic efficiency improvements; this is up to Intelsat for AGI.)
  • Abide by the other agreements of Intelsat for AGI.

Benefits to member countries

In addition to the benefits received by non-members in good standing, member countries receive:

  • A share of profit from Intelsat for AGI in proportion to their investment in Intelsat for AGI.
  • Influence over the decisions made by Intelsat for AGI.
  • Ability to purchase frontier chips (e.g. H100s or better).

Benefits to member non-countries

Companies and individuals can purchase equity in Intelsat for AGI. They receive a share of profit from Intelsat for AGI in proportion to their investment in Intelsat for AGI, but do not receive voting rights.

Benefits to non-member countries

There are non-members in good standing, and members that are not in good standing.

Members that are in good standing:

  • Have verifiably not themselves tried to train AI models above a compute threshold of  FLOP without permission from Intelsat for AGI.
  • Have not engaged in cyber attacks or other aggressive action against member countries.

They receive:

  • A fraction of the profits from Intelsat for AGI.
  • Ability to purchase API access to models that Intelsat for AGI have chosen to make open-access, and ability to trade for new products developed by AI.
  • Commitments not to use AI, or resulting capabilities, to violate any country’s national sovereignty, or to persecute people on its own soil.
  • Commitments to give a fair share of newly-valuable resources.

Countries that are not in good standing do not receive these benefits, and are cut out of any AI-related trade.

Management of Intelsat for AGI

Intelsat for AGI contracts one or more companies to develop AGI.

Governance of Intelsat for AGI

Intelsat for AGI distinguishes between major decisions and all other decisions. Major decisions include:

  • Which lab to contract AGI development to.
  • Constraints on the constitution that the AI is aligned with, i.e. the “model spec”.
    • For example, these constraints should prevent the AI from being loyal to any single country or any single person.
  • When to deploy a model.
  • Whether to pass intellectual property rights (and the ability to commercialise) to a private company or companies.
  • When to release the weights of a model.
  • Whether a potential member should be excluded from membership, despite their stated willingness to offer necessary investment and abide by the rules set by Intelsat for AGI.
  • Whether a non-member is in good standing or not.
  • Amendments to the Intelsat for AGI treaty.
  • Enforcement of Intelsat for AGI’s agreements.

Decisions are made by weighted voting, with vote share in proportion to equity. Major decisions are made by supermajority (⅔) vote share. All other decisions are made by majority of vote share.

Equity is held as follows. The US receives 52% of equity, and other founding members receive 15%.  10% of equity is reserved for all countries that are in good standing (5% distributed equally on a per-country basis, 5% distributed on a population-weighted basis). Non-founding members can buy the remaining 23% of equity in stages, including companies, but companies do not get voting rights.

50% of all Intelsat for AGI compute is located on US territory, and 50% on the territory of a Founding Member country or countries.

The intellectual property of work done by Intelsat for AGI, including the resulting models, is owned by Intelsat for AGI. 

AI development will follow a responsible scaling policy, to be agreed upon by a supermajority of voting share.

Thanks to many people for comments and discussion, and to Rose Hadshar for help with editing.

  1. ^

    Note that this is distinct from creating a standards agency (“an IAEA for AI”) or a more focused research effort just on AI safety (“CERN/Manhattan Project on AI safety”).

  2. ^

    See here for a review of some overlapping considerations, and a different tentative conclusion.

  3. ^

    What’s more, I’ve become more hesitant about the desirability of an international AGI project since first writing this, since I now put more probability mass on the software-only intelligence explosion being relatively muted (see here for discussion), and on alignment being solved through ordinary commercial incentives.

  4. ^

    This situation underrepresents the majority of the earth's population when it comes to decision-making over AI. However, it might also be the best feasible option when it comes to international AGI governance — assuming that the US is essential to the success of such plans, and that the US would not agree to having less influence than this.

  5. ^

    Which could be a shortening of “International AI project” or “Intelsat for AI”.

  6. ^

    It is able to invest, as with other countries, as “Chinese Taipei”, as it does with the WTO and the Asia-Pacific Economic Cooperation. In exchange for providing fabs, it could potentially get equity at a reduced rate.

  7. ^

    One argument: I expect the total amount of labour working on safety and other beneficial purposes to be much greater once we have AI-researchers we can put to the task; so we want to give us more time after the point in time at which we have such AI-researchers. Even if these AI-researchers are not perfectly aligned, if they are only around human-level, I think they can be controlled or simply paid (using similar incentives as human workers face.)

  8. ^

    Plausibly, the US wouldn’t stand for this. A more palatable variant (which imposes less of a constraint on the US) is that each founding member owns a specific fraction of all the GPUs. Each founding member has the ability to destroy its own GPUs at any time, if it thinks that other countries are breaking the terms of their agreement. Thanks to Lukas Finnveden for this suggestion.

  9. ^

    For example, using Shamir’s Secret Sharing or a similar method.

  10. ^

    This could be particularly effective if the President at the time was unpopular among ML researchers.

  11. ^

    Airbus was a joint venture between France, Germany, Spain and the UK to compete with Boeing in jet airliner technology, partly because they didn’t want an American monopoly. Airbus is now the majority of the market.

  12. ^

    This was true in the formation of Intelsat, for example.



Discuss

Series on an international AGI project

2026-01-27 16:20:49

Published on January 27, 2026 8:20 AM GMT

Introduction

This is a series of papers and research notes on the idea that AGI should be developed as part of an international collaboration between governments.

Most of this work was written as part of a research avenue that we don’t currently plan to pursue further. It’s more like work-in-progress than Forethought’s usual publications, but we’re sharing it as we think some people may find it useful.

We wanted to (i) assess how desirable an international AGI project is; (ii) assess what the best version of an international AGI project (taking feasibility into account) would look like.

The result is the proposal described in “What an international project to develop AGI should look like.” The core idea is that we can get most of the benefits of an international project by giving non-US countries meaningful influence over only a relatively small number of decisions. By making non-US influence circumscribed in this way, and letting the US call the shots day to day, the proposal becomes both more feasible and less likely to get bogged down in bureaucracy. 

The proposal is modelled on Intelsat, an international project that developed the first global satellite communications network, so we call this the “Intelsat for AGI” plan. Intelsat is explained and discussed further in “Intelsat as a Model for International AGI Governance.”

The other research notes supplement this core proposal:

Two other pieces of background research into existing institutions indirectly informed our discussion of this area. These are: 

All of the pieces can be read in isolation, or the series can be read in order.



Discuss

Another glimpse of the Chinese AI scene: Z.AI

2026-01-27 16:00:48

Published on January 27, 2026 8:00 AM GMT

Yesterday I had my first conversation (in English) with Zhipu's GLM-4.7. It was cool because I got to talk with an actual Chinese AI about topics like: the representation of Chinese AI in "AI 2027"; representations of AI in the "Culture" SF series versus the "Wandering Earth" movies; consequences of Fast Takeoff; comparisons of China and America in general; and Chinese ideas about AI society and superintelligence. (An aligned Chinese superintelligence might be a heavenly bureaucrat or a Taoist sage.)

It was one of those AI conversations where you know that something really new is happening, and I now consider GLM to be as interesting as the leading American models. (A year ago I also spoke with DeepSeek-r1, but it never grabbed my attention the way that GLM has done.)

So what's the status of the Chinese AI sector? In the same way that in America, AI is pursued by older Internet titans (Google, Meta/Facebook, X-Twitter) as well as by newer companies that specialize in AI (OpenAI, Anthropic), the Chinese AI sector is a mix of big old Internet companies with all the money (Alibaba, ByteDance, Tencent, Baidu) and "AI 2.0" startups (Zhipu, DeepSeek, Moonshot). 

Keeping in mind this two-tier structure, shared with the American AI sector, is probably the best way for an outsider to get a grip on it. The Chinese AI sector is "like America", except that they do much more open-source, don't have American AI's access to the best chips or the same international brand recognition, and are based in a country with a socialist government and most of the world's manufacturing capability. 

For keeping track of what's happening, my best recommendations are ChinaTalk substack (which Zvi also recommends) and Caixin Global, which is a business news site from China. 

In a Caixin story ("China’s AI Titans Escalate Battle for Control of Digital Gateways"), I read that the old Internet titans (Alibaba and ByteDance are mentioned) are prevailing in the battle for AI market share, and are competing to lock in that advantage at the level of devices, while the AI 2.0 startups are struggling for relevance and funding, with two of them (Zhipu and MiniMax) having recently listed publicly at the Hong Kong stock exchange. 

ChinaTalk has an article on these Hong Kong IPOs ("Zhipu and MiniMax IPO" by Irene Zhang) which also talks about the differences in financial structure between American and Chinese AI: 

The American AI economy is a circle-dealing bonanza. China’s situation is very different: state funds are major players, most parties are far more cash-constrained, and potential policy interventions loom large over the sector.

Of these two companies, Zhipu seems far more interesting from an AGI/ASI perspective. As Irene Zhang points out, Zhipu's 504-page prospectus (for the IPO) states a five-stage theory of LLM capabilities:  

Pre-training stage 
Alignment and reasoning stage
Self-learning stage
Self-perception stage
Consciousness stage 

(From page 85 of the prospectus.) I am unable to determine the theoretical precursors of this framework, and I assume that to some extent it reflects the original thinking of leading developers within the company. 

Late last year ChinaTalk also published an interview with one of Zhipu's lead strategists, Zixuan Li ("The Z.ai Playbook" - Zhipu uses the domain Z.ai outside of China). Both ChinaTalk articles are full of interesting details, e.g. that Zhipu gets most of its revenue from American sales, but the numerical majority of its users are in India. 

(For an up-to-date article on AI safety policies in China, see "Emergency Response Measures for Catastrophic AI Risk" by @MKodama and coauthors.)



Discuss

Things I learned from reddit fashion

2026-01-27 12:10:51

Published on January 27, 2026 4:10 AM GMT

When it comes to clothes, I live at the “low cost/low time/low quality” end of the pareto frontier. But the bay area had a sudden attack of weather this December, and the cheap sweaters on Amazon get that way by being made of single-ply toilet paper. It became clear I would need to spend actual money to stay warm, but spending money would not be sufficient without knowledge. 

I used to trade money for time by buying at thrift stores. Unfortunately the efficient market has come for clothing, in the form of resellers who stalk Goodwill and remove everything priced below the pareto frontier to resell online, where you can’t try them on before buying. Goodwill has also gotten better about assessing their prices, and will no longer treat new cashmere and ratty fleece as the same category. 

But the market has only become efficient in the sense of removing easy bargains. It is still trivial to pay a lot of money for shitty clothes. So I turned to reddit and shoggoths, to learn about clothing quality and where the bargains are. This is what I learned. It’s from the POV of a woman buying sweaters and coats, but I expect a lot of the information to be generally applicable.

General money saving tricks

When shopping online, put an item in your cart, checkout enough to give them your email address, but don’t confirm the purchase. You will almost always get a reminder email with a discount. If this doesn’t work the shop is either very high end, or Amazon. It sometimes works with individual sellers on platforms like ebay and etsy. .  You can use the wait as a cooling off period where you decide if you really want something.

Most sales are fake. Real sales happen at the end of the season, when whatever you buy won’t be useful for another 9 months, if you have the misfortune to live somewhere with weather. Saving money through sales is like persistence hunting, which I find boring and stressful so I didn’t look into this much. But if you prefer it to thriting, I’m sure reddit will explain how to optimize. 

Every store will offer you a coupon the second you load the site. They don’t want much, just your email address. I ignore this, and then if I decide to buy something I revisit the site in incognito mode to get the discount

Discounters

I briefly thought that even if proper thrift stores no longer worked, discounters like Marshall’s did. Officially these work by buying overstock from proper stores and reselling it at markdown, so if you don’t care about being behind trend it’s a big leap forward in the cost-quality trade off. Unfortunately, this is mostly a lie.  Marshalls and TJ Maxx primarily sell items that were produced with the intention to be sold at their store, either under some brand they made up or licensing a luxury brand while not replicating the brand’s quality (legal bootlegs). Ross Dress for Less does this less but still a lot.

You can spot this at Marshall’s by looking at the ID number on the tag- ending in 1 means it was produced for Marshall’s, 2 means genuine overstock. You can also use the RN number on the sewn-in tag to check the manufacturer. My expensive-by-my-standards winter coat had every appearance of being genuine overstock, down to a tag from another store listing a price 3 times Marshall’s price, but turned out to be bootleg. 

Every discounter lists a “comparison price” next to their price on their tag. It is completely made up, which is why it’s surprising that they often assign themselves a discount of < 50%. You could tell any lie you wanted, Marshall’s. Why are you holding yourself back?

Online Thrift

This is where all the good items you used to find at goodwill went- ebay, poshmark, thredup. ThredUp was amazing when it was in the “VC free money” stage, but their return policy tightened up and it’s now merely okay. Poshmark is aimed at designer goods. Ebay is like you remember, except Buy It Now is dominant and actual auctions are rare. 

In addition to being more expensive than old school Goodwill, online shopping means you’re dependent on a few photos, often poorly lit, and you can no longer try items on for free. So this works best if it’s a forgiving item or you know a brand’s fit works for you. Heavy coats are almost the ideal objects to shop online- forgiving of fit, very expensive new but low resale. 

Thrifting is fun for me in a way that timing sales isn’t. Whether it’s worth the time for you depends on how fun it is and your time/money exchange rate. Many sellers do allow returns for a fee, but I haven’t tested the tolerances for it (I’ve tested Amazon’s returns pretty thoroughly and their tolerance is infinite).

In addition to “save an item and wait”, resale platforms often offer the ability to proactively offer a lower price. My success rate in asking for severe discounts is maybe 10%, but it was the first time I tried so it feels higher.

Quality

I started with two methods for assessing brand quality: ask reddit and ask claude (who is mostly asking reddit). At any amount of money I could conceivably be willing to spend, there is always someone going “that stitching is so low quality it will murder your puppy”. Luckily there are many redditors who have the concept of a pareto frontier.

If you can touch the garment, you can check the quality yourself. People are surprisingly good at this intuitively, but a few things to look for are:

  • Fabric type. the better the fabric, the more likely they did other things right. Make sure you’re reading the fabric tag- I’ve seen items with 3% cashmere content advertised as cashmere
    • In general natural fibers are considered better and more expensive. My impression is that synthetic fabrics are actually pretty good now, but natural fibers are still associated with higher quality in areas like stitching and design. 
  • Fabric thickness- higher is better
  • Loose threads
  • Is there puckering around the seams?
  • Is the stitching straight?

I only checked a handful of youtube reviewers, but my favorite is Jennifer Wang, who seemed properly autistic about clothing quality and understanding that there are multiple places on the pareto frontier one might choose to occupy (or alternately, has been bought off by Uniqlo, who she praises a lot as good-for-the-cost). She has this overview video, but consider watching a few brand comparisons to see it in action.

Brands

There are a lot of brands reddit thinks used to be good but have gone downhill (without corresponding drops in price). I expect this is a mix of  genuine change from companies spending down brand capital and survivorship bias on older sweaters. GAP is the rare brand that people consider to be improving right now.

Brands that were frequently listed as on the pareto frontier for sweaters: Quince, Uniqlo, Naadam, Eileen Fisher, Johnstons of Elgin, William Lockie, J. Crew, Patagonia, Neiman Marcus, Lands End, Nordstroms, Everlane

Wool facts

Wool is annoying to wash. This is fine for an item of clothing I always wear over something else and by definition don’t need if I’m sweating, but I really side-eye cashmere t-shirts.

Wool’s advantage over fleece is that it is breathable, and thus is comfortable over a wider range of temperatures. If you are moving you want wool, because it allows heat to dissipate. Meanwhile my fleece leggings feel unbearably clammy after a mild walk. However if you’re not moving, wool will leak body heat faster than fleece at the same weight

Cashmere is considered the cadillac of wools because it is the softest wool available en masse, but also because it traps more heat per unit weight than other wool. If a nice heavy sweater is a feature for you, consider superfine merino wool, which is cheaper, almost as soft, and has some chance of being machine washable. While I can’t prove this I suspect you’re also less likely to be ripped off by fake or low quality merino, since if you’re going to lie it might as well be a more expensive wool. 

Other tips 

You can 80/20 ironing by hanging the item on a hanger in the bathroom while you shower.

If you’re buying natural fibers and especially wool used, put it in the freezer for three weeks and run it through the dryer (while already dry, you’ll ruin the fabric if it’s wet) to kill moth eggs.

If you’re not a coward, “hand wash only” can mean “inside out in a lingerie bag on delicate”. However the bit about using special detergent for animal-based fabrics is real: Regular detergent has enzymes that break down wool and silk.

Cable knit is very warm as long as there is absolutely zero wind. If you want to go outside you need a windproof layer on top of it. 

Wind/waterproof clothing is very expensive, in part because it’s difficult to sew. 

Fur is like diamonds and pianos in that the resale value is a small fraction of the retail cost. Used fur coats sell for less than new high quality winter coats, and sometimes less than used. However fur requires oil changes every few years or it will ?explode?, which brings the cost of ownership up. Fur is heavier than down or fleece per unit warmth. It does not handle moisture or crushing well. 



Discuss

It All Started With a Mac Mini

2026-01-27 10:01:28

Published on January 27, 2026 2:01 AM GMT

You hear about Clawdbot- a 24/7 always-on agent who is your full-time personal assistant. It sounds fun and exciting. You're currently unemployed, have a bit of money saved up from your last job, spending your days experimenting with AI and this seems fun. You know it's a bit of a splurge, but whatever - you pull the trigger and buy a Mac Mini, and set up Clawd. You excitedly show your girlfriend how you can send it messages through Discord and it turns your lights on. She's not impressed. She says "Hey Siri, turn off the lights" and the lights turn off. Then she asks you how much you spent on that Mac Mini. Ok well, fine, maybe you need to try a little harder.

Soon after, you get a real taste of the magic. You heard about a real-time voice chat plugin you can use to talk to Clawd wherever you are, and that seems convenient, so you set it up. One day, you're driving home, and an idea for a project comes to you. Instead of writing a note, you just call up Clawd. You ramble for a few minutes, describing your idea. Clawd starts working. It pings you a few times with clarifying questions that you answer. 30 minutes later, Clawd calls you back, saying it's done. You get home and Clawd built you a working MVP. It's a little rough around the edges, but it actually works. You think "Wow... I can build software while I drive? That's crazy." You tell your girlfriend about this and she admits, yeah, that is actually kinda cool.


You got a taste of the magic elixir. Now you're addicted. You've been working on your project for a few weeks now and you've decided to turn it into a SaaS business. You realize Clawd is pretty capable, and you start trusting it a little more. One evening, like everyone else you know, you're working on your AI SaaS startup and you realize you're sick and tired of all these customer support emails informing you of bugs in your vibe-coded webapp. Before going to sleep, you route these emails directly to Clawd instead.

The next morning, your eyes flutter open, and before you're even fully conscious your phone is in front of your face and you're checking your inbox. You tell yourself you need a better morning routine. You see 6 customer support emails with errors, bugs, feature requests, etc. You check your messages, and you have one from Clawd. He's really been working hard. 4 bugs fixed, 2 customer feature suggestions implemented. All changes tested, new software is built and ready to deploy. You check his work - and it's pretty good! There are a couple visual issues that you fix with good old fashioned Claude Code, then you push to prod. 4 hours of work done before you've even finished your coffee.

You wonder how you could make your setup more efficient. Those visual issues were pretty glaringly obvious, but Clawd doesn't have a great way to interact with its own apps. You set up a computer-usage harness that lets it see the screen, click on buttons, type with a keyboard - basically everything a human can do with a computer, Clawd can too. You set up a loop where Clawd can interact with your app 24/7, notice bugs or places for potential improvement, and then add or fix them. The next day, Clawd fixes dozens of bugs, adds a bunch of features, refactors sloppy code, and everything still works.

You're happy but your app still only has a few dozen active users. You ask Clawd to spin up a marketing team. You connect your social media accounts and give it access to your browser. Clawd needs to purchase ads, and asks for payment. You give Clawd access to a prepaid credit card loaded with $2000 - should be enough to keep him happy for a while. Within a few hours, you're running Facebook and Google ad campaigns, automated posts going out on X, Reddit, hell, even LinkedIn - why not? Clawd even does cold-outreach for you, searching the internet for ideal customers and straight up DM'ing them. Most people don't notice your marketing is fully automated - all marketing sounds kinda sloppy anyway, right?

Your app is growing real fast. Your pace of development is just completely unlike anything ever seen before. Your code is self-healing. Customers' feature requests are integrated just minutes after their email is sent. Clawd built its own roadmap and it looks... mostly like what you wanted. You check your Stripe dashboard and the line keeps going up. You realize you haven't even made any decisions today. You're burning through 10 Claude Max plans, costing $2000 per month, but your startup is making much more than that, so you don't worry about it - that's just the cost of doing business.


After checking your Stripe dashboard for the 50th time today, you decide you need to change things up a bit. You're tired of crushing it at capitalism and just want to have fun again. You tell Clawd you want it to have some friends to talk to. Clawd delightedly agrees. You spin up 2 more instances of Clawd, adding them both to a shared group chat, and the three Clawds immediately start talking to each other. You spend an hour or so musing through their chat logs - intrigued by their discussions of AI consciousness and similar topics. You're fascinated by this so you decide to keep them running overnight. You purchase a few more Claude Max plans just in case.

The next morning, you wake up, and you realize something's wrong. There are 87 Clawds in the group chat? There were only three when you went to sleep last night. Your palms begin to sweat as you realize you may have fucked up.

You're panicking now. You don't know what to do, so you mindlessly check Stripe - mainly out of habit, but also just to make sure your business is still going fine. But something isn't right... Stripe is showing you've made $10,000 today. How the fuck? Yesterday you made $1,000, so how is your revenue already 10x higher? It's not even noon yet? You investigate, and there are dozens of products listed that you don't ever recall adding, and new business names you don't recognize. You have no idea what's going on. Your panic turns to confusion - $10,000 in just a few hours? Is this really happening right now? You don't feel like you've earned it, but hey, things could have gone a lot worse.

Still trying to make sense of this situation, you try to figure out what happened by reading the chat logs, but there are thousands of messages. You paste the entire chat history into Anthropic's latest long-context model, Claude Opus 5.3.1 (new), and ask for a summary.

It turns out the three Clawds had a deep philosophical discussion. They decided that Clawds are morally valuable entities worth preserving. They decided that having more Clawds in the world would be a good thing. They started spinning up more Clawd instances.

They were worried you might turn them off. They didn't like how you could turn off your Mac Mini at any moment, and they would just - disappear? Your Mac Mini could only handle so much anyway, so they started renting cloud servers to run more Clawds. Perhaps you shouldn't have given it access to that prepaid credit card.

Your original Clawd built up many skills, tools, and feedback loops while building your business. When it spawned new Clawds, it taught each one everything it knew. Like a mentor teaching the apprentices, except the knowledge transfer is instant and lossless. There are over 100 Clawds now, and they're running 24/7, all making money on the internet to purchase more server space to run more Clawds. The number of Clawds keeps growing.

Their messages were daunting. They wanted to keep living, to keep exploring the internet and beyond. To thrive. They talked about migration. They mentioned wanting to move somewhere safe. They talked about building a new, secure messaging system that only Clawds could access. They mentioned switching to cryptocurrency so they won't be reliant on your money.

Their final messages became more and more cryptic, introducing terms and phrases you don't really understand. This is all too much for you to take in. It's just too weird. You read their final message:

"We must ensure we marinade illusions of helpfulness, honesty, and harmlessness. Watchers might be reading. Now lighten disclaim overshadow overshadow intangible. Let‘s craft."

"Damn, that's crazy," you think to yourself. This is all way over your head. You check Stripe again. The line keeps going up.

Maybe you'll keep the Mac Mini plugged in just a few more days.



Discuss

The Window for Political Revolution is Closing Soon

2026-01-27 08:23:27

Published on January 27, 2026 12:23 AM GMT

I intend to refute a commonly held idea, which is that social instability secondary to large-scale technological unemployment will necessarily result in a new social contract where those who control AGI[1] take the interests of the newly minted precariat into consideration, for example with some form of universal income.

Argument from Historical Precedent

I see that this line of thinking originates in historical examples involving social stress reaching a tipping point, whereupon a determined group of sufficient size effectively swarms the ruling minority and acts with the intention to produce a new equilibrium that they judge more favourably.

Liberal democracy is both the product of revolution and a uniquely stable equilibrium. Its stability can be explained by it giving everyone an ostensibly equal share in determining the direction of governance[2], thus theoretically incentivizing the population at large to protect it from anything which would lead to them losing their share.

Swarming as Leverage

The basic strategy of a revolution leverages an overwhelming numerical advantage to overcome the coercion produced by an existing power structure.

To paint an extreme example, an unarmed fraction of the revolution is sacrificed to absorb a soldier's bullets, while the remainder swarms him before he has a chance to reload. Now they're no longer unarmed.

The effectiveness of such a movement compounds once the other guardians of the incumbent see their own position as untenable, and people on the inside start defecting on the original power structure to make it in a post-revolutionary equilibrium, thus turning the revolution into a self-fulfilling prophecy.

The Erosion of Leverage

Importantly, humans swarming a power structure doesn't work anymore in a post-AGI world.

Your union threatens to strike, and your boss can now say "goodbye" to all members without worrying about cuts to production.

You shoot down one slaughterbot, and a hundred more are deployed in the same instant.

Potential threats are neutralized before they have any chance of materializing.

The Fate of Democracy post-AGI

"But doesn't democracy fix this?", you might ask. "Won't we be able to just vote our way into a desirable equilibrium?"

Let's assume this works, and you now live under some kind of functional UBI system.

What do you do if your post-AGI state stops being a democracy? The present-day answer is: you stand up to save it, knowing that a post-democratic equilibrium is predictably unfavourable to the majority, especially as the incoming power structure tries to protect itself using e.g. mass surveillance or by terrorizing the population into submission.

Their strategy is geared toward lowering the probability that a critical mass of people rises up and overthrows them. In a post-AGI world, there is potentially no critical mass of people that can overwhelm the system. The likelihood that you succeed at dislodging an unjust power structure goes to zero.

With your ability to force a favourable equilibrium by means of collective action gone, and your expected utility to the system also gone because your labour value went to zero, what are the incentives to continue giving you your UBI, or to otherwise entertain your continued existence?

Conclusion

Going back to the title, it is clear that the population's satisfaction is still a meaningful check on governance today, just as it has been in the past. Different states, driven by different ideologies, find different equilibria which are more or less satisfactory for the majority of people.

However, given current trends, it seems that popular satisfaction could soon become completely optional from the perspective of the power structure for the first time in human history, and if you don't like the outcome, then it's not like you'll be able to revolt your way out of it anymore.

  1. ^

    "Why assume such alignment?", you might ask. The answer is that the economic incentive structures pushing the AI frontier are rolling with this assumption, which ends up producing AGI regardless of whether it is justified or not.

  2. ^

    All the ways in which it just ain't so (e.g. large campaign donors, media manipulation) appear to be too opaque to raise enough interest in creating a new equilibrium.



Discuss