2025-11-11 10:49:32
Published on November 11, 2025 2:49 AM GMT
People have been celebrating Secular Solstice for over a decade now, in our small community. Many different programs and versions have been collected at Secular Solstice Resources (and elsewhere). The amount of material can be overwhelming. Many Solstice programs are based around original material being written or updated, speeches that are specific to the speaker, and other things that make it challenging to reuse.
The goal for this program is to be an easy-to-follow, easy-to-reuse Solstice program for a group celebrating Solstice for the first time, or the first of a few times, possibly a small group, possibly without many resources.
Simply print one copy of the program per participant (or fewer and have people share), and follow the directions.
If you aren't familiar with the songs, you will need one or a few people to go through them first and help lead. You could host a listen-through party first to get your group familiar with the songs used. NOTE: I haven't made a specific playlist for this yet, due to time constraints, but reference materials for all the songs can be found at Secular Solstice Resources' Songs page. (Please express interest in the comments if you'd like this and I will try to make time!)
Additional editorial notes about this Solstice:
I wrote a script to extract these from the programs on Secular Solstice Resources. Here are the results, as of January 2025:
Most used songs:
| Song | Uses |
|---|---|
| Brighter_Than_Today | 38 |
| Uplift | 33 |
| Bitter_Wind_Blown | 22 |
| Hymn_to_the_Breaking_Strain | 21 |
| Time_Wrote_the_Rocks | 19 |
| Here_Comes_the_Sun | 17 |
| Five_Thousand_Years | 17 |
| When_I_Die | 16 |
| Always_Look_on_the_Bright_Side | 16 |
| X_Days_of_X_Risk | 15 |
| Bitter_Wind_March | 14 |
| Chasing_Patterns | 14 |
| Do_You_Realize | 13 |
| Bold_Orion | 12 |
| Still_Alive | 12 |
| Endless_Light | 11 |
| Here_and_Now | 10 |
| The_Sun_Is_A_Mass_Of_Incandescent_Gas | 10 |
| Blowin_in_the_Wind | 8 |
| Somebody_Will | 8 |
| Lean_on_Me | 8 |
| Voicing_of_Fear | 8 |
Most used speeches:
| Speech | Uses |
|---|---|
| Minute_of_Silence | 17 |
| Beyond_the_Reach | 13 |
| Road_to_Wisdom | 10 |
| 500_Million_But_Not_A_Single_One_More | 10 |
| Pale_Blue_Dot | 10 |
| Gift_We_Give_Tommorow | 8 |
| Call_and_Response_Defiance_Abridged | 8 |
| We_Are_Here | 7 |
| This_Is_a_Dawn | 6 |
| Litany_of_Tarski | 6 |
| No_Royal_Road | 5 |
| Origin_of_Stories_Morning_Edition | 4 |
| You_Cant_Save_Them_All | 3 |
| Communal_Meal | 3 |
| Toasts_Boasts_and_Oaths | 3 |
| Story_of_Winter | 3 |
| The_Goddess_of_Everything_Else_Abridged | 3 |
| Origin_of_Stories_Twilight_Edition | 3 |
| Call_and_Response_Defiance | 3 |
| Only_Human | 3 |
2025-11-11 10:26:39
Published on November 11, 2025 2:26 AM GMT
Many prominent figures, including Sam Altman and Elon Musk, have suggested universal basic income (UBI) as a solution when artificial intelligence renders human labor obsolete. Musk has even promised "universal high income." However, there are serious reasons to be skeptical of this vision. The framing assumes that alignment will be solved but the AIs are aligned to some elite group or they uphold anything like current capitalism with its property rights.
Elon Musk has said that "in the benign scenario, probably none of us will have a job" and promises not just UBI but "Universal High Income." However, when he took a government role with DOGE, one of his first moves was to defund foreign aid programs including PEPFAR (which provides HIV treatment), as well as programs fighting malaria and tuberculosis. A government memo estimated the cuts could cause millions of additional deaths, with one analysis projecting approximately 600,000 deaths, two thirds of the deaths being children. Bill Gates remarked that "the picture of the world's richest man killing the world's poorest children is not a pretty one." While some of these cuts were later reversed, infectious disease programs for HIV are still reduced to about 30% of pre-cut funding levels.
Your work is your leverage in society. Throughout history, even the worst rulers depended on their subjects for labor and resources. Mutual dependence is why society functions: why you can go to the supermarket and buy food, why housing exists for you, why society is "aligned" to provide you with space to live a decent life. And if you are more useful to others than the average person, you get rewarded proportionally. In a fully automated world, you become a net burden. If we assume that the police and military have also been automated, there is no realistic option to rebel or protest for change either. Resource dependent economies may give us an early glimpse at this, many resource rich countries have an extremely poor population and no democratic institutions.
If everyone is unemployed, that means all media is automated too. All the information you consume would be AI-generated and AI-curated. We're already seeing how AI-powered social media algorithms fragment society. These systems, currently optimized merely for engagement, have discovered that lies, misinformation, and conspiracy theories often generate the most interaction.
Now imagine a world where not just the algorithms but all the content itself is AI-generated. Imagine algorithms explicitly designed to mislead and divide. People couldn't organize their thoughts coherently enough to even conceive of rebellion or imagine alternative social structures.
It's extremely difficult to imagine a society where we're all unemployed and humanity is still around. Think about what's required for a mass unemployment scenario: an AGI system powerful enough to run society, automate all jobs including research, police and military. Such a system would obviously be lethally dangerous, therefore if we get it catastrophically wrong, we die. We only get one shot at this and without a promising plan the odds don’t look good.
Being useful keeps us alive. This is more fundamental than constitutional rights or free speech. In the real world, we're aligned with other humans largely because we need each other. Our jobs mean we're still useful to the economy, useful enough that others will provide us with the goods and services they produce.
But more fundamentally: we're unlikely to survive long enough to face this dilemma. The kind of AGI powerful enough to automate all jobs is the kind of AGI powerful enough to end humanity if misaligned. And we're rushing toward building it with no credible plan for alignment.
2025-11-11 10:19:24
Published on November 11, 2025 2:19 AM GMT
My post on the grapefruit-juice effect contains a ternary plot of citrus fruits. Here it is again (source):
Ternary plots are great! They're used in a number of specialized fields, but I think they would be more widely popular if people were more familiar and comfortable with them. Let's make it happen.
Look at the citrus fruit plot. Every data point on that plot corresponds to a species of citrus, whose ancestry is some mixture of mandarins, pomelos, and citrons. The three corners are the ancestral species themselves, which are 100% themselves and 0% anything else. To take an example from somewhere inside the plot, most lemons are about 20% pomelo, 30% mandarin, and 50% citron. You can read that off from the grid lines: mandarin ancestry is shown by vertical position, with 0%-mandarin fruits along the bottom of the triangle and the mandarin itself at the top. The "lemons" cluster is about 30% of the way up (between the yellow horizontal lines marked "20%" and "40%"), so it's 30% mandarin. Similarly, lemons are 20% of the way from the right edge (0% pomelo) to the bottom-left corner (100% pomelo).
Note that there are three variables (percent pomelo/mandarin/citron ancestry), but they are not independent; they always sum to 100%, so there are only two independent "degrees of freedom". This is why you can make a two-dimensional plot of the three-dimensional data.
The one confusing thing about ternary plots is that the axis lines are usually only marked on one edge. For example, the horizontal lines marking mandarin ancestry are labeled on the right-hand side; on the left, you instead have angled labels in blue, which correspond to the sloped lines for pomelo ancestry. Most ternary plots don't use different colors in the helpful way this one does; I think the best way to read them is to look at a corner (100% of something), identify the opposite edge (0% of that thing), find the lines parallel to that edge, and read off the labels on whichever side of the triangle is oriented to match those lines. (If the labels aren't helpfully rotated to match up with the lines, you can instead go with whichever side reaches 100% at the corner and 0% at the side in question.)
You can (and should) use a ternary plot to display any kind of quantitative data with:
Actually, if you instead have a constraint on a different linear combination of the components, you can still make (something like) a ternary plot; it just won't be an equilateral triangle. If the second rule (non-negativity) doesn't apply you can still make a ternary plot, but some points will fall outside the triangle, so you'll have to extend your grid lines.
The rest of this post is fun examples.
Most ternary diagrams are for mixtures of things, where the sum constraint corresponds to the fact that percentages have to add to 100%. The citrus fruit diagram is one example.
This mixing diagram for soil has gone somewhat viral:
Clay, silt, and sand have a technical definition in terms of grain size. Below .05mm, sand becomes silt; below .002mm, silt becomes clay. So this plot gives a sort of low-dimensional projection of the infinite-dimensional space of granular mixtures. In fact, you can think of an arbitrary granular mixture as having a sort of spectrum of grain sizes, described by a spectral density function, and this plot shows a two-dimensional quotient of the vector space of those functions...
Speaking of spectra, one can also make a ternary plot for RGB colors! Here it is, embedded in the broader space of all possible colors:
The colors outside the triangle are too saturated to be displayed on a standard computer screen. The horseshoe shape is a little harder to explain; maybe in a future post.
This plot shows the conditions for a mixture of methane, nitrogen, and oxygen to be flammable. (Note that the grid-line labels are oriented unhelpfully, so you have to be careful about using the correct scale when reading off percentages.)
And here's one for frequencies of different alleles.
QAPF diagrams are basically two mixing triangles glued together:
Sums and differences of the electronegativities of a pair of atoms can be used to construct a sort of ternary diagram of bond types, called a van Arkel--Ketalaar triangle:
(Hmm. One could apply that trick to quite a lot of things...)
The baryon decuplet is often drawn in a sort of ternary diagram of quark content -- although the chart slightly predates the idea of quarks:
(source)
Actually, there's some deep math behind that: the triangular lattice appears here as the root lattice of the Lie group , also known as . In most derivations, the lattice itself shows up as a diagonal slice through a 3D Cartesian grid; the lattice points are all the points with integer coordinates that sum to zero.
Ternary plots are also used in astrophysics, for non-gaussian statistics (measurements involving a triangle of three points in the sky at once). Unfortunately, these are usually displayed with two axes for ratios of edge lengths of the triangle, instead of as a ternary plot of the three angles, which sum to 180 degrees. Also, they're Fourier-transformed.
2025-11-11 10:14:53
Published on November 11, 2025 2:14 AM GMT
In order to cause a catastrophe, an AI system would need to be very competent at agentic tasks[1]. The best metric of general agentic capabilities is METR’s time horizon. The time horizon measures the length of well-specified software tasks AI systems can do, and is grounded in human baselines, which means AI performance can be closely compared to human performance.
Causing a catastrophe[2] is very difficult. It would likely take many decades, or even centuries, of skilled human labor. Let’s use one year of human labor as a lower bound on how difficult it is. This means that AI systems will need to at least have a time horizon of one work-year (2000 hours) in order to cause a catastrophe.
Current AIs have a time horizon of 2 hours, which means it’s 1000x lower than the time horizon necessary to cause a catastrophe. This presents a pretty large buffer.
Currently, the time horizon is doubling roughly every half-year. That means that a 1000x increase would take roughly 5 years at the current rate of progress. So, in order for AI to reach a time horizon of 1 work-year within the next 6 months, it would mean that the rate of AI progress would have to increase by 10x, leading to a doubling in time horizon roughly every two weeks instead of every 6 months.
It seems like a huge breakthrough is necessary to make 5 years of AI progress happen in less than 6 months. Has a breakthrough of this size ever occurred?
I can mainly think of two recent examples of AI breakthroughs that might be comparable in size to what’s needed to create dangerous AI in the short-term.
One is the invention of the transformer architecture. We don’t have time horizon estimates for any models before GPT-2, as even our easiest tasks are probably too difficult for earlier models. However, we can maybe get a sense of how many years of progress it is through other means.
Epoch AI estimates that transformers represented a compute efficiency gain of 10x-50x. According to Epoch AI, the historical rate of effective compute increases is around 3x per year, and algorithmic advances have accounted for roughly ⅓ of AI progress in LLMs since 2014. That means that transformers represent a 9-15 month jump in AI progress[3]. This places it way below the 5-year threshold required to get current models to a dangerous time horizon.
Also, the algorithmic progress from transformers wasn’t instant. It took two years to go from the Attention is All You Need paper to large transformers like GPT-2 being released. So it’s likely that the invention of transformers is neither large enough, nor sudden enough, that a similar invention would be very dangerous if it happened tomorrow.
In fact, if we round the impact of transformers to a 1-year jump in AI progress, then we’d need five transformer-sized breakthroughs compressed in a 6-month timespan to reach a 1-year time horizon.
The other example is AlphaFold. While AlphaFold is not a general AI architecture, it’s still useful for establishing an upper bound of how crazy breakthroughs can get.
I haven’t seen a good analysis that tries to answer the question “How many years of protein-folding progress did AlphaFold represent?” But judging from this analysis, it seems like it’s at least 5 years, and maybe in the decades.
This means that AlphaFold is an existence proof for at least 5 years of narrow AI progress being made in a short period of time.
Assuming transformer-sized breakthroughs happen every 10 years, and events are independent, then the probability of five such breakthroughs happening in the next 6 months is very, very small.
But AI breakthroughs aren’t independent events. They have the same inputs, and the occurrence of one breakthrough is an update towards the inputs being sufficiently high to produce additional breakthroughs quicker. Additionally, AI breakthroughs could feed into each other, as the “AI capabilities” output of AI R&D can feed back into the “intellectual labor” and “compute spend” inputs.
So, if multiple transformer-like breakthroughs are unlikely to lead to AIs with 1-year time horizons, what about AlphaFold-like breakthroughs? If we think that there could be a narrow AlphaFold-sized breakthrough in coding, then it becomes plausible that AI R&D could be automated in the short-term.
For any verifiable domain that AI researchers are trying to “crack”, I’d guess that AlphaFold-sized breakthroughs are less than 5% likely per year. And programming seems harder to “crack” than protein folding, as evidenced by the fact that companies have been throwing lots of money at it for a few years with no AlphaFold-sized breakthroughs to show for it. So, adjusting slightly downward, the probability that AI R&D is automated in the next 6 months seems less than 3%.
But even if AI R&D was automated tomorrow, this wouldn’t guarantee that we’d reach a time horizon of 1 year in less than 6 months. It’s more likely that the speed of AI progress would slowly ramp up as people and AIs found better ways to distribute labor and resources between automated and human AI researchers. And training runs sometimes take a while, meaning the breakthrough might take more than 6 months to fully take effect.
It also seems more likely than not that if any AGI company was on track to create AIs which have dangerously high levels of general capabilities in the next 6 months, they would be able to tell that this is happening, at least at the start. They would see that one year of progress has happened before all five years of progress have happened, at least assuming that there aren’t large discontinuities between checkpoints in training.
If a 1000x increase in time horizon routes through AI R&D automation, then the AGI company would definitely at least notice that AI R&D has been automated. If it doesn’t route through AI R&D automation, it’s likely we’d notice a 10x increase before the 1000x increase. So in any case, the AGI company is likely to have some sense that “something big might be happening” if they’re heading towards a major improvement in capabilities.
However, it is plausible that after some level of capabilities, the AIs would figure out how to subvert oversight mechanisms and make it look like they’re less capable than they actually are. So if capabilities measurements during training are too sparse, or easy to subvert, the researchers might not notice a sudden jump that enables oversight being broadly undermined.
So, judging from the size of breakthroughs needed, and from the sizes of some recent AI breakthroughs, it seems very unlikely (<2%) that AI will reach a 1-year time horizon in the next 6 months. The main pathway I see is a sudden breakthrough in coding, which would lead to automated AI R&D, which would lead to a large number of transformer-sized breakthroughs in quick succession. Accounting for unknown unknowns, I’d increase my probability to around 3%. If this does happen, I think it’s more likely than not than the AGI companies in question would have some awareness that it’s happening, instead of it being a complete overnight surprise.
I’m assuming away the possibility of a catastrophe caused by misuse of AI systems, like bad actors using AIs to create very potent biological weapons. I’ll only consider AI catastrophes caused by autonomous AIs.
By “catastrophe”, I mean an event where 100 million humans die, or something even worse happens.
Although I do feel kind of skeptical of this number. Surely transformers were a bigger deal than that? Without transformers I’d guess we’d be more than 1 year behind, but that’s probably a different operationalization than the one Epoch AI uses.
2025-11-11 08:43:23
Published on November 11, 2025 12:43 AM GMT
One important property for a style of thinking and argumentation to have is what I call galaxy brain resistance: how difficult is it to abuse that style of thinking to argue for pretty much whatever you want - something that you already decided elsewhere for other reasons? The spirit here is similar to falsifiability in science: if your arguments can justify anything, then your arguments imply nothing.
In this post, I will argue that patterns of reasoning that are very low in galaxy brain resistance are a common phenomenon, some with consequences that are mild and others with consequences that are extreme. I will also describe some patterns that are high in galaxy brain resistance, and advocate for their use.
Vitalik talks about styles of arguments that prove too much, pulling examples from the AI, EA, and cryptocurrency communities. As defenses, he recommends having deontological principles that override slick reasoning and avoiding incentives that would distort your reasoning.
His advice to would-be AI safety researchers:
- Don't work for a company that's making frontier fully-autonomous AI capabilities progress even faster
- Don't live in the San Francisco Bay Area
2025-11-11 07:59:16
Published on November 10, 2025 11:59 PM GMT
Why aren't all pencils the same quality, conditional on price?
When I go to a store to stock up on pencils[1], I'm presented with an array of choices and lacking any indicators of quality, I choose any old brand. For some strange reason, the quality of the pencil will be a crapshoot. The lead inside might break easily, or it might not. The mechanism may get stuck after a month, or it may not. The eraser may be a flimsy thing unworthy of the name, or it might match a Staedler. Why?
Is a mechanical pencil not a commodity good, which should be fungible no matter the make? Like fruit, it should be of consistent quality. Wait.
Fruit is not of a consistent quality. Nor are laptops, or shoes or paper, or really any of the goods that pop to my mind. E.g. go buy 10 random types of laptops at the same price point, and I'm sure they'll vary greatly in quality.
So we have a new mystery. Why are commodity goods not the same quality, conditional on price? (Another way to frame this mystery is: why does it pay to look for high quality in so-called commodity goods?)
I'm not talking about a high bar for consistent quality here. If most of the variance was explained by within brand-line variation, then a product would in my mind live up to the name of commodity good. But they don't.
My guesses as to why have the following structure:
1) Manufacturers mostly aren't optimizing for functional quality, but for other traits. And when you are optimizing for one trait, the other traits you do not optimize take random values. Only the moderate pressures for quality prevent greater variance in quality. This explanation is dual to the next.
2) Consumers care about something other than quality.
2 reminds me of a question Robin Hanson raised: why is there so much diversity in modern products? Why hundreds upon hundreds of different kinds of phones instead of just a few? His guess is that consumers want diversity to signal their own differences from others, which is a high-status behavior now common because we moderns are status mad.
I think this theory has some merits. Certainly, it says why there should be lots of variance amongst brands. And if manufacturers mainly optimize for diversity, then functionality varying greatly makes some sense. At the most extreme ends of fashion, you have clothing that is already torn to shreds.
But I don't think most people are signalling anything with their choice of mechanical pencils. Oh, some do, for example children who want dinosaurs on their pens. But that's a different segment of the market. For all of us who don't attach much care about the designs of their mechanical pencils, shouldn't we get boring, reliable quality?
Still, it's the best answer I've got. Stuff like "consumers just don't care enough to do the research needed for quality" are question begging.
I used to do this, but I've since remedied the problem by researching quality mechanical pencil brands. I've found the Pentel P205 0.5mm to be a worthy weapon with which to challenge Landau and Lifshitz.