2026-04-20 15:10:22
This is a weekly round-up of things I’ve posted in the last week.
InkHaven requires that I post a blog post every day, which is a lot. Especially for people subscribed to my blog. Someone requested I spare their inbox, so I haven’t been sending out every post.
So now you get to catch up! You can even be selective if you prefer :)
Diary of a “Doomer” (part 1) is about my experience getting into the field of AI and AI Safety (I started graduate school in 2013). A lot has changed since then. What used to be a fringe topic has become really mainstream! I’m talking about deep learning, of course… But seriously, AI researchers really dropped the ball, and owe society a debt they can probably never repay for failing to consider the consequences of their actions.
Contra Leicht on AI Pauses takes apart Anton Leicht’s piece arguing we shouldn’t try to pause AI. I first encountered Leicht when he was arguing against having an “AI Safety” movement at all last fall. I don’t think either of these articles are very good — I find the reasoning sloppy.
Post-Scarcity is bullshit is mostly about how certain things are fundamentally scarse; like land, energy, and status. I got a bit snarky here about the discourse around the topic, and how vague, incoherent, and/or unimaginative people’s visions of the “post-scarcity” world typically are.
From Artificial Intelligence to an ecosystem of artificial life-forms. If the AI race doesn’t stop, the natural end-point is the creation of artificial beings that proliferate, diversify, and radically reshape the world. This is one of my quick and dirty attempts to explain a part of my world view that really deserves a 30-page essay.
Idea Economics is a rare non-AI-related post about how and why I think people devalue ideas: Not because they’re easy to come by, but because they’re hard to hold on to if you share them. But then I ruin it by talking about the CAIS Statement on AI Risk as an example (it was sorta my idea).
Stop AI is an attempt to get the basic case for why we need to stop AI down in writing. It ended up basically just covering the risks and not why other solutions aren’t good enough (stay tuned, that might be the next post).
Stop AI Now argues against kicking the can down the road. I think that’s intuitively a bad idea, but here I give three particular reasons.
I did this as a bit of an experiment. Besides the person complaining to me directly, I did notice a dip in subscribers at some point after about seven posts in a row at the start. A blogger friend of mine with more of a following says they often lose followers after a post. I guess that makes sense… people don’t like their inbox being clogged.
I did still send out two of these posts as email notifications. The first one was deliberate, the second was an accident. You can see that the ones I sent out did get a lot more views. I’ll be curious to see how much this post makes up the difference!
Thanks for reading The Real AI! Subscribe for free to receive new posts and support my work.
2026-04-20 14:53:16
Or, the end of the world is no excuse for sloppy work
One morning when I was nine, my dad called me over to his computer. He wanted to show me this amazing Korean scientist who had managed to clone stem cells, and who was developing treatments to let people with spinal cord injuries – people like my dad – walk again on their own two legs.
I don't remember exactly what he said next, or what I said back. I have a sense that I was excited too, and that I was upset when I learned the United States had banned this kind of research.
Unfortunately, his research didn’t pan out. No such treatment arrived. My dad still walks on crutches.
Years later, I learned that the scientist, Hwang Woo-Suk, had been exposed as a fraud.
In 2004, Hwang published a paper in Science claiming that his team had cloned a human embryo and derived stem cells from it (the first time anyone had done this). A year later, in 2005, he published a second paper claiming that they managed to repeat this feat eleven more times, producing 11 patient-specific stem cell lines for patients with type 1 diabetes, congenital hypogammaglobulinemia (a rare immune disorder), and spinal cord injuries. This was the result that, if true, would have helped my dad.
None of this was real. The 2004 cell line did exist, but was not a clone; investigators concluded that it was an unfertilized egg that had spontaneously started dividing. The 2005 cell lines did not exist at all; investigators later found that the data reported for all eleven lines had been fabricated from just two samples, and the DNA in those two samples did not match the patients they had supposedly been derived from.
My dad was not the only person Hwang had given hope to. On July 31st, 2005, Hwang had appeared on a Korean TV show. The dance duo Clon had just performed; one of its members, Kang Won-rae, had been paralyzed from the waist down in a motorcycle accident five years earlier, and had performed in his wheelchair. Hwang walked onto the stage and told a national audience, with tears in his eyes, that he hoped “for a day that Kang will get up and perform magnificently as he did in the past” – a day that was coming soon. He made similar promises to other patients and their families.
I don't think Hwang was a monster who set out to commit fraud for international acclaim. I think he was a capable scientist with real results. (Some of his lab’s cloned animals were almost certainly real clones, including the world’s first cloned dog Snuppy.) But over time, he repeatedly took what he felt was his only option.
The 2004 paper may have started as a real mistake; it’s possible his team genuinely thought the parthenogenetic egg was a clone. But by 2005, with a nation watching and a Nobel on the table and a paralyzed pop star looking at him on live television, there was no version of "actually, we can't do this yet" that he could bring himself to say. So he didn't say it.
The way in which Hwang began his downward spiral is what sticks out most to me. He started out a good scientist, with good results and an important field of study. But with tens of millions of dollars of funding, thousands of adoring fans, and all the letters written to him by hopeful patients and their families, Hwang likely felt the weight of the world on his shoulders. He had to do what he had to do, in order to not let them down.
I work in AI safety. Many of the people I work with believe (and I believe) that the next decade will substantially determine whether and how humanity gets through this century. The stakes are literally astronomical and existential, and the timelines may be short.
That is the weight we carry. And I worry that when push comes to shove, our scientific standards will slip (or are slipping) in order to not let other people down.
For example, wouldn’t it be the right choice to just accept the code written by Claude, without reading it carefully? We don’t have much time left, and we need to figure out how to do interpretability, or monitoring, or how to align models with personas, and so forth.
Why investigate that note of confusion about the new result you saw? Surely with the stakes involved, it’s important to push forward, rather than question every assumption we have?
Why question your interpretability tools, when they seem to produce results that make sense, and let you steer the models to produce other results that seem to make sense? Why flag the failed eval run with somewhat suspicious results, when the deadline for model release is coming soon, and evaluation setups are famously finicky and buggy anyways? Why not simplify away some of the nuance of your paper’s results, when doing so would let it reach a much larger audience?
I worry that it’s tempting for us to take the expedient choice and let our standards slip, precisely because the stakes are so high. But it is precisely because the stakes are so high, with all the real people who will be affected by the outcome, that we need to be vigilant.
Yes, timelines may be short and we may not have time to do all the research that we want. But slipping up and producing misleading or wrong research will only hurt, not help. And if we need to say "actually, we can't do that yet", then we should say as much.
2026-04-20 14:48:16
I've designed a few spaces people seem to really love, most widely known Lighthaven.[1]
Most people, including me before I co-designed Lighthaven and its predecessors, have surprisingly bad introspective access into why a space makes them feel certain things. Often the most we can do is as shallow as "this place feels fake", or "this place has bad vibes", or "this place feels cozy". It took me many months of practice to get to the point where I could tell why a room made me feel off, or why it made me feel warm and cozy, or alert and energized. Luckily you don't need to do that because the answer most of the time turned out to be the same.
Usually a space feels bad because it is lit by low-quality lights
Our eyes evolved to see things illuminated by sunlight. Correspondingly the best proxy we have for whether the light in a room "works" is how similar the light in that room is to natural sunlight. The most popular way of measuring how much light differs from natural sunlight is the "Color Rendering Index" (CRI):
The everyday situation where I've found the effect of low-quality lighting starkest is to look at the face of a person illuminated by nothing but a computer screen. Computer screens emit extremely low CRI light, as a screen simulates white by combining the light from red, green and blue LEDs, producing a very low-CRI emission spectrum. Faces illuminated by nothing but screens often look off and have a slightly alien, plasticky, often blue-feeling quality to them, even if the screen that is emitting the light is almost fully white.
In short: If you want a space to feel natural, buy lightbulbs with at least 95 CRI, ideally 98.
But I thought my eyes can only see three colors?
Yes, and that is exactly why when you look at a computer screen directly, colors look real and vivid. Your computer screen emits light that (pretty precisely) stimulates the three kinds of cones in your eyes and so can produce basically arbitrary perceptual colors (it's not perfect, but it's quite good).
The problem occurs when light bounces off of other objects in the room. The color of an object is determined by how it absorbs, reflects, and changes light that hits it. For example, an object under sunlight might completely absorb orange light (~630nm), but fully reflect the red light and green light emitted by the LEDs in your computer screen. That object would look unnaturally bright under the light from your computer screen, because it basically reflects all the light from your screen, but under sunlight it would absorb all orange wavelengths.
So to a first approximation a light feels "natural" if its light emission spectrum is a smooth curve. Sunlight, as well as any other light created by burning or heating things to really hot temperatures, has a smooth emission spectrum, which maintains all color information as it bounces around a room.
You might have noticed a second number that keeps showing up in the widgets above, often formatted as "5000K" or "2500K". This is the "color temperature" of light. We call this "temperature" because it corresponds to what kind of color objects emit when you heat them to that temperature[2]. Objects that are hotter, emit light that is more blue. Objects that are less hot (e.g. "only" 2000 degrees Kelvin) emit light that is more red. Naturally we call red light "warm light" and blue light "cold light"[3].
If you are lighting a room with plenty of natural light, just use 2000K-3000K lights
People prefer bluer light during the day, but redder light during the evening and morning. Sunlight is really really bright, so what lamps you have in your room do not matter if you have large windows during the day. This means the primary purpose of your lights are to light things in the evening and morning. This means they should be warm.
If your room does not have much natural light, I recommend having bright overhead lights that are closer to 4000K, and dimmer floor lamps around 2000K-3000K.
The world got ugly when we invented LEDs
For basically all of human civilization up until very recently lighting quality was a complete non-issue. Why? Because all of our artificial light sources consisted of heating things to very hot temperatures, or burning things. When you do that, you basically always emit natural light with a smooth emission spectrum.
Lighting quality only became an issue within the last 100 years with the introduction of fluorescent lamps in offices. This is why "fluorescent lighting" has for many people become the best shorthand for fake or artificial lighting.
But people's homes, as well as any entertainment venues, bars or really anywhere where people socially congregated in the evenings were lit by incandescent light bulbs (or before then, candles and oil lamps) with perfectly smooth spectra.
But around 30 years ago home lighting LEDs were introduced, initially with truly terrible color rendering indexes, and most people unable to put words to the discomfort and alienness they caused, chose the energy-saving option and replaced their incandescent bulbs with LEDs. Eventually, in most of the western world outside of the US, incandescent lightbulbs were literally banned to promote energy saving policies.
This was the greatest uglification in history. Within two decades, much of the world that was previously filled with beautiful natural-feeling light started feeling alien, slightly off, and uncomfortable, and societal stigma around energy-saving policies prevented people from really doing anything about it.
But you, within your home, can fix this. LED technology has come along way and we can produce high-CRI LED bulbs (I recommend YujiLED or Waveform Lighting). The world really used to be much more beautiful and a much less harsh place in this one respect. You can restore the natural light, and the homeliness that all your ancestors felt, at least within the confines of your home. Just buy some high-CRI, warm color temperature light bulbs. There is a lot more to interior design, but it's honestly so much easier to iterate on than lighting.
But before that the Lightcone Offices, and I've also played a role in designing some of the most popular areas at Constellation. Also of course credit to my team for much of the work.
What kind of object? Well, turns out really any object, unless the thing you are heating undergoes some specific chemical reaction when you heat them that causes them to emit other wavelengths of light. The radiation curve that most objects tend to follow here is known as the "blackbody radiation curve". You can google it or ask your local LLM if you want to understand the physics behind this better.
This is a joke. This is indeed exactly backwards. You cannot imagine how much this makes explaining color temperatures to people more confusing. "Oh, just get the warmer light bulb, no not the one that has the higher temperature written on it why would you think that, that would produce much colder light". Grrrr.
2026-04-20 14:20:25
I think we need to Stop AI. Specifically we need to Stop AI Now. We can’t wait around. The standard metaphor is a runaway train heading towards a cliff. Let’s work with that.
World’s most-cited-scientist (and my Master’s supervisor) Yoshua Bengio says we’re racing into a bank of fog, and there could be a cliff. That’s about right. There are two implications of this: 1) maybe there’s no cliff and it will all be fine 2) the cliff could be anywhere, we can’t see it far enough ahead to stop unless we’re going very slow. So while a lot of people seem to think we’re going to see the risks clearly in time to stop, I’m not so sure.
The entire time I’ve been in the field, people have repeatedly been surprised by the rate of progress in AI. The people at the leading AI companies are an exception – the most vocal among them have been, if anything, overestimating how fast things move.
There’s a dangerous idea that’s caught hold that AI progress is predictable because of “scaling laws”. We’ve seen pretty consistent patterns in how quickly AI advances in terms of particular metrics as a function of time. But there’s a few problems with this: 1) The metrics don’t measure the things we care about, 2) There’s no reason why these trends should hold if there’s a paradigm shift. Indeed, RE (2), there’s already been a major shift with the deep learning era where massively more resources are being put towards AI year-on-year than before. The rate of progress changed.
There’s no reason this can’t happen again. Indeed, I think we should expect it to happen again for at least two different reasons.
First, at some point, when AI R&D really kicks into gear, we could discover learning algorithms that work much better than today’s. I think the current AI paradigm leaves much to be desired, with major improvements, e.g. in long-term memory and efficiency. And those could arrive suddenly, and take an AI system from “really useful, but still needs a lot of hand-holding” to “we’re not sure we can stop this thing, maybe we should, um… shut off all the computers?”
Second, at some point, AI agents could really take off (we may be in the beginning of this, already), and get very good at effectively and efficiently causing things to happen in the physical realm, and could then start to rapidly and autonomously scale up the amount of physical resources (e.g. energy) directed by AI towards accelerating both AI R&D, and this process of acquiring resources and influence.
Another dangerous trend is an increasing focus on capabilities that are obviously dangerous, such as bioweapons or cyber-attacks, to the exclusion of unknown risks.
*This RAND report is an exemplar. I previously wrote a detailed response, maybe I’ll post it soon.
The “unknown risks” argument is “When you play against a much better chess player, you know they will win, but you don’t know how”. The things you see coming, they also see coming. They do something else.
We should be worried about any system that is very smart posing a risk to us. Sometimes we can make a fairly strong case that a system lacks a particular capability, and that this makes it safe. For instance, an AI system that has only been trained to play games of Chess or Go is probably going to be safe, even if it’s an insanely good player.1
Arguments that might seem stronger than they are include:
It’s stuck in a computer, we can just unplug the computer.
Its memory is wiped after every interaction, it would struggle to make and execute coherent long-term plans.
The problem with both arguments is that they assume that the AI cannot use its influence over the external world to acquire new capabilities. For instance, a smart AI that notices it is limited by such things could pay people to help give it a robot body or a better external memory, or trick them into it.
In general, it’s hard to know what to make of a system that is clearly really smart, and not fully understood. A lot of experts (Yann Lecun, Gary Marcus, …) claim that current approaches to AI are fundamentally limited, but this is just them stating their opinion, which many other experts disagree with. The reality is we just don’t know.
But even if the system is fundamentally limited in some way, it could still cause massive risks. For instance, lacking a sense of smell probably wouldn’t stop an otherwise super intelligent AI from taking over the world if it wanted to.
What needs to happen, once “we” decide to stop? A rough list I have in my head is:
The US government decides to stop AI, and starts trying to broker an agreement with China and maybe a few other key players.
The US, China, etc. reach an agreement on how to stop AI.
The rest of the world gets on board with this agreement.
I expect these steps to take time, quite likely a lot of time. How do you actually stop AI? I have an answer, but there are still a lot of details to be worked out, and I don’t think we’ll really know the answer to this question until world powers actually start prioritizing this issue and are willing to make major sacrifices and compromises to achieve it.
A unilateral pause in the US could be implemented faster (but would still require navigating the politics of the thing, which could take arbitrarily long), and to be fair, I think this is what many people imagine a “pause” looking like: frontier AI companies suddenly cease their R&D operations; they send their researchers off on vacation, and stop their big training runs. And the US is ahead right now, so China wouldn’t immediately race ahead. How quickly might they catch up? Three considerations are: 1) How hard are they racing? 2) How far behind are they? 3) How reliant are they on copying US companies to make progress?
The problem with a unilateral pause is that it expires. You get a few months -- or a few years, if you’re lucky -- to figure things out, and then we’re off to the races again. But we can’t count on figuring things out in that amount of time! We don’t even know what we need to figure out. “Solving alignment” (as popularly conceived) may not be enough.
It’s getting harder and harder every day to stop. Every day we wait, AI companies get richer. AI gets more embedded into society and infrastructure. AI gets smarter. AI research advances. More people get AI psychosis. More AI computer chips are built. More datacenters.
There’s a sense in which AI is already out-of-control.2 The very people building it have repeatedly expressed concern, fear, anxiety, dread, apprehension, etc. about the risks it brings. They say they would like to slow down, if they could. Elon Musk says he has AI nightmares. They don’t seem to feel like they’re in control. As easy as it might seem for AI CEOs to say “oh, damn, this model really is dangerous, we’d better pause”, it’s not clear they will be able to at the critical moment. Maybe a dangerous AI has already escaped human control. But more generally, I am concerned that we will increasingly lack a “nexus” of control at all.
Despite the risks, a lot of people I talk to in the AI Safety community think that we should keep building more powerful AI until it’s more clear that it’s getting too dangerous. One argument is: Until AI literally kills everyone, it’s basically just great; let’s keep getting all the benefits for as long as we can. This is sort of mostly a vibes-y thing. These people are “pro-technology” and really, really don’t want to be mistaken for Luddites.
The more significant argument is this: the whole point of pausing is to do more safety/alignment research, and we can make the most use of wall-clock time when we have more advanced models to study, and to use to help us with the research. This is clearly a 12-D chess move. In order for it to work, we’d need to know where the precipice is, and we’d need to slam the breaks before we get there, and we’d need to make sure that nobody cuts the breaks in the meantime. I’m not optimistic about any of those things working out, but the plan requires all three of them to. I say: slam the breaks now, while we still can, and we just have to hope that we can stop soon enough.
Thanks for reading The Real AI! Subscribe for free to receive new posts and support my work.
Even such a limited system might in principle be able to discover things about the world outside the game and want to gain influence over it -- this would be a bit like the plot of “The Matrix”, but in reverse. This is an area where we do have some uncertainty, but where I’m comfortable saying “I don’t think we need to worry about that yet”.
See number five of Ten different ways of thinking about Gradual Disempowerment.
2026-04-20 13:34:09
Epistemic Status: Abstract claims, but grounded in data science... though the data science is somewhat stale. I wrote this on March 5th of 2026 based on memories of work I did and methods applied circa 2017, and pushed it out for publication after realizing that maybe there is appetite for it, after I saw this post.
Suppose we took a snapshot of each person in the US, and made a list of their "skills", as one might do with a D&D character.
I would like to report on what I expect would happen if this was attempted in real life, and why (until I get to the point that you understand the title of the essay about "Budgeting" skills being Important a little ways in).
I haven't done this recently, with modern data, but I felt that this was likely to be something in my brain that most people don't know about, and worth an article.
At the end of the essay there will be a call to action! I want to start at least one study group in the SF bay area to "Level Up Budgeting" so I could attend somewhere face to face and talk about books or essays or tools, and I'd be happy if various meetup communities around the world formed their own local study groups, so there can be cross-pollination and transient diversity and so on.
With a skill list for every person in the US, we would see a lot of lists for people whose chief skill is their ability to use a QWERTY keyboard and use a suite of Office Software to create spreadsheets, presentations, and prose.
It turns out you would see a lot of people who have a CPR certification, and almost everyone who gets that in modern times gets an AED certification as well.
(When I think of this, it reminds me of how rationalist!Harry went straight for a magical first aid kit on his first day of being able to buy magical gear. But also, it is interesting how "skills" blur very very swiftly into "certification of those skills" in the actual speech of actual people.)
In real life, writing down a skill list for every person would probably lead to variation in how people write.
Some people would write "Skills: word, cpr, aed, ..." and others might write "Skills & Certifications: MSOffice, AED, CPR, ..." and in some platonically perfect realm these would parse down and be applied to generate the same basic inference of the same basic capacities in the author of the list.
Languages already lump. To really dig into a skill, you don't just know "the name of the nameable skill" but actually can DO things.
There are probably subskills!
Some of the subskills might not even have names but still be something that can be transmitted and learned by someone saying "hold your hand <like this>... see?" and then "no, your pinky is getting in the way... more <like this>". We are interested in all of it, of course, from the microskills to the macro level.
Most people in real life can't cast a "healing spell" (because magic isn't real, so far as I know).
And "krav maga" is pretty rare, and not what people think of first.
Fighting is not central to real life in the modern world... mostly (cops and soldiers and bouncers and bail bondsmen do make up a non-trivial chunk of the jobs though).
Usually people think of the civilian job market, in "real life", when they actually make lists of skills and post them online.
Even in the military, they have complex software helping with the task of killing people effectively to serve diplomatic ends (or whatever), but in practice they end up needing to model and optimize the logistics of hiring 18 year olds and turning just the right number of them into "fuel-truck drivers-license test proctors" as fast as possible. In a deep sense: a lot of skills are pretty prosaic.
The "reality" that is "out there" is nebulous, of course.
Depending on how much lumping and splitting you do, so far as I could tell back when I had access to the data, there are roughly 50k to 200k total skills that people will think are distinct enough to write out as distinct phrases.
Some of the phrases will point to having a specific security clearance level, but often someone might just write "Abilities: office, cpr/aed, ... security clearance, ..." as if those two words "security clearance" were a skill, or pointed to skills, or something?
(Metonymy is the name for what happens when people hit the limits of their words, and just name a thing for whatever thing happens to be nearby that they already know the name of.)
One could probably get the number of skills down around 35k if you put serious effect into de-duping the phrases that people use, and ignore hapax legomena.
But if you only de-dedupe the top 100 most common skills by hand based on semantics, and de-dupe the skill phrases that show up in skill lists algorithmically (reducing plural and single forms into one form), and keep the skill phrases that only ONE PERSON EVER thought deserved to be in a skill list, and yet also hunted far and wide to gather all the English language resumes you could (from the 1980s to 2026) I bet you could get as many as 300k skills, without much trouble.
And if you took each one of these and then traced it to the person, and asked them to teach someone this "officially listed skill" I bet it would turn out to often have 10 subskills that can only be described with a phrase or sentence or paragraph... which means that there are plausibly ~3M skills that are super granular and would take a "paragraph of description" to point to, potentially? Things like "the best way to hold a nail in your other hand when hammering nails".
When I estimate ~3M such skills, I'm being pretty sketch and rough... I would be surprised if it was less than 500,000 of them, and I would also be surprised if it was more than 10 million.
A very reasonable person might think that this is crazy.
They might think "computers" is really "just one skill" and that "git" and "svn" are NOT meaningfully different skills.
They might assert that there is ONE "computer" skill, and that if you are super great at that good at the one skill (but better than a normie!) you could apply that skill in a tech support job, but then you might level that skill up until you could apply it to work in Computer Research.
This would be a very reasonable and pragmatic perspective, but in that case, there's a lot of prior classification work!
The reasonable people at the Bureau of Labor Statistics have already lumped things down to 832 narrow job categories fitting into 116 medial grained categories and just 21 basic categories.
Following them, seeking VERY lumpy lumps, you might reasonably say that the total number of "skills"...
...is either ~21 (some examples being "legal", "protection", "production", "management", "sales", and so on)...
...or else perhaps ~116 (some examples being "legal support skills", "fire fighting skills", "printing skills", "operational management skills", or "wholesale sales skills").
The BLS... is almost certainly too narrow.
Those aren't really skills, you know?
Those are really more like job categories... right?
But it does seem to be true that job categories sort of wink and nudge at the sense in which some skills might be more convergently useful than other skills, or have prerequisite skills, and be useful mostly only in concert with other skills.
Most people have played the piano at least a little, when killing time and kinda bored while waiting somewhere that a piano existed, and quite a few people have practiced specific piano skills (and there are many distinct such skills to practice separately (like just sitting properly, even)) but if we bow down to the lumpiest lumpers and their traditions, even using the full 832 BLS categories, all(!) of the piano skills (and so much more) are lumped into "27-2042 Musicians and Singers".
This is NOT GOOD ENOUGH to tell us what skills to spend 10 minutes practicing every day, or how to hire people who will pay the piano in the way that we really want to hear it played in a bar from just what can be found about them via online scraping of their data.
If we are going to HIRE for jobs, based on a gears level understanding of what cognitive or physical performance or capacity goes on in that job then we need something more granular than jobs to point at, or reason about, that "are the gears".
Here's a thought: maybe we take all the skill lists as is to create ~100k skills from some corpus (with regional choices, and choice of era, and some amount of de-duping) that someone ever mentioned.
We treat each unique skill skill as a node in a network (or as mathematicians might call it a "graph theoretic graph")... and then for each skill list in each resume, we draw lines between every pair of skills that occur in the the same list.
And then for the next list we do the same (making some lines +1 in strength if they were paired in a previous list, so AED and CPR are decently likely to have a three-weight-strong link after processing maybe a dozen random resumes).
This gives us a weighted graph, which comes up in a LOT of optimization problems like shortest path discovery and pagerank and so on.
I have done this before! That's why I am writing this essay ;-)
Lots of "clusters" of skills fall out of such an analysis in the form of dense cliques where many members of the clique are strongly linked to other members of the clique, and weakly linked to anything else. These cliques represent tasks that are demanded in the same job, or abilities taught in very common forms of general education, or sometimes tasks that show up in "standard career progresses" (where a drafting student becomes an engineering tech and eventually a licensed surveyor, and their skill list has things from that whole history).
Every so often you'll find skills that are very common, and that many cliques of skills ALL point to.
For example, there's a bunch of skills that teachers, in the field of education, are proud of, where, by mid career, almost everyone is bragging on their resume that they can do "curriculum design" (no matter whether they're a math teacher or a piano teacher or a kindergarten teacher).
There are other skills like "security clearance" that might show up in the protection area..
But then "cpr" is linked to both of these (and many others)!
This strongly suggests a high degree of convergent instrumental utility exists in skills like this, across a wide variety of fields, even though the skill is "narrowly a skill" that is more like "git reflogging" that can be taught and practiced and tested, but also "broadly a skill" in the sense that it comes up for practically everyone.
There are many measures of "centrality" in graph theory.
One way to arrange the math of it might pinch out the thing at the very center of the very biggest clique (which itself is perhaps at the center of the biggest macro-clique and so on)... but that won't give us these insanely broadly applicable skills that can be taught and learned!
The thing we want, if we're looking for very very broadly valuable skills in almost any domain or any job is betweenness centrality.
The way this works (roughly) is that we pick a lot of pairs of nodes, and spend compute to find the shortest path between them... over and over... and every time we do this we add a point to all the nodes that were on this "shortest journey" from node to node.
At the end, we find the chokepoint... the master node, the node from which you can go ALMOST ANYWHERE very very quickly.
"Curriculum design" is more central than "math curriculum" because it is invoked by more kinds of people with more diversity of skills. And "CPR" has still more betweenness centrality than "curriculum design" (because CPR is useful for cops with cop skills, and life guards with swim instructor skills, and fire fighters, but also elementary school teachers, and summer camp instructors and so on).
Here is the punchline: the thing with the most betweenneess centrality out of all skills is "budgeting".
Which... uh... you know... makes sense? Maybe? <3
Here are some thoughts:
FIRST, consider the holy grail of "rationality" (in the sense of "verbally transmissible cognitive practices that conduce to higher chances of success at nearly any goal" is skill transfer).
Finding skills with high skill transfer and broad applicability lets you spend the least amount of time leveling up, and gives you the most benefit from them.
The skill of "budgeting" makes sense here because like... finitude is everywhere? Tradeoffs are everywhere? Also time is real and ubiquitous. And "making tradeoffs over time in the face of scarcity" is basically the essence of budgeting.
"Budgeting" arguably deserves some halo, because it makes sense, from first principles, that almost every agent would need this skill, if you actually think about agency itself in a first principles way.
SECOND, this comes up in practice in business because the "mangle" of organizations often leads to a handful of people on like.. "the budgeting committee" (which no one wants to be on because it SOUNDS SO BORING) wielding enormous organizational power, and needing to be able to justify their use of power when budgets cause weekly, monthly, quarterly, yearly adjustments in what the organization can and will choose to do.
This will show up, predictably, at the HEART of many moral mazes. We would kind of expect, then, like with the "security clearance" skill, that "budgeting" is a pointer to skills related to the use of language in clarifying AND politically obfuscating ways, depending on who a budgeter is talking to.
The skill might be anti-inductively hard to master because it is woven into bizlore (not academia) and it is very political by default.
THIRD, the lists of skills from which this kind of conclusion can be drawn often come from a wide variety of people, posting resumes online and such. Some of them are older, and more advanced in their careers.
There is a decent chance that if you start in almost any profession or organization, and are good at anything, eventually you'll rise to a role of teaching, guidance, planning, and in a word "leadership". And almost all leaders of non-trivial organizations have accounting and bookkeeping to handle... and also this idea that "the procedures of accounting" could be used to map and understand and control and generally guide a team?
So maybe the betweenness centrality of budgeting comes from a real pinch point, that shows up over and over again in MANY careers, where "budgeting" (about your expertise, on behalf of a group that collectively wields that expertise) becomes essential.
FOURTH, we can compare Biden's semi-famous quote "Don’t tell me what you value. Show me your budget—and I’ll tell you what you value" with a lot of common topics on LW like money itself (the unit of caring!) and kelly betting (essentially iterated bet budgeting), and the relation of money to VNM rational utility functions and bayesian beliefs.
If the skill of "budgeting" turned out to be a super power wouldn't actually be that surprising, if you have a sense that more is possible.
This essay so far has been very high level. I haven't really substantiated it, except to casually suggest what MIGHT be observed, if someone hypothetically looked at certain ways of describing the world in an intensely operational way. Maybe you just don't think it makes sense?
I generally don't like talking about myself, but... uh...
I applied to work at Google in 2013 when I believed that they had the Mandate Of Heaven with respect to the Singularity.
(For several reasons. One big one was that I had previously thought The Singularity Institute had the Mandate Of Heaven earlier than that, but then they sold their banner and trademark and social outreach apparatus to Kurzweil in 2012, who worked at Google. This was before OpenAI was founded in 2014. If I had waited another year, I might have joined OpenAI, instead.)
By 2018 I had given up on Google, because it is a moral maze, and was clearly not going to save the world, and I moved on to working on a blockchain project with friends that was not much related to the Singularity or to AGI in any obvious ways, but it gave me an opportunity to design decentralized utopias and dystopias for decent pay, and that was just super super hard to turn down.
In the intervening time, from 2013 to 2017, I first worked inside the Google Brain "cultural silo" (because I had been told I could work on deep learning aimed at comprehensive proactive benevolence) but after about a year of working on bullshit (like trying to get people to watch longer YouTube content that ads could fit in the middle of) I changed to work on something that actually fucking matters.
So I hopped over into the "PeopleOps" silo by 2015, and instead of increasing the size of YouTube's "ad inventory" I helped build the job search engine that is one of many topic-focused engines existing inside the overall Google Machine for that overall technological macro object to wield according to high level metrics.
(Job search seemed like the closest thing I would be able to find to a sword of good, that The Google Machine would have a hard time abusing too egregiously. And it seemed like it could plausibly produce tens of thousands of dollars in consumer surplus for people who used "free job search" (supported merely by ad money, that is) to find better jobs.)
For about 18 months, between 2015-2017, I was doing data science on the theory and practice of searching for jobs specifically through the lens of "skills and education".
I didn't have the ability to contact people in the outside world about it (because we didn't want to leak that we were working on job search and so on) but my team had access to "all the resumes" and "all the job postings"... in English anyway (and it turns out that the language used in different job markets around the US was regionally dialectical, and so we optimized at the beginning for just a handful of cities, and when we tried to use machine translation to generalize it to Telugu or Japanese the relevance numbers just totally cratered because there is a lot of nuance in the euphemisms people use to talk about hiring and firing and such).
We could play with ways to parse and analyze them using now-old-fashioned pre-LLM NLP techniques. So for a bit more than a year, I could, in fact, do the kind of analyses I described above, but with real data.
And "budgeting" did, in fact, fall out of that actual data, back in ~2017, in the way described <3
Does anyone else want to do this?
I hope that people in the comments chime in in lots of ways because I still feel like "budgeting" is something I'm STILL learning to do well. I would love to hear good textbooks. I would love to hear war stories. I would love to hear ways to spend an hour a day practicing something for a week and become really good at "budgeting".
Does anyone know how to hire the best CFO who is a total wizard at the "budgeting" skill? I don't! I think that's an important thing to be able to do, and I can't reliably do it.
I would be open to driving to Berkeley once a week to talk about some shared reading, or report on homework we assigned ourselves, and hear about other people's challenges and growth at "budgeting, the most central of skills" <3
2026-04-20 09:56:41
I've been working on a mini-book on RLVR for the past few weekends, sharing the v0 now: https://rlvrbook.com
Please check it out!