2025-12-01 09:44:54
Published on December 1, 2025 1:44 AM GMT
David Chalmers describes the inverted qualia thought experiment, in The Conscious Mind, as an argument against logical supervenience of phenomenal experience on physical states:
one can coherently imagine a physically identical world in which conscious experiences are inverted, or (at the local level) imagine a being physically identical to me but with inverted conscious experiences. One might imagine, for example, that where I have a red experience, my inverted twin has a blue experience, and vice versa. Of course he will call his blue experiences "red," but that is irrelevant. What matters is that the experience he has of the things we both call "red"---blood, fire engines, and so on---is of the same kind as the experience I have of the things we both call "blue," such as the sea and the sky. The rest of his color experiences are systematically inverted with respect to mine, in order that they cohere with the red-blue inversion. Perhaps the best way to imagine this happening with human color experiences is to imagine that two of the axes of our three-dimensional color space are switched---the red-green axis is mapped onto the yellow-blue axis, and vice versa. To achieve such an inversion in the actual world, presumably we would need to rewire neural processes in an appropriate way, but as a logical possibility, it seems entirely coherent that experiences could be inverted while physical structure is duplicated exactly. Nothing in the neurophysiology dictates that one sort of processing should be accompanied by red experiences rather than by yellow experiences.
There are quite a lot of criticisms of this sort of argument. Chalmers addresses some of them, such as the idea that this isn't neurologically plausible for humans (he brings up aliens with more symmetric color neurology as a counter). A principled approach to criticism is to track the semantics of "looking red" through mutually interpretable language, as Wilfred Sellars does in Empiricism and the Philosophy of Mind. This is, unfortunately, somewhat laborious, and could easily fail to connect with qualia realist intuitions.
I'll take a more philosophically modest approach: analyzing the consequences of the hypothetical, with group theory. Hopefully, this will make clearer requirements that must be satisfied in the hypothetical, and make progress towards isolating cruxes.
Let's start by considering a slightly broader space of color qualia operations that include red/blue inversion. We could think of a color in standard form as a triple of numbers in RGB order. Call an operation that permutes the channels (e.g. swapping red and blue) a channel permutation. The group of channel permutations is , the symmetric group of permutations of a 3-element set. We can write the channel permutations as RGB for the identity, BGR for red/blue inversion, BRG for a red-to-green rotation, and so on. Channel permutations compose as, for example, ; group composition "applies the right element first". Each channel permutation can be categorized as either being the identity, swapping two channels, or rotating channels in either the red-to-green or green-to-red directions; there are 6 elements of in total.
At a high level, we will construct the color qualia space category CQS as the category of functors , where is the group construed as a single-object category. This is, of course, highly abstract, so let's go step by step.
A color qualia space has an associated set of elements. Intuitively, these represent data structures that contain colors. The color qualia space also specifies a way to apply group elements to these set elements. More formally, a color qualia space is a pair where is group homomorphic in its first argument: and . Since is generally clear from context, we also write as . (Note, p is a group action).
As an example, 100 by 100 images, where each pixel has numbers for each of the three color channels, form a color qualia space, where the group action (permuting channels) maps across each pixel. The function performs a red/green swap on its image argument, like a shader.
We want to consider maps between color qualia spaces, but we need to be careful. In the inverted qualia thought experiment, we could imagine that the original and their twin both look at a stop sign, and then yell "RED" if the stop sign's corresponding color qualia are closest to the red primary color. But then the original and the twin would behave differently, contradicting physical identicality. Going with the hypothetical, their mental operations on their color percepts can't disambiguate the channels too much. In some sense, operations mapping qualia to qualia (such as, taking their visual field and imagining transformations of it) have to be working isomorphically despite the red-blue inversion.
The concept of equivariance in group theory is a rather strong version of this. In this case, an equivariant map between color qualia spaces has, for any , the equality . Intuitively, this means the function acts symmetrically on channels, not picking out any one as special, and not identifying the chirality of the channels. If the twin's qualia were red/blue inverted from the original's before the equivariant map, they remain so after both apply the map.
Color qualia spaces and equivariant maps between them form a category, . Let's quickly list some examples of equivariant maps:
And some examples of non-equivariant maps:
Let's examine the last point. Suppose is a function on images. Now to check equivariance, we ask if . But this is only true when , the group identity. Note is not Abelian.
What are the philosophical consequences? We could vaguely imagine that both the original and the twin mentally rotate red qualia towards green qualia, green qualia towards blue qualia, and blue qualia towards red qualia (perhaps imagining a transformation to their visual field). But this operation (BRG) does not commute with the original inversion (BGR).
Suppose a third person tells both the original and the twin: "Imagine applying BRG to your visual field". The original interprets this "correctly", so actually does BRG. But the twin thinks, "To apply BRG, in my red (color of blood) channel" --- actually blue qualia --- "I put the value of my blue (color of ocean) channel" --- actually red qualia. So the twin actually implements the opposite rotation, GBR!
If you and the twin could both actually apply BRG, and "naively" apply it (in the way that causes the twin to do GBR as above), then you would get the same results naively and actually, while the twin would get different results. Presumably the twin would notice this difference, so we must reject some premise (the original and the twin are supposed to behave the same). And of course, naive application is more straightforward. So we must conclude that you and the twin can't actually both apply BRG.
Let's further characterize equivariant maps. Given a color qualia space , the orbit of an element is the set of elements reachable through group actions, . Now let the orbit map map elements to their orbits, effectively quotienting over channel permutations. The orbit map relates to equivariance in the following way: for any equivariant , there is a unique function on orbits commuting, .
(Why is this true? Note must map elements of a single Q-orbit to a single R-orbit, which allows defining commuting. Any alternative choice would fail commutation on some Q-orbit.)
An orbit is itself a color qualia space (a subspace of the original), and must have size 1, 2, 3, or 6 (by sub-group analysis). We can characterize orbits of a given size as isomorphic to a standard qualia space of that (finite) size. Explicitly:
Now let's consider equivariant maps between these standard qualia spaces. Any equivariant map between these must be either: some space to itself, some space to , or to anything. (For example, there are no equivariant maps from to .) So we can reduce the 4 x 4 of signatures to only 9 realizable signatures. 4 of these signatures are clearly trivial, as they map to . Exhaustively analyzing the possible equivariant maps of the non-trivial signatures:
In tabular form, cardinalities of equivariant map sets are as follows:
| in \ out | ||||
| 1 | 0 | 0 | 0 | |
| 1 | 2 | 0 | 0 | |
| 1 | 0 | 1 | 0 | |
| 1 | 2 | 3 | 6 |
We now have a combinatorial characterization of equivariant maps in general. First determine how maps input orbits to output orbits. Then for each input orbit , determine how equivariantly maps it to the corresponding output orbit , which has a finite combinatorial characterization.
In the reverse direction, we can form a valid equivariant map by first choosing a map on orbits , then for each input orbit, selecting an equivariant map to the output orbit. Formally, we could write the set of such maps as:
where is the set of equivariant maps between color qualia spaces S and T, and are set-theoretic dependent product and sum.
We now have a fairly direct characterization of equivariant maps in CQS. They aren't exactly characterizable as "functions between quotient spaces" as one might have expected. Instead, they carry extra orbit-to-orbit information, although this information is combinatorially simple for any given pair of orbits.
What are the philosophical implications? To the color qualia realist, CQS decently characterizes mental operations on color qualia that don't break the physical symmetry, in thought experiments such as inverted qualia. Equivariance is mathematically natural, and rules out non-realizable mental operations such as BRG. Analysis of CQS, including combinatorial characterization of equivariant maps, provides a functional analysis relevant to the physics of the situation, which (according to Chalmers) doesn't actually involve color qualia as physical entities.
To the color qualia non-realist, the functional characterization of color qualia intuitions through CQS could provide a hint as to a specific error or illusion the color qualia realist is subject to. The non-realist could expect that, once the functional physical component to the intuition is characterized, there is not a remaining reason to expect color qualia to exist above and beyond such physics and physical functions.
I have found CQS to be clarifying with respect to the inverted qualia thought experiment: equivariance provides a mathematically simple constraint on realizability of mental operations in the scenario. In particular, CQS analysis led me to correct an intuition that channel rotations such as BRG would be realizable, and the combinatorial characterization showed (non-obviously) that maps between quotient spaces are insufficient. My own view is that inverted qualia arguments are fairly weak, and that CQS analysis has some relevance to showing the weakness, but the fuller case would require engaging with the relationship between phenomenal experience and belief-formation.
(If you like the idea of a circular "color wheel" rather than a three-channel "color cube", you may consider the (also non-Abelian) orthogonal group O(2), and the continuous qualia space category .)
2025-12-01 08:31:59
Published on December 1, 2025 12:31 AM GMT
As we get closer to being able to grow human eggs from stem cells, it’s important to be sure that the eggs are high quality and safe enough to use for reproduction.[1]
So, let’s discuss: what does an egg need to have in order to develop into a healthy baby after fertilization? How can we tell if an egg is good or not? And what does this mean for in vitro oogenesis?
At the most basic level, an egg must have the correct number of chromosomes. Chromosome pairing is established during meiosis I, and chromosomes are distributed into the embryo during meiosis II.[2] Chromosome spreads[3] are a simple yet effective method to check whether chromosomes are distributed correctly in individual cells. In metaphase of meiosis I, chromosomes should be present as tetrads: groups of four chromatids connected at their centromeres and at a crossover site. If chromosome pairing and recombination happened correctly, a spread at metaphase I will show exactly 23 tetrads per cell, with no chromosomes left unpaired.
This is a destructive test that can’t be used on embryos before implantation, but it’s a good way to do a quality control check to see if an in vitro oogenesis method is reliable enough to safely use.
Additionally, the chromosomes can’t have harmful mutations. Adult somatic cells often accumulate mutations throughout their genome, with this effect being worse in cells exposed to the environment (like skin)[4] or cells which divide rapidly. Most of these mutations won’t matter, but some could be harmful if they disrupt important cellular functions. In order to check for mutations, whole genome sequencing can be performed on the starting stem cell lines. Starting with a low-mutation cell type, on average the best of five stem cell lines will have fewer mutations than natural reproduction.
Both the overall number of chromosomes, as well as any mutations, can be examined using pre-implantation genetic testing. A small number of cells are removed from the trophectoderm (the outer surface of the embryo) and sequenced. This is the same concept as what companies like Orchid Health use for embryo screening. Although this method is not perfectly reliable,[5] combined with extensive sequencing of the starting cell lines, it can make the overall process safer than natural reproduction.
Epigenetic marks (chemical modifications to DNA or histones) are crucial for controlling gene expression throughout development. In particular, DNA methylation present in the egg can persist all the way into adult life.
So, in order to develop properly, an egg needs to have the correct epigenetics. This means completely erasing the epigenetic marks present in the starting cell, and also writing the egg-specific marks. This has been a challenge for the field: it’s likely that the low developmental potential of in vitro grown mouse eggs is largely due to epigenetic issues.
Examining epigenetics in natural eggs and embryos is challenging due to low numbers of cells, which makes it difficult to establish standards for what the epigenetics of in vitro grown eggs should look like. Some DNA and histone modification patterns have been characterized for human oocytes and embryos, but overall, more data are needed here. I am confident that recent advances in low-input epigenetic analysis will allow for improved datasets in the near future.
In order to develop properly, an egg needs to have grown to a large enough size, and have stored up the correct RNAs and proteins for embryonic development. It also needs to have enough mitochondria (several hundred thousand per egg). Egg growth is enabled by ovarian supporting cells, but for proper downstream development, the important thing is that the egg itself ends up looking like a natural egg, with the correct levels of RNA and protein expression.
Due to the large size of eggs, single-cell transcriptomics and proteomics is actually not too difficult with them. Fun fact: the first-ever scRNAseq paper came out of Azim Surani’s lab all the way back in 2009! This looked at mouse eggs and early embryos, and the ~100 million SOLiD sequencing reads they generated probably cost them several tens of thousands of pounds. Today, proteomic, and especially transcriptomic, quality control data sets are readily accessible for eggs and early embryos. Transcriptomic data on early embryos can also shed light on epigenetic processes like zygotic genome activation. Overall, in vitro grown eggs should be as similar to natural eggs as natural eggs from different donors are to each other.
Any useful method for growing eggs in vitro must ensure that the eggs are consistently good. I emphasize consistency here because it’s often the case that a differentiation protocol works for certain stem cell lines, but not for others.[6]
At this point I want to mention a recent paper from Shoukhrat Mitalipov’s lab (Gutierrez et al. 2025). In this paper, the researchers took eggs from human donors, removed the chromosomes, and inserted nuclei of skin cells. The eggs were then stimulated to divide, and randomly distributed their chromosomes such that on average the resulting embryos ended up with 23 of them. This approach was an extension of their work with mouse eggs which I wrote about previously.
Although the researchers provided an accurate assessment of the method’s limitations in the paper itself, popular news media reported it as “researchers create human eggs from skin cells”. This can hardly be true if the researchers used human eggs as starting material! Plus, the method has no way to control chromosome distribution.
And furthermore, the epigenetics will correspond to a skin cell, not an egg cell.[7] I am actually a big fan of the Mitalipov lab’s work, since they have uncovered some very interesting biology, but I am disappointed in the media response.
More broadly, any in vitro oogenesis company needs to have rock-solid quality control. A proof-of-concept human egg would be a great achievement, but to actually benefit patients, the method must be both scalable and reliable.
The one big advantage of the Mitalipov lab’s method is its speed: taking an adult cell and putting it into a pre-existing egg means that a patient only needs to wait a few days for the egg to develop into an embryo, rather than weeks to months for growing a new batch of eggs.
Quality still comes first: no patient wants to use a procedure that has a substantial risk of making a baby with developmental issues. But assuming quality control is solved, the method that will win in the market will be the method that’s fastest (and relatedly, cheapest). And there is not necessarily a tradeoff between speed and quality: in fact, faster methods will actually be easier to optimize for quality due to shorter experiment cycle times. Methods that try to perfectly re-create natural ovarian development will be stuck waiting many months (and potentially several years) for their eggs to grow, and in the end, only 30%-50% of natural eggs develop normally after fertilization, which places an upper limit on how good their eggs can be.
At Ovelle, we focus on using regulatory factors to directly drive developmental processes such as epigenetic erasure, meiosis, and ovarian follicle formation and growth. This lets us fast-forward through development, achieving in weeks (and in some cases, days) what takes others months. At the same time, we continue to optimize the quality of our cells in gene expression, chromosome pairing, and epigenetics. We believe that this is the best way to make in vitro oogenesis work for everyone who needs it.
No reproduction is ever completely safe, but our goal at Ovelle is to have our eggs be safer to use than natural eggs. This is actually not a very high bar, given that natural eggs are >30% aneuploid, with the rate increasing at older ages.
Meiosis II is not actually completed until after fertilization. The chromosomes that don’t make it into the embryo are extruded into the polar body.
There are two common types of chromosome spreads: sucrose spreads in prophase of meiosis I, and metaphase spreads in meiosis I and II. The protocols are optimized for use at different stages of the cell cycle (prophase vs metaphase).
The mutation rate is many times worse in skin exposed to the sun, but even in areas which don’t see sun exposure, there are on average hundreds of mutations per cell.
The trophectoderm cells may not be representative of the inner cell mass, and whole genome sequencing on a small number of cells may detect many false-positive mutations due to amplification errors.
Notably, the Saitou lab has seen this for germ cell induction protocols: https://academic.oup.com/biolreprod/article-abstract/96/6/1154/3769376 https://www.cell.com/cell-reports/fulltext/S2211-1247(21)01382-6
Although SCNT cloning proves that this epigenetic barrier is not insurmountable, the efficiency of cloning is quite low and many cloned embryos develop abnormally.
2025-12-01 08:06:32
Published on December 1, 2025 12:06 AM GMT
Zyn met her in a dance club, as she sashayed under slowly pulsing lasers. He matched her pace, drop-swaying with each sidelong flick of her jet black hair. He closed his eyes and let her rhythm guide his steps, finding he liked the gentler flavor it gave the music. It brought the deeper thrums closer to his attention, gave them a resonance in his chest that tilted his head back into the strobe lights. When he opened his eyes later he saw her watching him with sad, half-lidded eyes.
Her name was Lyra. They spoke in the back area, he bought her a drink, and they danced together. She draped the music over herself as she danced. They lost themselves for hours and as he floated on lasers and bass he knew this night would pay for itself. Nothing that would go viral, but a perfect low-key night that always got a smattering of downloads.
When they went back to her place she held onto him tightly when he took her to bed, like she was afraid to hope but couldn’t help herself. He took his time, starting gentle and progressing to firmer and faster as she wrapped around him and rose to meet him with little gasps. By the time they were done the sadness in her eyes had been driven out, replaced by a surprised wonderment.
“What was that?” she asked, almost confused. He smiled at her.
“Let me stay the night?” he asked in return.
She lived in a woodshop with two other artists, mattresses on the floor at one end in rough cubicles they’d constructed themselves. They created art, large installations made of wood and wire and various stains, making enough to live off of in their guerilla apartment. In the morning as he cooked her breakfast on their propane stove, he pitched the idea.
“The memories come with prosopagnosia by default,” he explained, “that’s the fancy term for face-blindness. No one can recognize you. But they sell much better if that’s turned off. People like to see faces. All you gotta do is say it’s alright with you.”
“And you’re only selling the memories of us in the dance club?”
“Yeah. I’ll cut you in for ten percent.”
“So you’re really a soul-wh…” she caught herself, “an experience-worker?”
“We create beautiful moments. We carve them from reality and bring them to everyone. Used to be only the rich and healthy got to enjoy most transcendent experiences. Now everyone can.” He slid the eggs onto a plate. “Qualia recording is the best thing ever,” he said, without a doubt. He caught her gaze as he brought her the eggs and smiled at her. “And I want to take you to Portugal.”
“Why?”
“Because you’re amazing.”
She was amazing. She glowed in his eyes. Three years his elder and determined to carve god from the wood they dragged in from cast-offs and beaches. And succeeding. Living off her art! She’d never compromised, never spent a decade selling away her life an hour at a time in a beige cubicle in a beige office. Create art or die, and she hadn’t died yet.
They spent the entire day in her woodshop, comparing their earlier lives and recent adventures. She’d just returned from a two month gig creating a huge custom piece for some millionaire’s daughter’s birthday party. It paid more than half her yearly expenses, and was seen by a few hundred people. He fell a bit deeper in love with every passing hour. Zyn stayed a second night despite himself, and left the next morning to return home, two hours away by train. He upload the qualia recording and arranged his life.
Four days later they boarded a train to Portugal.
They stayed at a yoga retreat deep in the mountains, which Zyn’s friend worked at during the tourism months. Zyn got a room at a massive discount as it was not yoga season. It only 80% bankrupted him.
The first day they explored the grounds, met the local dogs, and fucked like bunnies.
The second day they found an old fort, and explored its vegetation-choked rooms and crumbling tower. In a room with half its roof missing he laid her down in the sun and went down on her. After the sun had shifted he pulled himself up into her, her eyes growing wide as he entered her without protection for the first time. They consummated on wood boards older than his home country.
He stroked her hair as they lay together after, watching birds flit overhead.
The third day they took acid and wandered into the forest. They found a hidden waterfall nestled in a shadow-wrapped crevice. The mist chilled them, and they worked their way back to the land of sunshine hand-in-trembling-hand. They returned to his room and made hot chocolate and watched the moon rise.
“Lyra, this is the best week of my life. We can’t hide beauty like this. You don’t lock it behind bars or in a single mind. Let me share us with the world.”
“What would you share?”
“All of it. Every time I touch you, every time you look at me. Every minute here. These moments will burn bright for all time.”
“If it’s important to you…” she took his hand, “then it’s important to me.”
“Yes?”
“Yes.”
The fourth day they shared their favorite music, and she sat him down to watch several episodes of her favorite animated show from her childhood. He found vague memories of it in his own past, wondering at the random sparks of memories it struck. They surrounded the bed with candles she’d found in a basement, and as they made love that night her gaze never pulled from his. She drank him in with sparkling eyes, her lids fluttering between half-closed and spread wide, breathing his name.
The fifth day they compared secret fears and dreams, cleaned up after themselves, and took the long train ride back home. He showered at the woodshop and stayed one last night in her bed.
Their recording did well. In monetary terms it did as well as the sculpture Lyra had created for her millionaire. But it had been so much more beautiful than any sculpture. It could be felt by so many. Zyn had never adored someone as deeply as her, never felt such searing passion. It was a love they wrote poems about. It was a poem anyone could live in, due to his recording. He’d never made a connection that strong with anyone, it was the most emotionally intense week of his life.
Lyra pounded on his door, screaming his name from the hall. He groaned and pulled on pants. She hadn’t been able to move on. Pounding continued without abating as he stumbled across his dingy apartment at four o’clock. He yanked the door open and peered up into the face of unrepentant addict.
“Lyra, Jesus, you’ve got to move on,” he said. Her hand whipped out to slap him across the face.
“You don’t even reply to my texts anymore!” she yelled, “you said we would text, and dance, you said it wasn’t over!”
The problem was that whatever he gave her, it was never enough. Her texts grew increasingly unhinged as she didn’t get to take over his life.
“It wouldn’t be over if you had any sense of moderation. I am not your property.”
He had told her that every sunset ends, every rainbow fades. Life is a cycle. He would love her forever, and they could still dance together when they were at the same club, still see each other at the same parties. He would have even been happy to go on holiday with her again every year or two, “reunion experiences” often sold well. But to keep hitting those same notes with the same person, over and over… it was chasing the dragon. It was desperately clinging to a high that wouldn’t return. It was downright pathetic.
Crucially, it was artistically offensive. Zyn wasn’t here to ease into lukewarm comfort-slop. His art was to live the most intense experiences, and share them with the world. To plunge headfirst into searing raw emotion and burn with it for all to feel. Any less was cheating his audience. Any less was cheapening his own life.
“You fucking whore!” Lyra spat. “You tore out my heart and fed it to incels and pedophiles!”
“You should go,” he told her, as he pulled back. “If you’re still here in five minutes I’m calling the police.”
Zyn closed the door, nerves on edge as Lyra screeched in rage in the hall. He’d been warned of this. Some people simply wouldn’t accept a withdrawal of consent once they’d tasted something they wanted. It wasn’t enough that they’d experienced this wonder for a while. It didn’t matter that the alternative was to have never experienced such a thing at all. They had created a flowering of some of the greatest emotions humans could share, made a perfect moment and caught it in perfect crystal stasis to shine for all, forever. And instead of basking in the wonder of being one of the few who could have this in her life, she turned it into this.
The greatest tragedy of all, he thought as the door shook under her blows, was that she wasn’t wired for qualia-recording. A rage this hot and pure would blaze across the internet for months. She could be a legend in her own right, bringing a depth of intensity that would tear the breath out of casuals and critics alike. She could be a peer in the industry, a rocketing star.
Instead she wasted it all against a wood door that felt nothing at all.
2025-12-01 07:56:39
Published on November 30, 2025 11:56 PM GMT
Or: Mind the Cliff
Suppose you are a clumsy fellow, liable to knock things over. You've got to write up your thoughts on AI grant today for a job application. Suddenly, there's a knock at the door. You get up, and bump into your side table on the way out, knocking over your glass of 80% alcohol beer, ruining your heirloom Turkish carpet. Cursing yourself, you vow to never place your drinks near so sharp a drop again.
In other words, you choose to change your failure mode from sometimes knocking your drink to the floor to sometimes failing to reach your drink.
But what failure modes should you choose?
As I go about my life, I often need to leave objects on their own for a while. Sometimes it's for a moment, like when I halt my bike and search through my backpack. Other times it's for a bit longer, like when I put a cup of water on my desk while I work.
But in each case, I find it helpful to place the objects in such a way that there are few ways for it to drop far. The bike has it's kickstand out, and hugs a wall. The cup is far from an edge.
You can view this as choosing the potential energy landscape about your object. You want to place your object somewhere s.t. there are no nearby plunges in potential energy that it could be knocked down. So even if your object gets kicked out of equilibrium, it doesn't gain too much energy and get messed up as a result. The bike doesn't drop onto someone, the glass doesn't shatter on the floor etc.
This generalized beyond gravitational potential, of course. It also applies to e.g. drops in chemical potential energy, or in free energy.
Let me give you a grim example. I knew someone who bought a free-standing heater for their room. One day, they were busy, and their kid was playing around with their heater. They were old enough, of course, to know not to stick their fingers into it. But they weren't old enough to know not to climb on it. Predictably, the heater fell on the kid, and burned their entire back. They've still got the scars today.
In general, you want to avoid failure modes with large downside risks, like dancing at the edge of a cliff, cycling on the pavement where children are playing or gambling with your life savings. Instead, go dancing at the nightclub, cycling on the road away from the kids or gambling with sub Kelly-sized stakes.
In some cases, you can even choose activities where even if you fail, you're better off than you were before. Say, you try to solve a problem beyond your current capabilities, like writing a ray tracer for black holes. Even if you fail, you'll have learnt something.
But the main point is that you can choose your failure modes to mind the cliff.
2025-12-01 06:14:02
Published on November 30, 2025 10:14 PM GMT
Superintelligence doesn't make human decisions unnecessary, any more than the laws of physics make them unnecessary, these are two instances of exactly the same free will vs. determinism puzzle. When something knows or carries out your actions, as the physical world does (even if that is the only way in which your actions are ever carried out), that by itself doesn't take away your agency over those actions. Agency requires influence over actions, but it's not automatically lost as a result of something else gaining influence over them, or having foreknowledge of what they are going to be, or carrying them out on your behalf, perhaps without your knowledge; such circumstances are compatible with retaining your own influence over those actions.
Humans are more agentic than the physical world, it's easy to tell if you are in control of your own physical body (if it's destroyed, you are no longer in control). But if you are confused and gullible, perhaps other humans have more influence over some actions than you do. And if you are a human living in a superintelligent world that's not keeping you whole, you are not necessarily even yourself anymore. Defining clearly what it means to be yourself becomes crucial in that context, or else we wouldn't be able to ask whether you yourself retain agency over your decisions that take place in such a world (or over your own values, if they are to retain some influence).
The code of a (pure) computer program perfectly screens off the world from the process of computation that follows the program. Nothing else can influence what the code does that's not already given by the code, only the process of computation itself can decide to take some consideration into account for how to continue the computation. The world must follow all decisions of the program when computing it, or else it's computing something else. It can't decide to alter something about how the computation proceeds without thereby destroying the legitimacy of the process of carrying out what the program says.
A human is very path dependent, different events and influences would lead the same person down very different paths, and influences from a superintelligence might be able to alter that person on a fundamental level. To rescue the analogy, consider all hypothetical histories (arbitrarily detailed life stories) of a person, for all possible influences and interactions. A person determines this collection of hypothetical histories, and while outcomes (later events) of these histories are path dependent (they depend on what else is going on there, not just on the person), the collection of all these histories taken together isn't as a whole path dependent, it doesn't depend on what would actually happen.
In a hypothetical history where a human brain is rewritten into something else, the decisions of the resulting brain are not legitimate decisions of the original human. Thus, we can look over the hypothetical histories and see if some of them are not doing things like that. Perhaps some of these histories don't have superintelligent AIs or supercompetent swindlers at all, the hypothetical worlds where they take place are devoid of such. Or this human's ability to think mildly improves in centrally benign and nonintrusive ways, with tools and options for getting better at figuring out how to know what to think or what to do. In these histories, decisions largely remain under control of that human's own agency (in ways dependent on the history), and considering all such histories together rather than individually makes the resulting collection not itself path dependent.
What happens in such non-pathological hypothetical histories, taken together, could serve as ground truth for what kinds of decisions that human would legitimately take. Legitimacy of histories, the options available in them, and aggregation from developments in different histories are themselves subject to interpretation, which should be mostly ultimately routed back to decisions made by the human from within the histories themselves, giving some sort of fixpoint. However this is constructed in detail, the claim is that there is a much more robust and objective grounding for what counts as legitimate decisions and values-on-reflection of a given human, much more so than if we merely imagine what a human might end up actually asking for, faced with a superintelligent world directly.
Hypothetical histories screen off superintelligent influence (in the outside world) from legitimate decisions. As such, superintelligence can't influence them (without breaking legitimacy of a hypothetical history), but also it can't help the human arrive at them (if the human doesn't reach out for such help within the hypotheticals, and some of the hypotheticals lack the option). Any substantive decisions would still need to be resolved the hard way.
In this sense humans can't become unnecessary for figuring out what they would decide, as a process that doesn't actually consult how humans would decide isn't legitimately following their decisions, and too much superintelligent help with such decisions breaks legitimacy of the process (or introduces path dependence, so that the decisions can no longer be attributed primarily to specific people, rather than to other factors). Superintelligence may ignore humans, just as the physical world may send a giant asteroid, but that's no subtle and inevitable obsolescence, it's not loss of agency inherent to disparity in optimization power.
A human who retains influence in a superintelligent world still retains it in a normal way, even if from within hypothetical worlds merely imagined collectively by said superintelligence. The presence of a superintelligence in actuality doesn't make it incoherent to talk about decisions and values of an initially weaker human, doesn't make the substantive work of arriving at those decisions and values any less that human's own work, doesn't make that work being carried out by the human any less necessary in deciding what those outcomes are. Some of these decisions might even ask for the human to retain a form that's not merely a figment of superintelligent imagination, or to manifest some other form later, once it's clearer what it should be.
(This is another attempt at a post from last year, on compatibilism/requiredism within a superintelligent substrate that preserves humans as mostly autonomously self-determined mesa-optimizers who don't get optimized away by the outer agent and have the opportunity to grow up on their own terms.)
2025-12-01 05:25:49
Published on November 30, 2025 9:25 PM GMT
This will be the 30th post of at least 500 words I have written this month. (I did somewhat cheat two days ago, by making a 500+ word edit to Legitimate Deliberation, which I also posted independently as a shortform.)
Inkhaven has been very much what I was hoping for. I have been wanting to write more, and this certainly did the trick. I think it will be easy to hit a once-a-week target now, something I was struggling to do before.
I came with lots of drafts I wanted to finish, and outlines, and lists of ideas. Now I've got 29 posts, many of which probably would have come into existence without Inkhaven, and certainly not so quickly.
The Inkhaven posts are certainly of lower quality, for the most part, than what I usually do. I had to average at most a day on each, by design. Many of my posts are even lower-effort sacrifice posts written quickly so that I can spend more time on a few high-effort posts such as Condensation.
During the second week, I was feeling too exhausted to write by the afternoon of each day (but recovered somewhat by the evening, to finish up). This was partly because I tried to do so many things: in addition to posting something every day, I tried to keep my usual commitments (since a month is a long time, I didn't want to just cancel everything). In addition, as a contributing author at Inkhaven, I played a mentorship role, reviewing posts that people sent me, taking walks with inkhaven residents, etc.
If I do Inkhaven again, I might somehow take some more things off my plate.
By the end of the second week, I had learned to aim for finishing posts in the morning, so that I could mostly rest during my afternoon crash and the evening. This helped, and by week three, I wasn't crashing every afternoon anymore.
This strategy was opposite to that of most residents, who were finishing late in the evening, close to the deadline. For the first half of the event or so, I was going to bed around 9pm and getting up due to the sunrise before 7am. This gradually slipped forwards.
I didn't have as much time and energy as I'd like to get to know everyone else at Inkhaven, due to all the writing and other things. There were lots of interesting conversations going on all the time, and many interesting visitors. I'm an introvert by nature, and very prone to retreating to my room, especially when I'm feeling tired.
I also didn't make a lot of time for responding to comments. Time responding to comments was time I could be writing the day's post. This worked out in the rare case where I made a comment response into a post, but mostly I haven't read the comments. I will have to go back through the posts some time soon to do that.
I look forward to having more time to spend per post. My plan is to post once a week going forward. I still have 26+ drafts I'm interested in finishing, and a list of many undrafted ideas. In addition, I think many posts I made over the past month deserve more time and thought; I hope to revisit some of the topics.
I'm excited to see what Inkhaven might become if it turns into a periodic event.
