An online forum and community dedicated to improving human reasoning and decision-making.
The RSS's url is : https://www.lesswrong.com/feed.xml
Please copy to your reader or subscribe it with :
2024-11-21 11:11:56
Published on November 21, 2024 2:08 AM GMT
A short story by Raymond M. Smullyan, written in 1982.
...(A few weeks later.) Frank is in a laboratory in the home of an experimental epistemologist. (You will soon find out what that means!) The epistemologist holds up a book and also asks, "What color is this book?" Now, Frank has been earlier dismissed by the eye doctor as "cured." However, he is now of a very analytical and cautious temperament, and will not make any statement that can possibly be refuted. So Frank answers, "It seems red to me."
Epistemologist:
Wrong!Frank:
I don't think you heard what I said. I merely said that it seems red to me.Epistemologist:
I heard you, and you were wrong.Frank:
Let me get this clear; did you mean that I was wrong that this book is red, or that I was wrong that it seems red to me?Epistemologist:
I obviously couldn't have meant that you were wrong in that it is red, since you did not say that it is red. All you said was that it seems red to you, and it is this statement which is wrong.Frank:
But you can't say that the statement "It seems red to me" is wrong.Epistemologist:
If I can't say it, how come I did?Frank:
I mean you can't mean it.Epistemologist:
Why not?Frank:
But surely I know what color the book seems to me!Epistemologist:
Again you are wrong.Frank:
But nobody knows better than I how things seem to me.Epistemologist:
I am sorry, but again you are wrong.Frank:
But who knows better than I?Epistemologist:
I do.Frank:
But how could you have access to my private mental states?Epistemologist:
Private mental states! Metaphysical hogwash! Look, I am a practical epistemologist. Metaphysical problems about "mind" versus "matter" arise only from epistemological confusions. Epistemology is the true foundation of philosophy. But the trouble with all past epistemologists is that they have been using wholly theoretical methods, and much of their discussion degenerates into mere word games. While other epistemologists have been solemnly arguing such questions as whether a man can be wrong when he asserts that he believes such and such, I have discovered how to settle such questions experimentally.Frank:
How could you possibly decide such things empirically?Epistemologist:
By reading a person's thoughts directly.Frank:
You mean you are telepathic?Epistemologist:
Of course not. I simply did the one obvious thing which should be done, viz. I have constructed a brain-reading machine--known technically as a cerebroscope--that is operative right now in this room and is scanning every nerve cell in your brain. I thus can read your every sensation and thought, and it is a simple objective truth that this book does not seem red to you...
2024-11-21 08:37:54
Published on November 21, 2024 12:37 AM GMT
Over the last couple of years, I’ve been trying to skill up a lot at resolving community complaints. This is a really irritating field to get good at. When I want to get better at writing code, I can sit down and write more code more or less whenever I feel like it. When I want to get better at guitar, I can sit down with my guitar and practice that D to D7 transition. For complaint resolution, even finding people to roleplay the skill with takes a little setup, and that’s a lot less like the real thing than you can get with code or guitar. For real complaints, they come in uneven batches, and worst, you seldom get confirmation if you did it successfully. So, like any good Aspiring Ravenclaw Rationalist, I read a lot of essays about conflict.
Taking a Safety Report, Taking Someone Aside, and Outcomes of a Safety Report provide a useful frame and better still useful steps and considerations on what to do when someone comes to you with a concern about an attendee. I keep being tempted to rewrite these in a numbered list, but that’s a stylistic preference. While the topic is very different, I found A Better Way To Ask for An Apology to be similarly easier to turn into concrete steps, and it feels like it has a useful overlap with the previous three. To have a good apology, you need to be specific. If you take only one sentence away from this links post, I want it to be “It is also important to clearly distinguish between things you've observed, things you've heard secondhand, and your interpretations of your observations.”
When to Ban Someone is a discussion of the decision to ban people from events. It has a fairly specific stance on where these decisions should come from, but wears that on its sleeve in a way I appreciate. Decentralized Exclusion goes over the situation of distributed communities banning people, even without formal structures. I like the way it looks at the constraints of such communities, but I’m not sure how I feel about the reliance on singular public statements. It puts a lot of onus on that statement to get things right, and it isn’t clear to me how a true statement works better than a false statement. (See Guided By The Beauty Of Our Weapons- a big statement is asymmetric, but not very asymmetric.) They're both a bit prescriptive though (When to Ban moreso than Decentralized Exclusion) whereas Whisper Networks, Callout Posts, and Expulsion: Three Imprefect Ways Of Dealing With Abuse on the other hand feels more descriptive to me. One way bans and exclusion statements are often written is by a panel; I'm a little more bullish on panels than most people I talked to with more experience than I have, but given that my predecessor's writeup on the subject is titled The Impossible Problem of Due Process you should not take that as my ringing endorsement of the strategy.
Concerns with Intentional Insights, Details behind the InIn Document, and Setting Community Norms and Values: A Response to the InIn Document provide a useful three-beat case study of one such exclusion. The Details post is the most unusual, and while there aren’t as many details I appreciate laying out how much work goes into doing this and how it points out that the subject had a couple of opportunities to comment.
There’s also Safety Committee Resources, which is itself a linkspost. The gold mine here for me is the 12 Months of Cases part of The EA Community Health Team’s Work, Tales of a Dance Monarch, and Tales From the Front Line of Ombuds Work. I appreciate these for giving a sense of what comes up and what a ‘normal’ tour as a point person for community safety work is like. While I’m sharing case studies, Takes From Staff At Orgs With Leadership That Went Off The Rails is vaguer and more about formal organizations, but I liked having that perspective in my head too.
Run Posts By Orgs, and Productive Criticism: Running a Draft Past The People You're Criticizing are also written for more formal setups than your average community contact is going to have to deal with. I think it’s useful to import that norm back down to things event organizers do run into. If someone brings a concern or complaint to you about someone else, and wants you to act on it, then you should talk to the subject. The emphatic version of that is On Justice in the Ratsphere.
The person who takes safety reports and who holds the main responsibility is sometimes called a Safety Contact or Community Contact. CEA has a short writeup on what that’s like (EA Groups Resources: Community Contacts) but I also like A Contact Person for the EA Community and BIDA’s Safety Team’s How We Can Help. Most important on both the last two links is that they set the scope of what they do, which if you aren’t careful can balloon very easily into much more than anyone can handle. The coda here is My Life Would Be Much Harder Without The Community Health Team. I Think Yours Might Too. If you remember two sentences from this links post, I want the second sentence to be “And basically no one knows about any of those times they do things well, because why would they?”
I have more in mind to say about conflict, from direct experiences, interviews with people who are good at it, some books I've read, and some armchair theorizing, but this seemed a decent starting place.
2024-11-21 07:50:27
Published on November 20, 2024 11:50 PM GMT
DeepSeek-R1-Lite-Preview was announced today. It's available via chatbot. (Post; translation of Chinese post.)
DeepSeek says it will release the weights and publish a report.
The model appears to be stronger than o1-preview on math, similar on coding, and weaker on other tasks.
DeepSeek is Chinese. I'm not really familiar with the company. I thought Chinese companies were at least a year behind the frontier; now I don't know what to think and hope people do more evals and play with this model. Chinese companies tend to game benchmarks more than the frontier Western companies, but I think DeepSeek hasn't gamed benchmarks much historically.
The post also shows inference-time scaling, like o1:
Note that o1 is substantially stronger than o1-preview; see the o1 post:
(Parts of this post and some of my comments are stolen from various people-who-are-not-me.)
2024-11-21 07:28:21
Published on November 20, 2024 11:28 PM GMT
In Scott Garrabrant's excellent Geometric Rationality sequence, he points out an equivalence between modelling an agent as
And as we'll show in this post, not only can we prove a geometric version of the VNM utility theorem:
Which in and of itself is a cool equivalence result, that maximization VNM rationality maximization. But it turns out these are just two out of a huge family of expectations we can use, like the harmonic expectation , which each have their own version of the VNM utility theorem. We can model agents as maximizing expected utility, geometric utility, harmonic utility, or whatever representation is most natural for the problem at hand.
The VNM utility theorem is that an agent satisfies the VNM axioms if and only if there exists a utility function which:
Where is a probability distribution over outcomes .
The first property is easy to preserve. Given any strictly increasing function ,
if and only if
So also represents our agent's preferences. But it's only affine transformations that preserve the second property that . And it's only increasing affine transformations that preserve both properties at once.
But what if we were interested in other ways of aggregating utilities? One of the central points of Scott's Geometric Rationality sequence is that in many cases, the geometric expectation is a more natural way to aggregate utility values into a single representative number .
We can represent the same preferences using the expected logarithm , but this can feel arbitrary, and having to take a logarithm is a hint that these quantities are most naturally combined by multiplying them together. The expectation operator can emulate a weighted product, but the geometric expectation operator is a weighted product, and we can model an agent as maximizing without ever bringing into the picture.
And as scottviteri asks:
If arithmetic and geometric means are so good, why not the harmonic mean? https://en.wikipedia.org/wiki/Pythagorean_means. What would a "harmonic rationality" look like?
They also link to a very useful concept I'd never seen before: the power mean. Which generalizes many different types of average into one family, parameterized by a power . Set and you've got the arithmetic mean . Set and you've got the geometric mean . And if you set you've got the Harmonic mean .
It's great! And I started to see if I could generalize my result to other values of . What is the equivalent of for ? Well, scottviteri set me down the path towards learning about an even broader generalization of the idea of a mean, which captures the power mean as a special case: the quasi-arithmetic mean or -mean, since it's parameterized by a function .
For our baseline definition, will be a continuous, strictly increasing function that maps an interval of the real numbers to the real numbers . We're also going to be interested in a weighted average, and in particular a probability weighted average over outcomes . We'll use the notation to denote sampling from the probability distribution .
Here's the definition of the -expectation :
Which for finite sets of outcomes looks like:
So for example:
An -utility function represents an agent's preferences:
if and only if
And agrees with the -expectation of
It turns out that for every -utility function , there is a corresponding expected utility function , and vice versa. We'll prove this more rigorously in the next sections, but it turns out that these are equivalent ways of representing the same preferences.
Here's the core insight that powers the rest of this equivalence result, which Scott articulates here:
Maximization is invariant under applying a monotonic function... So every time we maximize an expectation of a logarithm, this was equivalent to just maximizing the geometric expectation.
If we think of an agent as maximizing over , something under their control like their action or policy, we can write this as:
So for every geometric utility function , there is a corresponding expected utility function which gives the same result if maximized.
This equivalence can be generalized to all -utility functions, and it follows from exactly the same reasoning. We have a strictly increasing function , and so must be strictly increasing as well.
And so will ignore either one. Let's use that to simplify the expression for maximizing the -expectation of :
And this suggests a substitution that will turn out to be extremely useful:
There is a function whose expectation we can maximize, and this is equivalent to maximizing the -expectation of our -utility function . And we'll show that is indeed an expected utility function! Similarly, we can apply to both sides to get a suggestion (which turns out to work) for how to turn an expected utility function into an equivalent -utility function.
It turns out that for every expected utility function , there is a corresponding -utility function , and vice versa. And this duality is given by and .
We'll start by showing how to go from to . Given an expected utility function , we'll define to be
We know from the VNM expected utility theorem that
Let's plug both of those into the definition of
So agrees with . And since is strictly increasing, and represents an agent's preferences, so does .
if and only if
Which means is an -utility function!
This gives us one half of the VNM theorem for -utility functions. If an agent is VNM rational, their preferences can be represented using an -utility function .
We can complete the duality by going the other way, starting from an -utility function and showing there is a unique corresponding expected utility function . We'll define to be:
And we'll plug that and the fact that into the definition of .
And that's it! Starting from an -utility function , we can apply a strictly increasing function to get an expected utility function which represents the same preferences and agrees with .
This also gives us the other half of the VNM theorem for -utility functions. If an agent's preferences can be represented using an -utility function , they can be represented using an expected utility function , and that agent must therefore be VNM-rational.
So for every -utility function , we can apply and get an expected utility function . And the same is true in reverse when applying . Does this translation process have any collisions in either direction? Are there multiple -utility functions and that correspond to the same expected utility function , or vice versa?
It turns out that creates a one-to-one correspondence between -utility functions and expected utility functions. And a consequence of that is that all of these languages are equally expressive: there are no preferences we can model using an -utility function that we can't model using an expected utility function, and vice versa.
Another way to frame this duality is to say that our translation function is a bijection between its domain and its image . And this induces a structure-preserving bijection between utility functions and -utility functions .
To show this, we can show that is injective and surjective between these sets of utility functions.
An injective function, also known as a one-to-one function, maps distinct elements in its domain to distinct elements in its codomain. In other words, injective functions don't have any collisions. So in this case, we want to show that given two distinct -utility functions and , and must also be distinct.
Since and are distinct -utility functions, they must disagree about some input .
And since is strictly increasing, it can't map these different values in to the same value in .
And thus must be a distinct function from .
A surjective function maps every element in its domain onto an element of its codomain, and these functions are also called "onto" functions. So in this case, we want to show that given an expected utility function , there is an -utility function such that . And this is exactly the -utility function that picks out.
And that's it! induces a one-to-one correspondence between expected utility functions and -utility functions . We can freely translate between these languages and maximization will treat them all equivalently.
I also want to quickly show two facts about how -expectations combine together:
All of this is going to reduce to an expectation of expectations, so let's handle that first. Let's say we have a family of probability distributions and expected utility functions . And then we sample from a probability distribution I'll suspiciously call .
Taking the expectation over of the expectation over is equivalent to taking the expectation over pairs .[1]
This is one way to frame Harsanyi aggregation. Sample an agent according to a probability distribution , then evaluate their expected utility using that agent's beliefs. The Harsanyi score is the expectation of expected utility, and the fact that nested expectations can be collapsed is exactly why aggregating this way satisfies the VNM axioms. The Harsanyi aggregate is VNM rational with respect to the conditional probability distribution .
Knowing that, the general result for -expectations is even easier:
So for example, Scott motivates the idea of Kelly betting as the result of negotiating between different counterfactual versions of the same agent. In that framing, naturally captures "Nash bargaining but weighted by probability." If we geometrically aggregate these geometric expected utilities , the result is the same as one big geometrically aggregate over all counterfactual versions of all agents, weighted by . And the fact that we can model this aggregate as maximizing means that it's VNM-rational as well!
This is a very cool framing, and there are some phenomena that I think are easier to understand this way than using the expected utility lens. But since Geometric Rationality is Not VNM Rational, we know that the model won't have all the cool features we want from a theory of geometric rationality, like actively preferring to randomize our actions in some situations.
With all of that under our belt, we can now reiterate the -Utility Theorem for VNM Agents. Which is that an agent is VNM-rational if and only if there exists a function that:
We can model a VNM-rational agent as maximizing expected utility , geometric expected utility , harmonic expected utility , or any other -expectation that's convenient for analysis. We can translate these into expected utility functions, but we can also work with them in their native language.
We can also think of this equivalence as an impossibility result. We may want to model preferences that violate the VNM axioms, like a group preference for coin flips in cases where that's more fair than guaranteeing any agent their most preferred outcome. And the equivalence of all these representations means that none of them can model such preferences.
One approach that does work is mixing these expectations together. Our best model of geometric rationality to my knowledge is ; the geometric expectation of expected utility. See Geometric Rationality is Not VNM Rational for more details, but the way I'd frame it in this sequence is that the inner expectation means that the set of feasible joint utilities is always convex.
And maximizing the geometric expectation, aka the geometric aggregate, always picks a Pareto optimum, which is unique as long as all agents have positive weight.
Check out the main Geometric Utilitarianism post for more details, but I think of these equivalence results as telling us what we can do while staying within the VNM paradigm, and what it would take to go beyond it.
We went through the proof for discrete probability distributions here, but holds for all probability distributions.
2024-11-21 02:19:25
Published on November 20, 2024 6:19 PM GMT
I am grateful to Noosphere89 and Seth Herd for prompting me to start this discussion.
In human evolution, the fundamental goal was to survive and reproduce, passing on our genes to the next generation. But somewhere along the way, humans developed complex societies and technologies that go beyond just survival and reproduction. Now, we often find ourselves pursuing goals that aren’t always aligned with basic evolutionary drives. In fact, we've become worse at our evolutionary goal than our ancestors (birth rates are at an all time low).
Even if we manage to align our first AGI with human goals, how can we ensure that 1) it doesn't drift from these goals and that 2) it doesn't create an AGI which drifts from these goals (kind of how we drifted from the goals of our ancestors)? What are the current proposals for solving these issues?
2024-11-21 02:17:52
Published on November 20, 2024 6:17 PM GMT
Discuss
2024-11-21 00:48:47
Published on November 20, 2024 4:48 PM GMT
Crosspost of this on my blog.
'Cause everybody cries
Everybody hurts sometimesSometimes everything is wrong
—Rem “Everybody hurts”
You didn’t think the era of absurdly long posts was over, did you?
Prior to the 2024 election, there were a series of swing states that pundits forecasted could have gone either way. All of them ended up going to Trump by a fairly wide margin, as predicted by some people (e.g. Glenn, Keith Rabois). None of the states that were solidly blue went to Trump, but all the states that pundits and prediction markets weren’t sure about went red.
I have a similar bet regarding consciousness.
I suspect that all of the species that are currently of significantly debated consciousness—call them swing state species—are conscious. This would include crabs, lobsters, fish, and most insects, but it wouldn’t include oysters or AI. There’s fairly widespread agreement that such beings aren’t conscious. Oysters are like California.
Furthermore, I suspect that they all suffer fairly intensely—they don’t just have very basic slightly negative experience. This would include fish (though they are reaching the point where their suffering is beyond doubt), crustaceans (like shrimp, crabs, and lobsters), and insects.
Here is a thesis that seems decently intrinsically probable which I’ll refer to as the ubiquitous pain thesis: feeling pain is fairly ubiquitous across the animal kingdom, such that all the species of significantly debated sentience are sentient. I think, aside from the simplicity, plausibility, and naturalness of the thesis, there are five reasons to think it’s right that don’t require reviewing any complex behavioral evidence.
The first reason is evolutionary: it’s beneficial for a fish or insect to feel pain for the same reason it’s beneficial for you to feel pain; it encourages them to avoid dangers. If we hadn’t observed any creatures, we should expect decent odds that many creatures feel pain because of the obvious evolutionary reasons supporting pain.
Second, these creatures act in most ways like they’re in pain. If an insect or shrimp is exposed to damage, they’re struggle with great effort and try to get away. Either they are in pain or they evolved a response that makes them behave like they’re in pain. But if a creature struggles and tries to get away, a natural inference is that they’re in pain. This is why we’d naturally assume that, if we came across large aliens that responded aversively when we cut them, they probably felt pain, even if we hadn’t yet done studies on the nature of their brain.
If you look at how a dog acts when you hurt it, you’re justified in thinking it’s conscious even if you don’t know any neuroscience. For the same reason, we have justification for thinking that pain is widespread in creatures that act like they’re in pain.
Third, the inductive track record has clearly been in the direction of attributing pain to more and more creatures. Before 2002, the belief that fish couldn’t suffer was widespread. Over time, more and more creatures have been recognized as sentient, such that prior to the 1980s, it was widely believed that animals didn’t suffer at all. Thus, there’s a clear inductive trend towards more and more species being recognized as sentient. By induction, we should think we’re likely underestimating the scope of animal consciousness.
Fourth, there’s a probabilistic argument for this conclusion. Suppose one starts with a prior of .5 in consciousness being widespread across beings that act sort of like agents (including fish, insects, reptiles, and so on) and a prior of .5 in consciousness requiring specific brain regions. Conditional on consciousness being widespread, it’s guaranteed that there could be creatures like us that could sustain consciousness, while if consciousness is limited, then it might be that creatures like us can’t be conscious (perhaps only Reptile brains could give rise to consciousness or a kind of very advanced fish brain). Thus, given that we’re conscious, we should update in the direction of thinking more beings are conscious, just like upon seeing you have a bellybutton, ceteris paribus, you get evidence that more humans have bellybuttons (it’s unlikely you’d be the outlier).
Fifth, if one is a dualist, then they should have a preference towards attributing consciousness to fairly simple creatures, just as the other fundamental features of the universe show up at the very basic level. If one isn’t a dualist, then probably consciousness has to do with the right kind of information processing, yet even these simple creatures have brains that process aversive information.
When you combine these five arguments—behavioral, evolutionary, inductive, bias, probabilistic, and theoretical (BEIPT, for short)—they make a pretty strong prior that most of the animals of debated sentience are sentient. The thesis that they’re in defense of is simple and natural—that a creature with a brain that processes stimuli negatively is conscious is a very plausible hypothesis.
Thus, even before looking at the specific experimental evidence, I think we should think it’s decently likely that the species of debated sentience are all conscious (though unlike Eliezer, I don’t come to my conclusions about animal consciousness from the armchair without reviewing any evidence). The most convincing evidence for this thesis that I’ll present is behavioral—I’ll argue that across each debated species, there’s convincing evidence of sentience. But before I do that, I’ll defend the judgment that these animals don’t just feel a small amount of pain, but instead suffer relatively intensely.
Section summary: there are behavioral, evolutionary, inductive, probabilistic, and theoretical reasons to suspect that animals of debate consciousness are conscious. Given that the hypothesis that pain is ubiquitous is simple and parsimonious, these considerations make it decently likely.
The most basic reason to think animals suffer intensely is fairly simple: they act like they suffer intensely. They strain with great force to get away from harmful experiences. So long as we grant that they feel pain, from their behavior, it seems like they feel a lot of it. Note: here I’m not arguing that they do feel pain, just that if they feel pain it’s probably pretty intense. So just judging basically from animals’ behavior, they seem to feel pain relatively intensely. When lobsters are boiled alive, for instance, they struggle so much that chefs often leave the room because it’s so unpleasant to watch.
Suppose we were around in 1970, before much neuroscience on dogs had been done. We observed that when you put a dog in a cage, and then prod it, it strains with great force to try to get away. It would be reasonable to think that they suffer a good deal just because of their behavior. The same is true if we came across very brain-damaged humans—perhaps missing some important features of the brain and many neurons—aliens, or other animals. Yet this should also justify a presumption in favor of thinking animals feel a lot of pain.
The evolutionary considerations present in the last section give reasons to think that animals feel intense pain—being in a dazed, barely-conscious state isn’t evolutionarily advantageous. In fact, there might even be reason to think that animals feel pain more intensely than we do. If a creature isn’t very intelligent, it may need more pain to teach it to do something than if it can cognitively reflect. As Dawkins says:
Widely assumed human pain more painful than other species. But pain’s Darwinian purpose is warning: “Don’t do that again. Next time it might kill you.” So perhaps the more intelligent the species the less pain needed to ram warning home. Animals might suffer more than us.
Additionally, if a creature has a simple mind, it cannot think about anything other than the pain—when it hurts, its entire world would be pain. The worst sorts of pains are those that occupy our entire consciousness—yet for animals, this is probably true of literally every pain! This theme is explored well in Ursula Le Guin’s story Vaster Than Empires And More Slow. Ozy Brennan also depicts it well:
I am not sure if you have ever experienced 10/10 pain—pain so intense that you can’t care about anything other than relieving the pain, that you can’t think, that your entire subjective experience consists only of agony. Pain so intense that you are begging to be allowed to commit suicide because death would be better than enduring one more moment.
I have.
It was probably the worst experience of my life.
And let me tell you: I wasn’t at that moment particularly capable of understanding the future. I had little ability to reflect on my own thoughts and feelings. I certainly wasn’t capable of much abstract reasoning. My score on an IQ test would probably be quite low. My experience wasn’t at all complex. I wasn’t capable of experiencing the pleasure of poetry, or the depth and richness of a years-old friendship, or the elegance of philosophy.
I just hurt.
So I think about what it’s like to be a chicken who grows so fast that his legs are broken for his entire life, or who is placed in a macerator and ground to death, or who is drowned alive in 130 degrees Fahrenheit water. I think about how it compares to being a human who has these experiences. And I’m not sure my theoretical capacity for abstract reasoning affects the experience at all.
When I think what it’s like to be a tortured chicken versus a tortured human—
Well. I think the experience is the same.
Brian Tomasik also makes the point clear, arguing that what might matter to suffering is not how many signals there are aversive but what proportion of a brain’s signals are aversive. Such a thing would favor pain in smaller creatures who spend more of their cognitive architecture on feeling pain. Tomasik writes:
It may be that the amount of neural tissue that responds when I stub my toe is more than the amount that responds when a fruit fly is shocked to death. (I don't know if this is true, but I have almost a million times more neurons than a fruit fly.) However, the toe stubbing is basically trivial to me, but to the fruit fly, the pain that it endures before death comes is the most overwhelming thing in the world. There may be a unified quality to the fruit fly's experience that isn't captured merely by number of neurons, or even a more sophisticated notion of computational horsepower of the brain, although this depends on the degree of parallelism in self-perception.
Tomasik gives an example where 5 “I don’t like this” messages are processed by two individuals, one a human, the other a snail. For the human, these messages are slight in comparison to other things it’s experiencing, but for the snail, these are the only thing it’s experiencing. Intuitively it seems likely that the 5 “I don’t like this” messages would be felt more intensely by the snail, because it’s brain is unified in receiving that pain as its only signal, while for the human, it’s a small part of a larger picture. He then cites Daniel Dewey for the following point:
if the simplicity of the notional world an animal brain/mind/computer has decreases, it seems possible that each computation is more "intense" in that it represents a larger subset of the whole of possible spaces that mind can be in. So a simple pain not-pain system would experience cosmic amounts of pain. Whereas a complex intricate system, like us, would experience pain as a small subset of its processing.
A version of the argument from marginal cases can also be made in favor of thinking animals suffer intensely. Imagine we came across a human with very severe brain damage, missing much of his brain, with many higher functions eroded. However, despite this, he reacted like he was in pain, desperately struggling to get away. We wouldn’t assume that he was only barely conscious. So why do we assume this about animals?
Additionally, if we take the inductive trend discussed in the last several sections seriously—that we persistently underestimate animal sentience—such a thing should bias us in the direction of thinking that more animals are conscious rather than feer.
Finally, the most detailed report ever compiled on the subject of animal sentience concluded that animals feel fairly intense pain. Shrimp, for instance, were estimated to feel pain, on average, 19% as intensely as we do. Furthermore, I think this is likely an underestimate—if one simply looks at the number of proxies that we expect to correlate with pleasure and pain or sentience, rather than (in my view arbitrarily) squaring or cubing things or relying on neuron count, they get an even higher estimate. To be clear, this isn’t a criticism of the report which I think was quite good, especially for being modest and trying to incorporate lots of different ways of measuring sentience—I think my judgments would line up more with the higher estimates, for reasons I’ve described.
The main way people argue that animals don’t feel much pain is by relying on neuron counts (see Anatoly Karlin, for instance). Because animals tend to have many fewer neurons than we do, it’s argued that they don’t feel pain. The problem is that neuron counts are a totally bogus way of measuring sentience, as Rethink Priorities argues convincingly.
In the brain, there’s just no correlation between neuronal activation or brain mass and sentience. No serious neuroscientist has proposed that we can measure the valence of experience by looking at the number of neurons, and often there’s an inverse correlation between neuronal activation and intensity of experience. Psychedelics, for instance, produce more intense experience by shutting down many brain regions. As the Rethink Priorities report notes:
For there also are a large number of studies showing an inverse relationship between brain volume and the intensity of particular affective states. In particular, when it comes to chronic pain, by far the most commonly cited relationship between brain volume and chronic pain is a decrease in brain volume in regions commonly associated with the experience of pain. For example, see the Davis et al. (2008) article “Cortical thinning in IBS: implications for homeostatic, attention, and pain processing”.
These seem to show that what matters for intensity of valanced experience is the functions performed by different brain regions rather than simply whether brain regions exist. But animals have brains that perform similar functions (e.g. fish brains perform similar functions to mammalian brains but in a different way). In addition, as RP notes, while neuroscientists have tended to think that changes in brain regions can detect whether a person is in pain, there are no indications that such things can be used to determine relative pain between different humans. Doctors don’t decide which of two people needs analgesia more by looking at their numbers of active neurons.
Were we to rely on neuron counts, we’d conclude that elephants have experiences much more intense than humans (for they have around 260 billion neurons and much greater brain size and mass). On this picture, despite pigs and elephants behaving mostly similarly, elephants would matter more than 500 times as much as humans. Children tend not to have many more neurons than adults, though they seem to suffer much more intensely.
Such a view also fits poorly with cases involving humans. There are numerous cases of people missing large chunks of their brains without it seeming to affect their experience. In one case, about 90% of a man’s brain was liquid (though it’s hard to know how many neurons were destroyed) yet he still seemed to experience normally. About half of another man’s brain was removed after a deadly accident, removing many of his neurons, yet this didn’t seem to affect his consciousness.
For this reason, we shouldn’t have a strong bias in favor of denying animal consciousness based on neuron counts or brain mass. It’s a totally bogus metric supported by almost nothing and undermined by lots of neuroscientific evidence. While it would be nice if it worked, the fact that it would be nice if P doesn’t, in fact, mean P.
Section summary: Animals probably aren’t just a bit conscious but feel pain intensely. This is supported by considerations involving behavior (they struggle as if they’re in a lot of pain), evolution (feeling only a bit of pain is less adaptive for it produces less significant action), learning (pain serves to teach animals not to do things, but a dumber animal may need a stronger signal to be taught a lesson), totality (for simple creatures, pain make up their total experience), inductive (we’ve consistently underestimates animal pain), marginal cases (we’d assume humans with similar behavior and dysfunction felt a lot of pain), and relative signal strength (for simpler creatures, pain signals make up a bigger part of their cognitive architecture. The pithy acronym is beltimr. The main reason for denying this relies on using neuron counts as a proxy for moral worth, but these are a horrible, terrible, no good, very bad proxy.
Okay, technically fish are a swing species in that there isn’t complete agreement, but there’s a pretty broad consensus. That fish feel pain is recognized by The American Veterinary Medical Association, The Royal Society for the Prevention of Cruelty to Animals, and the RSPCA. The broad scientific consensus is that fish feel pain.
Lynne Sneddon released a report attempting to likelihood of pain in aquatic animals. The result—everything that we’d expect to correlate with feeling pain is exhibited in fish.
(Little sneak peak into the case for crustacean suffering).
One study found that when fish were injected with painful venom on their lips, they behaved differently, rubbing their lips against the side of the tank, using their lips less, swimming less, and rocking back and forth (engaging in repetitive tasks when in pain is also observed in humans and other mammals). I’d also behave differently if some asshole injected painful venom into my lips!
Their heart rate also went up, like ours does when we’re in pain. However, this effect was eliminated by giving them a painkiller. Such results are naturally made sense of on the hypothesis that fish feel pain that’s dulled by painkillers.
Another study also by Sneddon found that fish that were in pain would swim to a location they liked less if they were consistently given a painkiller while in the region. This means fish actively seek out painkillers, which makes a lot of sense if they’re in pain, but is otherwise rather mysterious. A report (also by Sneddon lol—this woman is everywhere!!!) finds:
Fish learn to avoid electric shocks usually in one or a few trials (e.g. Yoshida and Hirano, 2010). This avoidance behaviour persists for up to 3 days (Dunlop et al., 2006), but after 3 days of food deprivation fish will risk entering the shock zone to obtain food (Millsopp and Laming, 2008). This demonstrates that teleost fish find electric shocks so aversive they alter their subsequent behaviour.
One effect that pain tends to have is that it distracts people from other things. For example, if I’m trying to watch a soccer game, and then you kick me in the face, I’ll stop focusing on the soccer game. One way to test pain in fish, therefore, is to look at whether it distracts them from other things. Normally, trout are scared of new objects. However, when they’re in pain, they’re so distracted by the pain that they won’t pay attention to new objects, and won’t react aversively to them.
Trout also like hanging out with their friends. They make tradeoffs between this and receiving electric shocks, so that as long as the electric shocks in a region where their friends are hanging out are mild, they hang out with their friend. If they get too powerful, however, they avoid their friend (this is what I do with my friend—I’ll endure up to a 5 hz electric shock to hang out with
). This effect is eliminated by morphine.
They also make similar sorts of tradeoffs between eating and electric shocks, making complex tradeoffs between degree of electric shock and food eaten. If they like food better, they’ll endure a greater electric shock to get it. Me too! I’ll endure a lethal dose of electricity to get a sufficiently good thing of vegan curry.
Fish also have pretty insane cognitive abilities. They can solve mazes and do all sorts of other complex tasks. Ozy notes:
Other cognitive abilities also matter. Cichlids and Siamese fighting fish can remember the identities of fish that they've watched fight. Cichlids are, in fact, capable of transitive inference. In one study, a cichlid "bystander" was allowed to watch other cichlids fight: A beat B, B beat C, C beat D, and D beat E. The cichlid was then placed in a tank with, say, B and D. The cichlid will reliably swim towards D, the weaker fish, even though they’ve seen both B and D win once and lose once.6 It’s not a status communication: a cichlid who hasn’t observed any fighting, if placed in a tank with B and D, is equally likely to swim towards either of them.
Fish also have various features that we’d expect to go with consciousness, being able to communicate and remember for months or years areas where they encountered predators so that they avoid them. While it’s conceivable that a creature could communicate without being conscious, being conscious certainly makes such capabilities more likely.
Additionally, there’s decent evidence that fish dream. But to dream, they have to be conscious. If they’re conscious, they can feel pain. Sure, it’s possible that they look and act like they’re dreaming but aren’t really dreaming, but it’s also possible that you look and act conscious but aren’t really conscious. If it looks like a duck and acts like a duck…
There are a few broad classes of objection to the idea that fish feel pain. The first involve claims that fish can’t feel pain because they don’t have the right brain regions. Such a view is quite controversial—Culum Brown writes:
The vast majority of commentaries, however, do not, and argue that fish most likely feel pain. Most agree that Key’s argument is flawed at best, and his evidence of how pain works in humans is selective, simplistic, misleading and outdated (Damasio & Damasio; Merker; Panksepp; Shriver). One emerging consensus, however, is that no single line of scientific evidence should over-rule any other, and many of the respondents agree that behavioural studies, such as conditioned place aversion, are an important component to understanding pain in human and nonhuman vertebrates alike.
The primary message from these commentaries is that Key’s argument is fundamentally flawed from an evolutionary perspective. He argues (although later denies it) that human brain architecture is required to feel pain. The mechanistic approach centres around the role of the cortex in human pain. But as Dinets, Brown, Ng and others point out, the human cortex has taken on a huge number of roles that once were the domain of other brain regions. To suggest that fish don’t feel pain because they lack a cortex, one would also have to write-off consciousness (Seth) or indeed any cognitive function that occurs in the humancortex. One example of applying this logic would be to conclude that fish are not capable of learning either. Clearly this argument is absurd (see Brown et al. 2011; Brown 2015 for reviews). Jones, Mather, Striedter, Elwood and Edelman all point out that Key’s approach also denies any possibility of convergent evolution, but surprisingly Key accepts that birds might provide an example of an independent evolution of a cortex-like structure. Haikonen, Manzotti, and Seth take it one step further and suggest that we don’t even understand pain in humans yet, and it is far too early to be making judgement calls on other taxa. The role of the cortex in human pain is still debated (Stevens; Damasio & Damasio) and is most likely not the “on-off switch” for consciousness (Segner). Walters argues that high levels of consciousness are not required for pain perception, a view which is consistent with evolution being a gradual process rather than occurring as all-or-nothing leaps and with the notion that pain is likely an ancient evolutionary trait.
Thus, it seems like the consensus view is that the brain regions that fish have probably are enough for consciousness (when a mammal’s cerebral cortexes are removed, they continue to function normally, and some humans are born without cerebral cortexes). Additionally, even if human pain requires certain brain regions, that doesn’t mean that pain in general does. Humans and fish see in very different ways, yet both see—consciousness could be similar.
The other popular argument against fish pain is that after being injured, fish will go back to feeding normally within a short time. But this is entirely explicable on the assumption that fish feel pain—fish have more A fibers than C fibers. A fibers are responsible for quick, shooting pain, while C fibers are responsible for long-lasting dull-aching pain. Thus, fish might not as much dull long-lasting pain, but they certainly feel pain when hurt.
It’s true that fish nociceptors don’t always respond to the same stimuli as human nociceptors, but this isn’t an argument against fish nociception. Some mammals don’t have nociceptors that respond to acid, yet they’re still surely conscious.
Some fish, like sharks, are known as elasmobranchs—they’re bony (no offense). In elasmobranch fish, nociceptors have yet to be detected, and elasmobranch fish seem to shrug off injury to an insane degree. You can commit a super egregious microaggression against elasmobranch fish, and they’ll ignore it. Thus, one might think that elasmobranch don’t feel pain and use this as evidence against fish pain. But I think this is a bad argument (see also this post on the EA forum).
First, even if they don’t feel bodily pain, this tells us little about whether they can be in unpleasant states. The reason why they’d evolve to not feel pain is that they’re constantly getting bitten and injured in insane ways—e.g. they’ll often eat things with giant spikes. For such creatures, feeling pain wouldn’t be adaptive, so perhaps they no longer feel pain when injured. However, it’s still plausible that they’re conscious and can suffer.
Second, the reason why they might not feel pain is that there might be direct evolutionary pressure for them not to feel pain. But such a thing doesn’t apply to other fish more broadly. Thus, inferring from these creatures not feeling pain that others must not is a bad inference.
Third, we don’t really know if they have nociceptors. We haven’t discovered them yet, but that doesn’t mean they aren’t there (it took until 2002 to discover nociceptors in fish at all).
Fourth, the fact that they shrug off injury doesn’t show that they don’t feel pain. Some mammals shrug off injury to a great degree but still feel pain. Perhaps they need to really be injured to feel pain. As Max Carpendale notes in the above EA forum post, probably the sharks that don’t feel any pain would go the way of humans who don’t feel pain, being susceptible to constant injury.
Section summary: fish respond to painkillers, remember and avoid places where they were hurt, actively seek out painkillers, respond behaviorally as if they were in pain by exhibiting distress and rubbing the injured area, have physiological responses characteristic of pain, avoid using injured areas, and make complex tradeoffs between pain and gain. Everything about fish behavior when injured seems to suggest pain.
(While we’re on the topic of shrimp, I’ve recently made the case that you should give money to the shrimp welfare project—in total, that article triggered a chain of events that helped almost half a billion shrimp. Some of this section—though not most of it—is repeating points I’ve made there).
Shrimp have most features that we’d expect to go with consciousness (like being my friends😊—I have a great relationship with the shrimp, they tell me “sir, you’ve really been doing a beautiful thing in terms of—swp, they call it swp, and that’s really a tough thing if you think about it, they’re ripping off the shrimp in a way that nobody could believe”). They communicate, integrate information from different senses into one broad pictures, make trade-offs between pain and various goods, display anxiety, have personal taste in food, and release stress hormones when scared. When shrimp are injured, they rub the wounded area in the way way animals do when in pain. Other crustaceans behave like we do when we’re in pain, being likelier to abandon a shell when they’re given more intense electric shocks, such that their abandonment is a function of both the desirability of the shell and greatness of the shocks. One study looked at the seven criteria that are the best indicators of pain:
(1) a suitable central nervous system and receptors, (2) avoidance learning, (3) protective motor reactions that might include reduced use of the affected area, limping, rubbing, holding or autotomy, (4) physiological changes, (5) trade-offs between stimulus avoidance and other motivational requirements, (6) opioid receptors and evidence of reduced pain experience if treated with local anaesthetics or analgesics, and (7) high cognitive ability and sentience.
It concluded that crustaceans, which include shrimp, lobsters, and crabs, score highly on every criteria. They have a complex nervous system that bundles together many kinds of information; exhibit avoidance behavior, so that they remember and avoid places where they were hurt; nurse injured areas and try to remove sources of pain; have physiological changes like dilated pupils in response to pain, just like we do; make trade-offs between pain and other goods; have opioid receptors and respond positively to painkillers; and exhibit complex behavior.
Their complex behaviors were especially cool. Lobsters demonstrate incredible navigation—having a much better sense of direction than I do. If a lobster is going somewhere, and you move it quite significantly, it can sense how it’s being moved and reorient towards where it’s going. I can’t do that! Hermit crabs use several different sources of information to learn about new gastropod shells, integrated into a broad picture, and can remember crabs they fought for up to four days. Whenever I fight a crab, I can’t tell it apart from the other crabs! Hard to imagine them doing that without being conscious.
In 2021, the London school of economics carried out an extremely detailed 107 page report on invertebrate sentience. While they ended up uncertain in many cases of whether invertebrates possessed some correlate of consciousness, in every case it was because of lack of data. Not a single piece of evidence they reviewed turned up evidence against the non-sentience of any crustacean. At the end of the report, they recommended extending legal protections to every invertebrate investigated.
A report (cited earlier when discussing fish) by Sneddon that looked at whether we have evidence that decapods (the kind of animal that includes shrimp, crabs, and the like) concluded that they meet almost every criteria that we have reason to expect goes with pain:
Consistently, either there’s no evidence about crustaceans or it ends up confirming their sentience. It never seems to go the other way. Like fish, if crabs are shocked in some location, they avoid it in the future, remembering it. This makes sense on the supposition that they feel and remember pain but is otherwise largely mysterious.
One thing that we’d expect to go with consciousness is individual personality differences. Humans have different personalities as do other animals that we know to be conscious. Oysters have no personality (sorry to all the oysters reading this).
Shrimp demonstrate behavioral differences, with some being more dominant and others being more docile. This provides more evidence against shrimp being merely complicated machines. Individual shrimp also have differing levels of risk aversion. Mere mechanisms tend not to display personality differences. Decapods (this includes shrimp, lobsters, and crabs) also nurse their wounds the same way that humans do.
Crabs display quite complex behavior regarding shells. One study by Robert Elwood found that hermit crabs make complex and impressive tradeoffs. Specifically they:
Decapods also exhibit a stress response. Shrimp and crayfish both seem to exhibit a stress response that produce chemical changes and risk aversion, and crayfish respond to anti-anxiety drugs given to humans. On the supposition that decapods don’t suffer, it’s utterly mysterious that they respond in every way—from reacting to pain killers to getting frightened to make tradeoffs—as if they were conscious.
One study even found that shrimp become exhausted, so that if poked once, they flick their tail, but if flicked many times, eventually they stop flicking their tail. They even seem to prioritize tending to their wounds over eating, delaying feeding for around 30 minutes when in pain. Most shockingly, crayfish self administer amphetamine, which is utterly puzzling on the assumption that they aren’t conscious. Hard to imagine that a non-conscious creature would have a major preference for drugs.
Not sure how much this counts, but around 95% of farmers in India, who deal with shrimp regularly, think they’re conscious.
Most convincingly is that nearly every investigation has turned up evidence for decapod pain. While it would be one thing if there were many studies pointing up in both directions, the general trend is that studies go the way that one who believed in decapod consciousness would expect, and their critics are left explaining them away. So what do those who deny crustacean consciousness say?
It’s often suggested that decapods like shrimp and lobsters don’t have brains. This isn’t true. Even penaeidae (the shrimp we generally eat) have brains, though they’re relatively primitive. You might be surprised that penaeidae shrimp brains give rise to consciousness, but it’s similarly surprising that human brains give rise to consciousness! If you look in a brain, you’ll never find qualia!
Others note that crayfish, for instance, only respond to certain types of painful stimuli. They don’t react aversively to chemical stimuli or low temperature stimuli. But this tells us little, as some mammals like the African mole rat have nociceptors that don’t respond to certain kinds of stimuli. It’s highly likely that decapods have nociceptors given that they’re ubiquitous across the animal kingdom, including in insects, and there’s behavioral evidence for nociception.
The standard arguments against invertebrate pain are weak. It’s just repeated that they have simple brains without very many neurons. But as described before, we don’t have good reason to think that simply brains mean dull consciousness or that neuron counts correlate with intensity of valenced consciousness. What matters more than number of neurons is the sorts of functions they perform, yet crustaceans nervous systems perform similar functions to ours.
Finally, one might think that only some decapods are conscious. Because penaeidae have simple brains, perhaps they can’t suffer but, say, crayfish can. This judgment could be supported by the limited research on shrimp. Here, however, three things are of note.
First, given the similarity between the various decapods, probably either they’re all conscious or none of them are. It would be surprising if only some decapods turned out to be conscious. This also best explains the consistent pattern across decapods of possession of features that correlate with consciousness.
Second, as discussed before, simple brains don’t necessarily mean simple consciousness.
Third, much of the evidence discussed has regarded shrimp. When hurt, shrimp show long-lasting unusual behavior and groom the affected area, yet this is reduced by anesthetic, and they have protocerebral centers linked to information integration, learning, and memory. They also display individual personality differences and taste preferences.
Fourth, my core claim in this article is that the evidence supports the ubiquity of pain thesis. If a creature thrashes and tries to get away when you hurt it, there should be a strong presumption in favor of thinking it feels a good bit of pain. This thesis is supported by suffering in fish and widespread suffering in decapods, and then if it’s right, it lends support to the consciousness of any particular creature.
Section summary, crustaceans (decapods more specifically) probably feel pain given that they:
I was doubtful of insect pain for a long time. My skepticism came mostly from hearing the sorts of claims initially made by an influential paper by Eisemann et al. Barrett summarizes these claims:
1. Insects lack nociceptors.
2. Endogenous opioid peptides and their receptor sites do not necessarily indicate the capacity for 91 pain perception.
3. Because insects have simple brains with relatively few neurons compared to mammals, they cannot generate the experience of pain.
4. Insects do not protect injured body parts.
5. Insects often behave normally after severe injuries.
6. Insect behavior is inflexible in a way that suggests that pain would have little adaptive value for them.
The problem is that all of these claims have been undermined by more recent research as Barrett and Fischer argue in an important paper. Nociceptors have been found in many different species of insects. Given the similarity of function across insects, probably many different insects have nociceptors. As for their second claim, while insects may lack endogenous opioid peptides that modulate pain in humans, they may represent pain in other ways.
As for the third claim about brain size, some insects have roughly similar brain size and neuron counts to some very small mammals, and we haven’t even studied the brains of the biggest insects. This becomes especially true when we consider the giant insects in the fossil record. Additionally, as discussed before, and furthered here, brain size is a poor measure of cognitive complexity.
As for the fourth claim, insects often show a dramatically decreased willingness to eat and mate (sometimes, in their case, at the same time) after receiving an injury. Insects increase grooming of an injured area, yet this effect vanishes if given morphine, a painkiller. It comes back if they were given naloxone which cancelled out the morphine. After getting a leg injury, insects, in fact, place less stress on the injured leg and often limp and move more steadily.
Barrett and Fischer also provide an explanation of where other studies might go wrong: often there’s a delay between the insects being harmed and the tests on them. Insofar as insects, like perhaps fish, feel momentary shooting pain but not chronic pain, perhaps by that time, the injuries are no longer painful.
It’s true that sometimes insects sustain significant injury and keep going about their business as if nothing is wrong. However:
To summarize: the central plank of Eisemann’s case against insect pain is based on unsystematic observations, without due consideration of prevalence, mechanism, context, or biomechanics, of a limited class of negative stimuli that lead to behaviors that are ripe for misinterpretation. However, when we recall key differences in morphology and biomechanics between insects and other animals, these behaviors may be less surprising in soem cases. Additionally, more systematic observations of insects reveal more familiar responses to mechanical damageand demonstrate context-dependent repair. Finally, a broader view of the relevant noxious stimuli—notably, shock and heat—demonstrate that insects do not behave “normally” in response to injury. Though there is more to say here, these points suffice to show that the behavioral evidence against insect pain is not nearly as robust as Eisemann suggests.
Regarding Eisemann’s final claim that insect behavior is inflexible and mechanical, this has been overturned by more recent research. Insects make tradeoffs between injury and reward and are capable of learning.
As for the positive evidence that insects feel pain, Gibbons et al released a detailed 75-page report on insect sentience. Across 8 criteria for consciousness, insects met either 6 or 3-4 of the criteria. Of the criteria they did not meet, however, it was never because there was contrary evidence, but always because not enough research had been done. No research ever suggested an insect failed a criteria. Building on previous work, the authors suggested the 8 criteria most relevant to assessing feeling pain are:
1. Nociceptors: The animal possesses receptors sensitive to noxious stimuli (nociceptors).
2. Integrative brain regions: The animal possesses integrative brain regions capable of integrating information from different sensory sources.
3. Integrated nociception: The animal possesses neural pathways connecting the nociceptors to the integrative brain regions.
4. Analgesia: The animal’s behavioural response to a noxious stimulus is modulated by chemical compounds affecting the nervous system in either or both of the following ways:
a. Endogenous: The animal possesses an endogenous neurotransmitter system that modulates (in a way consistent with the experience of pain, distress or harm) their responses to threatened or actual noxious stimuli
b. Exogenous: Putative local anaesthetics, analgesics (such as opioids), anxiolytics or anti-depressants modify an animal’s responses to threatened or actually noxious stimuli in a way consistent with the hypothesis that these compounds attenuate the experience of pain, distress or harm.
5. Motivational trade-offs: The animal shows motivational trade-offs, in which the disvalue of a noxious or threatening stimulus is weighed (traded-off ) against the value of an opportunity for reward, leading to flexible decision-making. Enough flexibility must be shown to indicate centralised, integrative processing of information involving an evaluative common currency.
6. Flexible self-protection: The animal shows flexible self-protective behaviour (e.g., wound tending, guarding, grooming, rubbing) of a type likely to involve representing the bodily location of a noxious stimulus
7. Associative learning: The animal shows associative learning in which noxious stimuli become associated with neutral stimuli, and/or in which novel ways of avoiding noxious stimuli are learned through reinforcement.
8. Analgesia preference: The animal shows that they value a putative analgesic or anaesthetic when injured in one or more of the following ways:
a. Self-administration: The animal learns to self-administer putative analgesics or anaesthetics when injured.
b. Conditioned place preference: The animal learns to prefer, when injured, a location at which analgesics or anaesthetics can be accessed.
c. Prioritisation: The animal prioritises obtaining these compounds over other needs (such as food) when injured.
Regarding the first, as discussed before, many insects have nociceptors. Even in cases where we haven’t yet detected nociceptors, this is mostly because the subject is underexplored. As for the second, insect brains integrate together many kinds of information, and nearly all adult insects have “mushroom bodies” (neither mushroom nor body—who named these things!!??) which give them the ability to learn, remember, and process sensory information. Regarding the third, studies on cockroaches have shown that their brains integrate the nociceptive information. While people often remark on the fact that insects can survive for a bit without a head, cockroaches no longer have coordinated responses to harmful stimuli after missing their head, showing that the head integrates pain signals. (Also, chickens can survive for a bit without a head, but that doesn’t mean they’re not conscious).
The fourth provides the most evidence: insects respond to noxious stimuli by trying to get away. However, this affect is blunted by a painkiller, which makes insects more willing to suffer harm. If given a painkiller, they’re more likely to endure heat for the sake of a reward. Bees avoid areas where they’re given electric shocks, yet this effect is reduced if given a painkiller.
Fruitflies also seem to get depressed. If they’ve been treated badly, when placed in water, they lie dormant for a bit, not moving. This is very similar to how depressed humans often lay in bed and remain inactive. Legend has it that when lying in the water, fruitflies like to eat icecream and binge their favorite shows. However, giving them anti-depressants causes them to start moving more quickly. They also display states akin to anxiety; after subject to danger, they flee to dark areas and arena edges.
Regarding the fifth, insects make complicated tradeoffs between pain and gain (I know I have already used that phrase twice, but it’s really a great phrase—it rhymes). When deciding whether to jump across heat to access food that they enjoy, they make tradeoffs between the degree of the heat and the light. Bumblebees make complex tradeoffs between sugar water and pain, such that which place they choose to fix is a function both of the sugariness of the water and the heat they need to endure to get to it. Note that in this case they remember the tastiness of the water and the degree of the heat, which they learn to identify from the color of the feeder, and then use this to make future decisions.
Regarding the sixth, insects engage in flexible self-protection, the way many other creatures do. Many insects nurse their own wounds and, in the case of ants, the wounds of other injured ants.
Regarding the seventh, insects display associative learning. If an adult fruit fly is shocked after being exposed to a smell, they later consistently avoid that smell. Similar things are true across many different kinds of insects, including flies who, after being exposed to just one painful stimuli, remember what was happening. Gibbons et al conclude:
Overall, there are multiple studies demonstrating nociceptive associative learning in adult Blattodea, Diptera, Hymenoptera, Lepidoptera, and Orthoptera, as well as last instar juvenile Diptera and Lepidoptera, so we have very high confidence that these orders fulfil this criterion at these life stages.
Rethink Priorities commissioned a very detailed report on the probability and degree of insect sentience, analyzing both bees and black soldier flies (the primary ones that are being farmed and fed to farmed fish). They found a large amount of evidence for insect sentience. One study they cite involved making bees engage in complex tasks that required backtracking to get reward:
We therefore trained bumblebees with two types of task that we believe represented challenges unlike any they have evolved to respond to. These involved dragging various-sized caps aside and rotating discs through varying arcs away from the entrances of artificial flowers (via detours of up to three body lengths), to access a reward hidden beneath. Further, we successfully trained bees to use disc colour as a discriminative stimulus predicting whether rotating discs clockwise or counterclockwise would reveal the reward. This true, complex operant conditioning demonstrated that bumblebees can learn novel, arbitrary behavioural sequences, manipulating and moving items in ways that seem far from any natural task that they would encounter, and doing so flexibly in response to specific discriminative stimuli. This adds to growing evidence of impressive behavioural plasticity and learning abilities in bees, and suggests new approaches for probing their cognitive abilities in the future.
Not only can they rotate disks in complex formations that they didn’t evolve to do, honey bees communicate, including through a complex waggle dance that takes into account the location of the sun. As Peter Curry (substack here) writes:
The waggle dance is a line dance performed by honey bees - an individual bee discovers a food source, and boy, is she excited (the vast majority of bees are female). She arrives back in the hive, and begins ferociously waggling while running in a line, then does a semicircular loop and starts the dance again. The angle that the bee runs in relation to the vertical is the angle that the food source is relative to the sun. If a bee runs straight upwards, the food source is in the direction of the sun. The distance the bee runs is proportional to the distance to the food source.
As you may have surmised at some point, the sun moves. The dancers factor this in, and as they spend longer performing the dance, they change the angle of their dance in order to factor in the movement of the sun. And if the sun is behind a cloud? Bees are sensitive to polarised light, and can infer the location of the sun from this light.
An important detail here is that it is generally dark inside hives. Other bees can’t actually see the dance. Instead, bees that dream of foraging put their feelers on the dancer's abdomen, and hold them there while the dancer shimmies. It’s probably very arousing if you’re a bee. Bees have specific dialects of their waggle dance language, and some bees can learn the dialects of other bee languages if they spend enough time on Duolingo and don’t get killed by the foreign bees.
(Sidenote: Peter should have won the ACX book review competition. He would have if it weren’t for MASSIVE AMOUNTS of MAIL IN BALLOTS. He just needs 11,000 votes, Brad, give me a break).
Curry also notes that bees have awareness of their bodies—when flying through an area they shift position depending on whether they need to approach it diagonally, sideways, or go around. Additionally, bees seem to dream in a way similar to mammals. He writes:
Bees need to sleep. If they don’t get their beauty sleep, their dance moves get worse. If you expose bees to odours while they are in a phase of deep sleep, they begin to consolidate memories from the previous day, and their memories improve. Rats do a similar thing, repeating activation patterns from the previous day in their hippocampus. Does this suggest these creatures are dreaming?
Most impressively, bees figure out tool use. If a lever manipulates a pully, making it possible to get sugar when pulled, many bees learn to pull it. Yet the dumbass bees who didn’t learn to get the sugar initially are able to learn from watching the bees around them. In addition, bees integrate multiple senses together to form one cohesive picture of the world.
The Rethink Priorities report concluded that almost every consciousness proxy has been shown to be displayed in bees. For the ones that aren’t, with the exception of transitive inference, this is only because of an absence of evidence, rather than evidence showing that they don’t engage in some behavior. This includes:
Now, bees are among the most impressive insects. But even black soldier flies, a generally unremarkable kind of insect (that we’re tormenting by the trillions) meet a lot of these. Specifically, black soldier flies have the following consciousness proxies:
People find it unintuitive that insects suffer intensely, but I think this is most likely the result of bias. Insects are very small—probably the reason people sympathize more with lobsters than bugs is that bugs are very small. We have very different intuitions about giant prehistoric intuitions like the one below than, say, houseflies, though they’re similar:
Additionally, as Brian Tomasik notes, caring about insects is inconvenient. It makes morality messy, where everything we do is constantly killing creatures that can feel (though I suspect this doesn’t count against it as most insects live quite wretched lives). Based on the fact that insects look weird—in the words of Kendrick Lamar, “they not like us,”—caring about them is inconvenient, and they’re very small, we’re predictably biased against them.
In sum, then, we have lots of evidence that insects are conscious: they display complex behaviors that provide evidence of consciousness, act as we’d expect them to act if in pain (making tradeoffs, responding to anesthetic, nursing wounds, remembering pain and avoiding painful areas in the future, displaying associative learning), and much more.
Here I have argued for what we might call the ubiquitous pain thesis: all the creatures of significantly debated consciousness are conscious. Probably every creature that has a brain and nociceptors is conscious, and such things are most likely ubiquitous. Additionally, I’ve argued there’s a reasonable probability these creatures suffer a lot. While some of the evidence is species specific, there are broad trends—nearly all the creatures investigated:
I’ve provided 5 a priori arguments for the ubiquitous pain thesis, which, being simple and parsimonious, should start out pretty plausible: ubiquitous pain is evolutionarily beneficial, such a thing would explain the behavioral reactions in animals of trying to avoid harmful stimuli, the odds that we’d be conscious are higher if consciousness is more widespread than more limited, consciousness being widespread makes better sense under dualism (where consciousness is fundamental) and physicalism (where it probably just involves some type of information processing), and the inductive trend shows that humans tend to underestimate the frequency of animal suffering. In addition, I’ve reviewed experimental evidence from many creatures of debated consciousness—fish, insects, and crustaceans (decapods, to be precise)—and argued that these all support the ubiquitous pain thesis.
The thing that was most convincing—more than any specific experimental result—is the broad trend. As the data rolls in, it consistently provided more and more evidence for consciousness in the species of debated consciousness. There were many cases of creatures seeming to exhibit consciousness but no cases of any solid evidence against species exhibiting consciousness. One who started with the assumption that animals aren’t conscious would have consistently incorrectly predicted experimental results, while the opposite is true of those who started with the assumption that animals are conscious.
I’ve also argued that pain in simple creatures is likely pretty intense. If pain serves to teach creatures a lesson, then simpler creatures would need to feel more pain, and creatures with simpler cognition would have their entire consciousness occupied by pain. In addition, the behavioral evidence reviewed by Rethink Priorities seems to suggest that it’s likely that many animals experience a lot of pain—they react quite strongly for stimuli, not like a creature in a dull, barely-conscious haze. I’d bet at 66% odds that most insects, fish, and decapods are conscious and would say that conditional on them being conscious, there’s a 70% chance they feel relatively intense pain. If this is right, then it makes a strong case for taking insect and shrimp suffering quite seriously and giving to organizations working on reducing their suffering.
Sadly, it seems like all the swing state species are conscious.
2024-11-20 21:40:08
Published on November 20, 2024 1:40 PM GMT
Previously: Long-Term Charities: Apply For SFF Funding, Zvi’s Thoughts on SFF
There are lots of great charitable giving opportunities out there right now.
I recently had the opportunity to be a recommender in the Survival and Flourishing Fund for the second time. As a recommender, you evaluate the charities that apply and decide how worthwhile you think it would be to donate to each of them according to Jaan Tallinn’s charitable goals, and this is used to help distribute millions in donations from Jaan Tallinn and others.
The first time that I served as a recommender in the Survival and Flourishing Fund (SFF) was back in 2021. I wrote in detail about my experiences then. At the time, I did not see many great opportunities, and was able to give out as much money as I found good places to do so.
How the world has changed in three years.
This time I found an embarrassment of riches. Application quality was consistently higher, there were more than twice as many applications, and essentially everyone is looking to scale their operations and their spending.
Thus, this year there will be two posts.
This post contrasts between this experience and my first experience in 2021.
The other post will be an extensive list of charities that I believe should be considered for future donations, based on everything I know, including the information I gathered at SFF – if and only if your priorities and views line up with what they offer.
It will be a purely positive post, in that if I don’t have sufficiently net helpful things to say about a given charity, or I believe they wouldn’t want to be listed, I simply won’t say anything. I’ve tried to already reach out to everyone involved, but: If your charity was in SFF this round, and you either would prefer not to be in the post or you have new information we should consider or share, please contact me this week.
This first post will contain a summary of the process and stand on its own, but centrally it is a delta of my experiences versus those in 2021.
Note that the speculation grant steps were not present in 2021.
Or:
You are given well over 100 charities to evaluate, excluding those that did not get a speculation grant, and you could yourself recruit others to apply as I did in 2021. There are several times more charities than there were last round in 2021 when time was already tight, and average quality has gone up, but your time available is the same as before. On the order of $1 million is on the line from your decisions alone.
Claude helped, but it only helps so much.
I assume most everyone spent substantially more time on this than the amount we committed to spending. That was still not remotely enough time. You are playing blitz chess, whether you like it or not. You can do a few deep dives, but you have to choose where and when and how to focus on what matters. We all did our best.
For the majority of organizations, I read the application once, selectively looked at links and documents offered, decided the chance they would hit my bar for funding was very low given my other options, and then evaluated them as best I could based on that since the process has one recommender who is the average of everyone’s rankings, so your evaluations all matter. And that pretty much had to be it.
For others, I did progressively more diligence, including emailing contacts who could provide diligence, and for a number of organizations I had a phone call to ask questions. But even in the best case, we are mostly talking 30-60 minutes on the phone, and very few opportunities to spend more than that off the phone, plus time spent in the group discussions.
The combination of tons of good options and no time meant that, while I did rank everyone and put most organizations in the middle, if an organization did not quickly have a reason it rose to the level of ‘shut up and take my money’ then I didn’t spend too much more time on it, because I knew I wasn’t even going to get through funding the ‘shut up and take my money’ level of quality.
When did I have the most confidence? When I had a single, very hard to fake signal – someone I trusted either on the project or vouching for it, or a big accomplishment or excellent work that I could verify, in many cases personally.
Does this create an insiders versus outsiders problem? Oh, hell yes. I don’t know what to do about that under this kind of structure – I tried to debias this as much as I could, but I knew it probably wasn’t enough.
Outsiders should still be applying, the cost-benefit ratio is still off the charts, but to all those who had great projects but where I couldn’t get there without doing more investigation than I had time for, but might have gotten there with more time, I’m sorry.
There is now a speculation grant requirement in order to be considered in a funding round. Unless you get at least $10k in speculation grant money, you aren’t considered in the main round, unless someone actively requests that.
That greatly raised the average quality of applications in the main round. As a speculation granter, you can see that it is a strong quality filter. This was one reason quality was higher, and the multiplier on number of charities worth considering was very large.
Another huge problem continues to be getting people to take enough risks, and invest in enough blue sky research and efforts. A lot of the best investments out there have very long tailed payoffs, if you think in terms of full outcomes rather than buying probabilities of outcomes. It’s hard to back a bunch of abstract math on the chance it is world changing, or policy efforts that seem in way over their heads but that just might work.
The problem goes double when you’re looking at track records periodically as organizations seek more funding. There’s a constant pressure to Justify Your Existence, and not that much reward for an outsized success because a lot of things get ‘fully funded’ in that sense.
A proposed solution is retroactive funding, rewarding people post-hoc for big wins, but enthusiasm for doing this at the necessary scale has overall been quite poor.
Others paid a lot of attention to salaries, worried they might be too high, or generally to expenses. This makes obvious sense, since why buy one effort if you can get two similar ones for the same price?
But also in general I worry more that non-profit salaries and budgets are too low not too high, and are not able to either attract the best talent or give the best talent full productivity – they’re forced to watch expenses too much. It’s a weird balance to have to strike.
This is especially true in charities working on AI. The opportunity cost for most involved is very high, because they could instead be working on AI companies. If those involved cared primarily about money, they wouldn’t be there, but people do care and need to not take too large a hit.
A great recent example was Gwern. On the Dwarkesh Podcast, a bunch of people learned Gwern has been living absurdly cheaply, sacrificing a lot of productivity. Luckily in that case once the situation was clear support seemed to follow quickly.
I also strove to pay less attention than others to questions of ‘what was fair’ for SFF to fund in a given spot, or who else had said yes or no to funding. At some point, you care anyway, though. You do have to use decision theory, especially with other major funders stepping back from entire areas.
Back in 2021, time did not feel short. If there were not good enough opportunities, I felt comfortable waiting for a future date, even if I wasn’t in position to direct the decisions involved.
Now in 2024, it feels like time is short. AGI and then ASI could arrive scarily soon. Even if they do not, the regulatory path we go down regarding AI will soon largely be set, many technical paths will be set, and AI will change many things in other ways. Events will accelerate. If you’re allocating for charitable projects in related spaces, I think your discount rate is much higher now than it was three years ago, and you should spend much more aggressively.
A distinct feature this round was the addition of the Fairness and Freedom tracks. I was part of the Freedom track, and instructed to put a greater emphasis on issues of freedom, especially as it interplays with AI, as there was worry that the typical process did not give enough wait to those considerations.
The problem was that this isolated the three members of the Freedom track from everyone else. So I only got to share my thoughts and compare notes with two other recommenders. And there were a lot of applications. It made it hard to do proper division of labor.
It also raised the question of what it meant in this context to promote freedom. You can’t have freedom if you are dead. But that issue wasn’t neglected. Do forms of technical work lead us down more pro-freedom paths than others? If so which ones?
Many forms of what looks like freedom can also end up being anti-freedom by locking us into the wrong paths and taking away our most freedom-preserving options. Promoting the freedom to enable bad actors or anti-freedom rivals now can be a very anti-freedom move. Failure to police now could require or cause worse policing later.
How should we think about freedom as it relates to ‘beating China’? The CCP is very bad for freedom, so does pro-freedom mean winning? Ensuring good light touch regulations now and giving us the ability to respond carefully rather than with brute force can be very pro-freedom by heading off alternatives.
The most pro-freedom thing in the last five years was Operation Warp Speed.
Everything is complicated.
In our first session I asked, how much should we emphasize the freedom aspect of applications? The answer was some, but not to the exclusion of other factors. And there were not that many applications that had strong freedom arguments. So I still looked at all the applications, and I still allocated to what seemed like the best causes even if they weren’t directly linked to freedom, but I did substantially elevate my ranking of the more directly and explicitly freedom-focused applications, and ensured that this impacted the ultimate funding decisions.
My biggest top-level cause prioritization decision was to strongly downweight anything meta or any form of talent funnel, based on a combination of the ecosystems seeming funding constrained and time constrained, and because I expected others to prioritize those options highly.
I did not similarly throw out research agendas with relatively long timelines to impact, especially Agent Foundations style alignment approaches, because I do have uncertainty over timelines and pathways and I think the expected marginal value there remains very large, but placed less emphasis on that than I did three years ago.
Last time I extensively discussed the incentives the S-process gives to organizations. I especially noted that the process rewards asking for large amounts of money, and telling a legible story that links you to credible sources without associated downside risks.
This time around, I saw a lot of applications that asked for a lot of money, often far more than they had ever spent in the past, and who strove to tell a legible story that linked them to credible sources without associated downside risks.
I do not regret my statements. It did mean I had to adjust on all those fronts. I had to watch for people gaming the system in these ways.
In particular, I did a deliberate pass where I adjusted for whether I thought people’s requests were reasonably sized given their context. I tried to reward rather than punishing modest asks, and not reward aggressive asks.
I especially was sure to adjust based on who asked for partial funding versus full funding, and who asked for funding for shorter versus longer periods of time, and who was projecting or asking for growth faster than is typically wise.
There was a key adjustment to how the calculations go, that made it much easier to adjust for these issues. In the past, we had only first dollar value, last dollar amount and a concavity function. Now, we were asked to evaluate dollar values as a set of linear equations. This made it easy to say things like ‘I think Acme should get $100k with very high priority, but we should put little or no value on more than that,’ whereas in the past that was hard to do, and Acme asking for $500k almost had to make it easier to get the first $100k.
Now, we had more freedom to get that right. My guess is that in expected value terms asking for more money is correct on the margin, but not like before, and it definitely actively backfired with at least one recommender.
There were a number of good organizations that were seeking far more funding than the entire budget of an individual recommender. In several cases, they were asking for a large percentage of the entire combined round.
I deliberately asked, which organizations are relatively illegible and hard to fund for the rest of the ecosystem? I did my best to upweight those. Versus those that should have a strong general story to tell elsewhere, especially if they were trying to raise big, where I downweighted the value of large contributions. I still did place a bunch of value in giving them small contributions, to show endorsement.
The best example of this was probably METR. They do great work in providing frontier model evaluations, but everyone knows they do great work including outside of tradition existential risk funding sources, and their budget is rapidly getting larger than SFF’s. So I think it’s great to find them more money, but I wanted to save my powder for places where finding a substitute would be much harder.
Another example would be MIRI, of Eliezer Yudkowsky fame. I am confident that those involved should be supported in doing and advocating for whatever they think is best, but their needs exceeded my budget and the cause is at this point highly legible.
Thus, if you are looking to go big and want to be confident you have made a solid choice to help prevent existential risks from AI (or from biological threats, or in one case nuclear war) that can absorb large amounts of funding, you have many good choices.
If this seems like an incomplete collection of thoughts, it is again because I don’t want to be restating things too much from my previous overview of the S-process.
There were a lot of worthwhile individual charities that applied to this round, including many that ultimately were not funded.
Again, there will a second post next week that goes over individual charities. If your charity was in SFF and you either actively do not wish to be included, or have new information on your situation (including major changes in funding needs), you can reach out to me, including at LessWrong or Twitter.
2024-11-20 19:48:14
Published on November 20, 2024 11:48 AM GMT
Authors: Anonymous (I'm not one of them).
Abstract:
Most analysis of transformer expressivity treats the depth (number of layers) of a model as a fixed constant, and analyzes the kinds of problems such models can solve across inputs of unbounded length. In practice, however, the context length of a trained transformer model is bounded. Thus, a more pragmatic question is: What kinds of computation can a transformer perform on inputs of bounded length? We formalize this by studying highly uniform transformers where the depth can grow minimally with context length. In this regime, we show that transformers with depth O(logC) can, in fact, compute solutions to two important problems for inputs bounded by some max context length C, namely simulating finite automata, which relates to the ability to track state, and graph connectivity, which underlies multi-step reasoning. Notably, both of these problems have previously been proven to be asymptotically beyond the reach of fixed depth transformers under standard complexity conjectures, yet empirically transformer models can successfully track state and perform multi-hop reasoning on short contexts. Our novel analysis thus explains how transformer models may rely on depth to feasibly solve problems up to bounded context that they cannot solve over long contexts. It makes actionable suggestions for practitioners as to how to minimally scale the depth of a transformer to support reasoning over long contexts, and also argues for dynamically unrolling depth as a more effective way of adding compute compared to increasing model dimension or adding a short chain of thought.