2026-01-10 21:37:33
Published on January 10, 2026 1:37 PM GMT
The topic of acausal extortion (particularly variants of Roko's basilisk) is sometimes mentioned and often dismissed with reference to something like the fact that an agent could simply precommit not to give in to blackmail. These responses themselves have responses, and it is not completely clear that at the end of the chain of responses there is a well defined, irrefutable reason not to worry about acausal extortion, or at least not to continue to do so once you have contemplated it. My question is if there is a single, reasonably clear reason, which does not depend much on the depth to which I may or may not have descended into the issue, which would be more persuasive than proposed reasons not to pay the 'pascal's mugger'. If there is one, what is it?
2026-01-10 11:46:17
Published on January 10, 2026 2:17 AM GMT
I'm excited to share that my team and I won 1st place out of 35+ project submissions in the AI Forecasting Hackathon hosted by Apart Research and BlueDot Impact!
We trained statistical models on the AI Incidents Database and predicted that AI-related incidents could increase by 6-11x within the next five years, particularly in misuse, misinformation, and system safety issues. This post does not aim to prescribe specific policy interventions. Instead, it presents these forecasts as evidence to help prioritize which risk domains warrant policy attention and deeper evaluation.
2026-01-10 11:22:19
Published on January 10, 2026 2:21 AM GMT
Cross post, adapted for LessWrong
Several challenges add friction to finding high signal people and literature:
We reapply PageRank to Twitter, which naturally weights “important” people higher. If Ilya Sutskever follows only three accounts, a puppy fan page among them, perhaps we should sit up and take notice. The approach is very similar to the existing LessWrong work analyzing AI discourse on Twitter/Bluesky, but instead of categorizing p(doom) discourse, we want to find "important" and “underrated” people.
Approach:
Six 'famous' users were used to bootstrap PageRank, chosen for high quality public contributions. After a round of convergence, the top ranked handle is added (removing organizations), repeating until we have ~200 "core" handles. Finally, we cut the list down to top 749 and rerun one last time. The full table with additional columns can be found at https://thefourierproject.org/people
"Influential" People
Let’s look at the results! Unsurprisingly, Sam Altman is rank 0, clearly a center of gravity in the field, with other famous people trailing. How do the original 6 rank?
We can also see some well known LessWrong members also in the untruncated ~60,000 list (rankings slightly different).
"Underrated" People
"Underrated" handles should have low rank and low follower count, resulting in a high discovery score, where:
For example @csvoss is ranked 111 suggesting her importance at OpenAI but “relatively” undiscovered at 12,275 followers.
"High Signal" People
However, high influence or high discovery may not imply high signal. Highly public people cannot tweet freely and everyone has personal interests. To estimate signal with a dystopian approach, we prompt Gemini Flash 3.0 Thinking with each user's 20 most recent tweets and the following:
Are these tweets high signal? I'm sifting for hidden gems on the internet. I am looking for any of the following:
Critically, the author should not:
Credentials can be a weak signal, but must be relevant to the topic and are not obsolete or fake. It is extremely critical to discern popularity due to stoking emotions in polarizing topics versus pieces that are actually sharp and high quality. Strongly penalize pieces that rely on emotional manipulation. Think very critically - do not use a categorized points based rubric, consider each tweet holistically.
Given Gemini's love for the term 'smoking gun', see if your taste aligns with its prompted interpretation of high signal, as ratings can vary over different runs. It is potentially useful as a starting point, and a external perspective to force reconsideration when I disagree with it's judgement.
The Lack of Correlation Between Rank and Discovery
Lastly, we examine and find that neither rank or discovery score appear to correlate with signal, which suggests that it’s possible that the LLM signal rating is more random, however my personal spot checks (Andrej Karpathy, Lilian Weng, Sam Altman) seem about correct.
Gaps in the approach and possible additional explorations
In the future I would like to tackle related fields such as semiconductors, robotics, and security in their separate clusters. If anyone had good bootstrap handles or field prioritization suggestions I would greatly appreciate it.
Thanks for reading!
2026-01-10 10:42:46
Published on January 10, 2026 2:24 AM GMT
Crossposted from https://substack.com/home/post/p-183478095
Epistemic status: Personal experience with a particular failure mode of reasoning and introspection that seems to appear within different philosophical frameworks (discussed here are rationality and Tibetan Buddhism), involving intolerance of felt uncertainty, over-indexing on epistemic rigour, compulsive questioning of commitments, and moralisation of "correct" thinking itself.
If you do this correctly, you’ll be safe from error.
This is the promise I’ve been chasing for years: across Sufi treatises, Western philosophical texts, Buddhist meditation halls, rationalist forums. Each framework seems to offer its own version: think rigorously enough, examine yourself honestly enough, surrender deeply enough, and (here my anxiousness steps in, with its own interpretations) you’ll finally achieve certainty. You won’t ask yourself whether you’ve got it all wrong anymore. You’ll know that you’re doing it right, taking the right path, being the right sort of person.
This isn’t what I believed consciously. I would, confidently, say that certainty is unattainable, and that it's better to keep one’s mind agile and open to new evidence. This seemed like the only respectable position to me. My behaviour, however, has suggested a less relaxed attitude: relentless rumination, nose-length scrutiny of my motives, endless reassurance-seeking through rumination and feedback, and an inability to fully commit to, but also fully leave behind, any framework I’ve encountered.
This has come with some heavy costs.
The primary consequence: a sort of analytical paralysis in my spiritual commitments. For a long time I saw this as avoiding premature foreclosure, but now I suspect that it actually comes from needing certainty before acting: needing to be as sure as possible that this is the right path, the right community, the right teacher, before committing fully (which for me meant not only practicing it seriously, as I did, but also explicitly excluding other possibilities). I've long had a deep anxiety about my beliefs, as if I had to protect them against every possible objection so that the ground itself isn't pulled away from beneath my feet, demanding a radical re-evaluation of my entire identity.
There was a lot of distrust. Staying in only one tradition and learning nothing about the others didn’t seem okay, because what if it wasn’t the right tradition for me, or what if sticking rigidly to any tradition is spiritually sub-optimal, or what if it’s actually an ancient cult that can only perpetuate its belief system by isolating its members? And yet, sampling all the meditation techniques, and choosing whichever technique seems to work best, also didn’t seem okay, because that would be mystical window-shopping and therefore disrespectful to the traditions (so went my thinking).
At the same time, I couldn’t allow myself to drop any tradition before going deep enough into it (in thought or practice), which would take at least ten years or so, because then I would be abandoning it prematurely too.
In my attempt to figure out the perfect balance of curiosity and commitment, self-reliance and trust for my teachers, respect for tradition and intellectual openness, I ended up oscillating between frameworks of judgment. Whichever one I was in, I doubted whether I should be there.
Another consequence has been an exhausting internal hyper-vigilance. It is like having an internal surveillance system that inspires constant terror: a terror of self-deceiving, of being hypocritical or insincere, of seeing myself as more morally motivated than I am, of doing good things for other people to virtue-signal rather than to do good. After a few hours of introspection, I can often (mostly) reassure myself of my sincerity; but in the moment, while standing before another person, I am confused, distrustful of myself, afraid that I will unintentionally lie to them.
Reading my recent series on introspection again, there is, at the moment, nothing significant I would change in what I have written (although I'm less sure that I’ve been embodying the spirit I described—the spirit of attending to moment-to-moment experience without judgment or analysis). I still think openness is crucial, still believe in seeking out criticism, still think self-deception is better avoided.
But now I am much more conscious of the ways this approach—of internally saying something about the self, wondering about what we’ve said, then circling back to it again and again, to paraphrase Escher—can harm someone. Rigorous self-examination can easily turn into unreasonable discomfort with uncertainty, and that, I suspect, contributes to the very mental noise I have been trying to cut through.
Here is the trap: you start examining your thinking to assure yourself that you aren't self-deceiving. The examination reveals spots of darkness, like shadows on an X-ray, so you look even closer. This reveals even larger, more ominous spots of darkness, and the longer you look, the more they expand. Confused by your inability to ascertain what they are, you turn your gaze on your examination, and see a space of darkness there too.
The issue isn't that you haven't thought about it carefully enough. It's that "carefully enough" has no floor, and you're falling. If you don't allow yourself to stop, you will never land on solid ground.
It has also made it much more difficult to pursue my spiritual practice.
Especially when I was more firmly rooted in Tibetan Buddhism, I worried a lot. Is this paralysing doubt, or inquisitive doubt? Are my reservations about a practice something I must overcome? Are my criticisms of a given teacher or practice justified, or are they just my ego seeking faults, my pride giving me excessive confidence in my thinking, my cultural conditioning with its fixed ideas about critical thinking and authority? Perhaps I would have to cultivate radical epistemic humility, abandon any notion of figuring things out myself, in order to learn anything. Sometimes I convinced myself of this, telling myself I was too confused to see anything clearly. Any extreme proved unsustainable.
From a journal entry during this period, during a two-month retreat: "My identity is composed of components that seem to be arbitrarily chosen. There are a few things I am deeply committed to, like my values, but everything else seems anything but necessary. I am painfully aware that for every pursuit I undertake, I can come up with a dozen reasons not to do it. The only thing that seems obviously, undeniably meaningful to me is my spiritual practice, and yet I am unable to focus on it or pursue it continuously because I don't fully believe in this tradition, and have paralysing doubts about which practice is right for me."
Reading this now, I can see the trap: the very fact that I couldn't be sure I was making the best possible choice made my commitments feel arbitrary. If I couldn't prove a choice was absolutely necessary, with impeccable justification, it felt illegitimate, lacking in weight, too easy to destabilise by new ideas or counter-evidence. I could get temporary relief through my self-criticism, but then my mind would find another weak spot, another objection I had to defend against.
Since no choice could meet my standard of justification, I was stuck in perpetual internal vacillation. The problem wasn't that I was aware of other traditions, but that it wasn't actually possible to choose one correctly enough for my taste. I practiced Tibetan Buddhism as traditionally as possible for five years, but despite my outer committedness, doubt haunted me throughout. The uncertainty I felt made my commitment feel dangerously provisional and fragile.
When I started learning about rationality, it was partly with the intention of engaging more carefully and honestly with the mystical traditions I’d been studying. But it soon morphed into something else. The new refrain became: am I being epistemically rigorous enough?
Am I engaging in motivated reasoning? Can I still treat spiritual experiences as epistemically valuable, or is that suspect too? Am I, given what I’ve learned about confabulation and introspective bias, thinking in the right way about my own thoughts and feelings? And so on and so forth. I replaced one idée fixe with another. And I haven’t dropped it yet: a part of me still believes I just haven’t thought about any of these topics carefully or long enough, that with some more thinking, I’ll figure it out.
As a result, I am in a rather complicated liminal space: multiple internal frameworks, all with their own value, but all of them suspicious of each other and of themselves.
If this isn’t scrupulosity, it may just be perfectionism: even when I try to keep my mind open, that becomes another task I must perform to the highest standard, requiring continual self-monitoring to ensure that I am keeping it open in the optimal way. Paradoxically, the very thing making me want to submit to the Tibetan Buddhist worldview seems to be what makes me so concerned with epistemics now. I suspect that, if I were Sufi, I would be wondering whether my thoughts are whispers from Shaitan.
Perhaps the problem isn’t the correctness of any given framework, but this compulsive, certainty-seeking relationship to frameworks themselves; this demand that, whatever system I engage with, I must be certain that I am thinking and acting and doing everything in the correct way—either thinking fully for myself, or being perfectly open and epistemically humble, or maintaining exactly the right balance between openness and self-trust.
But where did this compulsion come from? What made me believe that perfect thinking was possible, or that it should be my highest priority? To answer this, I’ve been examining something that I’ve been taking as self-evident: the value of truth-seeking itself.
For me, this meant asking myself: do I actually care that much about Truth, and Truth alone?
I am not so sure anymore. In principle, I would much prefer to have beliefs that accord with reality, and I strongly prefer not believing that an AI-generated video is real, and if God exists, I would much rather believe that He does, especially if Pascal was right and Hell exists. But I’m not so sure that any of these preferences are really about a cool-minded pursuit of Truth.
What I notice, instead, is something that seems more intuitive and independent of any conscious reasons: an urge to interrogate certain questions more deeply, a difficulty letting certain questions go, a need for my beliefs to not contradict each other logically and a sense of disturbance when they do, a desire to understand the world better so that I can change it in what I feel is a more positive direction, and a sort of fascination with the unknown within myself and within others, for which I have never needed justification.
And beyond this, I know I have multiple other values and desires. I want a heart that is open to others, and that I am deeply in touch with, rather than solely with the head; I want to experience beautiful sights and be moved by them; I want a mind that is wakeful and responsive to new information, but also not so weighed down by its own self-scrutiny; I want to reflect when it is time to reflect, and act decisively when it is time for that; I want to be able to connect with all kinds of people and be a positive force in their lives; I want to feel deep commitment to something in life.
This, to me, sounds much more like different facets of another overarching, more heartfelt desire: I want to live well. And it seems that my fascination with everything from philosophy and rationality to Sufism and Tibetan Buddhism, have been ways of serving that. But the belief that I should want to have correct beliefs over and above all these other things, this sense that I cannot hold any belief that has not survived brutal interrogation, has been overpowering everything.
For the first time, I am wondering: if optimising for epistemic hygiene gets in the way of honouring these other values, perhaps it isn’t always Truth that should win.
What would it mean to live in this way, seeking truth in service of life? I don’t really know. It may well be a cognitive skill issue, something I can learn with time. Or perhaps my mistake is treating introspection and reasoning both as moral purity tests, not as means to a more expansive end. There’s probably a way of relating to them which is more effective and less psychologically costly; if there is, I would love to hear about it.
If you recognise this pattern in yourself, here's what I wish someone had told me earlier: you don't need to be sure that your motivations are pure before you can help someone. You don't need to purify yourself of every negative thought for others to be safe around you. You don't need to resolve every meta-level epistemological doubt before you can commit to something. You're allowed to spend some time thinking about it, find a position that's good enough (for now), and then act.
For now, I am starting with some questions to remind myself of what matters when I’m lapsing back into this pattern, and which I hope will be valuable for others with a similarly compulsive, perfectionistic thinking style:
Notice when you’re optimising your reasoning as a way to feel as secure as possible, not just to be incrementally less wrong.
Notice when your truth-seeking has become compulsive, something distressing, something you believe you must do to feel okay.
Notice when your demand for airtight arguments hinders you from taking action or committing to something—whether to a path, a profession, a partner, or anything else.
These observations won't end the pattern. Still, I have hope that naming it can lessen the power of the inner morality police, which currently patrols our thoughts and punishes us for our supposed mental transgressions. When you can see it for what it is—an anxiety-driven compulsion, not a necessary guarding of your conscience—you can see through the illusion that one more hour of research, one more year of thinking, one more decade of exploration, will finally give you the clarity you think you need. They won't.
2026-01-10 08:01:17
Published on January 10, 2026 12:01 AM GMT
(I'm reposting this here from an old Dreamwidth post of mine, since I've seen people reference it occasionally and figure it would be easier to find here.)
So people throw around the word "impossible" a lot, but oftentimes they actually mean different things by it. (I'm assuming here we're talking about real-world discussions rather than mathematical discussions, where things are clearer.) I thought I'd create a list of different things that people mean by "impossible", in the hopes that it might clarify things. Note -- because we're talking about real-world things, time is going to play a role. (Yes, there's not really any universal clock. Whatever.)
I'm listing these as "levels of impossibility", going roughly from "most impossible" to "least impossible", even though they're not necessarily actually linearly ordered. Also, some of the distinctions between some of these may be fuzzy at times.
Level 0. Instantaneously inconsistent. The given description contains or logically implies a contradiction. It rules out all possible states at some point in time, in any universe. People often claim this one when they really mean level 2 or level 3.
Level 1. Instantaneously impossible (contingently). In the actual universe we live in, the given description is instantaneously impossible; it rules out all possible states at some point in time. I think in most discussion that isn't about physics this isn't actually strongly distinguished from level 0.
Level 2. Non-equilibrium. The described system fails to propagate itself forward in time; or, if a system extended in time is described, it contains an inconsistency. This is one that people often actually mean when they say something is "impossible".
Level 3. Unstable equilibrium or possible non-equilibrium. The described system is not resilient to noise; it will not propagate itself forward in time unless exceptional conditions hold continually. This is another one that people often really mean when they say something is "impossible".
Level 4. Unachievable. The described system is unreachable from our present state -- it may make sense on its own, it may not be inconsistent with the way the world evolves in time, but it's inconsistent with the initial conditions that hold in the real world. Yet another one that people often mean when they say "impossible".
Level 5. Not "stably achievable". The only path from the current state to the described state is not resilient to noise and requires exceptional conditions to hold, possibly for an extended period of time. We might also want to require that in addition that the failure modes of such a path leave us worse off than we were before or somehow prevent the same path from being used again (so that you can't just try over and over for free).
I'm not sure that this is really complete; but, overall, the point is that when you say something is "impossible", you should think about whether you're actually talking about instantaneous impossibility, non-equilibrium / instability, or unachievability (and then yeah I've introduced some finer distinctions here).
2026-01-10 07:48:05
Published on January 9, 2026 11:48 PM GMT
I’m not quite sure how unequal the world used to be, but I’m fairly certain the world is more equal (in terms of financial means) than the world was, say, in the 1600s.
There are many things that enormous wealth allows you to buy that’s out of reach for middle-class American consumers, like yachts, personal assistants, private jets, multiple homes, etc. You can frame these things in terms of the problems they solve e.g. private jets solve the problem of travelling long distances, multiple homes solves the “problem” of wanting to go on vacation more often. Note that the problems persist across wealth brackets, it’s just that the ultra-wealthy have different methods of solving those problems. While the ultra-wealthy might solve the problem of “vacation travel” using a private jet, those without extreme wealth might travel using commercial airlines. The ultra wealthy introduce novelty into their lives by purchasing multiple homes, while everyone else goes on vacation and stay in a hotel or similar.
If you cluster goods and services based on the problem they solve, most seem to be available at wide range of prices, with the higher end being around 2 or maybe 3 orders-of-magnitude greater than the lower end. For example:
I have low confidence that the difference between the lower- and the higher-end of goods/services to solve a problem is precisely 100x to 1000x, but I’m very certain that it’s orders of magnitude greater and not 1.01x to 2x greater.
However, you do get some products/services which do not exhibit this behaviour: Even the wealthiest man in the world will use the same iPhone, play the same video games, read the same books[1], and watch the same movies[1] as a middle-class American. What gives? What is the underlying factor that means some goods and services have 1000x range in value, and others have basically no difference? Put plainly: Why are there no incentives to build a $100k iPhone[2]? I have some hypotheses about the constraints:
Option 1 seems a little fishy to me. I’m not sure, but I doubt there’s nothing Apple could do to put more high-end features in a $5k iPhone. $100k does start to push the limits, but $1k seems low considering the frequent critique of iPhones. And there’s a very wide range of non-technical features (different colours, different sizes, various customisations) that are present in other high-end luxury goods (cars, watches, jewellery, etc) but aren’t present in iPhones.
Option 2 also doesn’t quite sit right with me. It seems okay on its own, but I don’t see why iPhones would have this dynamic but not watches, furniture, cars, etc.
One third option that I think is closer to the truth:
Consumer technology regularly experiences game-changing innovations, much more so than cars/furniture/housing/etc. So under option 3 we’d expect to see that goods or services with a very small financial range (like iPhones) to be things that are undergoing rapid innovation and change (including but not limited to tech products).
This hypothesis seems to hold up: laptops, smartphones, internet services (like YouTube, Netflix, Gmail, etc), Starlink, all have a very small financial range, some of them you can’t buy a more expensive variant even if you wanted to.
Our hypothesis also predicts that goods/services which experience little innovation should have a wider range of prices. One example of this would be gas-powered cars, which have been stagnant for several decades[3]. There have been improvements in comfort, efficiency, and safety, but I’d argue that these improvements stem more from increased demand for these features rather than previous inability to innovate in these features. Most recent innovation in cars have come from changing preferences, rather than static preferences which undergo more thorough innovation.
I suspect there’s something deeper here which might be empirically useful. If hypothesis 3 is true, then we could use the range of prices for a given product/service as a measure of the innovation in that field: A narrower range of prices means that there’s much more innovation. And this could be impactful, as it gives a way to measure how much the economic market expects a field to be innovating. It would be foolhardy to try sell a $100k iPhone if you suspect a competing product might be released that has significantly better features but with a lower price tag.
The large price associated with luxury goods effectively serves as a bounty for someone to come along and innovate.
We can also make some empirical predictions from this. Smartphones during the 2000s and 2010s underwent a lot of innovation, but they have become stale in recent years: every black slab of glass is nearly identical. I suspect the luxury smartphone businesses take a while to boot up, and it’s possible that new battery technology could cause a resurgence in smartphone innovation. So I’d predict, in the next ~10 years or so, we’ll either see a luxury smartphone business start up[4] and stays in business, or we’ll see significant innovation in smartphones (something like 10x better batteries or other components).
Thanks to Jasmine Li, Jo Jiao, Desiree J, Jay Chooi, and the MATS 9 blogging and writing channel for reviewing drafts of this essay.
I'll note that books and movies might be a slight outlier here: centi-millionaires absolutely can pay or sponsor creative professionals to produce work that they specifically wish to exist. This might be explicit (contracting a director to make a specific film), but more likely this is implicit (funding a film studio, sponsoring an artwork, organising meet-and-greets with powerful donors).
You might debate the precise duration of the stagnation, but the past 20 years of gas-powered car innovation is certainly less impressive than the last 20 years of mobile phone innovation.
one that produces a smartphone ~10x more expensive than the most popular iPhone
