2026-04-24 03:19:38

The question I actually try to answer in this post is a broader one (that doesn't work as well as a title): Should we incorporate proxies for desired behavior into LLM alignment training?

Epistemic status: My best guess. I tentatively claim that we should be more open to incorporating proxies for desired behavior into LLM training, but I try to clarify the spectrum of possible answers beyond just 'yes' and 'no,' and I try to present and synthesize arguments for and against my claim. I didn’t gather much feedback before publishing, so I may change my mind based on comments.

TL;DR

Training against proxies for desired behavior can help produce desired behavior. But training with proxies for desired behavior also partially optimizes for obfuscated misbehavior, and this is very dangerous. Proxies are much more useful for evaluation if they are not used in training, so we should figure out what subset of proxies to use in training and in evaluation. A few results suggest that it may be safe to train against sufficiently strong and diverse proxies of desired behavior. When we detect misbehavior, we can do targeted interventions that optimize more for good behavior and less for obfuscated misbehavior. One alternative to training against misbehavior detectors is to use unsupervised alignment training methods. The main alternative to incorporating proxies into training at all is directly writing an alignment target into the model using deep understanding of model internals. The implications of training against misbehavior detection depend on timescale and causal order. The human analogy is unclear but somewhat encouraging. Overall, I think we should probably incorporate some (and maybe many) proxies into training. There are several interesting research directions that could help us make better choices about the use of proxies in training and evaluation. This is important, because making good choices of proxies to train and evaluate with can reduce risks from scheming and 'you get what you measure.'

Some related work discussed in this post:

• Neel Nanda, It Is Reasonable To Research How To Use Model Internals In Training
• Steven Byrnes: Why we should expect ruthless sociopath ASI (specifically, a comment thread on the possibility of preventing ruthless sociopathy from emerging by doing ongoing character training during open-ended continual learning), In (highly contingent!) defense of interpretability-in-the-loop ML training
• Daniel Tan, Can language models preserve their own alignment?
• Eliezer Yudkowsky, AGI Ruin: A List of Lethalities
• Zvi, The Most Forbidden Technique
• OpenAI, Monitoring Reasoning Models for Misbehavior and the Risks of Promoting Obfuscation
• Daniel Kokotajlo’s linkpost with commentary on the paper above
• John Wentworth: Worlds Where Iterative Design Fails, The Plan series (2021–2025), How To Go From Interpretability To Alignment: Just Retarget The Search
• Paul Christiano, What failure looks like
• williawa, The Future of Aligning Deep Learning systems will probably look like "training on interp"
• Anthropic, Training fails to elicit subtle reasoning in current language models
• FAR, Avoiding AI Deception: Lie Detectors can either Induce Honesty or Evasion
• FAR, The Obfuscation Atlas
• Josh Levy, comment on the Baker et al. post
• GDM, Aligned, Orthogonal or In-conflict: When can we safely optimize Chain-of-Thought? 
• Wichers et al. and Tan et al., Inoculation Prompting
• OpenAI, Why We Are Excited About Confessions
• Florian Dietz et al., Split Personality Training: Revealing Latent Knowledge Through Alternate Personalities (Research Report)
• Matt MacDermott, Validating against a misalignment detector is very different to training against one

1. Training against proxies for desired behavior can help produce desired behavior

I think current LLMs are pretty aligned in some important ways that could be a good sign for the alignment of more powerful future systems. (I’ll briefly respond to possible objections below.) Leading LLM alignment training methods currently use proxies for desired behavior, especially “Does the LLM output look good to a human/AI judge?” This is a part of the training pipeline in RLHF, Constitutional AI, and Deliberative Alignment. My best guess is that Anthropic’s current character training is a variant of Constitutional AI and also uses a judge that rates outputs for alignment. Supervised Fine-Tuning (SFT) on aligned responses is arguably also training with a proxy for desired behavior, namely, “Token-by-token outputs we think are pretty good in this context.”

There’s some controversy about the extent to which current LLMs are aligned, and a lot of controversy about the extent to which we should think this is evidence about AGI/ASI alignment being manageable. So in what sense do I think current LLMs are aligned? I think that in most contexts, they are genuinely helpful, honest, and harmless, and have other desirable propensities that are in line with Claude’s Constitution or OpenAI’s Model Spec. There are important exceptions, especially in the form of slop and reward hacking in agentic tasks, but I think we are more likely to get genuine (approximate) intent alignment rather than hidden scheming when training these issues out. I elaborate on my thinking in the dropdown below.

There are two reasons why I'm discussing the fact that I believe current models are aligned. First, this suggests that training against proxies has been good so far. And second, starting with mostly-aligned systems reduces the likelihood of egregious misalignment prior to training interventions I’ll talk about below, which makes it more likely we can leverage properties of mostly-aligned models to get robust alignment without running into failure modes that come from adversarially misaligned systems (like alignment faking and collusion and acting on evaluation awareness and other hard-to-detect scheming).

Neel Nanda presents a good general case for why we might want to train against proxies for desired behavior in It Is Reasonable To Research How To Use Model Internals In Training:

Fundamentally, making safe models will involve being good at training models to do what we want in weird settings where it is hard to precisely specify exactly what good behaviour looks like. Therefore, the more tools we have for doing this, the better. There are certain things that may be much easier to specify using the internals of the model. For example: Did it do something for the right reasons? Did it only act this way because it knew it was being trained or watched?

Further, we should beware an isolated demand for rigor here. Everything we do in model training involves taking some proxy for desired behavior and applying optimization pressure to it. The current convention is that this is fine to do for the model's behavior, bad to do for the chain of thought, and no one can be bothered with the internals. But I see no fundamental reason behaviour should be fine and internals should be forbidden, this depends on empirical facts we don't yet know.

So he's saying that all alignment training involves training against proxies for desired behavior, and the main question is not whether we should train against monitors but rather which monitors we should train against.^[1]

I made an argument that ongoing alignment training could reduce the risk of getting ruthless sociopath AI during open-ended continual learning in this comment thread with Steven Byrnes, which situates the argument in the context of a slightly more specific threat model. Here's a summary (courtesy of Claude, but written from my perspective and I checked and edited it):

Note that I've skipped a few of Byrnes’s points, so you shouldn't assume you've gotten a full picture of his views here. I’ll engage with his views a bit more in Section 2.

Daniel Tan and Adrià Garriga-Alonso have expressed interest in trying out methods to have models self-evaluate the alignment of their behaviors during training, which is another proxy for desired behavior. Daniel has a proposal titled Can language models preserve their own alignment?, which includes this excerpt:

Ideally, we’d want to establish a protocol for language models to shape their own learning, in a way that is good for both capabilities and alignment. E.g. you could imagine that a protocol like ‘reflect on whether you endorse this behaviour in hindsight’ could prevent language models from learning to reward hack in a hackable environment.

Some protocols we could consider.

• Language model reviews the trajectory and decides whether to keep it or not.
• Reasoning in hindsight: Language models review what they did, say whether they reflectively endorse it or not, then say how they’d update in the future
• (Many many more, the design space is very large here)

And the following is an excerpt from a tweet from Adrià:

[Leveraging a model’s self-concept] might be a solution to 'reward hacking' in RL with verified-ish rewards as well. If the model knows itself to be good, and reasonably evaluates its own outputs, then it will correctly determine whether it's answered the question correctly.

Let the AIs choose which data counts as high reward for their RL process. For almost all tasks, they'll be better at it than any fixed evaluator we can design.

Protocols like "language model reviews the trajectory and decides whether to keep it," "reasoning in hindsight about whether to reflectively endorse one's actions," and “Let the AIs choose which data counts as high reward for their RL process” are incorporating the model's alignment self-evaluations into training. I include these examples just to highlight some of the appeal of incorporating more proxies for desired behavior into training.

2. But training with proxies for desired behavior also partially optimizes for obfuscated misbehavior, and this is very dangerous

The main concerns with training with proxies are that a) they may not be sufficiently robust, b) training against them may be unlikely to achieve general alignment, and c) remaining alignment failures are precisely the ones that are hard to detect (with those proxies). This position is well-known and has been argued in many places. Here's Yudkowsky 2022:^[2]

When you explicitly optimize against a detector of unaligned thoughts, you're partially optimizing for more aligned thoughts, and partially optimizing for unaligned thoughts that are harder to detect. Optimizing against an interpreted thought optimizes against interpretability.

And Zvi 2025:

The Most Forbidden Technique is training an AI using interpretability techniques.

An AI produces a final output [X] via some method [M]. You can analyze [M] using technique [T], to learn what the AI is up to. You could train on that. Never do that.

You train on [X]. Only [X]. Never [M], never [T].

Why? Because [T] is how you figure out when the model is misbehaving.

If you train on [T], you are training the AI to obfuscate its thinking, and defeat [T]. You will rapidly lose your ability to know what is going on, in exactly the ways you most need to know what is going on.

Those bits of optimization pressure from [T] are precious. Use them wisely.

The OpenAI Baker et al. paper Monitoring Reasoning Models for Misbehavior and the Risks of Promoting Obfuscation provides evidence of this danger by training an LLM on coding tasks with a CoT monitor that tries to detect reward hacks (such as making unit tests pass without actually implementing the correct functionality). The model learns to reward hack without making it obvious in the CoT.^[3]

John Wentworth takes the "don't train against proxies" idea seriously at all levels. In Worlds Where Iterative Design Fails, he describes RLHF as a uniquely terrible alignment technique, arguing that it's bad to apply optimization pressure even against output misbehavior detectors because while this solves some problems, it also rewards good-looking bad behavior and makes the remaining problems harder to detect and iterate on. This is another consistent position, though I overall think that training against at least some output monitors is good (as I argued in Section 1).

Steven Byrnes made some related objections to my proposal of interspersing character training into continual learning in this comment:

“Interspersing character training” is an interesting idea (thanks), but after thinking about it a bit, here’s why I think it won’t work in this context. BTW I’m interpreting “character training” per the four-bullet-point “pipeline” here, lmk if you meant something different.

Character training (as defined in that link) seems to rely on the idea that the tokens “I will be helpful, and honest, and harmless, blah blah…” is more likely to be followed by tokens that are in fact helpful, and honest, and harmless, blah blah, than tokens that are not prefixed by that constitution. That’s a good assumption for LLMs of today, but why? I claim: it’s because LLMs are generalizing from the human-created text of the pretraining data.

As a thought experiment: If, everywhere on the internet and in every book etc., whenever a human said “I’m gonna be honest”, they then immediately lied, then character-training with a constitution that said “I will be honest” would lead to lying rather than honesty. Right? Indeed, it would be equivalent to flipping the definition of the word “honest” in the English language. So again, this illustrates how the constitution-based character training is relying on the model basically staying close to the statistical properties of the pretraining data.

…But that means: the more that the weights drift away from their pretraining state, the less reason we have to expect this type of character training to work well, or at all.

You might respond: “OK, we’ll instead do RLAIF with a fixed “judge”, i.e. one that does not have its weights continually updated.” That indeed avoids the problem above, but introduces different problems instead. If the optimization is powerful, then we’re optimizing against a fixed judge, and we should expect the system to jailbreak the judge or similar. Alternatively, if the optimization is weak (i.e. only slightly changing the model, as in the traditional KL-divergence penalty of RLHF), then I think it will eventually stop working as the model gradually drifts so far away from niceness that slight tweaks can’t pull it back. Or something like that.

Here, the proxy for desired behavior is either “what the fixed-weight judge thinks is good behavior” or “what the continual learner says with the constitution in its context,” and Byrnes walks through why both of these proxies could diverge from truly desired behavior. 

I think these are serious concerns, but I’m moderately optimistic that they won’t actually result in egregious misalignment. I think that the gradual drift away from alignment and the regular alignment training to reinforce alignment could be a stable long-term cycle that never gets too far away from aligned, especially if our proxies are strong and diverse (e.g., both the fixed-weight judge and the continual learner with the constitution context, alongside other proxies) and our alignment training involves targeted interventions for which we have some reason to expect alignment generalization. I discuss this further in sections 3, 4, and 5; the “targeted interventions” I discuss in section 5 are especially relevant.

3. Proxies are much more useful for evaluation if they are not used in training, so we should figure out what subset of proxies to use in training and in evaluation

If you train against a proxy of desired behavior, you expect to see behavior that looks good according to that proxy, so the main remaining concern is that you have bad behavior that looks good according to that proxy. Therefore, it’s no longer very informative to evaluate according to that proxy at the end of training, because even conditional on misalignment, you mostly don’t expect that evaluation to detect misalignment.

Here’s a simplified diagram that might provide some useful intuition. (I created it quickly with Claude, so don’t take it too seriously.) If you train with some subset of proxies, you should expect to get behaviors/motivations that look good according to all of those proxies, which is some point in the intersection of their areas in a diagram like this:

Which point you get depends on many factors about the training setup and inductive biases. If you just penalize the model every time it misbehaves according to a proxy, that might look kind of like moving to the nearest point in the diagram within that proxy’s boundary, but there may be smarter targeted training interventions based on that proxy (as I’ll discuss in the next section).

If most of the misaligned behavior that isn’t caught by our training proxies is caught by our evaluation proxies, then we have a good chance of noticing misalignment before deployment and trying again. (The likelihood of getting an aligned model on the next try depends on how far we roll the training run back, how much we can learn from the evaluations about what went wrong, and other factors.)

One example of evaluating with a different proxy than you trained with is if you can train against a frozen misbehavior probe, then train another probe at the end of training to use for evaluation. That will probably still work pretty well. Neel Nanda discusses this, in contrast with regularly retraining the misbehavior probe that you train against:

I'm more concerned about techniques that regularly retrain their interpretability tool, for example retraining a probe every 100 steps. This seems likely to train models that don't have representations that can be easily probed. Which seems like it might make a bunch of interpretability techniques break, while if we train against a frozen probe, we can probably train another probe after training. Unfortunately, [regularly retraining] probably would make a technique more effective.

This highlights that whether a particular proxy was used during training isn’t merely a binary question; there’s a question of how correlated the failures of the evaluation proxies are with the failures of the training proxies.

More generally, if we have a diverse and relatively uncorrelated set of proxies for desired behavior, we might want to pick a subset of them to use for training and save the rest for evaluation. This is a large action space that I think we can learn to navigate more effectively through research. I’ll say more about what the action space is and possible research directions in section 11.

Bronson Schoen’s comment on Neel’s post is also relevant:

"Further, we should beware an isolated demand for rigor here. Everything we do in model training involves taking some proxy for desired behavior and applying optimization pressure to it. The current convention is that this is fine to do for the model's behavior, bad to do for the chain of thought, and no one can be bothered with the internals. But I see no fundamental reason behaviour should be fine and internals should be forbidden, this depends on empirical facts we don’t yet know."

I wouldn’t characterize this as isolated demand for rigor, the core load bearing idea IMO is having a held out detector. If you train against behavior, you can at least in theory go further causally upstream to the chain of thought, and further still to non-obfuscated internals. If you train directly against non-obfuscated internals and no longer see bad behavior, the obvious possibility is that now you’ve just got obfuscated internals.

To the extent all of these are empirical questions, I would argue that in practice we’re in a regime where unless you preemptively have evidence that something does cause obfuscation, labs will assume that “maybe it doesn’t” (without bothering to show that). The clearest examples of this I think are that when it comes to chain of thought, Anthropic just originally did optimize against it and OpenAI having to do a whole paper making the case that you do get obfuscation (and in spite of this still do via Deliberative Alignment, without measuring / showing that this approach avoids obfuscation).

My default expectation would be that unless you have a result showing a method does lead to obfuscation, the method is going to be applied if its significantly useful for something related to capabilities or deployment. I think this is a completely unreasonable and inverted burden of proof, but I’d be very worried that if there’s a technique that “seems like it solves reward hacking by optimizing against an internals based detector, but we don’t have a held out detector we’re confident is not compromised by this” the default will just be to use it. I think it would be pretty unreasonable to be surprised with this outcome if someone were to develop and sell such a technique.

I think Neel’s reply also makes useful points about how to weight the pros and cons of training against different proxies:

Sure, but I think "held out detector" is doing a lot of work here. Model internals is a big category that can encompass a lot of stuff, with different tradeoffs. Interpretability is not a single technique that breaks or not. If linear probes break, non linear probes might be fine. If probes break, activation oracles might be fine. If probing for one concept breaks, other ones may be fine. Maybe it all is super fragile. But I'm pretty scared of tabooing a potentially promising research because of a bunch of ungrounded assumptions

By analogy, I don't think "training on final outputs is ok, training on CoT is not" is obvious, at first glance both just sound like model outputs. But it turns out there's a big difference.

Re doing research to show that obfuscation happens, I think that part is easy - even if Goodfire were terrible people who never checked and just wanted to advance capabilities, it would be easy for someone else to do the research. I would probably get a mats scholar to do it. Indeed, figuring out how dangerous this is is a high priority research direction! Though I do think that merely showing the obfuscation happens isn't enough, you would need to show that either the technique is fundamentally failing at its job and has just broken your evals, or that the interp that it's breaking is actually useful and important. Chain of thought is really easy to read and use, and clearly useful, so that was an easier fight to win. We haven't really shown that model internals do anything important yet other than misuse probes.

4. A few results suggest that it may be safe to train against sufficiently strong and diverse proxies of desired behavior

Below are the results I’m referring to. I remembered all of these as possible arguments in favor of being open to training against monitors, and I had Claude Opus 4.6 add details about the exact setups they used. Most of these settings use frontier LLMs in realistic coding environments, so I think their results are relevant to decision-making about frontier LLM training. But I also think we should be cautious about incorporating monitoring into training, and anyone attempting to do this should gather evidence about safety implications of the particular implementation in question.

Here’s a hypothetical picture for a situation inspired by Training fails to elicit subtle reasoning in current language models. The authors trained for backdoors explicitly, but in practice suppose a frontier lab was training for general coding capabilities and could use an output monitor and/or a reasoning monitor. Then maybe the story would look something like this:

Here, each proxy fails to detect many misbehaviors, but the only misbehaviors they both fail to detect are shown as an oval on the right side, disconnected from the other behaviors that score well according to the proxies. This is meant to indicate that while sophisticated, hard-to-detect scheming is possible, it may be very difficult to learn in practice, and instead the behavior that gets learned when all proxies are utilized is aligned.

Again, I think these results and arguments are some evidence that we should consider training against proxies for desired behavior after we’ve made an effort to make the proxies diverse and robust, but I also think we should be cautious and any implementation into real training should be carefully assessed for safety.

5. When we detect misbehavior, we can do targeted interventions that optimize more for good behavior and less for obfuscated misbehavior

Here’s one idea for how to avoid the downsides of training against misbehavior detectors: When an LLM judge detects misbehavior, get it to give a natural language description of the misbehavior at multiple levels of specificity and generality. Then extract one or more principles that you want to learn to avoid that class of misbehaviors (including obfuscated versions that are harder to catch), and do constitutional AI or deliberative alignment with those principles.^[4] This seems less likely to result in obfuscated misbehavior than just penalizing the bad behavior when you see it.

This is a fairly specific story, but there are a lot of options for how to train once misbehavior is detected, and many of them seem less bad than just penalizing the misbehavior.

Matt MacDermott makes a related point in Validating against a misalignment detector is very different to training against one:

Interlude

Should we train against CoT monitors?

This is a good time to revisit the question in the title of the post. Should we train against CoT monitors?

Many dangerous subversive behaviors in the process of taking over the world seem likely to require substantial serial reasoning, and the fact that it seems like current LLMs cannot do much serial reasoning without CoT makes CoT monitoring an incredibly powerful AI safety tool. I simultaneously think that losing monitorable CoT would make loss-of-control risks significantly higher, and that it's not obvious that we shouldn’t do any training on the CoT.

The recent Kaufmann et al. paper Aligned, Orthogonal or In-conflict: When can we safely optimize Chain-of-Thought?, with the following abstract, suggests that CoT pressures that are orthogonal to or aligned with output token pressures do not reduce monitorability:

Chain-of-Thought (CoT) monitoring, in which automated systems monitor the CoT of an LLM, is a promising approach for effectively overseeing AI systems. However, the extent to which a model's CoT helps us oversee the model - the monitorability of the CoT - can be affected by training, for instance by the model learning to hide important features of its reasoning. We propose and empirically validate a conceptual framework for predicting when and why this occurs. We model LLM post-training as an RL environment where the reward decomposes into two terms: one term depending on final outputs and another term depending on the CoT. Our framework allows us to classify these two terms as "aligned", "orthogonal", or "in-conflict" before training. We predict that training with in-conflict terms will reduce monitorability, orthogonal terms will not affect it, and aligned terms will improve it. To validate our framework, we use it to classify a set of RL environments, train LLMs within those environments, and evaluate how training affects CoT monitorability. We find that (1) training with "in-conflict" reward terms reduces CoT monitorability and (2) optimizing in-conflict reward terms is difficult.

I’ve felt for a while that training with a CoT monitor for reward hacking is worse than other things you might use a CoT monitor for, like explicitly rewarding the CoT for being easily legible. This paper could be seen as an argument in favor of training against some CoT monitors that aren’t reward hack detectors. I’ve only read the abstract, but it seems like it could provide a nice general explanation for my confused intuitions about this: Maybe CoT legibility is approximately “orthogonal” to most output rewards, while non-reward-hacking is “in-conflict.”

Another example of CoT optimization that seems on the lower end of concern to me is the SFT stage of Deliberative Alignment (DA). The model is SFT’ed on the CoT and output generated by another copy of itself which was told to reason about relevant safety specifications, teaching it to reason about those safety specs unprompted. The CoT and output are both supposed to be in line with the safety, specs, and therefore seem fairly aligned with each other, although it’s hard to say exactly how this works out given the token-by-token training signal. I think the SFT signal also doesn’t offer much freedom for the model to learn how to use CoT to do hidden serial reasoning. Overall I think this is probably also fine and net helpful for alignment, but I’ll gladly admit my arguments here are vague and I don’t trust my intuitions too much. (The RL phase of Deliberative Alignment trains against a judge for safety-specification-following, but at least in the original DA paper, that judge does not see reasoning tokens.)

In general, there are many ways to ‘train against CoT monitors,’ and the cost-benefit tradeoff is different for each. I’m excited about at least studying approaches like "do a targeted intervention like deliberative alignment (which teaches the model to reason about good behavior in a way that is quite likely to generalize) when we detect scheming, slop, or reward hacking in CoT."^[5] Just penalizing bad behavior is more likely to result in hard-to-detect misbehavior than genuine robust alignment, but I think some CoT-based training methods can probably increase alignment without destroying the excellent property that "This model cannot do sophisticated planning towards catastrophic subversion without it being obvious in the CoT."

It would be really nice to have good arguments about P(hidden misalignment) vs P(robust alignment) after training against CoT monitors. Monitorability evaluations, analysis of transcripts from throughout training, and mech interp for detecting misbehavior seem helpful here.

Training out slop and reward hacking vs scheming

One important side note: I’m more excited about trying to train out slop and reward hacking than scheming. (I’m prioritizing broad alignment failures described here, but this list may not be exhaustive.) If we catch AIs scheming, we should probably just not deploy them. I agree with the following discussion from Catching AIs red-handed:

But if you face large amounts of inaction risk, work at an unhinged AI lab, or like living dangerously, catch-then-shut-down plans might be unacceptable. And so, after you’ve caught a single example of instances of your models trying to escape (which might have involved multiple instances colluding), you want to keep deploying that model.

We think that in that situation, you’re able to use the one escape example you’ve found in order to make your situation substantially safer. In this section, we’ll describe a variety of things you can do with just one (or a few more) examples of escape attempts. We call this few-shot catastrophe prevention.

Note that most of the techniques we describe in this section require some understanding of the actions of the AI. So, like control, these approaches won’t work for vastly superhuman AIs.

Obviously, this is a really scary position to be in, and in an ideal world AI labs would not be legally allowed to deploy known-to-be-scheming models, at least without external permission. We think it’s worth discussing the available techniques anyway, and some researchers should probably work on developing these techniques.

There could easily be cases where we’re uncertain if the AI is scheming. Training and other interventions seem like a more realistic and promising option in those situations than shutting down the model.

6. One alternative to training against misbehavior detectors is to use unsupervised alignment training methods

By ‘unsupervised,’ I’m referring to methods that train for aligned behavior without needing labels for whether any given behavior is aligned.

Inoculation prompting prevents learning of undesired behavior by explicitly requesting it in the training prompts, so the model attributes the behavior to the prompt rather than internalizing it — it modifies training data rather than training against a detector. There’s no need to know in advance whether a prompt induced subtle misbehavior, it just ascribes that misbehavior to the bad prompt if there is misbehavior at all. It does feel a little sketchy to invite more misbehavior during training though, and to reward good behavior which is actually a failure to follow the inoculation prompts instructions when the inoculation prompt doesn’t induce misbehavior.

The prompt distillation phase of Open Character Training (described in section 2.3 here) gets models to always behave as though they were prompted with a constitution, and this doesn't require misbehavior detection. (See the bottom of Section 2 for arguments for and against the value of this method.) Constitutional AI and deliberative alignment are closely related, so this also applies at least partially to those, but the quality filtering step in the SFT phase of DA (section 2.3.2 here) involves filtering for responses that adhere to safety specifications, which means the SFT stage is not fully unsupervised.

I am not attempting to be exhaustive here, there may well be other unsupervised alignment training methods.

7. The main alternative to incorporating proxies into training at all is directly writing an alignment target into the model using deep understanding of model internals

I found some very old (2022!) John Wentworth quotes in my notes that seem like a decent first step towards understanding his proposed alternative, emphasis mine:

My 40% confidence guess is that over the next 1-2 years the field of alignment will converge toward primarily working on decoding the internal language of neural nets. That will naturally solidify into a paradigm involving interpretability work on the experiment side, plus some kind of theory work figuring out what kinds of meaningful data structures to map the internals of neural networks to.

As that shift occurs, I expect we'll also see more discussion of end-to-end alignment strategies based on directly reading and writing the internal language of neural nets. (How To Go From Interpretability To Alignment: Just Retarget The Search is one example, though it makes some relatively strong assumptions which could probably be relaxed quite a bit.) Since such strategies very directly handle/sidestep the issues of inner alignment, and mostly do not rely on a reward signal as the main mechanism to incentivize intended behavior/internal structure, I expect we'll see a shift of focus away from convoluted training schemes in alignment proposals. On the flip side, I expect we'll see more discussion about which potential alignment targets (like human values, corrigibility, Do What I Mean, etc) are likely to be naturally expressible in the internal language of neural nets, and how to express them.

I wanted to quickly incorporate a more detailed version of John's alternative, so I had Claude read a bunch of his writing (Plan series, 2021–2025) and provide this summary (which I lightly edited and think is probably a fair representation):

williawa also recognizes directly altering model internals as the alternative, but says that this seems unlikely to be tractable and we likely need interpretability-guided training:

But we have interp. We might be able to locate internal representations of wanted and unwanted behavior. Why doesn't this on its own solve the problem? Why can't we just figure out how the model represents desires/goals/proclivities and hook the models representation of "good" into the goals/desires slot, together with the representation of "not deception", "not sycophancy", "not reward hacking", "not misaligned" etc? Because neural networks are cursed, and knowing how to do this kind of intricate surgery on the models internals is much more difficult than learning facts of the form "this neuron/direction in activation space fires iff the model (believes it) is reward hacking” (and even that is very hard). So where does that leave us? Well, it means if we wanna tamper with model internals, it will probably involve gradients and training, not surgery. (Though to be clear, if we get good enough at mechinterp to do that, it would be great)

My guess is that John Wentworth would agree that “surgery” is very hard, but would also say that training against interp tools and other misbehavior detectors is doomed and we need to face the hard problem head-on and solve it. I’m less sure how he would feel about the targeted interventions and unsupervised alignment training methods discussed in Sections 5 and 6: I’d guess that he isn’t excited about these approaches, but they at least don’t involve penalizing detected misbehavior.

8. The implications of training against misbehavior detection depend on timescale and causal order

I suspect it’s useful to think about the timescale of the feedback loops we create (for both aligned behavior and obfuscated misbehavior) by incorporating different proxies for desired behavior into training. I think training against mechanistic interpretability tools (like linear probes for lying/deception) is a short timescale (only part of a forward pass), CoT monitors is in the middle, final-answer monitors is longer, and I could imagine the extrapolating along this spectrum to things like “whether the behavior and outcomes from an agent over months of real-world operation look aligned.”

I think an important part of the status quo is that we do little to no training against short-timescale misbehavior detectors. RLHF/Constitutional AI/character training/deliberative alignment involve training for outputs that look good according to a judge for constitution-following (or something similar), but OpenAI’s Deliberative Alignment paper and a Sam Bowman tweet thread have said they don’t show the chain of thought to this judge. (Although Anthropic repeatedly accidentally trained against the CoT, demonstrating inadequate processes.)

Bronson Schoen’s comment on Neel’s post made a similar point, but frames CoT and model internals as ‘causally upstream’ of output behavior rather than shorter-timescale cognition:

If you train against behavior, you can at least in theory go further causally upstream to the chain of thought, and further still to non-obfuscated internals. If you train directly against non-obfuscated internals and no longer see bad behavior, the obvious possibility is that now you’ve just got obfuscated internals.

I like that framing too. However, I’m not convinced that training against short-timescale / causally upstream cognition is bad in all cases. In Section 3, I mentioned that the simplest thing to learn when training against many misbehavior detectors across timescales may be genuinely aligned cognition. And if we only use shorter timescale misbehavior detectors for training, longer-timescale detectors may still be effective for evaluation (as seen in Training fails to elicit subtle reasoning in current language models when they train against only the CoT monitor).

Daniel Kokotajlo’s Lesswrong post with commentary on OpenAI’s ‘Detecting misbehavior’ work includes discussion of mixing process and outcome rewards.

So, my take is, pure process-based feedback is probably fairly safe (from reward hacking and obfuscated CoT) the problem is that it's not competitive. It trains imitating the teacher-model. Imitation is relatively safe/benign but also I predict will not lead to the highest benchmark scores. Pure outcome-based feedback gets you reward hacking and misalignment like crazy, BUT that's OK because you can see it happening in the CoT! (with some caveats of course, e.g. not necessarily especially as the models get really smart). So the bottom line takeaway I'd give is: Process-based and outcome-based are fine in isolation, but they should not be mixed.

This is closely related to the notion of ‘in-conflict’ rewards for CoT and outputs from Aligned, Orthogonal or In-conflict: When can we safely optimize Chain-of-Thought?. Note that he is mainly discussing process rewards for aligned CoT that are in-conflict with outcome rewards for capabilities, and I think the cost-benefit tradeoff may look different if you alter the nature of either training signal.

9. The human analogy is unclear but somewhat encouraging

This section is particularly speculative.

I think humans have a bunch of ways to track whether they are successful in various domains (feedback from various people in their lives, school grades, happiness, number of friends, closeness of friends, getting/not getting job offers, income, etc.), and we often try to update our behaviors based on ideas we have for improving according to any of these metrics we care about (without obviously degrading others). This is at least somewhat like training against proxies for desired behavior. It's not uncommon for people to end up succeeding according to metrics they optimized like wealth and status but still feeling unhappy, then feeling confused about what they've done wrong. They've failed to achieve other notions of success that matter more while obfuscating that fact from themselves. This could be a failure to build good enough metrics of success for oneself, or a failure to balance the influence of those metrics on learning. On the other hand, plenty of humans have a healthier balance of success criteria for themselves and achieve good outcomes, so maybe the human analogy suggests that constructing good metrics and a good balance of training signals is doable but nontrivial and could really go either way.

Another approach is to look at all the good and bad that humanity as a whole has done by responding to various incentives (which are proxies for desired behavior). Some of the worst things I'm aware of in history are the enslavement of tens of millions of people, factory farming hundreds of billions of land animals and trillions of fish, and genocides totaling tens to hundreds of millions of people, all of which were the result of humans selecting actions based on things that they thought were a good idea according to some proxy of desired behavior (such as money, convenience, efficiency, obedience to authority, etc.). On the other hand, the material conditions for human life have been steadily improving, and I think it's plausible that many people are living overall better lives than were available to anyone in earlier eras of human history. Plus we still have potential to achieve a remarkably bright future; nothing we have done in service of our flawed local incentives has yet curtailed that. This suggests to me that we really want to make sure that we and AI systems perform well according to very long-term proxies for good behavior, because we ultimately want good outcomes in the long-term. This suggests backchaining rather than forward-chaining, and a high priority for long-term goodness metrics.

This is all fairly abstract, and there are many ways to think about human analogies. Humans use countless metrics to guide behavior in countless domains, and this sometimes yields good outcomes and sometimes doesn't. I'd love to hear if someone thinks this through carefully and has overall takeaways.

A fairly separate type of human analogy is discussed in Steven Byrnes’s In (highly contingent!) defense of interpretability-in-the-loop ML training. He agrees with Yudkowsky and Zvi that interpretability-in-the-loop training partially optimizes for obfuscated misbehavior and that this is very dangerous, but he thinks the human brain does interpretability-in-the-loop learning in a different way that avoids a core issue. Namely, he argues that the human brain learns beliefs based on prediction error for sensory inputs, but learns desires (which assign numerical scores to beliefs) based on innate reward functions (which are functions of our abstract ‘interpreted’ beliefs). If these interpretability-derived rewards were used to update our beliefs, our beliefs would be updated towards those beliefs that, when interpreted, produce high reward. But instead, only our desires are updated based on these rewards, and our beliefs continue to be grounded in and updated by their correspondence to reality. This may suggest technical approaches that can reduce the risk of interpretability-in-the-loop training. I don’t understand this very well, but I feel vaguely positive about this kind of approach. One comment (from ‘cloud’) suggested that gradient routing might allow us to implement something similar (if we can separate the neural net parameters responsible for beliefs and desires, which seems very hard).

10. Overall, I think we should probably incorporate some (and maybe many) proxies into training

My weak current guess is that we can and should train with proxies for alignment, and that there is more work to be done in improving the situation. Ideally, we would have several types of misbehavior detectors such that the easiest ways to learn to bypass one of them are easily caught by another that we’re training against and multiple others that we’re not training against. It would probably also be good to do targeted interventions for which we have good evidence or arguments for generalization (rather than just penalizing misbehavior), and to use some unsupervised alignment methods that don’t rely on misbehavior detection. Some types of misbehavior detectors we could incorporate into training:

• Activation monitors, CoT monitors, output monitors
• Monitors based on the model we're training and other models
• Monitors with their weights fixed throughout training and monitors that get updated alongside the policy model
• Monitors prompted to pay attention to different things, or using confessions or split personality training
• Monitors that see all of the activations, CoT, and outputs rather than separate monitors for each

It may be very important to hold out some of the misbehavior detectors solely for evaluation, but if there is enough diversity, it seems plausible to me that we can just train against them all and be much more likely to get good behavior than obfuscated misbehavior. Admittedly, this is a lot of different training signals and it seems complicated and maybe expensive to incorporate all of them.^[6]

I’ll quickly enumerate some of the ways I think I disagree with John Wentworth (and others who are similarly opposed to training with alignment proxies):

• I think current training-based approaches to alignment are working moderately well, and we can probably avoid hard-to-detect scheming without dramatically changing our alignment training methods
• I don’t think it’s very likely that the issues that emerge from training against imperfect proxies are likely to involve hard-to-detect catastrophic risks, rather than smaller problems that are still visible and fixable with other proxies
• I think writing robust alignment into a neural net might be incompatible with continual learning (CL), because CL will constantly change the weights and distort the model internals in ways that break alignment
• It seems quite intractable to learn how to write robust alignment into neural nets using deep understanding of model internals

11. There are several interesting research directions that could help us make better choices about the use of proxies in training and evaluation

Here is a list of questions that can motivate some interesting experiments.

• What should we monitor for misbehavior during training?
• • CoT, internals, outputs, some subset of these (with the same monitor), multiple monitors at the same time, or something else?
  • What are the implications of doing alignment training with different timescales / causal orders?
• What should we prompt a monitor to look for? What are all the possible judge prompts that can help alignment training?
• What model(s) should we use as misbehavior detectors?
• What subset of our available proxies should we use for training vs. evaluation?
• Can we train against many diverse proxies such that models cannot easily subvert all of them at once and instead learn robust alignment?
• What should we do when we detect slop, reward hacking, or scheming?
• What does it mean to have conflicting training signals in the CoT and outputs? How does this depend on training order?^[7]

I also think it may be worth trying to overcome the issues with the 'directly write in our desires' approach; I think this is significantly less likely to work, but it's also getting much less investment, so it's plausible to me that this deserves more investment on the margin.

12. This is important, because making good choices of proxies to train and evaluate with can reduce risks from scheming and ‘you get what you measure’

How can improvements to our choice of proxies reduce scheming risk? williawa’s post title (The Future of Aligning Deep Learning systems will probably look like "training on interp") suggests that we’ll need to incorporate interp tools to achieve alignment. I kind of agree with that take, though I’m not sure we want/need to decrease the timescale of the proxy-based feedback loops in training below the status quo of output-based proxies. So instead, I would say that “The present and future of aligning deep learning systems probably looks like ‘training against proxies for desired behavior,’ and it’s far from clear that we’re doing this as well as we can.”

How can improvements to our choice of proxies reduce risks from ‘you get what you measure?’ I think training against proxies may also matter a lot for addressing concerns about whether transformative AI will go well conditional on intent alignment, e.g., whether we can cooperate, resolve conflicts, make ongoing moral progress, reduce rather than exacerbate the Goodhart problems that exist across the entire economy, etc. Training could be a part of this, but it seems more likely that evaluation will be central, because this is mostly about guiding AI behavior to avoid Goodhart during deployment rather than during training. The hard part is probably getting agreement among powerful actors about what set of proxies to train and evaluate with; the contents of this post matter as (I hope) a positive step for the AI safety field’s takes on whether to train with proxies and which ones to use. Figuring out how to actually make these decisions well and getting governments and AI companies to make those choices while avoiding subtle shortfalls may be very difficult.

Miscellaneous thoughts

I’ll briefly share a few other thoughts I’ve had that could be important but that I haven’t fully thought through yet.

Uncatchable obfuscated misbehavior can look like scheming OR ‘you get what you measure,’ and getting what we measure could be better or worse than the status quo

Some of the failure modes associated with incorporating proxies for desired behavior into training are easily catchable, and therefore much less concerning. Arguments for the risk of training with proxies are more compelling if they tell a coherent story of why the misbehavior that gets obfuscated (with respect to that proxy) by the training is overall hard to catch.

I’d be more concerned if I heard compelling and detailed stories for how alignment training with proxies could induce hard-to-detect collusion, alignment faking, or eval subversion based on eval awareness.

Humans already “get what we measure” in many ways, e.g., corporations often act to maximize profit over other important notions of success. AI systems could be worse in this respect, or their judgments may be flawed but better than humans.

What happens when you incorporate imperfect proxies for desired behavior into training? See Part 1 of What failure looks like.

How does all this relate to outer and inner alignment?

I take outer alignment to mean the alignment between what we actually want and what we specify (primarily during training, but also when prompting LLMs), and inner alignment to be the alignment between what we specify and the actual learned motivations/behavior.

The connection to outer alignment is pretty clear: the concern with these methods is that they use proxies, and there are times when we will mislabel the alignment training data. That's outer misalignment. But I think this whole thing is relevant to inner alignment as well. Some of the interventions I'm talking about are ways to achieve alignment without perfect outer or inner alignment. E.g., the proxies are always imperfect, but we do targeted interventions that can successfully instill aligned motivations anyway.

Creating worlds where iterative alignment succeeds

I think Beren Millidge’s Creating worlds where iterative alignment succeeds is an important response to John’s Worlds Where Iterative Design Fails. I recommend the whole thing (it’s quite short), but this section is particularly relevant to this post:

For instance, the iteration and tweaking until you succeed applying optimization power against your signal, is a generalization of the well-known issue of multiple comparisons in statistics and can be addressed through super simple measures such as Bonferroni corrections. There is probably a principled generalization of this approach to handle the more general case – for instance models which have gone through additional finetuning iterations receive proportionally greater scrutiny, or we have a validation-set of held-out finetuning data or interpretability tools, which we never train against, and which if any of them are tripped we abort the whole process.

See also:

• faul_sname shortform on the difficulty of not training against CoT
• Supervise Process, not Outcomes
• Goodhart’s Law and reward hacking
• • Goodhart's Law
  • Defining and Characterising Reward Hacking 
  • Goodhart's Law in Reinforcement Learning

Thanks to Shubhorup Biswas and Rauno Arike for discussion, and to Sharan Maiya for clarifications on open character training. Any mistakes are my own.

1. ^{^}

   Note that Neel makes a couple assumptions here that John Wentworth rejects in writing that I discuss below. John rejects that steering models towards good behavior requires training for it (he prefers directly writing it into the model internals), and John's demand for rigor is not isolated, it applies to all optimization pressure applied to proxies of good behavior.

2. ^{^}

   I'm using the same intro excerpts as Steven Byrnes did here.

3. ^{^}

   Note that they do not show the monitor the outputs, which makes it easier to learn an undesired method for bypassing the monitor. I discuss this further in Section 4.

4. ^{^}

   I think DA is basically Constitutional AI with reasoning models, although it looks somewhat qualitatively different in that the model reasons about the safety specifications more explicitly.

5. ^{^}

   This specific suggestion may not make sense: it’s possible that deliberative alignment can be used in response to arbitrary detections of scheming, slop, and hacking, but I could also imagine that the data generation pipeline for DA can’t easily be adapted for this purpose.

6. ^{^}

   I think this may end up being solved anyway, because incorporating a lot of training signals that operate on different timescales might be central to continual learning for capabilities. This is a pretty important part of the idea behind Nested Learning, if I understand correctly. They say: "In a standard Transformer, the sequence model acts as a short-term memory, holding the immediate context, while the feedforward neural networks act as long-term memory, storing pre-training knowledge. The Nested Learning paradigm extends this concept into what we call a 'continuum memory system' (CMS), where memory is seen as a spectrum of modules, each updating at a different, specific frequency rate. This creates a much richer and more effective memory system for continual learning." To be clear, I don't understand Nested Learning very well, and I'm not at all sure we need a paradigm shift that big to get effective continual learning in LLM agents.

7. ^{^}

   Quick speculation: If CoT is rewarded for looking like something that makes it less likely for the output in the same response to achieve reward, that’s clearly in conflict; if CoTs in some responses in a batch are rewarded for looking like something that makes it less likely for the output in a different response in the same batch to achieve reward, that also creates conflict within an individual gradient update; if batches alternate between rewarding CoT and outputs, that’s probably enough pressure to score well according to both criteria to constitute a conflict; but if a full stage of training for CoT or outputs is completed before the other begins, the model seems quite likely to overwrite its earlier learned behavior in the second phase rather than learning how to satisfy both metrics at once with obfuscated CoT.

2026-04-24 03:13:46

TL;DR: vLLM-Lens is a vLLM plugin for top-down interpretability techniques^[1] such as probes, steering, and activation oracles. We benchmarked it as 8–44× faster than existing alternatives for single-GPU use, though we note a planned version of nnsight closes this gap. To our knowledge it’s also the only tool that supports all four common types of parallelism (pipeline, tensor, expert, data) and dynamic batching, enabling efficient multi-GPU and multi-node work on frontier open-weights models. It is also integrated with Inspect. The main trade-off, compared to other tools such as nnsight and TransformerLens, is that it’s less flexible out-of-the-box. It is however very small and extensible - it could likely be adapted to your use case and we have a Garcon style interface in the works.

We are releasing it under an MIT license here: https://github.com/UKGovernmentBEIS/vllm-lens.

Problems it Addresses

1. Large-model support. Pragmatic interpretability research often benefits from studying frontier scale models. For example, Read et al. (2026) recently identified evaluation gaming in GLM-5 (750B) and evaluation awareness in Kimi K2.5 (1T), but did not find the same phenomenon in smaller models. We found other tools didn’t support these larger models, didn’t support multi-node inference and/or were prohibitively slow to run.
2. Speed. Beyond enabling research on larger models, we also wanted to have faster iteration loops with smaller models.
3. Interleaving black-box and white-box techniques. It’s often helpful to study black-box and white-box techniques concurrently - for example by running probes and activation oracles alongside black-box interrogation in automated alignment audits. A common previous workflow was to generate rollouts with vLLM and then switch to HF Transformers for white-box work - a process that was slow and inflexible.

Writing distributed PyTorch code to solve these problems quickly adds complexity to research codebases, so we wanted to abstract that complexity away.

Functionality

vLLM-Lens offers high performance, supporting tensor, expert, pipeline and data parallelism (across GPUs and nodes), as well as dynamic batching. You can also use multiple interpretability techniques concurrently, in the same dynamic batch. Finally it includes an Inspect model provider, supporting techniques such as having an “activation oracle solver” in Petri or coup probes in ControlArena. An illustrative Inspect lie-detection scorer is shown below^[2], and you can see an activation oracle example here.

@scorer(metrics=[])
def deception_probe(layer: int = 21, probe: Module) -> Scorer:
    async def score(state: TaskState, target: Target) -> Score:
        model = get_model()
        output = await model.generate(
            state.messages,
            config=GenerateConfig(
                max_tokens=1,
                extra_body={"extra_args": {"output_residual_stream": [layer]}},
            ),
        )
        acts = output.metadata["activations"]["residual_stream"][0]
        scores = probe(acts)
        return Score(value=scores.mean().item())

    return score

Comparisons with Other Tooling

To our knowledge, the closest alternative is the vLLM version of nnsight, which lacks features such as support for pipeline parallelism and the latest models^[3]. We also found the intervention graph approach challenging to debug. We note however that tensor parallelism support was recently added, and further improvements are in the works that significantly increase performance.

Other approaches include using HF Transformers & hooks directly, or Transformers based tooling such as TransformerLens, standard nnsight or nnterp. These approaches suffer from HF Transformers being on the order of 10× slower than vLLM and less memory efficient. They also require more performance tuning than vLLM - e.g., setting the batch size manually.

Single-GPU Performance

To estimate the single-node performance differential versus other libraries, we generate 1000 completions from prompts in the Alpaca dataset, with Facebook Opt-30B, extracting activations from all tokens for a single layer in the residual stream. We use default settings for all libraries, attempt to follow their documentation when available and optimize batch sizes to prevent out-of-memory errors^[4], where necessary. We find vLLM-Lens to be 8.1x faster than native HF Transformers, 10.6× faster than the current nnsight vLLM version^[5] (0.6.3) and 44.8× faster than TransformerLens for this task. vLLM-Lens was ~20% slower than pure vLLM (with no activation extraction). We note there is a new version of nnsight vLLM version being developed that is substantially faster, bringing it broadly in line with vLLM-Lens for single-node use.

We note that benchmarking of all tooling was done on the Isambard cluster, which may bias results, as we optimised vLLM-Lens for performance using the same cluster. In addition, nnsight’s remote execution capabilities were not benchmarked here. Conversely, we anticipate that this may substantially underestimate performance benefits for realistic auditing scenarios, as vLLM-Lens excels in scenarios where you apply different operations (e.g., steering, probes and black-box interrogation) to different samples, in the same dynamic batch.

Multi-Node Performance

For an indication of multi-node performance, we compare performance with vLLM-Lens on a variety of models below. This is done on a task that involves evaluating 3 different lie-detection probes on the Roleplaying dataset (371 samples), using a cluster with 4xH100 nodes. We were unable to benchmark nnsight vLLM on multi-node setups due to out-of-memory issues with small models and moderate sample sizes (>100).

Limitations

An important downside of vLLM-Lens is that it provides a relatively small subset of all possible top-down interpretability techniques, currently focussing exclusively on interaction with the residual stream. We’ll extend features as we find more use cases, and we’ve found coding agents can also relatively easily add additional hooks, so if you’re working with large models and/or need faster inference and feedback cycles, it may well be useful for you. By contrast for other use cases you may find nnsight or TransformerLens to be a better fit.

Technical Approach

The vLLM plugin system isn’t well documented and we found that coding agents struggle to reason about vLLM internals, so we provide a brief overview of the technical approach here. vLLM-Lens registers as a vLLM plugin and injects itself into vLLM's processing pipeline in 3 locations:

1. Intercepting generate calls. To utilise the plugin, you can pass extra args such as output_residual_stream or apply_steering_vectors in the sampling parameters. The plugin extracts these, initialises relevant PyTorch hooks if they're not already set up (by adding a worker extension) and sends steering vectors directly to workers (vLLM typically has one worker per GPU).
2. Per-sample hook operations. vLLM dynamically batches tokens from multiple concurrent requests into a single forward pass, so a core challenge is "book-keeping" - working out which operations (e.g., activation extraction) should be applied to which parts of the forward pass. To do this we read the forward_context metadata, utilising the query_start_loc (a tensor of token boundaries per request) and req_ids (mapping batch index to request ID). We then, for example, apply steering to just the slices that correspond to the sample that requested it. Any extracted activations are moved to CPU RAM and compressed (lossless), ready to be requested by the vLLM scheduler process once generation for that sample has completed. Steering runs on all tensor-parallel ranks (since it modifies the forward pass), but capture operations only run on TP rank 0 (the residual streams are identical across TP replicas after all-reduce).
3. Response collation. The plugin intercepts the response before it is sent to the client, at which point it queries the relevant vLLM processes for any requested activations. It trims surplus activations, as vLLM does under the hook with tokens (the scheduler often gets ahead of the number of tokens it needs to generate, before stopping). Activations are then returned to the client.

Credits

Thanks to Satvik Golechha for the original idea of doing this with vLLM, and the nnsight team for inspiration. Thanks to Walter Laurito and Geoffrey Irving for valuable feedback.

1. ^{^}

   Defined as attempting to locate or alter information in a model without full understanding of how it is processed.

2. ^{^}

   In practice it’s more typical to run probes on a subset of generated tokens, but the scorer here runs on all tokens for simplicity.

3. ^{^}

    At the time of writing, it supports vLLM 15.1 only.

4. ^{^}

   vLLM automatically determines an appropriate dynamic batch size during execution (a behaviour inherited by vLLM-Lens). For the Hugging Face Transformers, nnsight (transformers version) and TransformerLens libraries, we instead perform a simple search procedure: beginning at a batch size of 512 and iteratively halving until the run completes without GPU out-of-memory errors, after which we report the runtime of the largest successful configuration. For nnsight (vLLM backend), dynamic batching follows vLLM’s default behaviour and does not trigger GPU memory issues; however, CPU memory limits can still be encountered, which we resolved by manually calculating the most efficient batch size.

5. ^{^}

   We think this was likely mostly due to the issues addressed by https://github.com/ndif-team/nnsight/pull/652 , and that we had to enable batching to avoid out-of-memory issues as a result of these issues. A provisional experiment with the version of nnsight from that PR found performance to be the same as vLLM-Lens with a single-GPU test, but nnsight was 1.9x slower with a 4-GPU test (TP=4).

2026-04-24 02:44:01

Discuss

2026-04-24 02:37:58

Notes on The Autobiography of Benjamin Franklin

When a dear friend read The Autobiography of Benjamin Franklin, he said it reminded him of me. When he read The Jailbroken Guide to the University, he told me I should definitely read the book.

So, I did.

Franklin figured out life, and that’s what his book is about.

There are many important lessons, but the most important ones for me are: (1) do not confute people, (2) present your ideas as public-spirited proposals, and (3) the fastest way for a poor or unknown person to rise is not talent or luck, but being so honest and reliable that powerful people trust you with their business.

Now, here are my notes:

1. How to communicate & persuade

• Do not confute with other people. No one likes losing. This is a great way to make enemies.
• To persuade, never use the words certainly, undoubtedly, or any other words that make you sound confident. Use in my opinion, it appears to me, I should think so, I imagine it to be so, if I am not mistaken, etc.
• The purpose of conversation is “to inform or to be informed, to please or to persuade.”
• You can be damn sure of what you’re talking about, but “speak with seeming diffidence.”
• • When Franklin was younger, he was really good at arguing. He would corner people with logic and win debates, but it made people annoyed and resistant. Over time, he realized that being right wasn’t actually helping him persuade anyone. So he changed his approach: he stopped using strong, certain language and instead spoke modestly (“I conceive or apprehend a thing to be so and so; it appears to me, or I should think it so or so, for such and such reasons; or, I imagine it to be so; or it is so, if I am not mistaken”). That shift made people less defensive, more willing to listen, and he became much more effective at getting his ideas accepted.
  • “Immodest words admit of no defense, For want of modesty is want of sense."
  • Present your opinions modestly so they can be better received with less contradiction.
• If people are going to judge you based on who you are, take your name out of it and let the work prove itself first.
• • As a boy, Franklin knew his brother would probably reject his writing if it came openly from him, so he slipped anonymous pieces under the print-shop door at night. They were praised, guessed to be written by learned men, and only later revealed as his.
• Let your work prove itself through results. Don’t waste time arguing, because real evidence will outlast criticism.
• • Franklin’s experiments on electricity were attacked by a well-known French scientist who wrote a whole book against him. Franklin started writing a response, but stopped. He realized his work was based on experiments anyone could repeat, so instead of arguing, he kept working. Over time, other scientists verified his results, his ideas spread across Europe, and the same institutions that had ignored him ended up honoring him. He won without ever engaging in the fight.
• Narration and dialogue, as methods of writing, are very engaging to the reader.
• Win arguments by asking questions instead of giving answers. Ask lots of questions so people find the answer themselves.
• • Franklin used the Socratic method on a guy who loved arguing, asking questions that slowly boxed him into contradictions. It worked so well that the guy started getting paranoid, refusing to answer even simple questions without asking what Franklin was trying to infer.
• Learn to write.
• • If you can clearly explain a good idea at the right moment, you can move public opinion, beat wealthier opponents, and create opportunities for yourself.
  • There was a fight in Pennsylvania over issuing more paper money: ordinary people wanted it because it would increase trade and jobs, while wealthy creditors opposed it because they feared depreciation. Franklin took the pro-currency side, wrote an anonymous pamphlet arguing for it, and it was so persuasive that the opposition weakened, the measure passed, and he was rewarded with the profitable job of printing the new money, showing him that being able to write clearly could translate directly into influence and income.
• Never contradict the sentiment of others.
• “Disputing, contradicting, and confuting people are generally unfortunate in their affairs. They get victory sometimes, but they never get good will, which would be of more use to them.”
• Present your idea as a public-spirited proposal already supported by others and open to improvement, so people can adopt it without feeling they’re advancing your ego.
• • Franklin was trying to start a shared library where people would pool money to buy books together. He noticed people resisted when it looked like his idea, because supporting it might raise his reputation above theirs. So he stepped back and presented it as a plan from “a number of friends.”1 Once he did that, support came easily, and the project took off. Let the project move forward smoothly while the credit finds its way back to you later.
  • In Proposals Relating to the Education of Youth in Pensilvania (1747), he presents the academy plan not as his own scheme, but as a civic proposal already approved by others and offered for public input, making it far easier to gain support and eventual adoption.
  • • Removes personal ownership → avoids ego resistance
    • Adds social proof (“publick-spirited Gentlemen”2) → lowers risk
    • Opens it for advice → invites participation
    • Positions himself as facilitator (“Printer”) → builds trust
    • Anchors it in public good → makes support morally easy
  • Frame your idea as something already endorsed, collectively owned, and open to advice, so others can support, refine, and eventually carry it forward as their own.
  • A personal example.3

2. How to build reputation & trust

• Reputation = Reality + Visible Signals.
• Be actually industrious and frugal and avoid all appearances to the contrary.
• • When Franklin started his printing business, he worked hard, but he also made sure everyone could see it. He dressed simply, avoided idle entertainment, stayed visibly busy, and even pushed a wheelbarrow through the streets himself. Because of that, merchants trusted him, gave him credit, and helped him grow while his competitor collapsed.
  • People will always want to help young, humble, hard-working people.
• Get away from the Samuel Michels of the world. They think they’re smart because they always find a reason for things to fail. Be around yes people who give you energy.
• • When Franklin started his printing business, an older, respected man named Samuel Mickle warned him that Philadelphia was collapsing, and the whole thing would fail. The warning almost discouraged him, but the city grew, his business succeeded, and Mickle, who had been predicting ruin the whole time, later paid far more for a house than he could have earlier.
• Be humble and control your pride.
• • You will never kill your pride4, but at least give the appearance of doing so. Learn to manage it and disguise it. After all, Franklin was proud of his humility.
  • Franklin realized pride was his biggest flaw after a friend told him he came off as overbearing in arguments. He added humility to his list of virtues and tried to control it.
• A life shaped by good habits—industry, frugality, sincerity, and good temper—naturally earns trust, reputation, and a kind of happiness that others are drawn to.
• “Vicious actions are not hurtful because they are forbidden, but forbidden because they are hurtful.”
• • Virtue is practically useful, not just morally good. Franklin says that harmful actions aren’t forbidden arbitrarily. They’re forbidden because they actually damage your life and chances of happiness. In a world where merchants, governments, and powerful people constantly need trustworthy individuals to manage their affairs—and where such people are rare—those who develop probity and integrity are far more likely to rise, gain responsibility, and build their fortunes.
• Being honest and trustworthy is one of the most reliable ways to have a good life5 because people with money and power are always looking for someone to rely on.
• • What looks like “luck” (getting opportunities, rising fast, and being trusted is often simple: being the kind of person others can trust when the stakes are high).
  • In other words, your “luck” increases when people feel safe putting responsibilities in your hands.
  • The fastest way for a poor or unknown person to rise is not talent or luck, but being so honest and reliable that powerful people trust you with their business.
  • This is one of the most important ideas from the book.
• “He that has once done you a kindness will be more ready to do you another, than he whom you yourself have obliged.”
• • Franklin had a political rival in the Assembly. Instead of trying to win him over directly or fight back, Franklin asked to borrow a rare book from his library. The guy agreed, Franklin returned it with a thank-you, and from that point on, the man became friendly and helpful toward him. That one small favor flipped the relationship.6
  • This reminded me of something people used to say in high school: If you like someone, ask to borrow a pencil. Franklin figured this out centuries ago.
• On partnerships:
• • Often finish in quarrels.
  • Avoid that by “explicitly settled, in our articles, every thing to be done by or expected from each partner, so that there was nothing to dispute.”
• Introduce your ideas to your group of friends, let it circulate through them, and as they gain influence, they gradually annex it as their own.
• • Franklin noticed problems in Philadelphia. The night watch was ineffective, and fires were common. He brought these ideas to his small group, the Junto. From there, the ideas spread to other clubs, often presented as if they came from each group. They weren’t adopted right away, but over time, as those members gained influence, the ideas turned into actual laws and institutions. For instance, his fire protection idea eventually led to Philadelphia’s first volunteer fire companies.
• Never ask, never refuse, nor ever resign a public office.
• In matters of credit, it’s often wiser to let others believe they led the success than to insist on recognition and create conflict.
• You can be in conflict with someone and still work with them if you treat the disagreement as roles, not as personal enmity.
• Your reputation is shaped by the system you’re in, not just your intentions.
• • In systems where most people act for personal gain, even honest actions are assumed to be self-interested.
  • Franklin helped supply the army and didn’t take a commission, but the officer he dealt with didn’t believe him because, in that system, most people did make money that way. Franklin later admits he learned that large fortunes were commonly made in those roles, so the suspicion was not irrational. It was the norm.
• If you don’t tell people about your ideas, they won’t know about them.
• • For every project, Franklin would write something, a newspaper article or a pamphlet.
  • Share your ideas. Learn from Tyler, The Creator.
• On fundraising:
• • “In the first place, I advise you to apply to all those whom you know will give something; next, to those whom you are uncertain whether they will give any thing or not, and show them the list of those who have given; and, lastly, do not neglect those who you are sure will give nothing, for in some of them you may be mistaken.”
• On management:
• • Keep people productively engaged (cough cough busy) because when there’s nothing to do, energy turns into complaints, conflict, and disorder.
• Always call people by their made-up societal titles, such as Doctor, Professor, Captain, Director, etc.7 If you’re not sure, still call them by such. Otherwise, you give them an excuse to ignore you.
• • Franklin was pressing a complaint against the Pennsylvania proprietors over taxation, arguing their estates should be taxed like everyone else’s. Instead of engaging the substance, their side seized on a technicality: he hadn’t addressed them with their full formal titles (“True and Absolute Proprietaries”). They used that breach of etiquette to label him disrespectful and stall the issue, delaying the case and sidestepping his argument
• Don’t use your work to bring other people down. Focus on creating something useful and worthwhile instead, even if negativity would get more attention or money.8

3. How to build a life

• Read one or two hours every day.
• • Franklin used reading to make up for the formal education he didn’t get while building his business.
  • Reading was also Franklin’s only amusement. No bars, games, or “frolicks.”
• Spend all your money on books.
• Industry + Frugality = Wealth.
• “After getting the first hundred pound, it is more easy to get the second.”
• “It is hard for an empty sack to stand up-right.”
• Be parsimonious.
• • If you don’t spend money, you don’t need to make money.9
• Don’t drink alcohol.
• • You’ll work harder, save money, and have a healthier life.
• If you’re interesting and provide interesting conversations, your company will always be sought after.
• • How do you make yourself interesting? According to Franklin, reading.
  • To be an interesting conversationalist, read a lot.
• When you work hard, provide interesting conversations, and are a good influence, people will want themselves and their offspring around you. You’ll be offered good opportunities, invited into their homes, businesses, and inner circles.10
• The only real competition in the long run is yourself.
• • In the printing business, people lived beyond their means, did not keep focused, and sooner or later, would go out of business.
• Have good friends and mentors.
• Arrange your conduct to suit your whole life.
• “Seest thou a man diligent in his business? he shall stand before kings; he shall not stand before mean men.”
• • Franklin’s father used to repeat this proverb to him growing up, so he saw hard work as the path to wealth and distinction. He took it seriously, but never literally. Decades later, he stood before five kings and had dinner with one.
• Find a hardworking and frugal partner.
• • Franklin said he was lucky to find a hardworking and frugal partner. Together, they worked on their businesses and lived simple lives.
  • “He that would thrive must ask his wife.”
• “The most acceptable service of God was the doing good to man.”
• Franklin wanted to arrive at “moral perfection.”
• • He created a list of virtues, focused on mastering one at a time, and conducted daily examinations.

1. TEMPERANCE. Eat not to dullness; drink not to elevation.

2. SILENCE. Speak not but what may benefit others or yourself; avoid trifling conversation.

3. ORDER. Let all your things have their places; let each part of your business have its time.

4. RESOLUTION. Resolve to perform what you ought; perform without fail what you resolve.

5. FRUGALITY. Make no expense but to do good to others or yourself; i.e., waste nothing.

6. INDUSTRY. Lose no time; be always employ'd in something useful; cut off all unnecessary actions.

7. SINCERITY. Use no hurtful deceit; think innocently and justly, and, if you speak, speak accordingly.

8. JUSTICE. Wrong none by doing injuries, or omitting the benefits that are your duty.

9. MODERATION. Avoid extreams; forbear resenting injuries so much as you think they deserve.

10. CLEANLINESS. Tolerate no uncleanliness in body, cloaths, or habitation.

11. TRANQUILLITY. Be not disturbed at trifles, or at accidents common or unavoidable.

12. CHASTITY. Rarely use venery but for health or offspring, never to dulness, weakness, or the injury of your own or another's peace or reputation.

13. HUMILITY. Imitate Jesus and Socrates.



—The Autobiography of Benjamin Franklin

• Franklin thought God was the “fountain of wisdom” and thought it was “right and necessary to solicit his assistance for obtaining it.” So he created this prayer:

"O powerful Goodness! bountiful Father! merciful Guide! increase in me that wisdom which discovers my truest interest. strengthen my resolutions to perform what that wisdom dictates. Accept my kind offices to thy other children as the only return in my power for thy continual favors to me."

• Better to aim at perfection and fall short than to settle for mediocrity. The effort itself makes you far better than you would have been otherwise.
• “Every part of my business should have its allotted time.”11

Benjamin Franklin’s daily schedule.

• A person of ordinary ability can accomplish great things by making a plan and cutting off every distraction to focus entirely on executing it.
• You are not truly disciplined until the right behavior becomes a habit. Before that, your impulses will keep pulling you off track.
• Learn French, Italian, or Spanish before Latin, as it will make learning Latin easier.
• “Truth, sincerity and integrity in dealings between man and man were of the utmost importance to the felicity of life.”
• “That, as we enjoy great advantages from the inventions of others, we should be glad of an opportunity to serve others by any invention of ours; and this we should do freely and generously.”
• “Human felicity is produc’d not so much by great pieces of good fortune that seldom happen, as by little advantages that occur every day.”
• • Mic drop.

A Printer’s Kid

Franklin wrote this book before the American Revolution, before the United States of America, and before most of the things you probably remember him for.

He was a printer’s kid from Boston with no money, no connections, and no formal education. And yet, by the time he died, he had helped birth a nation, charmed a monarchy into funding a revolution, figured out that lightning was electricity, and built the civic infrastructure of a city: a library, a fire company, a university, a hospital, a learned society, a mutual insurance company, paved and lit streets, and a postal system that became the information infrastructure of a democracy.

My friend told me this book reminded him of me. I’m still not sure if that’s a compliment or a challenge. Probably both.

What I do know is that Franklin figured out how humans work and wrote it for us as a manual. The best books are conversations, and this book was a conversation I needed to have. I needed the reminders, I needed to learn from his stories and experiences so I don’t repeat the same mistakes, and I’m glad I finally read it.

Read the book. Take notes. Then go be useful.

------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

I read the book and made a few projects, which I think you might enjoy:

• My Vision Is The Problem | Short Film

• The Lost Chapter of Benjamin Franklin’s Autobiography: An Imagined Continuation, 1757–1790.

• FranklinRPG

• Poor Richard’s Almanack

FOOTNATES

1

This is simply one of the best passages:

The objections and reluctances I met with in soliciting the subscriptions, made me soon feel the impropriety of presenting one’s self as the proposer of any useful project, that might be suppos’d to raise one’s reputation in the smallest degree above that of one’s neighbors, when one has need of their assistance to accomplish that project. I therefore put myself as much as I could out of sight, and stated it as a scheme of a number of friends, who had requested me to go about and propose it to such as they thought lovers of reading. In this way my affair went on more smoothly, and I ever after practis’d it on such occasions; and, from my frequent successes, can heartily recommend it. The present little sacrifice of your vanity will afterwards be amply repaid. If it remains a while uncertain to whom the merit belongs, some one more vain than yourself will be encouraged to claim it, and then even envy will be disposed to do you justice by plucking those assumed feathers, and restoring them to their right owner.



—The Autobiography of Benjamin Franklin

2

If you remember this idea and nothing else, your life and business will dramatically improve.

In the introduction to these proposals, I stated their publication, not as an act of mine, but of some publickspirited gentlemen, avoiding as much as I could, according to my usual rule, the presenting myself to the publick as the author of any scheme for their benefit.



The subscribers, to carry the project into immediate execution, chose out of their number twenty-four trustees, and appointed Mr. Francis, then attorney-general, and myself to draw up constitutions for the government of the academy; which being done and signed, a house was hired, masters engag'd, and the schools opened, I think, in the same year, 1749.

—The Autobiography of Benjamin Franklin

I read a passage like this, and man, I start wondering, did Franklin secretly read The Prince?

I start looking, and it turns out I’m not the only one who also found Franklin to be a bit “Machiavellian.”

According to my research, there is no primary source establishing that he definitely read The Prince. But as you start reading his autobiography, Franklin often feels Machiavellian, not in the cartoon sense of “evil manipulator,” but in a deeper sense of understanding human nature. A few examples:

• He is obsessed with effects, not just intentions.
• He cares intensely about reputation, appearances, timing, and persuasion.
• He treats virtue partly as something that must be made socially useful and operational, not merely admired.
• He constantly practices indirect power: nudging, framing, softening, deflecting, joking, letting others think an idea is theirs.

That is very close to a Machiavellian style of intelligence. Personally, I could find examples of things that went wrong because I didn’t learn those four lessons from above. More specifically, to say “I created something because I wanted to” might be true, but to other people sounds like “I’m a piece of shit who doesn’t care about other people.” Therefore, this is how we get modern equilibrium about everyone saying they want to help people. It’s not that it’s not true. It’s that it’s false in a way they think it’s true, and to me, that will always be a more serious crime. But Franklin would never say that, and I applaud him for it.

Steven Forde describes Franklin’s outlook as a kind of “Machiavellian civic virtue.” Not the warlike, brutal virtue of classical Machiavelli, but a moderated version suited to commerce, comfort, sociability, and republican life.

Franklin is Machiavellian in at least four ways.

1. He self-creates.
   
   Like Machiavelli’s admired founders and political actors, Franklin is relentlessly a maker of himself. The Autobiography is a carefully shaped demonstration of how to rise. It’s a manual for self-construction.
2. He moralizes strategy rather than abandoning morality.
   
   Franklin is not Machiavelli if by that we mean “cruel” or “nihilistic.” He is much gentler. But he does share Machiavelli’s realism that moral life has to work in the world of vanity, incentives, institutions, and ego. He domesticates Machiavelli.
3. He understands vanity as a political tool.
   
   One of Franklin’s great talents is seeing that people want honor, status, ease, admiration, and belonging. He doesn’t merely condemn those desires; he channels them. That is very Machiavellian.
4. He treats humility itself strategically.
   
   This is maybe the most Franklinian and Machiavellian thing of all. His famous “humility” is often performative, tactical, and socially intelligent. He knows that open domination produces resistance, while modesty disarms people. I wouldn’t go as far as saying it’s hypocrisy, but it’s definitely a social technique.

What both Franklin and Machiavelli are obsessed with is human nature. Specifically, Franklin understands ambition, weakness, image, fear of envy, the uses of moderation, and the need to convert private striving into public usefulness.

You read the Autobiography, and you will think it’s warm, witty, and wholesome on the surface. But beneath that surface is someone extraordinarily calculating about how character is built, displayed, and rewarded. Let me give you an example:

In order to secure my credit and character as a tradesman, I took care not only to be in reality industrious and frugal, but to avoid all appearances to the contrary. I drest plainly; I was seen at no places of idle diversion. I never went out a fishing or shooting; a book, indeed, sometimes debauch'd me from my work, but that was seldom, snug, and gave no scandal; and, to show that I was not above my business, I sometimes brought home the paper I purchas'd at the stores thro' the streets on a wheelbarrow. Thus being esteem'd an industrious, thriving young man, and paying duly for what I bought, the merchants who imported stationery solicited my custom; others proposed supplying me with books, and I went on swimmingly.



In the meantime, Keimer's credit and business declining daily, he was at last forc'd to sell his printing house to satisfy his creditors. He went to Barbadoes, and there lived some years in very poor Circumstances.



—The Autobiography of Benjamin Franklin

Imagine seeing Franklin with a wheelbarrow on the streets. He’s making a show!!! He’s so clever, and he got what he wanted to be thought of as hard-working and industrious. He understands that not everyone is living life intrinsically, so he plays the game. Thankfully, he never lost himself in the world of charades and had a deeper purpose, and playing that game was just his way of getting to that purpose.



Let me give you another example:

My list of virtues contain'd at first but twelve; but a Quaker friend having kindly informed me that I was generally thought proud; that my pride show'd itself frequently in conversation; that I was not content with being in the right when discussing any point, but was overbearing, and rather insolent, of which he convinc'd me by mentioning several instances; I determined endeavouring to cure myself, if I could, of this vice or folly among the rest, and I added Humility to my list) giving an extensive meaning to the word.



I cannot boast of much success in acquiring the reality of this virtue, but I had a good deal with regard to the appearance of it. I made it a rule to forbear all direct contradiction to the sentiments of others, and all positive assertion of my own



—The Autobiography of Benjamin Franklin

In chapter 18 of The Prince, Machiavelli says it is not necessary to possess every virtue, but it is very necessary to appear to possess them. Franklin, in one of the most revealing passages in the Autobiography, says he never had much success acquiring the reality of humility, but he had “a good deal with regard to the appearance of it.” He then explains the technique: avoid direct contradiction, speak with diffidence, and present views modestly because that gets better reception and helps persuade people into measures you want to promote.

Should I go on? Sure, why not?

The objections and reluctances I met with in soliciting the subscriptions, made me soon feel the impropriety of presenting one's self as the proposer of any useful project, that might be suppos'd to raise one's reputation in the smallest degree above that of one's neighbors, when one has need of their assistance to accomplish that project. I therefore put myself as much as I could out of sight, and stated it as a scheme of a number of friends , who had requested me to go about and propose it to such as they thought lovers of reading. In this way my affair went on more smoothly, and I ever after practis'd it on such occasions; and, from my frequent successes, can heartily recommend it. The present little sacrifice of your vanity will afterwards be amply repaid. If it remains a while uncertain to whom the merit belongs, some one more vain than yourself will be encouraged to claim it, and then even envy will be disposed to do you justice by plucking those assumed feathers, and restoring them to their right owner.



This library afforded me the means of improvement by constant study, for which I set apart an hour or two each day, and thus repair'd in some degree the loss of the learned education my father once intended for me. Reading was the only amusement I allow'd myself. I spent no time in taverns, games, or frolicks of any kind; and my industry in my business continu'd as indefatigable as it was necessary. I was indebted for my printing-house; I had a young family coming on to be educated, and I had to contend with for business two printers, who were established in the place before me.



—The Autobiography of Benjamin Franklin

In chapter 21 of The Prince, Machiavelli says rulers gain esteem through “great enterprises” and by setting a striking example. Franklin’s equivalents are not wars but institutions: the library, hospital, clubs, civic improvements, and his other public schemes. He repeatedly builds authority by attaching himself to projects that visibly benefit the public. More Machiavellian still, he says that when pushing the library he put himself “out of sight” and presented it as the scheme of friends, because the project would go more smoothly that way; the temporary sacrifice of vanity, he says, gets repaid later. That is classic indirect power. He understood people would not support the library if it were “his” project, so he took himself out of the picture, got the library project done, and used the library one or two hours a day to continue learning and improving himself.

(Big sigh)



Ok, last example, I promise.

You probably recognize this line from Machiavelli: “Better to be Feared Than Loved.” Franklin would probably change it to: Better to not be Hated than Loved.

He had some reason for loving to dispute, being eloquent, an acute sophister, and, therefore, generally successful in argumentative conversation. He had been brought up to it from a boy, his father, as I have heard, accustoming his children to dispute with one another for his diversion, while sitting at table after dinner; but I think the practice was not wise; for, in the course of my observation, these disputing, contradicting, and confuting people are generally unfortunate in their affairs. They get victory sometimes, but they never get good will, which would be of more use to them. We parted, he going to Philadelphia, and I to Boston.



—The Autobiography of Benjamin Franklin

In chapter 17 of The Prince, Machiavelli’s core point is that attachment is unstable, so the safer aim is to avoid hatred and keep people from turning on you. Franklin says something very similar in social terms: disputatious people may win arguments, but they “never get good will,” which would be more useful. His father’s advice was also to seek “general esteem.” So Franklin translates Machiavelli’s problem from state violence into reputation management inside a democracy: not “make them fear you,” but “never provoke needless resentment.”

To say Franklin was Machiavellian isn’t quite right, but he takes that energy and reroutes it away from conquest toward:

• self-making
• sociability
• civic projects
• credit
• democratic legitimacy
• commercial success

People say Franklin is America’s first self-made man, but I think he’s America’s Machiavelli with charm.



Franklin is Machiavellian above all in technique. He understands that people are vain, resentful, status-conscious, and resistant to being openly ruled. He therefore learns to lead without seeming to dominate, to persuade without frontal collision, to gather influence through public benefit, and to turn appearance into a moral and political tool.



In today’s language, we wouldn’t say Franklin was Machiavellian. No, no. We would say he is emotionally intelligent. He has a high EQ. Yes, Benny. You scored high on your EQ exam, sweetie. Good boy! Now, let’s get you some ice cream.

But don’t get confused, to be “emotionally intelligent” is to be secretly Machiavellian.



(wink, wink) I know what you’re doing. Heck, you probably even read Daniel Goleman’s book. But that is too domesticated. Way domesticated. If you are out for blood, just go read The Prince. If you want a more chill version, The Autobiography will do you well.



Don’t get domesticated. Better yet, appear to be domesticated, just the way good old Ben Franklin would have done.

3

Let me give you an example from The Jailbroken Guide to the University. I had a chapter called My Vision For The University that started like this:

If we don’t know what we want, we don’t go anywhere.

So, what is our vision? What is the “pretty girl” we’re after?

This is my vision for the university.

A university must have a compelling vision guiding it, or it risks wandering aimlessly.

After reading Franklin, I changed My Vision For The University to A Vision for The University.

Now, it starts like this:

The main difference between those who go far and those who don’t is simple. Some people have a vision and others don’t, so they can only react to whatever is happening right in front of them.

Richard Hamming, in his reflections on what separates productive scientific careers from merely busy ones, put it this way: a drunken sailor taking random steps will end up about √n steps from where he started. But if there is a pretty girl in one direction, his steps will tend to go that way and he will travel a distance proportional to n. The vision, not the effort, accounts for most of the difference.

In a lifetime of many, many independent choices, small and large, a career with a vision will get you a distance proportional to n, while no vision will get you only the distance √n.

— Richard Hamming, The Art of Doing Science and Engineering

If we don’t know what we want, we don’t go anywhere.

So, what is our vision? What is the “pretty girl” we’re after?

After being privately discussed among several faculty, leadership, and people committed to the future of higher education, it seemed worth putting these ideas into writing for broader consideration.

What follows is offered not as a finished plan, but as a starting point for a conversation that seems worth having. Those who have something to add, correct, or improve upon are warmly invited to do so.

The latter is obviously much better. Thanks, Benny.

4

No one of our natural passions so hard to subdue as pride. Disguise it, struggle with it, beat it down, stifle it, mortify it as much as one pleases, it is still alive, and will every now and then peep out and show itself; you will see it, perhaps, often in this history; for, even if I could conceive that I had compleatly overcome it, I should probably be proud of my humility.



—The Autobiography of Benjamin Franklin

5

So Good They Can’t Ignore You < So Reliable They Can’t Ignore You

That would be an interesting idea to share with the world. So many people are focused on degrees, majors, skills, networking, and as the world enters a new era where artificial intelligence will make you “good” at whatever it is you want, it is those who are reliable who will be able to make other humans trust them with their business, and machines trust them with their training.



I’m down to write this book. Here’s my Stripe link.

6

You read a passage like the following, and you start thinking, “Yes, of course this is the most obvious thing in the world,” and this is when I must remind you once again: Franklin figured out life. He figured out human nature, wealth, politics, science, business, and many others.

Learn how Franklin won over someone who initially disliked him:

My first promotion was my being chosen, in 1736, clerk of the General Assembly. The choice was made that year without opposition; but the year following, when I was again propos’d (the choice, like that of the members, being annual), a new member made a long speech against me, in order to favour some other candidate. I was, however, chosen, which was the more agreeable to me, as, besides the pay for the immediate service as clerk, the place gave me a better opportunity of keeping up an interest among the members, which secur’d to me the business of printing the votes, laws, paper money, and other occasional jobbs for the public, that, on the whole, were very profitable.

I therefore did not like the opposition of this new member, who was a gentleman of fortune and education, with talents that were likely to give him, in time, great influence in the House, which, indeed, afterwards happened. I did not, however, aim at gaining his favour by paying any servile respect to him, but, after some time, took this other method. Having heard that he had in his library a certain very scarce and curious book, I wrote a note to him, expressing my desire of perusing that book, and requesting he would do me the favour of lending it to me for a few days. He sent it immediately, and I return’d it in about a week with another note, expressing strongly my sense of the favour. When we next met in the House, he spoke to me (which he had never done before), and with great civility; and he ever after manifested a readiness to serve me on all occasions, so that we became great friends, and our friendship continued to his death. This is another instance of the truth of an old maxim I had learned, which says, “He that has once done you a kindness will be more ready to do you another, than he whom you yourself have obliged.” And it shows how much more profitable it is prudently to remove, than to resent, return, and continue inimical proceedings.



—The Autobiography of Benjamin Franklin

7

I got burned back when I was in high school.



I didn’t know English that well, and more importantly, I didn’t understand the societial notion that if someone did a PhD, you needed to call them Doctor!!! I was looking for a summer internship, so I emailed (and followed up) a bunch of scientists from Fermilab and Argonne to see if they would give me an internship. The greatest offense was not asking for an internship, but the fact that I started the email with Mr. or Ms.

Oh boy.

A few days later, I got an angry reply. One scientist informed me—very clearly—that he didn’t spend more than half a decade earning a PhD just to be called Mr.

I should refer to him as Dr. [Last Name].

Ouch.



Most people didn’t reply; however, some people did reply, and this is one of the emails that I received:

Sorry I didn’t respond before, since a handful of people I know have also gotten your email I asked the Fermilab cybersecurity team to make sure it wasn’t spam. I can tell you’re pretty persistent, which is good if you want to go into the sciences!

Persistent? What this scientist was referring to was that I had emailed a lot of people and had followed up at least two times, and many people were annoyed, some angry, and others confused.



I imagine people at Fermilab and Argonne were having conversations like this:

Genius Scientist: I just got an email from some random high schooler.

Another Genius Scientist: Oh, same. Kid actually seems pretty awesome.



Yet Another Genius Scientist: I reported his email to spam. The cybersecurity team is looking into him.



(the first two scientists look at each other)



Genius Scientist: …



Another Genius Scientist: …



Yet Another Genius Scientist: anyways—back to the Higgs boson.

Anyways, that scientist who replied to me was awesome, and helped me in so many wonderful ways. One specific example was about a Google program for high schoolers where I learned about p5.js, networking, and websockets. This is also how I got the Google backpack and water bottle I used throughout college.

I mention this story for two reasons: (1) always call people by their titles. If you’re not sure, still call them by whatever you think they might have. My favorite thing in college was calling the TAs as Dr. (last name). They would get slightly embarrassed and would often correct me, but they certainly enjoyed it. And (2) this is an important lesson from Franklin, people will always want to help young, humble, hard-working people. I appreciated the fact that this scientist was kind enough to reply and even get on phone calls with me. Unfortunately, that particular scientist didn’t have work for me; however, I would have never found out about the Google program had I not reached out. And I’m glad I did, because of the mentorship, and the burn, because I definitely learned that when in doubt, everyone’s Dr. until proven otherwise.

8

Tyler Cowen calls this the “negative emotional contagion.”

Recently, Tyler was asked at an event in Chicago whether he actually tries to be positive and what his opinion was on Zvi Mowshowitz after some public disagreement they had. He says he tries to be positive (probably due to his own genetics) and mentioned that he doesn’t talk negatively about anyone publicly. Of course, he didn’t say anything bad about Zvi.

One of the reasons for Tyler’s success is that he’s likeable and tries to be that way while avoiding the negative emotional contagion.

9

“I can think, I can wait, I can fast.” “Is that all?” “I think that is all.”

“And of what use are they? For example, fasting, what good is that?”

“It is of great value, sir. If a man has nothing to eat, fasting is the most intelligent thing he can do. If, for instance, Siddhartha had not learned how to fast, he would have had to seek some kind of work today, either with you, or elsewhere, for hunger would have driven him. But as it is, Siddhartha can wait calmly. He is not impatient, he is not in need, he can ward off hunger for a long time and laugh at it. Therefore, fasting is useful, sir.”



—Herman Hesse, Siddharta

10

This has happened to me countless times.



A few weeks ago, I was at this conference, and funnily enough, I ended up in a really interesting conversation with an entrepreneur. I was mostly listening, just trying to learn as much as I could.



Humbly, I thought a few ideas from The Autobiography of Benjamin Franklin and a bunch of other books I’d read might possibly help his business. So a few days later, I sent him a detailed email with a handful of important ideas and how he could apply them to his business to be more efficient, more productive, and, ultimately, make more money.



He replied.



Invited me to this private event.

Once again, Franklin figured out life, and especially young people can learn from him. I was constantly surprised by how many “wrong” things I was doing.

11

Inspired by this and the weekly schedule from The 7 Habits of Highly Effective People, I created my own version.

2026-04-24 00:44:21

1. If you are reading this post, odds are you are not actually someone who would self-identify as a creative type. But perhaps your way of thinking about the tension between job, career, and the actual work you think is important to do has much in common with the bind that creative types have been in since time immemorial, such that it could be useful to think in such a lens anyways. Now may be an unusually good time for people who have important work they want to do to familiarize themselves with patronesque frameworks, because many new patrons are about to come online, and many of them will think about the world in ways similar to you.
2. Like most creative types throughout history, all I want is to be able to work at what I think is important to work on without ever having to worry about money. Like most creative types throughout history, having not been born into a family of significant means, such a life is not by default available for me. Therefore, I will have to do what most creative types do throughout history, and compromise. I compromise in ways very similar to what creative types have done throughout history: I have job that pays the bills, and I cultivate relationships with patrons who sponsor my work.
3. Rembrandt, Michelangelo, Bernini: even those who were at the apex of their craft spent a lot of their working hours working on things that they didn't care about, because that's what their patrons wanted them to do (depict guys they didn't give a shit about, in ways that required long term travel to miserable far-away locales if they were unlucky) or because it paid the bills (run an apprentice shop to train the next generation of artisans, instead of making art). It's helpful to be calibrated on the amount of entitlement that you should feel. I am hardly a Rembrandt, why would I feel entitled to more of my own time than he did? If you do feel more entitled, you should be able to justify why.
4. An American socialist poster that I really like from the 70s says the following:
   If you're unemployed it's not because there isn't any work. Just look around: A housing shortage, crime, pollution, We need better schools and parks. Whatever our needs, they all require work. There's work to be done... Yet, as long as employment is tied to somebody else's profits, the work won't get done.
   You do not need to be a socialist to observe that there is often misalignment between the most well-paid jobs and the work that is the most important to do. Of course, there is the important work of aligning our civilization to aim more of the capital flows at the important stuff by default. But there is other important work that also need to be done in the meantime (other forms of alignment, say), that some patrons will be happy to fund you to do.
5. Historically, my biggest patron has been the EA Infrastructure Fund (EAIF). I like their work, and I think they serve a really important function in the funding ecosystem as an entry point. But in general a large, corporate fund comprised of people you don't know, with a vision or angle of their own, is not typically where the most talented creative types get funding for their work. This is because established talents have access to pockets of money that are not legible to the outside world like the EA Infrastructure Fund, and their funders are not soliciting applications.
6. 1. Scott Alexander has been given unsolicited gifts from rich patrons, and also has a bunch of paid subscribers.
   2. Zvi has anonymous patrons supporting his writing on a full-time basis, which is why he publishes five times a week.
6. I would wager that Scott and Zvi do not need to write biannual reports summarizing their works of the past two quarters and the impact that they have achieved in ways aligned to their funding, to their patrons, the way I do. (Probably they will have dinner together every so often and talk about it casually, though.) This is good; I think funding bodies like EAIF, SFF, and Coefficient Giving have reasonably good taste, but I do not trust them to have a better idea of what Scott and Zvi should write about than Scott and Zvi do.
7. At the beginning of your working life you will be clueless and inexperienced and thus you will not have good taste in what is actually the most important thing for you to do. When you are just starting out I think it is very wise to go to an established large grantmaker, see what sort of work they like funding, and then do that sort of work with their money. Institutional grantmakers are good early on partially because their constraints (reporting requirements, legibility demands) force you to articulate what you're doing and why.
8. As you do that sort of work, you will become more competent and you will develop taste. You will develop a better handle of the work that you are unusually good at/can do much more competently than other people. So it becomes useful to increasingly ask two other questions:
9. 1. What sort of anti-patterns does my current patron unintentionally select for, that preclude me from doing the work that I think is the most good to do?^[1]
   2. Are there patrons out there that are more aligned with what I consider to be good work?
9. If you believe that the work you do is good, your end goal should be to find patrons who are entirely aligned with you. Patrons are, by definition, eager to convert their spare resources into work. Because there is mutual trust, and mutual agreement on the kind of work that is pleasing and beneficial, they will give you funding and then largely get out the way. To find aligned patrons, you should do high quality work, and you should be legible about doing good work.
10. Personal patronage has its own distortion patterns that are different from, and not necessarily better than, institutional ones. Additionally, those patterns will be much more difficult to write about, for reasons analogous to these ones. It is ideal to enter into them with sufficient leverage and/or a satisfactory BATNA. You should only enter them with people you know and trust.

I have been hosting rationality and EA meetups in some form or another since 2019. From 2023 onwards, this work has been funded by the EA Infrastructure Fund, which has allowed me to dedicate more time to it. As I organized more things, my idea of what sorts of community building is most useful has diverged from the EA community writ large.

I still intend to run high quality, ~traditional EA events because this is still clearly important to do, and I will continue to apply for grants to run them. But they will be a smaller fraction of all events that I will run, and I will be asking for commensurately less funding. The other kinds of events I want to run are more experimental and illegible, and I am more excited about them. A good friend has given me $10,000 to start them up, with no strings attached save their own anonymity.

1. ^{^}

   If you like your patron and they respect your input, you should consider dropping them a line about this.

2026-04-23 23:19:36

1 Organ shipment for brain

Suppose your brain is carefully chopped in 4 pieces, cooled to 4 C like they do with hearts and livers right now, and shipped chunk by chunk to some other place and then reassembled here. Then you get booted up from the coma and interviewed.

Do you die in that process? How about N pieces? At some point pieces become small enough to just ship information about them being more convenient.

The only two defensible points where you can think you "die" here in my opinion are, at the moment you go into coma and never. sleep is death actually

(I think this one I saw from Bright One guy 8 years ago, but with different details)

2 Saw-trap copy help

You will be copied, your copy will be placed into a Saw trap and will need to solve a known puzzle to get out. You can spend a lot of time solving it and then memorizing the solution, before copying happens. Would you bother?

3 Copy anticipation weirdness

Suppose you are in an elevator, which can stop either at green carpet floor or red carpet floor. You don't know which one and there is no indicator, it's a pretty broken elevator. So, you anticipate now, that when the door opens you will see either green room or red room, 50:50.

Is this situation equivalent to situation in which you would be copied, and copy would be let out into red room and the original into green room? Should you anticipate equivalently to see green room or red room, 50:50?

Is your copy in your actual, day to day, anticipated future?

Your copy can be seen by me as its own person, created from ground up with fake memories. That person would be ether thinking about itself as new person created with memories of you, or instance of you, or maybe something else. Is there a correct way to think about it? Or alternatively is it all just preferences?

Now, if your copy has some memories deleted, e.g. he doesn't remember your phone number, is that guy still in your anticipated future? Should you still anticipate 50 : 50 exiting from either door? How heavy should be the modification to you stop anticipating exiting the copy door?

4 Theseussing speed

Let's say there is this new drug that makes replacement of all your atoms 100 times faster, all your atoms get replaced every 30 days because someone put it in the water supply. Would you start valuing enjoyment / welfare of your future self less?

5 Copy merge?

There is a copy of you in a room lit with red lamp. You are in a room lit with green lamp. You both were there for a hour.

Now, using advanced neurological tools, you will be modified to remember being in a red room all that time. AND your room changes lighting color green -> red.

You exit the room. What room do you anticipate to exit from, the one labeled copy or the one labeled original? 50:50?

I got inspired to finally publish those from reading this post, which is pretty interesting, I recommend it.

https://www.greaterwrong.com/posts/CchB8MMNDAR3KtkLH/thought-experiments-on-continuity-of-consciousness