2025-12-14 07:39:24
Published on December 13, 2025 11:39 PM GMT
So, it's recently come out that Oliver Sacks made up a lot the stuff he wrote.
I read parts of The Man Who Mistook His Wife for a Hat a few years ago and read Musicophilia and Hallucinations earlier this year. I think I'm generally a skeptical person, one who is not afraid to say "I don't believe this thing that is being presented to me as true." Indeed, I find myself saying that sentence somewhat regularly when presented with incredible information. But for some reason I didn't ask myself if what I was reading was true when reading Oliver Sacks. Why was this?
The main reason I can think of is that the particular domain of Sacks, which I'd call neurology or the behavior of brain damaged patients, is one in which I had prior belief that A. incredible stuff does happen and B. we don't really understand. In particular, we have stuff like the behavior of split hemisphere patients and people like Phineas Gage. So my prior is that incredible things really do happen, and nothing Sacks said was any more unbelievable than these phenomena.
Also, for Musicophilia, the "domain" could additionally said to be music or humans' reactions to music, which again is something I think is pretty incredible and that we don't understand. Like, music is really powerful, why do we have such strong reactions to it? Why does it exist at all? Let me put it this way: music is so weird that if I hadn't experienced its effects first hand, I'd be inclined to think that the entire thing is "made up" and humanity is under some sort of mass delusion, confusion, or fraud.
The second reason I can think of is that something... the approach or voice or worldview or something else... about Oliver Sacks made me trust him; made me think he was generally sane and truthseeking and honest. I'm not entirely sure why this is. I'll be thinking about this more.
If you were like me and you were insufficiently skeptical of Oliver Sack's claims, it's worth asking: why did I make this mistake? Certainly this thing is relevant to the general rationalist project, to the goal of being less wrong. Or maybe you weren't like me, and you didn't believe Sacks. Well, why not? Don't just say "This isn't actually hard," because this is actually hard. Epistemics is hard! Under what principles or knowledge of the world did you not believe Sacks while also believing that split brain patients were a thing?
2025-12-14 07:16:11
Published on December 13, 2025 11:16 PM GMT
One of the main complaints I heard about the Inkaven Residency is that it put too much pressure on people to write too quickly. The fellowship was based on the premise that people who publish every day have historically gotten very good at writing. The problem is that ability to publish every day is strongly correlated with latent potential as a writer and ability to publish good things every day; people tend to publish only what they feel good about, so if someone is publishing every day, then many of them can likely produce things they feel good about very quickly.
When you actually try to make people write every day in hopes of making them into people who can publish high-quality thing frequently, then you run into Goodhart's law:
Any observed statistical regularity will tend to collapse once pressure is placed upon it for control purposes.
Or more commonly,
When a measure becomes a target, it ceases to be a good measure.
There is, however, a countervailing force, which made Inkhaven less bad than the above reason would suggest: practicing writing often makes you better at writing. Not always enough to make up for immense time pressure, but enough to do something at all.
The Halfhaven camp took the Inkhaven format but modified it to be fully remote and with posts required every other day, intended for those who "have a school / job / family / other excuse, and can't simply take a month off."
It's hard for me to tell with this rich and plentiful data, but I also suspect that Halfhaven created higher quality posts, since there are more people who can publish a good post every other day than there are people who can publish a good post every day. Some people take longer to think of good things, or tend to write about things that take more effort and time per unit blog-goodness, such as research or talking to a thousand people[1].
For the past ~4 months, since early August 2025, I've been writing a post every week, with two exceptions, once for a reasonable excuse and once for simply failing to Do The Thing. Instead of being kicked out of the program for failing to post, I simply lost[2] a small amount of money, calibrated to be the least amount of money that would get me to reliably post something.
I feel like this has allowed me to make much higher-quality writing that often requires significant research and developing new ideas, while also pressuring me to put out my shorter-form ideas.
Going for writing extremely often is not always the best strategy, but writing as fast as you can sustain makes you better at writing. Balancing frequency with quality standards is important. I would like to see more people doing commitment devices, but I don't want people to be scared off by the reports of Inkhaven being too difficult for some people.
Although that particular author seemed to have done that very quickly, so maybe this isn't a good example.
Specifically, paid two of my friends for the service of embarrassing me for not publishing anything.
2025-12-14 07:05:13
Published on December 13, 2025 11:05 PM GMT
AI is changing the way we interact with online content. There are the gloomy waves of AI slop, the TikTokification of video content, the ad-maxing-prompt-injection-cyberwar, yes, but also the prospect of fulfilling the web’s promise: making content come to life.
From TV to YouTube to …
I am particularly interested in how AI can open an ecosystem of new ways content can be presented. The web broke the centralization of content production enforced by TV channels and newspapers, allowing anyone to become a content creator. The big platforms offer different expression media: blog posts on Substack, videos on YouTube, short forms on TikTok, photos on Instagram, etc. But platforms continue to enforce an economic centralisation, controlling the redistribution of revenues from advertisements. And beyond their economic power, platform continue to enforce a centralization on the presentation of the content.
Content is forced to fit within the constraints of the platform: a video, a blog, a short form, a tweet, etc. Everything surrounding the content is the territory of the platform. There is the feed controlled by recommendation algorithms, the visual design of the website (with maybe some minimal control to change the theme). Some creators experiment with new content forms, like interactive visualizations, but they are published on their own small distribution platforms, like personal websites.
On top of that, platforms distribute content in a static form. Even if a video creator finds a cool new way to add subtitles to their video, you will not be able to apply the same effect to another video. The choice of presentation of the video and the content of the video are packaged in a single file and cannot be decoupled.
AI for just-in-time combination of content and presentation.
AI presents the potential to break free from this centralization of presentation. It could allow end users to “re-flow” content to be displayed in another frame, like applying a font to change the presentation of a book. The new frame doesn’t have to follow the constraints of the platform where it was originally posted, nor the intention of the creator.
AI has the potential to break the centralization on content presentation.
This ability to “re-flow” can give rise to a new niche of online creators: framemakers—specialists in crafting content presentations, rather than content producers. Frames could be a new form of talent specialization, removing the need to find the intersection of people who are good at knowing what to say, as well as how to say it. The what and the how of the content can be produced independently and combined at the very end of the distribution pipeline to fit the preferences of the end user (the red arrows in the diagram above).
Alongside online creator remuneration, we would need to invent new systems to remunerate the contributors to the frame in which a given piece of content has been displayed. Moreover, as we needed to design recommendation algorithm to filter the content presented to a user, we would need to create frame markets that match a given piece of content with the appropriate frames to render it. The frame-content matching would take into account the user’s frame preferences and the content creator’s frame recommendations.
I am excited about the potential of framemaking for differential technological development. Good frames for engaging with complex topics have the potential to make difficult ideas more widely accessible and raise the quality of public discourse.
This vision of framemaking has been one of the guiding principles for my work on AI interfaces over the past year. I worked on adding new options to interact with ideas in static text: Bird’s eye view re-flows corpora of text to render them as an interactive 2D map, and breaking books turns the content of a non-fiction book into a collaborative board game.
In the rest of this post, I present a list of micro-visions for new forms of online experiences. For most of them, you can imagine the backend as a web browser extension that manipulates the content of a webpage before it is rendered to the user by calling multimodal input-output AI models. The AI models are faster, cheaper, and more reliable than those available today, but not radically more intelligent.
Though, I believe many of them can be realized with today’s technology. If you feel inspired to implement one of these visions, please go forth, and feel free to get in touch!
Micro-visions that preserve the original form of the content but add to it. Today’s examples include community notes that add important context to controversial tweets, or Orbit, a tool to integrate spaced repetition prompts into a text.
Interactive event timeline.
You are reading an online article describing a succession of historical events. It could be discussing the formation of the ocean in Earth’s history, the rising political polarization in U.S. politics, or the eventful weekend leading to the OpenAI battle of the board.
As you follow the article, an automatically generated interactive timeline remains on one side of the screen. When you hover over a piece of text, the corresponding part of the timeline lights up. When you click on the timeline, you can jump to the parts of the blog that discuss this piece of the story.
Interactive demos in scientific papers.
You open an ArXiv paper modelling the impact of climate change on rising sea levels. Along the text, an interactive demonstration is rendered in your web browser, showcasing the quantitative model proposed by the authors. It has been created on the fly from the open-source code and the content of the paper. You can tweak the CO2 emission curve, and see how the land mass on the world map shrinks as a result. This allows you to check the conclusions of the paper and identify blind spots in their analysis.
Interactive articles are an effective medium for explaining complex ideas. However, they are incredibly challenging to create, as they require the combined skills of designers, scientists, and communicators. In this vision, designers and communicators creates “demo prompts” that are applied to classic PDF scientific papers to transform them into interactive mediums.
Just-in-time diagrams.
You open a long X post making an argument about the money flow between various AI companies, claiming that AI is an economic bubble. The names of the different companies are highlighted in different colors, with their logos attached. Along the text of the tweet, there is a diagram generated to illustrate the flows described.
Distillation through analogies.
You are an economist reading a biology paper about genetic drift. The paper is enriched by analogies that bridge the two fields, allowing you to apply your economic intuitions to the concepts presented. The paper explains that when a population of organisms is too small, suboptimal genes might end up being propagated through random luck due to a lack of genetic diversity.
The analogy-maker adds that it is similar to the lack of competition in a market; when a certain good is produced by a few producers, the end product tends to be suboptimal in quality or price because of the lack of competition. You can expand the analogy to see the mapping: gene fitness <=> product quality and price, economic competition <=> natural selection, population size <=> market size. You also read the limitations of the analogy: the lack of competition can be exploited by companies to intentionally compromise on quality and raise prices leading to even worst products, while the effects of genetic drift occur through random luck.
From reading to deck-building.
You open a long essay making the case for a decrease in energy availability over the next few decades.
The text is broken down by quotes extracted from the essay and rendered on cards with a visual background that supports the vibe and content of each quote. All the cards in the article have a coherent style; it’s as if the author designed a visual identity for their work. Concepts introduced in the article, such as “energy descent” and “link between fossil fuel and GDP,” are displayed with symbols near the keywords and reused in the card visuals.
As you read, you can add cards to your inventory. These are the cards you want to take away from the article. However, you only have a few slots available. If you want to pick up more cards, you need to let go of others. This friction forces you to compare the different quotes to find the best ones.
Near the cards, you see small versions of related cards that you picked from previous readings. You can choose to reinforce the link of the suggested card or remove them if you find the connection irrelevant. You also see cards that come from a group collection you are part of. You discover that a friend has been reading another piece that makes the case for nuclear fusion reactors capable of supplying energy at scale in 20 years, contradicting your current article. You decide to bookmark this other article for later and send a message to your friend to ask what they thought of it, initiating a debate.
Every month, you curate the cards from your gallery to find the most important quotes you have read over the past weeks. The content of the article comes back to mind quickly as you recognize the visual style of the articles and their associated symbols. This allows you to reflect on the impact that the different pieces have had on your opinions. You curate the cards into a monthly deck, which gets added to your profile and sent by email to your subscribers.
Groups of interest also hold regular retrospective sessions to collectively curate the content its member have been reading.
Micro-visions that don’t preserve the original presentation of the content can be both promising and potentially scary, as the reframing has the potential to strip the content of its intended meaning. These visions are conditioned on good execution of the UX and reliable AI models.
Contemplative reading
You open Vitalik’s My techno-optimism manifesto. Instead of being presented as a blog post, you see a single page filled with a watercolor depicting a little boy running away from a bear. The page contains simple sentences and plenty of whitespace. There is no wall of text in sight; you have space to hold all the text in your head and contemplate what is being said. You click through the pages, navigating a distilled version of the article, blending quotes from the original text, high-quality generated text, and beautifully relevant illustrations.
For an example of an online contemplative reading experience meshed with generated images, see Amelia Wattenberger’s Interface Lost Their Sense. Robin Sloan’s Fish a Tap Essay is also a good example of a contemplative reading experience.
Socratic content
You open an article, and you see a single sentence stating “Books don’t work” with a text area underneath. The sentence is a provocation, and you must provide your opinion to continue reading. You argue against the opening statement: books work; they are a very effective medium for storing our collective memory. The article responds to your points by asking further questions and showing quotes from the original text, clarifying what is meant by “don’t work”: reading a book doesn’t necessarily translate to internalized knowledge.
You go back and forth to explore everything the article has to offer in a debate. This process allows you to skip the points you agree with, and surface the most load bearing points that challenge your initial opinion.
This process allows you to skip the points you agree with and surface the most load bearing points that challenge your initial opinion.
This can also be seen as an interactive unpacking of a Sazen, a short sentence compressing a precise insight. It is obvious to those who have it but puzzling to people who have never gone through the process of having the insight.
Content cairns.
After interacting with the socratic content, your contributions are added to the memory of the article. Your own arguments and counterarguments might be presented to new readers. As the article is read more frequently, the content integrates these contributions.
The initial author retains some editorial power. For instance, they can decide whether a certain disagreement counts as a valid limitation of the article or if it should be treated as a misinterpretation.
Some content cairns take a life of their own such that it doesn’t make sense to say they have been authored by anyone. They are living memes that gain life every time they are used, are tagged in online conversations, and respond without being bound to any platform.
Content to frame.
You just finished the excellent “On Green” by Joe Carlsmith. You learned the color / value association from Magic the Gathering: White: Morality. Blue: Knowledge. Black: Power. Red: Passion. Green: environmentalism? It’s complicated, that’s what the article expands.
You are interested in using this association in another context, and you add the essay to your frame library. Weeks later, you read the review of Seeing Like a State, a book about the limits of the high modernist, centralized state and the merits of local knowledge rooted in practical experiences. The “On Green” frame awakens and colors different parts of the article: the drive for control from the USSR in black, the need for making the territory legible through maps in blue, while the discussion of local knowledge is in green.
Multi-source content patchwork.
Instead of reading newsletters and social media feeds in silos, you read a single thread of content that smoothly integrates excerpts from long-form pieces with short tweets. AI news seamlessly gives way to the economic trends, which transitions into political discourse.
At the end of the thread, you have a good overview of the information landscape of the day and can decide which sources are worth digging into further.
Inspiration: Zvi Mowshowitz’s AI newsletters
No-screen online interface.
Every morning, you receive a newspaper made just for you. It contains a curated version of your social media feeds and newsletter subscriptions. You can use a barcode scanner to give feedback on the curation process to signal what you liked. You can also write on the newspaper, and the same barcode scanner will take a picture of your handwritten response and post it as a comment in the right place. Using the same mechanism, you can also publish handwritten pieces to your personal blog.
Some online creators have decided to go all the way and work in a no-screen online environment. They specialize in discussing slow-moving cultural trends, calling themselves the “trees” of the online ecosystem.
Inspiration: The Screenless Office, DynamicLand
2025-12-14 05:40:38
Published on December 13, 2025 9:40 PM GMT
tl;dr: How does an agent decide whether it is worth doing work to get something, as opposed to not working and not getting it? I'm a neuroscientist and my lab did an experiment to explore this question. Rats did voluntary work to earn water in a closed economy, while the wage rate varied. Rats showed characteristics of rational economic agents, which could be captured by a parsimonious utility model. The main results are described below. Links to the original paper, data and code are provided here[1].
The question
In the wild, animals generally have to work to get the things they need. Research on reward-based motivation has revealed a lot about how animals choose between two or more available (mutually exclusive) goal-directed actions, such as two different tasks that give food or water rewards. Other things being equal, animals will choose the actions that offer more reward, or require less effort, or deliver reward with shorter delay, or deliver reward with less uncertainty. Optimal Foraging Theory addresses another set of questions about explore-exploit trade-offs. When animals think the yield in their current foraging patch is sufficiently below the average yield in the global environment, they forego immediate foraging to go search for a better patch, taking into account how far the other patches are and how likely the yields in other patches have changed since they last checked.
But very little research has addressed the question of how animals decide whether to perform any of the available actions for some given reward, versus none of them; whether to keep foraging, as opposed to stopping. Perhaps it's simply homeostasis: they just keep doing whatever the best available option is, until they have enough. But the overall opportunity in the environment can vary drastically over time. The average yield of all available foraging patches, or the best-yielding of all available actions, can be lean in some years or seasons, rich in others. Or sometimes there is only one patch to forage in, or only one action that can yield the thing they need. Even when there’s only one option on the table, animals still have the choice to take or refuse the option. How do animals decide when they have “enough”, and when it’s not worth working to get more?
The experiment
The rats lived in custom cages whose only source of water was a computer-controlled visual classification task. The rats could perform a “trial” at any time (request a visual stimulus and assign it to one of two classes), and receive a drop of water if they provide the correct answer. The rats succeeded only 80% of the time, which shows both that the task was difficult (well below 100% correct), and that they were trying (well above 50% chance). The rats did not do the task if there was no water reward or if they had unlimited free water, which shows that they only did trials because they were “paid”. By this definition the task is considered work or “labor”[2]. The expected amount of water per trial (probability of success x size of drop) is the “wage rate”. This represents – in a highly stylized way – the fact that animals must generally do something effortful or costly (such as forage) to get a resource they need (such as water).
Every few weeks the size of the water-drops, and thus the wage rate, was randomly changed. This represents the fact that seasons or weather patterns can make a resource more or less abundant at different times. To ensure animal welfare, animals were checked daily; if any animal didn't consume enough water, lost weight, or showed any physical or behavioral signs of dehydration or distress it would have been removed from the study and given free water[3]. However, the study only used drop sizes for which it had been previously determined that such interventions would not be required.
Within a few days after a change, rats adopted a new steady-state trial rate. In nature, environmental conditions can last long enough that animals need to find a sustainable strategy in each condition. To ensure the observed strategies were sustainable, in the experiment the rats were monitored until it was verified that the number of trials, water consumed per day, and body weight were stable before changing to a new wage rate.
The observations
From reading the reward literature, one might think rats would be more motivated and therefore do more trials whenever the reward was larger. But that would be a bad strategy in nature. Animals have to increase the rate of effortful goal-directed behavior (like water foraging) precisely when that behavior is least rewarded (as in a drought), if they are to meet their basic needs. Indeed, rats did more work when wages were low, compensating for the lower yield and always meeting their physiological needs. When the water-drops were very large, rats did very few trials per day. In macroeconomic terms, this is called a backwards-bending labor curve, because the labor supply (willingness to perform work for pay) declined as wages increased. In human markets this is explained in terms of workers preferring to take wage increases at least partly in the form of increased leisure.
It’s not surprising that rats would do as many trials as necessary to get the amount of water they needed. If this were the whole story, however, rats should have consumed the same total amount of water in all conditions, regardless of the size of the water drops. This was not observed.
Instead, rats were willing to work for much larger total amounts of water (up to 3x more) when the water was easier to get. In macroeconomic terms, this is called price elasticity of demand: consumers consume more of a commodity when it is cheaper. This finding was more surprising, because core survival needs like water are often assumed to be “inelastic”. Apparently when unlimited free water is available rats drink a lot more water than they actually need. Part of it is physiologically necessary, but most of it is optional or hedonic. But rats strategically self-limited their water consumption to just meet their basic needs when water was scarce, and indulged in optional extra water when it was abundant – while also taking it easy (doing less overall work). Pretty smart.
Taking inspiration from classical economic theory, a utility maximization model was proposed to quantitatively account for and normatively explain the rats’ total effort and total water consumption as a function of the water drop size. There are only two free parameters: one for a rat's satiety point for water, and one for its aversion to the work. The model makes a number of testable predictions which the paper lays out in detail.
If one assumes the rat's choice to do a trial at any moment is a sigmoidal function of the instantaneous marginal utility, the model further qualitatively predicts when in time rats did the trials.[4] The shape of the marginal utility curve strikingly matched the activity profile of neurons known to be necessary and sufficient for driving water-consumption behavior in mice, providing a testable neural circuit hypothesis for how marginal utility of work for water might be computed.
Conclusion
Animals don't just choose between alternative ways to meet a need. They also regularly make the choice to refuse all available offers, even when unclaimed rewards are on the table. They decide this based on when they have “enough”. But their definition of “enough” is flexible and contextual – even for survival-essential resources. When resources are scarce, they consume less and work more; in times of plenty they enjoy a bit of excess, but also take time off.
Related work and Future Directions
Generalization: The described study tested a highly domesticated strain of rats. One generalization of interest would be to see if the same results hold for wild rats, or if it is specific to species domesticated by humans. Another would be to see if the results hold for phylogenetically diverse species. Does rational economic utility maximization require a neocortex, or would the same results be found in birds? lizards? bees? For the latter, one could further distinguish the individual decision-to-work from the hive-level organism decision-to-work. One would have to use a commodity other than water to test aquatic species but then: what about fish? shrimp? cephalopods?
The overall approach used here could be applied to how animals decide whether and when it is worth doing work to gain other resources, like food. It is speculated in the paper that the details would be different for food, because animals can store excess consumed energy as glycogen or fat, thereby buffering fluctuations in availability. By contrast, most animals including rats can't store much excess water in their tissues, and instead eliminate excess fluid as urine. Therefore, with food rats might to a greater extent rely on stored energy reserves to avoid excessive labor during food scarcity, and overconsume to replenish reserves during food abundance[5].
Given that excess water cannot be stored in body tissues, it is something of a mystery why the rats even want to consume excess water when they can. One testable hypothesis is that drinking extra water enables them to eat extra (dry) food, effectively trading a non-storable commodity for a storable one. This would be easy to test by measuring or limiting their food consumption in such an experiment.
Extensions: It would be quite interesting (but complicated) to simultaneously model allocation of effort toward two or more competing goals, including how they interact dynamically. For example, the same experiment could be done where food pellets were also earned by another kind of labor, like lever-pressing. Then the rats could choose to work for water, work for food, or neither. The utility functions would have to include cross-terms because the marginal utility of water increases with the amount of dry food consumed and vice versa. The interpretation of marginal utility as an instantaneous determinant of motivation might be enough to recapitulate observed oscillation between bouts of drinking, bouts of eating, and other activities.
Alternatively, if the cost for switching between tasks is significant, one might cast goal-switching as a generalization of patch-switching, and apply Optimal Foraging Theory.[6]
Applications: In related work, it was shown that the observed gap between physiological need and hedonic drive for water can be leveraged to motivate animals to perform behavioral trials for fluid rewards without the need for water restriction. It turns out rats will consume exactly, but only, the physiologically necessary amount of water if it contains Citric Acid (CA), a harmless additive. Animals tested in the same environment described in the paper were provided with an unlimited supply of free CA water, and still performed substantial voluntary work for water rewards. This has been widely adopted as an animal welfare refinement in research settings. From a theoretical standpoint it would be quite interesting to expand the utility model to explain how rats distribute voluntary consumption between costly water and a free but inferior substitute commodity. If anyone has any ideas how to do the math, I'd be happy to share the data.
For more details about the model, other extensions and predictions, alternative models, and a candidate neural mechanism, see the original paper, supplementary materials, data and code.
This needs to be said because sometimes animals will play with toys or games just for fun; mice will even pay for the chance to run on a running wheel. So not all effort is labor.
These study-specific checks were in addition to standard animal welfare measures, which included ample space, daily food replenishing, regular cage cleaning, a rodent-preferred light cycle, at least two toys, daily health observations, and and daily positive human contact whenever group housing was not possible. The baseline criterion for welfare is that animals are "bright, alert and responsive".
It's a bit too complicated to fully explain here, but the model predicts that the marginal utility of doing one trial falls quite steeply with number of trials completed, correctly predicting that animals will dramatically slow down after just a couple dozen trials, even if the ingested water hasn't reached the bloodstream yet (which takes 10-20 min), and even though they may only have consumed 10% of the total water they need to fully restore hydration. This pattern was key for identifying a possible neural mechanism. See the paper for a more complete explanation.
This might explain why humans need GLP-1 inhibitors: overconsumption when calories are abundant and cheap was likely adaptive for most of our evolutionary history; there hasn't been much time for selective pressure against the maladaptive consequences of obesity, and societal interventions supersede strictly biological selection pressures.
All of the suggested generalizations and extensions were in fact proposed in a grant proposal, but the grant was not funded after a couple of rounds and the lab has moved on. So these experiments are not currently planned, at least in my lab.
2025-12-14 05:19:11
Published on December 13, 2025 9:19 PM GMT
I've been thinking about specificity recently and decided to re-read SotW: Be Specific. In that post Eliezer writes the following:
S. I. Hayakawa called this the ladder of abstraction. I'm not sure if understanding the following section will really help with the skill of Being Specific, or help anyone construct exercises for the skill of being specific. But a better theoretical understanding does sometimes prove useful. So I will now digress to explain that abstraction isn't really a ladder, but a lattice.
I think I understand some of what he's saying. I think about it in terms of drawing boundaries around points in Thingspace. So the concept of "Sunny Days" is drawing a boundary around the points:
{ Sunny, Cool, Weekday }{ Sunny, Cool, Weekend }{ Sunny, Hot, Weekday }{ Sunny, Hot, Weekend }And the concept of "Sunny Cool Days" is drawing a narrower boundary around the points:
{ Sunny, Cool, Weekday }{ Sunny, Cool, Weekend }And so we can say that "Sunny Cool Days" is more specific than "Sunny Days" because it draws a narrower boundary.
But I still have no clue what a lattice is. Wikipedia's description was very intimidating:
A lattice is an abstract structure studied in the mathematical subdisciplines of order theory and abstract algebra. It consists of a partially ordered set in which every pair of elements has a unique supremum (also called a least upper bound or join) and a unique infimum (also called a greatest lower bound or meet). An example is given by the power set of a set, partially ordered by inclusion, for which the supremum is the union and the infimum is the intersection. Another example is given by the natural numbers, partially ordered by divisibility, for which the supremum is the least common multiple and the infimum is the greatest common divisor.
Maybe a lattice is simply what I described: a boundary around points in Thingspace. But I get a sense that "lattice" involves order in some way, and I am not seeing how order fits in to the question of how specific a concept is. I also think it's plausible that there are other aspects of lattices that are relevant to the discussion of specificity that I am missing.
2025-12-14 04:22:16
Published on December 13, 2025 8:22 PM GMT
One of my favorite AI papers is “Lets Think Dot By Dot”, which finds that LLMs can use meaningless filler tokens (like “.”) to improve their performance, but I was overestimating the implications until recently[1] and I think other people might be too.
The paper finds that LLMs can be trained to use filler tokens to increase their ability to do parallel reasoning tasks[2]. This has been compared to chain of thought, but CoT allows models to increase sequential reasoning, which is more powerful[3]. I now think this paper should be taken as evidence against LLMs ability to perform long-term reasoning[4] in secret[5].
This means that if a problem can be broken down into sub-problems, but the model isn’t wide enough to process it in one pass, the model can instead parallelize across multiple filler token positions and then combine the results. However, if the problem requires step-by-step thinking and the model isn’t deep enough, filler tokens don’t help. In comparison, Chain of Thought helps in both situations.
My metaphor for this is that filler tokens allow a model to dynamically increase the size of layers, but CoT allows the model to dynamically add layers.
Every layer in an LLM operates in parallel, so all input data must come from a previous layer. Attention allows layer n to collect data from layer n-1 from elsewhere in the context.
i, i+1 and i+2 all receive exactly the same inputs except for positional data and filler tokens.To continue sequential reasoning from a previous layer, the model either needs a deeper layer which can attend to the previous output directly, or the previous layer needs to output a meaningful token which can be fed in from the top.
This is a problem for filler tokens, since the network has the same depth at every input, and the only new information is the filler token and the positional information.
It’s possible[6] to exploit the positional information to process the same inputs differently (this is what the dot-by-dot authors do), but it’s not possible to process it for additional steps. An n-layer network only gets n layers of processing, no matter what path the data takes through it.
To be fair to the authors, they say all of this in the paper. I just didn’t understand it.
Specifically, they find that on a problem where the model needs to check the sum of up to 364 triplets with only 6 attention heads, it's able to spread the work across filler token positions and then select the triplet which sums to zero.
Any parallel algorithm can be executed sequentially, but not all sequential algorithms can be parallelized.
Inituitively, if your algorithm is "think about apples and oranges at the same time", you can turn it into "think about apples and then think about oranges"; but if your algorithm is "look at the last the word and then think about it", there's no way to parallelize that since the second step depends on the first step.
Don't take "not all" too strongly though. Many sequential algorithms can be turned into parallel algorithms, especially if you're willing to take an efficiency hit. For example, "do x and then do [y1, y2, y3, ...] based on the results" can be turned into "do [(x, y), (x, y2), (x, y3), ...] in parallel and then discard most of the results".
Long-term meaning reasoning that carries forward between positions. Some frontier models have over 100 layers, so the amount of hidden sequential processing they can do is still non-trivial.
Although steganography is still an option, at least in theory.
This is separate from my main point, but it’s also really hard to train a model to parallelize like this. The authors of the paper had to use hand-tuned training examples and custom positional information to make it work. And even then, it only learned to do this for one problem.
It’s theoretically possible for an LLM to learn a general method of parallelizing computation across positions using filler tokens, but I would be surprised if they were able to learn something this complicated by accident through RL.