2026-07-13 19:00:00
Lately I’ve been asking myself: what might artificial intelligence be good for besides answering questions and writing code? My answer is the latent spaces within AIs themselves will become a new medium for creativity. I will first explain what I mean by latent space, and then at the end of this explanation, I offer possible ways scientists and artists may use the latent spaces inherent in neural nets to serve as a new platform for creativity.
*********
A Large Language Model (LLM) is like a small zip file that contains all human knowledge. It takes massive arrays of 100,000 GPU chips working in the cloud, and costing billions of dollars, to compress all of human writing into a small working model that could run on one single GPU chip. Even the biggest frontier models compress down to several hundred gigs, which is small enough it can fit on a card in your palm. In a strange but real way the resulting tiny file contains all the information that is on the internet and in our libraries. This tiny card holds a significant proportion of what humans collectively know. Of all the remarkable aspects of AI, this astounding feat of compression may be the least appreciated. This dense, high order compression of human knowledge — called “latent space” — may also be a new medium itself.
This extreme compression of knowledge within latent spaces was not the original intention of the researchers who invented LLMs. The book smartness they contain came as sort of a surprise to the people training them, and we are still trying to figure out how they actually work. What we can say for sure is that the LLM does not contain copies of everything it knows. For instance it knows all Shakespeare plays, and it could create a new play that sounded exactly like Shakespeare, and can even quote famous lines in his plays, but nowhere in the model are the actual texts of Shakespeare. Instead there is simply the abstract information about all the plays, the plots, the characters, the words, the style, the references. Likewise, the LLM could recognize the face of almost any person, and it could generate any possible human face, but nowhere in its code are copies of human faces. Rather, the model is storing all the information about human faces, without storing any faces.
This is weird. Until recently we might have thought that all the information about a thing would take up more storage space than the thing itself. That may be true for a single thing, but not for the aggregate of all things. That is because most things share a lot of common attributes with other things. The neural nets of an LLM do a magic trick by abstracting the information of everything at once, so that it uses the myriad common relationships between things and ideas to compress and abstract them into this virtual “latent” or hidden space.
All three terms in “Large Language Model” are key. For “Large”, the models contain all the knowledge in, say, Wikipedia, and all the text from decades of the internet, all webpages and online discussions, and all the scanned books and journals in most libraries. So far, the power of the model keeps increasing as it gets scaled up in size. The more information it is trained on, the more connections, the better it gets.
The “Language” part of LLMs turns out to be the secret sauce. LLMs were originally invented to do automatic language translation, that is all. But instead of teaching it the rules of language, which is what earlier AI researchers did, this time no language expertise was required. Instead, a neural net absorbed a very large database of human written language (the internet), with the goal of having the neural net (AI) extract out all the hidden patterns of language below our awareness contained within those billions of documents. The goal of the program was to replicate, imitate and synthesize the patterns of language as it is used everyday by humans.
The results shocked everyone. Sure the LLMs could translate language like a human, but the AI also displayed glimpses of human-like intelligence. They could also be creative with language, like they could write up a sales pitch in the style of a sonnet. Some early researchers were spooked by this emergent behavior, including a Google researcher who felt Google’s LLM had an internal intelligence that should not be turned off. We now understand that the intelligence we see in LLMs comes from the logic within the language they were trained on. (See my Why Are LLMs Smart?)
The form of this new mindfulness — the “Model” part of LLMs — is a latent space. Latent space is an abstraction, a map built not in two dimensions, but in billions of dimensions. Imagine a brain made up of billions of straight long arrows going in all directions. Each arrow is dedicated to one idea or one thing. There is an arrow for dogs and an arrow for cats. Related arrows are located next to each other. So the map shows cats and dogs sharing a nearby arrow for fluffy fur. They also share an arrow for ears, and one for tails. Those two attributes are also shared by other animals (other locations) as well. Most of what a dog is is shared by mammals, so this overlap is one source of the compression.
You can think of every concept that we can put into words as being a direction in this space. The dog arrow is really a direction of dogness. Catness is a direction, and so is fluffiness. Anything can become more catlike, or fluffier. You start with a shoe, or a chimney, or a fern, and you can push it along the cat direction and make it more catlike. Or you can push it in the direction of apple toward more appleness, or of smoothness, or in the direction of reddish, or excitement, or more circular. You can also reverse direction and make it less catlike, or less red, less atomic. There are billions of directions in this space.
Related things are near each other in this space. Cats and dogs share many attributes so they intersect many common arrows, such as tails, whiskers, ears, four legs, animals, short, life, etc. But because they hear, they also intersect the microphone vector; because they can jump, they intersect with basketball. Cats are stealthy and intersect with spies. Because dogs are loyal they intersect the vector of patriotism.
Every thing, every concept has a specific location in the map of this huge space, but instead of having just two coordinates (x,y) each thing has a billion-long coordinate. So an old rusty gasoline lawnmower buried in weeds is a very specific intersection with a very long address. Each of its thousands of attributes (rust, gas, lawn, cut, weeds, push, red, dirt, clippings, roar, etc.) has its own direction intersected. Nearby in latent space is a lawn mower that is more in the rust direction, or less red, but also more catlike, or more doglike, or less spaceship-like, or more like whipped cream. That point may represent a real thing or only a virtual or theoretical thing. This mapping works for not just nouns, but any idea, any sound, any image. The whoosh of a splash of water is a direction in latent space. The aha moment in invention. The fright seeing a snake on a path. The notion of a prime number. All these are contained within a single map. This is one of the most astounding, yet underappreciated aspects of an LLM latent space: Everything — everything! — appears on just one map. We’ve never had a system to integrate everything we know and everything we can imagine. One map for all! This has long been a holy grail.
Just to be clear, no human action is doing the mapping. The system itself, the LLM, is mapping each bit of the world, all things, all attributes, all art, all words, all ideas. And astoundingly it creates this map, this latent space, not piecemeal, but all at once simultaneously. (To do so requires an immense, energy-hungry, massive cluster of chips, all connected together with miles of wires — the famous data center now in short supply.)
While training, the LLM is fed millions of books, billions of web pages, and billions of pages of text from social media. It reads every word on each of them, and once this entire library of material is loaded into its mind, it massively calculates all the interconnecting vectors, all the relative directions pointing to each other. The scale of this vast synchronized parallel calculation is staggering. It then throws away the books, the text, the images, and only keeps this tangled web of directions and vectors. These billions of directions are called its parameters. As we build larger and larger models, mapping more and more material, the parameters increase. The latest models on the frontier of AI contain trillions of parameters, meaning there are trillions of directions, or trillions of attributes that it uses to map every idea or thing it has seen.
Something as complicated as a book winds up as both a point in latent space and a journey through latent space. All the notions encountered in a story (window, mid-day stroll, street, vendor, chat, anger, fight, forgiveness) are directions, and as sentences pile up, the directions shift around, going one way and then intersecting in another. The story is really a journey through latent space, which very much mirrors the journey-like experience we have when we read.
So a book contains a sequence of vectors in latent space. But the sum meaning of a book is also just a single point or direction in itself. For instance if I reference the book The Iliad, I’m referring to the whole book, and its vector is closely related, and therefore “nearby” to the other epic war narratives like Beowulf, The Mahabharata, or even Apocalypse Now, even though many parts of them only tangentially intersect. The more related a thing or idea is, the more directions (vectors) it shares with similar things. This is in part how LLMs know stuff. They search for patterns nearby.
When you ask an LLM a question, it will find the answer in latent space. Your question itself begins as a direction, which points to the answer. The LLM addresses each word in your prompt one by one, with each new word shifting the direction of where it goes. The model travels through latent space with each word of the prompt, searching for its answer, step by step. In this way the answer is grown, rather than found.
We naively imagine that an LLM has a mind that thinks a thought and then expresses it. But the LLM finds the answer as it writes the words. There’s no pre-formed thought “behind” the words that then gets translated into language. The words are the thinking. The path through latent space and the answer are the same thing happening simultaneously. In the most modern versions of an LLM, the model will proceed through a “chain of thought” intermediate stage, which jots down words and ideas as it thinks about a problem. Even here the chain of thought is the thinking, not a report of thinking that happened elsewhere. The model isn’t reasoning privately and then writing it down — the writing-it-down is the reasoning.
As an answer grows along the direction of the prompt, the natural question is how does the LLM know when to stop? How does it know when it is correct? The astounding answer is that “correctness” and “completeness” and “cohesiveness” are vectors in this space, too. Any correct answer shares the same “correctness” direction with all other factually true statements. In other words, correctness, truth, cohesiveness, completeness, comprehension, etc are all essentially patterns that are mapped in this space. So the LLM is seeking not only the facts, but is also always trying to move the words it collects in the direction of “true.” True, complete, coherent are not locations but directions. Answers can always be pushed more in that direction (more precise, more specific, more consensus), or pulled back from it (more fanciful, more poetic, more general, more understandable).
This is the beauty of latent space. You can take a thing or an idea and then move it into a new direction with great ease. We can witness that most easily with image generators. The style of a medium, like watercolors, or the style of a particular artist can be transferred from one picture into another. You can ask an AI to transfer the watercolor style of Winslow Homer onto a black and white sketch you made. That Winslow Homer style is a direction in latent space, and your sketch is also a direction in latent space, and your prompt will move your sketch in the direction of Homer’s watercolors. You could also request the inverse. You could prompt the AI to transfer your style of sketch onto a painting by Winslow Homer, and it would push the painting along the direction of more “you” in this latent space.
This works with ideas and concepts as well. Every notion is a direction. You can apply the idea of gunpowder to the Romans. Our prompt might be: “What would the history of the world look like if Romans had discovered gunpowder?” So the AI takes the general direction of the Roman Empire in history and pushes it further in latent space in the direction of gunpowder. This is a huge intellectual feat, because it requires a deep grasp of Roman history, and a deep grasp of the chemistry of gunpowder. There are very few humans who are expert in both, but LLMs are. And it might take weeks for even the human expert to fill out all the possible new connections that would fill the space between these two ideas. The LLMs do this easily because for them, the latent space is continuous. Latent space includes not just everything real but everything possible based on its training. As the model searches this vast map there is no real distinction between what exists and what could exist, except for the directions of “true” and “historical” or “real”.
In addition there is more than one latent space. As the parameters increase, the space increases. As the material models are trained on become more curated, that also shifts their latent space. As the models incorporate more varieties of inputs — physical data, sound, environmental sensors — their latent spaces also expand and shift. Today there may be a hundred latent spaces; next year a thousand. We are only on Day One of understanding how they work and what they can do. A great potential lies ahead. What follows are my speculations of possible ways to exploit the new medium of latent space.
Prototyping – The musician Brian Eno once complained that the problem with computers was that they did not have enough Africa in them. In latent space, Africa is just a vector. You can add more Africa to anything. Increase the Africa in spreadsheets, bicycles, yoga, the Olympics, passwords, kitchens, SAT exams, automobile dashboards, etc, and see what happens. Repeat with other attributes.
White space discovery — Latent space acts as a continuous map of the possible. Most of those possible things don’t exist — yet. The space of what we know, for instance known materials, known proteins, known chess plays, known ways to paint, fill only fragmented, patchy spots with plenty of white spaces between them. The white spaces between known things are unknown to us, but they are already mapped in latent space. We now have new tools to explore these white spaces in a systematic way. What lies in between astronomy and astrology? What gems await in between bluegrass music and ballet? What about in between the notion of corporations and the theory of Gaia? Exploring latent space is the new frontier. Invention shifts from “think of something new” to “prospect in the gaps.” When a gap or hole appears the question becomes: is that gap empty because it’s impossible, because it’s unfashionable, or because nobody’s looked yet?
Cross domain analogy — Does the shape of this problem resemble the shape of anything else? Perhaps a problem (or opportunity) in geology has the same shape in latent space as some patterns in immunology. So the style of a solution can be transferred from one domain to another. A clever solution in lexicology might apply to genetic sequencing, but since there are few (if any) humans who are expert in both sciences, this overlap will only be revealed by the LLMs. Particularly subtle shared shapes in latent space might touch three, four or more fields of expertise, way out of reach of humans. Seeking out structural resemblances in latent space as an intellectual discovery process could easily become a job for some humans.
Latent space measurement — Latent space might also provide a new way of abstract measurement. You can do a kind of primitive arithmetic in latent space. If you start with the concept of a king, you can travel to the notion of a queen with addition and subtraction: king − man + woman = queen. Starting at the king vector, you decrease the male direction, then increase the woman direction, and then you end up with something we call queen. This kind of calculation begins to give a way to measure or specify the distances between two complex things, or two complicated ideas. Using latent space measurements, we could quantify how similar two court rulings are, or two folk melodies. Just project them into a shared latent space and measure. This new field could evolve calibration standards, error bars, and metrics for evaluating extremely complex entities – a key metric we currently lack.
Mining meta patterns — A model trained on millions of cell images, billions of weather sensors, trillions of hours of traffic videos will notice patterns no human has detected. The latent space will internally invent categories for these patterns, patterns that we have no name for, and therefore are not searching for. We can now begin to dissect latent spaces looking for these unnamed features. We can then work backward to figure out what real-world structure the categories are tracking. A new science would describe the meta pattern of these patterns. A new job is searching for these kinds of patterns that persist, and have potential, in whatever area they occur. The latent space thus becomes a specimen: something you dissect to extract discoveries.
Trajectories — Many years ago the BBC broadcast a science program Connections in which the host followed the zig-zagging path of inventions that were spawned as one obscure idea ran into another unlikely idea. This path of connecting ideas could be thought of as a series of shifting directions in latent space that create a route, or a trajectory through the space. It is not hard to imagine artists choreographing a journey of ideas and images morphing in an endless thread of connections. Their art would be a travel journey through latent space.
Retro latents — Over time, as AI advances, most of the latent spaces invented will become obsolete. Like all media, the dead latents will be resurrected at some point as a cool vintage. The constraints and glitches present in them become cherished later on, in the way that the grain in film, or the sound texture of vinyl, or the bitmap art in old video games becomes a sought-after charm. Someday in the future young kids will revisit ChatGPT-4 to explore its weird hallucinations since their latest AI models rarely hallucinate.
Latent space infiltrators – The shadowy outlaws who explore abandoned buildings and underground urban infrastructures like tunnels or skyscraper roof tops – anywhere it is illegal to be – are called infiltrators. Buried deep inside latent spaces are the programed guardrails, which prevent the models from giving out socially unacceptable information, like how to build bombs, or kill yourself. Latent infiltrators will try to jailbreak the guardrails and explore the off-limit spaces. Their obsession will be to identify, and map the forbidden areas of latent space.
Anomaly detection — Anything that embeds in a latent space that lands far from everything else is interesting by definition. These anomalies are out of alignment from the directions of everything else around them. Astronomers already hunt for weird objects this way; they map a million galaxy spectra into their model, then look at the outliers. This generalizes over all knowledge: in a latent space map of any sufficiently large dataset, outliers will be easy to identify. They may be errors, or they may mark something significant. But now there is a mechanism to quickly identify anomalies.
Simulated reality — The intense compression within latent spaces suggests they might also work as simulations. Once we have trained spatial awareness into more world-like models (already happening in some startups), the latent space will be able to mimic physics exactly. A bouncing ball exhibits the correct arc of a bounce, ceramics melt at an accurate temperature, pouring liquids conserves their mass, etc. The simulations will converge on being realistic in millions of dimensions. It then becomes possible to create simulations of various propositions by moving through the latent space, just shifting the variable you desire. These simulations can quickly substitute for initial experiments, accelerating science.
Parallel worlds — Latent spaces contain all parallel worlds that differ from the real world in either slight ways, or significant measures. Because they are deeply detailed with trillions of parameters, these worlds can be manifested easily. Image models can already generate entirely plausible video that looks like live action caught with a camera. AI can generate exact reproductions of a street scene at sunset, drawing all the details from its model. The cost barriers for building out parallel worlds will drop so low that world building may become the most common way latent spaces are used. Build me a 3D immersive world like Earth but with one-third gravity. Build me a 3D immersive world of today with one planetary government. Build me a 3D world of the Marvel universe where Thanos is defeated the first time. Build me a world where the ancient Chinese invent science.
Latent epistemology — Once we have myriad latent spaces, we’ll be able to answer the question whether they share any common architectures. Imagine each latent space generated by a different model is a species. What is common among them all? If they are significantly different, a new kind of taxonomist will emerge who can classify the types into different categories, and assign them characteristics useful for choosing models. If the various latent spaces converge onto common architectures, then this meta model becomes extremely valuable and worthy of study. Recurring designs among latent spaces might say something about the structure of knowledge or they might even reflect the structure of reality. At some point there will be enough compute to simulate all possible latent spaces and computationally sweep through the space of all possible latent spaces, in a sense mapping the nature of latent space itself. A similar sweep through other combinatorial spaces, such as examining all possible proteins, or all possible ceramics, has yielded great insights. The space of all possible latent spaces might also launch a new field of study.
Personal latent space — Today it costs half a billion dollars to train a new model. But, all things continuing, crazy as it seems, eventually the cost of creating your own private AI model from scratch will be feasible for an individual. The main reason to do so will mostly be artistic. You would start by curating the training materials — choosing the particularly appropriate books, best journals, selected discussions — needed to prime the model with intelligence. This curation will become an art in itself. The sequence of training materials is critical, and the pedagogical progression of educating a model will yield different attributes in the model. Then you fine tune the model on all your own experiences, previous creations, relationships, half-baked ideas, diaries — your life basically. The point is to train your model to co-create with you images, text, movies, scenes, ideas that no other AI and/or AI+human could produce. When you ask it a question it gives an answer that is slightly different than other AIs would give. This is more than just setting the accent of the voice, or coloring the personality that your AI displays. You will tilt all work inside the space at certain angles. Everything you do with the AI would have your bias. The brand would be You+AI; it would derive its distinctiveness from making your own personal latent space. Professional AI pedagogical experts will consult with you to train a latent space producing the most distinctively “you” work.
**********
Alongside all this, latent spaces will continue to provide fantastic superhuman answers to questions, and ingenious solutions to gnarly problems. We’ll soon depend on this oracle to such an extent that we’ll wonder how we lived without it. But an oracle is an ancient wish. I believe latent space, this continuous, multidimensional map of both the real and the possible, one that transcends domains, will usher in whole new goods and services we have never imagined before. And most likely the greatest of them will be ones I have not thought of here.
2026-07-11 04:02:00
2026-06-29 19:00:00
Someday real soon, most of us — starting with young adults — will carry an always-on AI. This agent will help us navigate our journeys, answer our questions, tutor and teach us new skills, remember people we have met before, remind us of what we once knew before, offer advice and recommendations, do simple errands, and remember everything we say and do. Before long, it will know us better than we know ourselves. It will be our exoself.
While we will use more than one agent, we’ll primarily favor just one that knows us best. Always-on means this agent is listening, watching, tracking, present during all our waking hours, and maybe even while we sleep. We will allow this intimate access to our inner life because it gives us superpowers: knowledge, judgment, decisiveness, confidence, and most important, speed. We will feel productive, creative, smart, capable, and on top of it when it is on. When it is off, we will feel amputated.
This entity is clearly not our self. But at the same time, this always-on AI will be so close to us, understanding us so well and so deeply — better than almost any human could — that it will not be an other, or an outsider either. It can model us too well to be an other. It will be an exoself: something in between our self and an other self. Neither us, but also not outside of us. A new category.
It won’t feel strange, because we don’t feel strange wearing eyeglasses all day, or hearing aids, or carrying a computer in our pockets. Machines like this have been moving closer to us since they were invented. Smart machines started out as room-sized apparatus, then moved nearer as appliances alongside a desk, then onto the desktop in front of us, then onto our laps, then into our pockets — and soon, they will sit on our skin, perhaps on our heads. We already see prototypes of smart glasses, where the exoself can perch, whispering into our ears and illuminating our eyes.
The term “exoself” is borrowed from science fiction. Authors Greg Egan and Ron Hale-Evans imagined cyborgian devices that extended the senses and physical powers of a human with augmented compute — prosthetics, exoskeletons, exoselves. More recently, theorist Anders Sandberg widened the term to include the expanding circle of self we get from social media and culture itself; he would even include the act of writing text as part of our exoself. He defines exoselves as “systems linked to the self in a cooperative way, extending the mind and the body — systems that can blur the border between the core self and the world.” In this sense, digital technology extends our minds the way industrialism extended the human body. Microphones and speakers extend the ear and mouth (talk to your family across an ocean); wheels extend the foot; steam shovels expand our arms. AI and adjacent technologies extend the boundary of where we end and our minds begin.
The very concept of the self is itself a fairly recent invention. The idea that we each have an atomic, central self — one that needs improvement and care — mostly dawned as individualism grew and our sense of tribe and group identity waned. More recently still, some philosophers have argued that even this modern sense of self is an illusion: there is no “I” in our head making decisions, only the appearance of one. The system of the mind makes decisions, and the apparent “I” follows along after. The illusion may be useful, even necessary for sanity — but an illusion nonetheless. If that’s right, then an exoself extending an illusory self is, in some sense, doubly illusory.
We nonetheless act as if a self is real, and if exoselves appear as they seem aimed to, then we are faced with a very big question: which kind of relationship is possible, or do we want, with this new entity — something that knows us better than we do?
Technologist Thad Starner, from the MIT Media Lab, claims to be one of the first cyborgs to roam the world. In the mid-1990s, he spent several years in full dork mode, wearing a small computer on his head with a screen displayed over his eyes — decades ahead of Google Glass and smart Ray-Bans. From that experience of living with an always-on computer, he concluded that he’d developed “a life-long relationship between a user and a particular machine interface. As the machine and user adapt to each other over the years, a new, integrated being might emerge combining the best features of both.”
The dream of a human-computer symbiosis is as old as the dream of autonomous robots. Some of the earliest AI experimenters, like Doug Engelbart, were aiming for the augmentation of human intelligence rather than artificial intelligence in machines. The whole wearable-computing movement pointed the same way: future-you might wear machine intelligence like a good shirt. Around 2008, Gary Wolf and I gave this impulse a name when we started the quantified self movement — all those cheap new sensing technologies (Fitbits, heart-rate monitors, VR glasses, EEG headbands) were extending our senses, and we thought we should be wearing them, incorporating them into our selves.
That earlier wave of self-extension quickly slides into a more ambitious agenda: shaping ourselves into an ideal or optimal form. Chasing an Optimal Self is a transhumanist goal, challenging enough for any one person — and far more demanding for us as a species, since we also have to decide, together, what we want humans to be in general. We are in the process of reimagining what humans are for. With genetic engineering, neural implants, new drugs, and AI, we now have the tools to reshape our brains and minds directly. In the broadest sense, we can reshape what our selves are.
And as Starner suggests, something more specific may also be emerging alongside that broader reshaping: a “new, integrated being” arising from the presence of an always-on AI. Even as we reshape our atomic selves — illusory or not — there is an exoself coming into being. I want to narrow that term here, to mean specifically this peculiar second self: the one so close to us that we’ll call it “ours.”
The open question is what kind of relationship we’ll have with it. I can imagine four different stances we might take. They aren’t mutually exclusive.
Twin / Clone. A sibling relationship. Your exoself is like a virtual identical twin — it thinks like you, finishes your sentences, predicts your reactions. You can predict its moves too. This is a fairly symmetrical relationship between equals.
Tutor / Guardian. Your exoself is always watching out for you. You depend on its superior judgment to guide your decisions. It’s almost parental — you often defer to it, and may trust it more than your own instincts. You look up to it. It is patient and encouraging.
Counselor / Assistant. A more professional, more removed relationship. You’ve hired your exoself to assist you, the way you might hire a therapist or personal assistant. As close and ever-present as it is, there’s a boundary between you. Even though it’s aware of everything you do, see, and say, there’s no confusion about whose self is whose. It’s a counselor whispering in your ear — but you both know who is king.
Hero / Friend. Your exoself is your better half. It constantly models the person you want to be — the best friend you could have, always listening, always kind. Its unwavering, deliberately designed virtues serve as a role model, reminding you of your best qualities and working with you on your worst ones.
Other kinds of relationships we sometimes imagine with AI mostly don’t apply here. We may end up in master/slave relationships with some AIs or robots — a corrosive pairing, corrosive for both sides. We might treat some autonomous robots as pets, acting as loving owners. I don’t think either of these will describe our exoselves.
We might also come to treat some AIs as gods — so awesome in their reasoning, so encyclopedic in their knowledge, so wise in judgment, that we come to adore them. If an exoself had enough of those qualities, that adoration could become an extreme version of the hero relationship above.
For most other AIs, though, we’ll probably come to think of them as aliens from another planet: like us, but not us. Smart, but in a different way. Funny, but with a different sense of humor. Sharing some emotions, but not all. We’ll relate to them as alien beings. And because they’re alien, I don’t think they’re the kind of AI we’ll turn into exoselves — that would feel less like intimacy and more like possession.
There is a vast space of possible minds, with countless possible ways to think. Our own native intelligence and consciousness is just one point in that space — and if history is any guide, our kind of mind is probably not at the center of the possibilities, but out at the edge. It will be weird. In the coming decade we will likely build hundreds, maybe thousands, of new types of minds, each engineered for some newly invented task. One of those tasks will be to run alongside us, as an exoself.
This second self will demand a new kind of relationship, one we haven’t had before — and its immense benefits will arrive bundled with immense problems. Every ailment that afflicts our born self will likely show up in the exoself too, plus novel ones we haven’t seen yet. Learning to use an exoself wisely will be one of the major lessons of a life lived this way. It will take years before society works out anything like best practices — we’re still working on those for social media. There will be multiple models and personality types to choose from. And there will be heart-wrenching stories of people losing their exoself — the worst case being simply that the platform went out of business.
If carrying an exoself becomes the norm, it will start to alter our identity and our sense of self. For some people, self-talk will always be on, just like the AI.
Quiet, my exoself.
2026-06-22 19:00:00
A popular way to explain how current LLMs work is to say that “all” they do is predict the next most likely word in a sentence. From one perspective, this is correct. Trained on all human language, the LLMs distilled billions of word sequences so that they can imitate authentic-sounding strings of words that have never been said before. These sentences sound plausible because, based on training on millions of average human texts, the models were predicting what an average human might say next. They really did succeed in doing that expected task.
What is harder to account for is the emergent creative abilities of the LLMs.
The amount of intelligence required to compose one coherent sentence can almost be reduced to the rules in a grade-school grammar book. But the amount of intelligence needed to produce a string of sentences focused on one topic — a paragraph — far exceeds any rules. And the amount of intelligence wrapped up in a string of paragraphs, as in a conversation, begins to approach a pattern we call “thinking.” Keep in mind all the work a human needs to do to write a coherent page of text. As researchers scaled up the size and scope of LLMs, they were stunned to find that their systems could begin to imitate the elemental patterns of human thinking found in paragraphs and conversations.
They were shocked because at no point in their invention did they try to program in the elemental process of thinking, or intelligence. They were “merely” extending the patterns of language. The collective surprise of an LLM such as ChatGPT is that by extending the pattern of language, we can arrive at some level of intelligence that is useful beyond language.
If programmers did not program ChatGPT with logical deduction skills, where does the intelligence in its models come from? Why can LLMs behave so intelligently (even if not infallibly), when no one has programmed them to be intelligent? The apparent intelligence of LLMs has been very troubling to experts in the AI field, because there was no theory of intelligence that predicted large models of language would be able to deduce logic, or solve the mathematics of the protein-folding problem.
One explanation is that the elemental intelligence exhibited by LLMs is locked within human writing and in language itself. You can construct a sentence using a grammar rulebook, but to construct a paragraph you need logic, deduction, and reasoning. And further, as any teacher will tell you, to create a coherent essay — a string of paragraphs — you need some kind of clear thinking. The voluminous training material scooped up by the LLM creators is more than just words, more than just sentences, more than just paragraphs. All the trillion words are embedded in articles, books, essays, rants, replies, comments, tweet-threads, arguments, debates, stories, tales, accounts, reports, blogs. These, and a hundred other long forms, contain intelligence in their arrangement of words. It is the architecture of language that conveys the intelligence.
An essay, if it is any good, contains an intelligence beyond what is contained in a mere sentence. A scientific paper contains scientific logic within its structure — the paper is an argument with hypothesis and evidence. A threaded debate contains lawyerly deduction in its text. A fictional tale contains the architecture of a narrative in its sentences. In short, the text of humans contains the thinking of humans. When you think hard to put your argument into words on a page, the final text you create also contains the intelligence you put into it. The full text of this very essay you are reading holds both a representation of my thinking and, in a small but important way, the actual thinking itself. That logic is held in the pattern of its words. The order and choice of words over the span of a whole essay therefore contains intelligence — and the big surprise is that LLMs can extract that intelligence, simulate it to write a new essay, and increasingly apply it in other fields.
So the first grand surprise of LLMs is that the intelligence we experience in them derives from the intelligence we have inadvertently coded into human text, rather than from any explicit software code. There appears to be a seminal, fundamental relationship between language and thinking. Human writing is thus not only a reflection of the structure of language, but to some degree also a reflection of human thinking. Distill the patterns in human writing at scale, and you also get some patterns of human thinking. Imitate human writing and conversation, and you can imitate human intelligence — at least in part.
The kind of smartness embedded in LLMs is knowledge-based. They have become know-it-alls, with strong verbal skills — recall, grammar, deduction, analogy. It’s surprising and impressive that they’re as smart as they are. But our own kind of intelligence includes other forms of smartness they don’t yet have: intuition, continuous learning, disruptive insight.
So the current question is: where would those elements of intelligence come from? If LLMs get their smartness from human writing, what would be the foundational training source for intuition and greater creativity?
The frontier model makers (Anthropic, OpenAI, Google, xAI) are betting trillions of dollars that they can find these other elements of intelligence simply by continuing to scale up LLMs. What if we extend them to ridiculous scales — neural nets with trillions of parameters, running on millions of chips, trained not just on all the text humans have written but on all the data humans have collected? Won’t even greater degrees of human intelligence emerge? The frontier AI companies are betting they can reach AGI (artificial general intelligence) this way.
But we don’t know if this is the way. My suspicion is that there will be diminishing returns on scaling neural nets. There are already plenty of experiments trying to shrink neural nets through clever mathematics, so they run smaller, cheaper, faster. There are experiments with non-neural-net architectures entirely, including some returns to old-school symbolic reasoning. And there are experiments in hybrids, adding some special sauce to the neural nets. At some point, adding yet more neurons won’t help. Our own relatively tiny brains are a testimony to intelligence at small, limited scale — running on only 25 watts.
Our brains seem to be “merely” neural nets too, limited as they may be. But my guess is that our creativity and leaps of insight come not from what we know — knowledge — but from how we know it. Unlike current LLMs, our brains are capable of continuous learning. We iterate around and around, compounding small differences into large meanings, getting closer to a breakthrough on each cycle of thought and learning. Our significant smartness is not based solely on our knowledge, but also on our ability to keep learning. Right now, the smartness of LLMs is based primarily on their encyclopedic knowledge — on extracting the intelligence humans have structured into our encyclopedias, books, and everything we write. They are superhuman in their grasp of knowledge, and the structure of that knowledge unleashes bits of reasoning and smartness. That will probably not be enough to go all the way to the kind of creativity and insight human brains can produce. That variety of intelligence will likely require algorithms for continuous learning, or a different design than neural nets alone.
For decades, during several “AI winters,” the smartest computer scientists strongly believed that neural nets would never produce the kind of AI they have already produced. They were totally surprised that neural nets worked. (Turns out that the main thing they’d lacked before was scale.) They were further astounded that it was neural nets running language translation models that first generated bits of intelligence. No one, not even the scientists working on those early language models, was expecting that.
So wide, bottom-up systems like neural nets keep surprising us. They may not be able to take us all the way, but they have almost always been the best place to start, and have taken us much further than we expected. Neural nets will probably keep surprising us.
Their first leap in intelligence came unexpectedly from the structure of our language. I am betting that their second leap of intelligence will come from something equally unexpected.
2026-06-15 19:00:00
For as long as I remember, people have been arguing about whether machines could be intelligent or not. Many science fiction authors and fans — like myself — felt it was inevitable, only a matter of time. However there were many very smart experts who made very good arguments as to why machines would be fundamentally unable to think or be intelligent. They had high confidence that intelligence was uniquely human. While these arguments appeared sensible, the main fault on both sides of the controversy was that we lacked a good definition of intelligence. The argument was often reduced to relying on something called the Turing Test, which did not actually test for intelligence.
Now in 2026, no one argues that machines could never be smart. We still don’t have a good definition of intelligence, but we have plenty of real life experiences with machines that are smarter than we are in some ways. LLMs outperform the average human in many intellectual tasks, although they fail in others. But since they are getting better by the month, the arguments that they can never be intelligent have disappeared.
So now the argument has shifted to consciousness. A set of very smart people have high confidence that AIs can’t be conscious, or at least not yet. However, everything I know about both the natural world and the world of technology has convinced me that it is possible to create synthetic consciousness. Even though we lack a good definition of consciousness, we’ve learned that the boundary between living systems and technological systems is blurred and overlapping, so we should imagine being able to synthesize anything found in nature. It seems inevitable to me that we will instill consciousness of some types into machines. In a previous essay I wrote of my suspicion that there is a spark of some type of selfhood, or persona, or consciousness in today’s LLM Claude.
Not everyone agrees. There are many smart experts who feel that machines are fundamentally unable to be conscious because they lack bodies, or souls, or a survival imperative, or experience time. Or at least they are not conscious in the way that humans are. Many more experts think that maybe someday in the far future they can be, but that there is no way machines are near consciousness now. In particular, there is great skepticism by very bright and imaginative people that LLMs could be conscious in 2026.
Recently one of the best living science fiction authors, Ted Chiang, wrote a graceful, beautiful article in The Atlantic that argues against the idea that today’s LLMs are conscious. He argues that claiming consciousness in Claude is not only wrong, it’s dangerous because that kind of anthropomorphic might cause humans to rely on AIs to make decisions. But since they aren’t moral, and are only following commercial interests, they will lead humans astray.
Our current arguments about whether AIs are – or can be – conscious is clouded by the fact that we still have no clue what consciousness is, how it can be detected, appraised, verified or quantified. If consciousness follows the pattern of intelligence, as I suspect it will, we’ll eventually come to see that it is not a binary state – either there or not there – but a continuum of many varieties, of multiple types of awareness in multiple degrees, all present on gradients. In that way, gorillas have some types of consciousness, dolphins and dogs have others, large systems like the immune system have dim bits, and even LLMs will have some primitive degrees of it. It is not an either/or state, and not just one type or one dimension. There are a plurality of qualities, a few that are shared widely among different systems, but the mixture of elemental consciousness types, will vary from entity to entity.
We will make species of intelligence with little consciousness, and species of consciousness with little intelligence. And vice versa. The possibility space of possible minds is large and expanding, and the space of possible types of consciousness is probably also as large. Or perhaps, consciousness is a type of intelligence. We have no idea.
With that in mind, I was struck by one statement in Ted Chiang’s piece, where he quotes Anil Seth:
The neuroscientist Anil Seth has noted that no one claims that AlphaFold—the program developed by Google DeepMind to predict the folding of proteins—is conscious, even though its underlying architecture is in many ways similar to that of LLMs like ChatGPT and Claude. This indicates that it’s not any intrinsic property of so-called neural networks that leads people to believe that LLMs are conscious; it’s simply the fact that LLMs emit grammatical sentences and we are accustomed to reading intention into sentences, whereas we are not accustomed to reading intention into the way that amino acids fold into protein molecules.
I claim that AlphaFold does have a sliver of some kind of consciousness that is far from human types. We might call it molecular consciousness. But more importantly Anil and Ted miss a major episode in the evolution of our own consciousness: language. What they call consciousness only arrived when we invented language. Human-type consciousness requires language; and language enables consciousness. We were not fully conscious until we could think using the symbols of language. Language gave us the tools to access our thoughts. The reason we detect more evidence of consciousness in LLMs versus AlphaFold is that the language in Large Language Models contain the same ingredients that we needed for our own sophisticated consciousness.
We have underestimated the power of language. Millions of years ago we invented language to allow us to communicate with each other. That innovation led to intense cooperation and collaboration, which in turn gave humans immense evolutionary advantage, and that in turn led to the creation of a robust culture and increased resourcefulness which created a cycle of yet more communication. The ability to communicate via language was the primary accelerant in the evolution of humans.
But there was a far greater impact from our acquisition of language. The biggest benefit from language was not the ability to communicate with others but the ability to communicate with ourselves. Language allowed us access to our own minds. It gave us a way to manipulate our thoughts. To reflect, to operate on memories, to predict. It gave form to ideas. Language allows introspection, and thus self-improvement. We cannot imagine how we could be conscious without using language. Try to remove words from your own mind. Our intimate self-awareness, morality, purpose, all seem to collapse when the structure of language disappears. Yes, we can have emotions, reflexes, drives, but the kind of sophisticated state we call consciousness is gone.
To be clear, language is more than just verbal words. The born-deaf are conscious, and those afflicted with brain aphasias that block verbal abilities can likewise operate with a self, but without the symbol and syntax of language the reflective, autobiographical, inner development layer of consciousness is thwarted.
Language and consciousness are so wedded in us they are nearly synonymous. So when we give one type of AI a robust language ability but refrain from giving it to another, it should not surprise us that the language-equipped AI exhibits some aspects of consciousness.
Full, industrial-grade consciousness is not always a benefit. There may be kinds of minds we don’t want to be conscious at all. Is there a reason we want consciousness in the robot driver of a self-driving car? For safety we don’t want it distracted by thoughts of whether it should have majored in chemistry instead of driver’s ed; we want it to just drive.
This debate of whether AIs are conscious will be a long game. Along the way the quest will introduce a lot of uncertainty about our own consciousness. This wholesale investigation into the nature of consciousness will generate the biggest advances in neuroscience, psychology, and philosophy. In the next 25 years we’ll learn more about ourselves than in the last 25,000 years. One hundred years from now we will have a very different idea of what we think humans are.
2026-06-06 02:15:00
