2026-09-10 22:00:00
As AI can agents tackle more work, we naturally assign more work to them. The most notable example this week was OpenAI's use of 10,000 concurrent agents to propose a solution to the Navier–Stokes Millennium Prize Problem. That's a lot of agents. How do you keep them all on task?
While I don't know how OpenAI coordinated their agents, I do know a lot about the large scale agent coordination techniques in Intent. Intent is primarily for software developers and therefore aligned with their workflows, but how it enables agent orchestration can underpin a wide range of domains. In fact, developers have used Intent's underlying system for reducing their electricity bill, making restaurant reservations, and more.
But first, what's agent coordination? I'd say: aligning lots of instances of back and forth messaging with AI models that have been trained to use tools in order to make progress on a unified task or goal. Coordination helps agents:
So how does Intent enable all this for developers?
Every task runs on its own copy of your files in a dedicated workspace. That isolation keeps agents from overwriting each other's changes. A living spec lives in each workspace and keeps the agents coordinated, recording what got decided and why along the way. The spec allows each agent picking up work to know what came before and what's next.
Human have different jobs (ideally based on what they're good at) and so should agents. Intent comes with a set of default agent roles: a coordinator breaks work into pieces and delegates them; implementer agents write the code; verifier agents check that code against acceptance criteria.
You can also add your own specialists. for example, if your team has conventions worth enforcing (a particular testing approach, a security review step), you can encode that as a reusable role and it shows up in the mix like any other agent.
When writing the spec for a task, a coordinator agent will outline how to get the work done: in what order, by whom, and how. As each agent makes progress, they can determine if need to wait for something else to happen and wake up only when needed. For instance, an agent can monitor a pull request in the background, answer review comments as they arrive, and push when everything's ready.
Handoffs can happen between agents as well. When an agent determines its work is done it can do a back-and-forth with a new agent to then move things forward. This allows new agents to only carry important information forward.
With isolated workspaces, focused context, agent roles, and handoffs, you can scale. Not just many agents per workspace, but many parallel workspaces, and many workspaces on multiple devices. Yes, that's a lot of work happening at once.
Not that long ago, a single agent finishing a coding task felt like magic. Now we're orchestrating thousands of them across devices. As with many things in AI, developer workflows and tooling are the most mature examples of large scale agent coordination. But the underlying approaches (focused context, agent roles, and intelligent handoffs) apply to a lot more than just writing code.
2026-09-08 22:00:00
Just about every chat-based AI application kicks things off with a question. What do you want to do? Ask anything! They just assume that people always know what to ask for and how. But what if we start AI-powered experiences with an answer instead?
To illustrate, the Ask LukeW feature on my Website (which answers digital product design questions using my writings and files) could follow the examples above and start with a seemingly simple interface: just a greeting and an open text field.
While it looks simple, it can be anything but. People regularly struggle with knowing what to ask. To address this, Ask LukeW includes an always changing set of suggested questions to give people both a place to start and a sense of the kinds of questions they can ask. In usability testing, we consistently found that even just reading suggested questions helped people understand the site's purpose and how they could use it.
But even with suggested questions, people still have to ask a question to get started. After updating the retrieval system powering Ask LukeW, I realized things could be different. The new system supported time-based questions like: what did you write about recently? After seeing the results for this question a few times, I decided to just turn it into an answer.
Each day, I grab my most recent tweets, articles, and files and compile a "what's Luke thinking about now" answer automatically. That answer shows up above the suggested questions and gives people something to read and/or respond to without needing to ask anything.
It's a small change but one that addresses one of the most common AI product issues: capability awareness. In other words, how do I know what this product can do? It also aligns with a larger theme I've been poking at: skip the tools and make the outcome. In this case, starting with an answer vs. requiring a question.
2026-08-31 22:00:00
Last year, I walked through the evolution of AI products to date starting with AI models behind the scenes and ending with an assertion that "the inevitable next step is for AI models to collaborate and interact to get things done." Today, we're firmly in that stage with agent orchestration. So what's next?
While an increasing number of software companies have adopted agentic features in their applications, as usual it is developers that are pushing things forward. The most forward leaning developers I work with today are supervising many agents on many projects at once. Hundreds and in some cases more agents coordinating to build, test, and ship software. Which, as you might expect, is a lot to manage. And developer applications born in the chat era of AI products aren't keeping up.
To account for this, we recently redesigned, rebuilt, and open-sourced our developer tool, Intent. Intent scales agent work by turning tasks into separate workspaces, each with an isolated copy of your files so agents don't step on each other's work. Each workspace runs on a living spec and a coordinator that breaks work down, delegates to specialized sub-agents, and keeps things moving without requiring babysitting. There's also a higher level agent to check-in and manage work across both multiple workspaces and multiple devices.
That's a lot of feature talk. But the point is between isolated workspaces, specialist agents, living specs, and more, the product is designed to provide observability (see what’s happening) and introspection (figure out why) over large amounts of agentic work.
While an increasing number of developers are now comfortable no longer manually reviewing AI agent code changes and just interacting with coordinator agents that manage other agents, they still need enough control to dive in, see what's happening, and course correct if needed. But as these systems mature and trust builds, we'll shift to another stage of AI product evolution.
Just like most computer users don't know or care about the code their applications run on their computer much less the deeper code their operating system runs to interact with a computer's hardware, they soon won't know or care how their agents use the applications on their computers. A new layer of abstraction will allow people to get work done in ways that don't require them to know about (and much less know how to use) the software we spend all our time interfacing with today.
In my last post, I highlighted how Apple's Knowledge Navigator video informed by John Sculley's conversations with Alan Kay (who was an Apple Fellow at the time) pointed the way toward human-centric computing. Increasingly feels like the next stage of AI products will feel a lot more like that vision as we inevitably add another layer of abstraction.
2026-08-27 22:00:00
Big AI models like ChatGPT and Claude get all the attention, but the continually increasing amount of fast, low cost models means today's software can embed little bits of intelligence anywhere they're needed. And when software can think, we can rethink the UI of software.
For most of its history, software has been deterministic. An application's behavior was thought through by developers and hard-coded in: how to check for errors, where to save data, and so on. This rigidity usually made its way to the user interface. Think Web forms with strict input requirements or a limited set of filters for search.
The end result was that people had to adapt their behavior to computers, not the other way around. We learned what little icons meant, remembered which menu item features were in, and tried to resolve (often cryptic) error messages when they came up. Not how most of us would want to spend our day.
Increasingly, though, we have technology that can flip this paradigm on its head. Fast and inexpensive AI models allow software to bend to humans instead. As always, examples help. Search and forms are a huge part of today's software, and both used to force people into computer speak. Today, semantic search replaces a rigid set of filters by turning your question into whatever query syntax is needed behind the scenes.
Multi-modal parsing does the same for input, taking images, documents, or web pages and turning them into the structured entries a database requires.
The common theme in both of these examples is the computer adapting to the human. Instead of making people speak and use computer, computer figures out what people want and gets it done. Add enough bits of intelligence into software and the process of using software becomes dramatically different.
The vision I keep coming back to (minus the bowtie) that illustrates just how different is Apple's 1987 Knowledge Navigator video. Put simply, it makes computing... natural.
There's no voice commands to memorize, it uses natural language. There's no query syntax to learn, it uses fuzzy search. There's no cursors, it uses direct manipulation through touch. All this together lets people act like people, not like computers. Which has always been the goal.
2026-08-17 22:00:00
The Ask LukeW feature on my Web site has been answering people's product design questions using my writings, talks, images, and videos for over three years. During that time, I've seen people ask lots of different kinds of questions. So we rebuilt the system to answer more of them, better. Here's how.
Not all questions are the same. Some are time bound: "What did you write about Web forms in 2010?" Some require specific a format: "Any videos where you talk about onboarding?" Some want a concrete number: "How many talks did you give in Seattle?" Some are just off topic: "Ever dance with a capybara?" Despite this, the way Ask LukeW previously found answers was always the same.
When someone asked a question before, the system ran through four searches at once: semantic and keyword across both titles and contents. Why so many searches at once? A relevant detail can sit deep inside an article whose title looks unrelated, and a strong title match can surface a whole relevant file instead of just passages. The results that come back get merged, deduplicated so we don't repeat the same content multiple times, and re-ranked before being used to answer someone's question.
This approach worked well enough to answer the 50,000+ questions people asked to date. But since it was architected three years ago, things have progressed. Very fast and capable AI models can now layer bits of intelligence into places they previously would have been to slow and expensive to use. Context windows for models have massively expanded in size and new models and ways to embed content open up better ways to do retrieval. So... it was time to rebuild.
We added a lightweight classifier that analyzes each question being asked and decides how best to search for results: a date range, a recency ordering, a comparison, a specific media type, a count, or a casual conversation. Because a question can be both temporal and media-specific at once (for examples) each classification can stack together. This allows us to better answer...
The old system was mostly blind to when I said something. Now it isn't. You can ask things anchored to a moment or a span of time. And it handles the harder version, where the question is about change over time.
Often people don't want an answer, they want a particular artifact. They half-remember it and describe it. The system now figures out you're after one specific thing and gets it.
These are closer to database queries than "find me relevant text" fuzzy text searches. We now recognize that and answer with actual counts and lists instead of a vague summary.
Some questions are best answered visually, and the response should be an image, not a paragraph describing one. We re-indexed every image posted on my site and can serve up to three per answer. We also cite specific PDF pages and drop in a slide when appropriate.
Sometimes people just want to have fun and ask about random topics. These used to get rejected when no result came back with a message saying I hadn't written about it. Now it recognizes a casual question and answers in kind, often with one of the thousands of LukeW character images people have generated thrown in for personality.
The new retreival system also breaks content into much smaller chunks (yes that's the technical term) to better match specific sentences instead of losing them in a big blob (not a technical term). But small chunks lose their surrounding text and thereby important context, so we pull also in neighboring chunks with each result.
There's still a re-ranking step to make sure we're using only the most relevant results to generate answers. But we tuned things to use 75% re-ranked results and 25% of results from our original embedding search, just to make sure the re-ranker is not over-powering the final result set.
Last but not least we diversify the results so there's not mulitple near-identical chunks that might overpower an answer. But because we now pull in neighbors, a small important set (say the 2008 results) can get washed out by a bigger pile from another result. So we diversify within each result separately.
Yes that's a lot of steps but it adds up to better answers. How do we know? Evals of course, this is an AI application after all. But as usual, automatic evals are not enough. Ours was confident the new system was better. But just to be sure, I compared 100 old vs new results by voting on them manually and guess what? The old system won.
Turns out we had a bug, which got fixed. But I mention it because, automatic evals tell you the things moved in the right direction. Human evals find the specific failures you'd otherwise be blind to.
Big thanks to Lukas Seklir, Jerome Paulos, and Sam Breed for this round of changes. I've written about many of the others updates to Ask LukeW over the years, if you want to go deeper into any of it, have at it:
2026-07-01 22:00:00
A few years ago I wrote about the upcoming shift from canvas-centric image and video editing tools to object-centric ones. At the time it was mostly early demos and research projects hinting at what object-centric editing interfaces could be. Reve's new layout model makes them real.
Digital image editing has always been canvas-centric. In Photoshop (used by something like 90% of creative professionals) you're adding, removing, or changing pixels. The application doesn't really know what it's looking at because there's no semantic understanding of the subject matter baked in. It's just pixels and, therefore, lots of pixel manipulation tools.
When every object in an image is identified and addressable, you can edit by changing the subject itself. Move the cat here. Make it a sunny day. Make the cat's eyes green. Popular tools like Photoshop were built before these kinds of capabilities existed so they put pixel manipulation front and center instead of behind the scenes. Object-centric editing flips this.
Most image models use text as their internal representation: a language model expands your prompt into a long description, and a diffusion model renders that into pixels. Text can say a lot, but it says it loosely, and loose input means loose control. To see that in action, just tweak a prompt and watch the whole image change.
Reve's new model is trained to think in layouts instead of prose. A layout is a structured, hierarchical description of an image where every element has a location, a size, a description, and attributes like color. The model takes layouts, instructions, and images as input, works out a layout in its thinking, and then renders the pixels. Layout is to an image what HTML is to a webpage.
When every image is built from a layout, all the elements in it are addressable. A designer can change the color of a car without regenerating the whole image. An interior decorator can move chairs around a room. Shadows, reflections, and perspective stay coherent because the model understands how the pieces relate, not just which colors land where on a canvas.
Because a layout is a readable, structured format, it can also be it a shared interface between people and AI agents. A agent can reason about a layout the way it reasons about code. That enables new forms of human machine collaboration that canvas-centric approaches can't. And we'll see a lot of new creative tools as a result.
