2026-08-27 22:00:00
Big AI models like ChatGPT and Claude get all the attention, but the continually increasing amount of fast, low cost models means today's software can embed little bits of intelligence anywhere they're needed. And when software can think, we can rethink the UI of software.
For most of its history, software has been deterministic. An application's behavior was thought through by developers and hard-coded in: how to check for errors, where to save data, and so on. This rigidity usually made its way to the user interface. Think Web forms with strict input requirements or a limited set of filters for search.
The end result was that people had to adapt their behavior to computers, not the other way around. We learned what little icons meant, remembered which menu item features were in, and tried to resolve (often cryptic) error messages when they came up. Not how most of us would want to spend our day.
Increasingly, though, we have technology that can flip this paradigm on its head. Fast and inexpensive AI models allow software to bend to humans instead. As always, examples help. Search and forms are a huge part of today's software, and both used to force people into computer speak. Today, semantic search replaces a rigid set of filters by turning your question into whatever query syntax is needed behind the scenes.
Multi-modal parsing does the same for input, taking images, documents, or web pages and turning them into the structured entries a database requires.
The common theme in both of these examples is the computer adapting to the human. Instead of making people speak and use computer, computer figures out what people want and gets it done. Add enough bits of intelligence into software and the process of using software becomes dramatically different.
The vision I keep coming back to (minus the bowtie) that illustrates just how different is Apple's 1987 Knowledge Navigator video. Put simply, it makes computing... natural.
There's no voice commands to memorize, it uses natural language. There's no query syntax to learn, it uses fuzzy search. There's no cursors, it uses direct manipulation through touch. All this together lets people act like people, not like computers. Which has always been the goal.
2026-08-17 22:00:00
The Ask LukeW feature on my Web site has been answering people's product design questions using my writings, talks, images, and videos for over three years. During that time, I've seen people ask lots of different kinds of questions. So we rebuilt the system to answer more of them, better. Here's how.
Not all questions are the same. Some are time bound: "What did you write about Web forms in 2010?" Some require specific a format: "Any videos where you talk about onboarding?" Some want a concrete number: "How many talks did you give in Seattle?" Some are just off topic: "Ever dance with a capybara?" Despite this, the way Ask LukeW previously found answers was always the same.
When someone asked a question before, the system ran through four searches at once: semantic and keyword across both titles and contents. Why so many searches at once? A relevant detail can sit deep inside an article whose title looks unrelated, and a strong title match can surface a whole relevant file instead of just passages. The results that come back get merged, deduplicated so we don't repeat the same content multiple times, and re-ranked before being used to answer someone's question.
This approach worked well enough to answer the 50,000+ questions people asked to date. But since it was architected three years ago, things have progressed. Very fast and capable AI models can now layer bits of intelligence into places they previously would have been to slow and expensive to use. Context windows for models have massively expanded in size and new models and ways to embed content open up better ways to do retrieval. So... it was time to rebuild.
We added a lightweight classifier that analyzes each question being asked and decides how best to search for results: a date range, a recency ordering, a comparison, a specific media type, a count, or a casual conversation. Because a question can be both temporal and media-specific at once (for examples) each classification can stack together. This allows us to better answer...
The old system was mostly blind to when I said something. Now it isn't. You can ask things anchored to a moment or a span of time. And it handles the harder version, where the question is about change over time.
Often people don't want an answer, they want a particular artifact. They half-remember it and describe it. The system now figures out you're after one specific thing and gets it.
These are closer to database queries than "find me relevant text" fuzzy text searches. We now recognize that and answer with actual counts and lists instead of a vague summary.
Some questions are best answered visually, and the response should be an image, not a paragraph describing one. We re-indexed every image posted on my site and can serve up to three per answer. We also cite specific PDF pages and drop in a slide when appropriate.
Sometimes people just want to have fun and ask about random topics. These used to get rejected when no result came back with a message saying I hadn't written about it. Now it recognizes a casual question and answers in kind, often with one of the thousands of LukeW character images people have generated thrown in for personality.
The new retreival system also breaks content into much smaller chunks (yes that's the technical term) to better match specific sentences instead of losing them in a big blob (not a technical term). But small chunks lose their surrounding text and thereby important context, so we pull also in neighboring chunks with each result.
There's still a re-ranking step to make sure we're using only the most relevant results to generate answers. But we tuned things to use 75% re-ranked results and 25% of results from our original embedding search, just to make sure the re-ranker is not over-powering the final result set.
Last but not least we diversify the results so there's not mulitple near-identical chunks that might overpower an answer. But because we now pull in neighbors, a small important set (say the 2008 results) can get washed out by a bigger pile from another result. So we diversify within each result separately.
Yes that's a lot of steps but it adds up to better answers. How do we know? Evals of course, this is an AI application after all. But as usual, automatic evals are not enough. Ours was confident the new system was better. But just to be sure, I compared 100 old vs new results by voting on them manually and guess what? The old system won.
Turns out we had a bug, which got fixed. But I mention it because, automatic evals tell you the things moved in the right direction. Human evals find the specific failures you'd otherwise be blind to.
Big thanks to Lukas Seklir, Jerome Paulos, and Sam Breed for this round of changes. I've written about many of the others updates to Ask LukeW over the years, if you want to go deeper into any of it, have at it:
2026-07-01 22:00:00
A few years ago I wrote about the upcoming shift from canvas-centric image and video editing tools to object-centric ones. At the time it was mostly early demos and research projects hinting at what object-centric editing interfaces could be. Reve's new layout model makes them real.
Digital image editing has always been canvas-centric. In Photoshop (used by something like 90% of creative professionals) you're adding, removing, or changing pixels. The application doesn't really know what it's looking at because there's no semantic understanding of the subject matter baked in. It's just pixels and, therefore, lots of pixel manipulation tools.
When every object in an image is identified and addressable, you can edit by changing the subject itself. Move the cat here. Make it a sunny day. Make the cat's eyes green. Popular tools like Photoshop were built before these kinds of capabilities existed so they put pixel manipulation front and center instead of behind the scenes. Object-centric editing flips this.
Most image models use text as their internal representation: a language model expands your prompt into a long description, and a diffusion model renders that into pixels. Text can say a lot, but it says it loosely, and loose input means loose control. To see that in action, just tweak a prompt and watch the whole image change.
Reve's new model is trained to think in layouts instead of prose. A layout is a structured, hierarchical description of an image where every element has a location, a size, a description, and attributes like color. The model takes layouts, instructions, and images as input, works out a layout in its thinking, and then renders the pixels. Layout is to an image what HTML is to a webpage.
When every image is built from a layout, all the elements in it are addressable. A designer can change the color of a car without regenerating the whole image. An interior decorator can move chairs around a room. Shadows, reflections, and perspective stay coherent because the model understands how the pieces relate, not just which colors land where on a canvas.
Because a layout is a readable, structured format, it can also be it a shared interface between people and AI agents. A agent can reason about a layout the way it reasons about code. That enables new forms of human machine collaboration that canvas-centric approaches can't. And we'll see a lot of new creative tools as a result.
2026-06-24 22:00:00
Large scale AI models can produce an innumerable variety of output (text, images, code). In most applications, though, teams want very specific output aligned with brand, design, and coding guidelines. An intentional steering layer can not only help but is increasingly needed in software today.
When building, and even more-so maintaining, websites and software applications, cohesion matters. A consistent brand enforced by intentional use of visuals, colors, fonts, etc. Coding standards, libraries and frameworks, a common development process, etc. Today's AI models can handle all that and more but how do teams make sure they handle it all the way they'd like? And do so repeatedly.
The answer boils down to context: prompting models with clear, comprehensive and relevant instructions. Better context, better results. So how can we encode context into our applications so not only every update to a Website or application stays aligned but every person on a team does too? In our recent projects we've relied on a steering layer.
So what's a steering layer? Put simply, it's how context gets enforced across a project, a team, and even, a company. Every Website or application at its core has a codebase and a set of tools for people to modify that codebase. That's how features ship, performance improves, and ideally software companies make money. A steering layer (conceptually) sits in between the tools and the codebase.
Examples, please. Let's start with a really simple one. My LukeW Character Maker is a small application that allows anyone to make on-brand (for LukeW) image assets. It consists of a tool that allows people to create, review, save, and share assets. A steering layer composed of guidelines, reference images, and prompt rewriting. And the code that runs everything.
The steering layer is responsible for enforcing an AI image model's output aligns with the LukeW character's style, colors, appropriate use guidelines, and more. The LukeW Character Maker tool allows anyone to easily stay on brand when making a LukeW brand asset. And the codebase makes it all happen.
At this point, it's worth noting that the steering layer often lives in the codebase. When it's just text files, reference images, etc. It makes perfect sense to leverage the version control, review process, and team collaboration native to code. But not everyone on a team is comfortable, nor should have to be, working as a developer. For the cases where designers, copywriters, PMs, and others should contribute to and maintain the steering layer, a UI for doing so separate from the codebase makes sense.
As another example, we recently launched a Website for our Agentic HR company, Sol. The steering layer for the website consists of a set of design tokens, development instructions in an agents.md file, and some agent skills. These live in the codebase with everything else. But anyone using AI agents to update the site (with a tool like Intent) is "snapped to" the design and development guidelines in the steering layer. This allows everyone on the team to make Website fixes and add new content without diverging from the design and development guidelines.
It's also worth noting that a steering layer isn't limited to a few text or image files. In the case of the Ask LukeW feature on this Website, the steering layer is composed of not only instructions and examples, but multiple retrieval systems I've iterated on for the past few years as well. Their job is to find the right context for any given question and dynamically apply them to steer results. There's also a whole set of admin tools for managing what's in the context they pull from but that's a whole other topic.
With each new project, we're looking at better ways to make the steering layer visible (and for some, editable) to teams. We're also building steering layer powered tools that enable more people in an organization to contribute to projects without sacrificing cohesion. So... more to come.
2026-06-18 22:00:00
There's a growing sentiment that AI lets you do everyone else's job: designers picking up Claude Code, developers spinning up Figma Make. But maybe the most interesting question isn't "what other jobs can I take on?" but "how do I make the thing I do well reach further?"
When I instruct a coding agent to write a large chunk of code, I can roughly tell what's happening, but I don't have years of professional software development under my belt. I can play a developer on TV. But in real life, I'm a pretty crappy one. The same is true in reverse: a developer can generate a layout, but they don't have the years of pattern recognition that tell a designer what matters and what to throw out.
In other words, the "do it all yourself" approach spreads us thinly across multiple complex and important jobs. That's not a superpower. Unless you want to live out the real-life adventures of Mediocre Man.
Years ago at Yahoo!, we built an internal class for designers called Board to Board (from the drawing board to the boardroom). The whole point was instead of convincing people design needs "a seat at the table" by trying to act like business leaders, figure out what designers bring that nobody else at the table has so you get invited. Pattern recognition. Visual communication that makes concepts clear. Those same skills turn out to matter enormously for telling the story of a company, not just a screen.
The lesson was double down on the thing you're uniquely good at and apply it elsewhere. Don't dilute it trying to be a passable version of other roles.
Similarly we can use AI today to do more jobs or to make our one job have much more reach. We've been building this into how we design and build Web sites and applications using collaborative steering. It allows a designer to put rigor and focus on the grid, the typography, motion rules, and color system of an application. They encode that into a shared context that both people and agents apply. A developer does the same with how code gets organized, written, and tested.
In practice this design and development "intent" ends up as text files outlining instructions that AI agents use when building software. Anyone using agents within a codebase with collaborative steering stays aligned with this intent because their agents make use of it when doing work.
With this approach, designers don't have to be present to influence how things are laid out, their intent is scaled with each AI agent. The grid is the grid no matter how many agents are writing code in parallel.
Collaborative steering doesn't mean dumping every possible bit of design or development context into agent instructions. It's being precise about what matters because with AI agents today, the more you pile in, the less useful it gets.
And that's where expertise kicks in. Knowing the three to five things that actually matter, in what order, and how they need to be done, is the value add. It's the part you can't fake by playing a role you never trained for. And it's the part worth encoding for everyone else.
AI doesn't mean you should go be everyone. It means you can scale the thing you're uniquely wired for.
2026-05-18 22:00:00
Today, AI tools are mostly solo sports. Developers write more code. Designers create more images. PMs crank out more docs. That's cool... but it's more cool to work together. So how does that work with AI?
Many AI-driven productivity tools have evolved from chatting with an AI model to guiding the work of agents capable of a lot more than just answers. But when everyone on a team runs their own agents they guide them towards the outcomes they want, towards their version of things. So in a agentic world, it becomes even easier for perspectives to drift apart. As one design leader recently put it at the Design Futures Assembly: when anybody can build what they want, you feel it in the product because you ship fifteen different ideas instead of one unified point of view.
One of the reasons this happens is that people use different ways to guide agents. Agent markdown files, skills markdown files, system prompts, agent prompts, memory, MCP servers, and so on. The combination all of these disparate elements influences the outcomes AI agents produce.
That's complicated enough for an individual but multiply it across a team and it becomes really difficult to work on the same thing together. Everyone's agents are optimized for their own perspective, not a shared one. And the elements influencing them are scattered across people's computers, codebases, and servers.
We need a different approach for simplifying context management not only for individuals but for groups as well. Think of it as collaborative steering: a mechanism for guiding agents that's collaboratively created, edited, and maintained by teams.
Why collaborative steering? Because, even with all-knowing AI, people have specific expertise and experiences that when brought together make products better. Designers versed in interaction design principles, brand voice, visual design integrity. Engineers focused on performance optimization, easy to maintain code structures, infrastructure choices. But ensuring these distinct roles produce a coherent whole has always been hard. AI can help.
In several of our recent projects we've used Intent, to define project-level context that steers agentic workflows toward shared goals, not away from them. We're currently applying what we learned to larger scale and more ambitious work, which I'll share in the coming weeks.
But after seeing how far we've gotten already, I'm pretty certain that the era of everyone on a team piloting their own disconnected agents can't be the end state. The tools that figure out how to make collaborative steering natural and lightweight are going to change how teams, not just individuals, build.
