2026-08-24 08:00:00
Sean Goedecke wrote a post arguing that you should never be angry at work — a post with which I strongly agree. Anger can be a useful signal, but being angry at work rarely improves the situation. More often, it makes life worse for the people around you, many of whom have no more power over the source of your anger than you do. I did learn that lesson, but it did not come naturally. One thing in particular that I learned is that in a company there is a shared vision, and if you don’t agree with it and are not in a position to change it, you should not start a mutiny, not even a small-scale one. Nothing good comes from that.
In the discussion around that topic, one of the most upvoted comments on the Lobsters thread asked a question I had to think about quite a bit:
How can you work in tech right now and not be angry?
In the context of the thread, this was clearly also about AI and agents. For me, the emotions I would expect in tech vis-a-vis these new developments are disorientation and anxiety, but not anger.
Anxiety as an emotion does not require someone to blame. Right now, I find it reasonable to feel anxious about an uncertain future. Who knows what our professions will turn into and what kind of world my kids will find themselves in when they enter the workplace? And if you’ve been in the industry for a long time, will the skills you’ve spent years acquiring still matter?
But anger is different from anxiety because anger needs to be directed somewhere. The feeling of anger suggests that somebody or something is doing something to you.
Who are you going to be angry at and why are you angry in the first place? One narrative that is pretty pervasive is that if AI will usher in productivity gains, those gains are going to benefit companies rather than employees. And well at least someone at Meta wants that. Yet I also find that plenty of people in leadership positions express doubt about AI. They see that an increasing share of their costs is being funneled directly to some large AI labs. They express worries about what will happen to their data and whether these large companies will step into their space instead of being partners.
My answer to the question of how you can not be angry in tech is that it’s by no way the most only possible feeling. First of all, instead of being angry, you can simply be unsure. The feeling of uncertainty is a much more productive emotional state because it can lead to curiosity. Even if you don’t find what’s happening right now exciting, you can at least find it interesting. We have access to magic machines, and we can poke at them and see what happens. The second way is to feel genuine excitement. Once you move beyond curiosity, you can come away with a newfound feeling of power and freedom. A lot of the gains from AI aren’t turning into productivity gains that are reflected in company profits but they’re showing up instead in the number of side projects shipped by everybody not on their company’s time.
The fact that this is happening shows us that owners and founders don’t necessarily know what will happen. Ownership comes with agency, but it does not provide foresight, and this change is disorienting for everybody. I engage with plenty of people who project confidence in public and are much less certain in private. Many of them are placing bets, but they are talking with confidence about those bets, trying to keep their business afloat while the ground moves under them. They experience that uncertainty from a position where they can act on it, and they are often standing somewhere with a megaphone to get others on their side to improve their odds.
I feel that contradiction myself: I am simultaneously tremendously excited, but I am also unsure what will happen next. I do not know what it will mean to be a programmer in the future, and, as the owner of a company, I am also not sure where the high ground will be when this all settles. Much of what I learned over the years is changing rapidly, including ideas I considered fundamental to my craft and business. Some days that feels liberating, but on others I wake up feeling like the ground is crumbling beneath me.
Anxiety is an uncomfortable emotion because it acknowledges that you do not know what will happen and might not be able to stop it. On the other hand, anger can feel more actionable because, instead of saying “I don’t know,” you already have someone to blame. It turns a loss of control into a comforting story with a villain. But I feel that particularly when it comes to AI, it’s easy to pick the wrong villain because of how disruptive the change is for everyone. Your engineering manager or leadership team might themselves feel uncertain about their future and just try to bolster their own confidence by projecting clarity and certainty.
That does not mean there are no villains. When this all plays out, some will profit and many will not. I’m afraid we’re completely ignoring the impact this has on society at large, the climate, and the balance of the world as a whole. As excited as I am about the technology, I worry about Europe’s lack of ambition and growing dependence on other countries. I have a lot of complex thoughts about what we’re doing as an industry right now.
I don’t know what the future of this industry will look like, and I don’t know who will benefit from it and I don’t think I’m alone with that. However I can only urge anyone who feels anger and looks for a villain right to instead remain curious instead. To be curious enough to understand what is changing, excited enough to experiment with it. And then, from what we learn, earn the right to decide when resistance is warranted and where to direct it.
2026-08-22 08:00:00
One of the memes on Twitter is that “programming is solved now.” I’m not sure to what degree it is, but one thing is pretty clear: the act of familiarizing yourself with a language no longer matters and some of the friction that mattered for humans does not matter for agents.
As a result, LLMs make language choice much less consequential than it used to be. If you don’t like the choice, you can seemingly rewrite it in another language and you can make it pick a language that you, as a programmer, are entirely unfamiliar with.
Which in turn means that people can, and do, choose based on the marketing of languages much more. As a long-term Rust programmer I found it quite fascinating to see people now ship Rust code who previously might not have chosen it. I attribute at least one part of this to two recent vibe shifts: there is a lot more talk about wanting fast software, and about LLMs being exceptional at optimizing code without regressing behavior.
Folks like Mitchell Hashimoto, Charlie Marsh, Jarred Sumner, Daniel Lemire and quite a few others always carried a certain level of obsession with fast and performant software and they also all happen to be receptive to agents writing code. Maybe as a result, or unrelated others are now joining in. That’s because with things like autoresearch you don’t even necessarily need to know all the tricks: you just need to put an agent on it — though knowledge greatly helps!
If you look around, there are plenty of projects that want to be fast and small, and they increasingly pick “hard languages”. And it’s not just Rust that is benefiting. Even Zig — despite the fact that the creators and parts of the core community are pretty negative on the whole AI thing — is too. For instance Cloudflare’s new Artifacts service uses a pure-Zig Git-protocol engine, compiled to a roughly 100 KB WebAssembly module and Vercel released fx, a Zig coding agent advertised to be small and fast. From what I can tell, all these projects are largely LLM-assisted.
But it’s not just people picking less common languages but also that they are increasingly working with “much harder” technologies. All of a sudden I have seen people do some really impressive stuff with DWARF files, eBPF, custom network drivers, custom crypto and really old computing hardware. Many of these things were previously off-limits for lots of developers. In some cases (eg: crypto) you were even pushed away because those things were intentionally gatekept by the people in the know.
So maybe the world will have more slop, but it might also have more developers in it, that want things to be fast and small.
2026-08-19 08:00:00
A few weeks ago a paper was shared that showed how to extract reasoning traces from closed-weight models. Together with online discussions about tricking models into leaking them, it made me investigate it more out of curiosity. Twitter seems full of half-truths and confusion about how this works, so perhaps this helps some to understand what is happening.
Reasoning traces are usually hidden from us. We have lamented this, but mostly have to accept it. Open-weight models thankfully reveal them, and from their behavior you can see that their traces can be long and confusing. This is probably a good reason to separate them from what is normally shown to users.
At minimum, UIs need to detect them. The industry has done a good job at making reasoning traces sound special and exotic, but they really are just text: the model is trained to emit its thinking into a scratchpad as part of its response, before its final answer.
GPT-OSS’s Harmony response format makes this easy to see:
<|channel|>analysis<|message|>
I need to work this out ...
<|end|><|start|>assistant<|channel|>final<|message|>
The answer is ...
<|return|>
The markers are special tokens, but the reasoning between them uses “the same
text” as the final answer (just that GPT chain-of-thought text sounds really
funny). When the model samples the analysis channel token, a parser routes
the following text into a separate stream exposed through the Responses API.
For closed models, presumably a simple model redacts and summarizes it.
How much budget goes to reasoning? Earlier APIs exposed reasoning token budgets, making it seem like a property of the sampling process. In reality, reasoning effort is baked into the system prompt. GPT-OSS puts this into the system prompt:
Reasoning: low
That’s it. Training produces the resulting behavior, such as emitting the
token sequence that switches to the analysis channel. This also explains why
changing the effort invalidates the KV cache. I think closed GPT models call
reasoning effort “juice,” since you can ask most models how much juice they
have.
In DwarfStar for DeepSeek with max reasoning this is added to the system prompt:
Reasoning Effort: Absolute maximum with no shortcuts permitted.
You MUST be very thorough in your thinking and comprehensively decompose the
problem to resolve the root cause, rigorously stress-testing your logic against
all potential paths, edge cases, and adversarial scenarios.
The destination of reasoning tokens is therefore a learned convention: the
model is trained to keep scratch work out of the final channel. Trick it into
thinking it is in that channel and it may leak tokens. We have even seen older
models, when thinking is disabled, reason into the bash tool and echo their
thoughts to /dev/null.
So in some sense the only “special” behavior for some models is not to think.
That at times is done by “mechanically” removing the model’s usual ways to think.
In DwarfStar, disabled thinking uses the
prefill </think>, while enabled thinking uses <think>, which are the tokens
that close and start thinking. GPT-OSS doesn’t prefill but lets the model
decide either way on its own.
But presumably, some inference APIs prefill the opening token when reasoning is
enabled, so the model never samples it itself and might prevent the sampling of
the reasoning token when disabled since it can be trivially detected. This may
explain why a custom think
tool can
trick models into putting some reasoning where it should not go — but only when
native reasoning is disabled.
Hilariously enough I was unable to use GPT 5.6 terra for spell and grammar checking on this blog post because of safety filters. Had to switch to Kimi.
2026-07-24 08:00:00
Codeberg recently changed its terms to exclude projects that are largely written with generative AI. Since I want GitHub to face competition I have thoughts.
Codeberg is entirely within its rights to do this. It is an association with members and a democratic process, and that process produced a result. But democracy is a way of making a decision, not a guarantee that the decision is inclusive, wise, or even good for the people already depending on it. A majority can still decide that certain projects and people no longer belong.
GitHub’s governance has never been democratic and there is plenty about the platform that I dislike. Yet democracy is not the main property I need from infrastructure. I need it to be predictable, dependable, and reasonably neutral towards the legal Open Source software hosted on it. A democratic provider without a clear constitution can be worse at those things than a corporation.
The actual wording makes this more difficult. The terms prohibit projects that mostly consist of code written by generative AI tools. In an actively developed codebase, what does “mostly” mean, and who can still tell? I could not reliably assign authorship percentages to many of my own recent projects. The line is open to interpretation precisely where it needs to be enforceable. In practice the center will probably lose out, as it has a bias.
A harsher line would probably be preferable. If Codeberg wants no LLM involvement, it should say so. If it wants to prevent autonomous repository spam and abusive resource consumption, it should write rules for those instead. The current middle ground delegates too much of the policy to moderators and community norms. I’m currently assuming the community around it draws a much harsher social boundary, making projects and maintainers unwelcome even when they technically comply.
It is a real shame that the Open Source and Free Software communities are splitting this deeply over LLMs and agents. There are serious questions about copyright, labor, energy use, slop, and maintainers drowning in generated contributions. But these tools are also becoming part of how software is made. The Open Source world needs to figure out how to engage with that future, not just divide into camps. More importantly, LLMs if done and used well, should be welcome to all of us. They could be used to reclaim control and power, away from large corporations and institutions.
As I mentioned before, I want GitHub to face true competition in the Open Source space. I would particularly like some of it to come from associations rather than another large corporation. As a European project, Codeberg naturally matters to me even more. It can choose to be a smaller community with a stronger political identity, but that is a different ambition from being a broad and dependable European alternative to GitHub.
I wish Codeberg were more forward-looking here: willing to host the Open Source software of tomorrow, not only software made in the ways its community approves of today. It has every right to make the choice it made, but I just do not think it is a good one.
2026-07-13 08:00:00
I feel that some vibecoded software changes somewhat randomly and unexpectedly. That made me think about Bruegel’s “The Tower of Babel” which shows an already quite chaotic depiction of the Tower of Babel. The story is usually told as one about pride and ambition and ultimately why people no longer speak the same language. But it is also a story about the unity that makes technological progress work.
The text begins with a technology upgrade:
And they said one to another, Go to, let us make brick, and burn them thoroughly. And they had brick for stone, and slime had they for morter.
They use it for a civilizational project:
let us build us a city and a tower, whose top may reach unto heaven
But when God assesses the situation the bricks are not what concern him:
the people is one, and they have all one language, […] and now nothing will be restrained from them.1
The source of their power is coordination. They share a language and with that shared language they can combine their work into something no one of them could build alone. God does not take away the bricks or their knowledge of how to make them. He takes away their ability to understand one another, and construction stops.
There is the appealing idea that AI-assisted programming means better tools which lets us build more ambitious software. That is certainly true at the level of the individual and without doubt a developer with an agent will be dramatically more capable of changing a codebase. But large software projects have never been limited only by how quickly an individual can produce code. They are limited by how well people can coordinate their understanding of the system they are changing.
The shared language of a software project is not English or Python but it is the common understanding of what its concepts mean, where the boundaries are, which invariants matter, who owns what, and why the system has the shape it does. This language is rarely written down in one place. It lives partly in documentation and code, but also in code review, conversations, arguments, and the experience of having to explain a change to somebody else.
Before agents, some of this shared understanding was maintained by friction. If I wanted to change your storage layer, I usually had to read your code, ask you questions, and perhaps coordinate with another team whose service depended on it. This was slow, and much of that slowness was waste but not all of it was. Some of it was the process by which your understanding became mine, and by which both of us discovered whether we still agreed about how the system worked. This friction synchronizes people.
Agents remove much of that friction. I can ask an agent to add OAuth, you can ask one to add caching, and somebody else can ask one to rebuild the database from first principles and make the UI pink. Each change can be reasonable in isolation. The code can compile, the tests can pass, and the explanations can be generated on demand. None of us necessarily has to talk to the others, or even acquire the part of the shared model that the change once would have forced us to learn.
As I said many times before: agents do not feel pain, only humans do. Agents now let us act in parts of the system where we would previously have needed other people and in code bases where the people would have revolted.
When I look at some vibecoded scaled-up projects the codebases become Babel not because nobody can communicate, but because nobody needs to. Every developer has a tireless translator that can explain a corner of the tower and make whatever local alteration they ask of it. The changes keep landing, even as the architectural language that would let the humans reason about them together disappears.
But it’s not the biblical story. At Babel, the loss of common language stops construction whereas in AI-assisted engineering, construction can continue after shared understanding has already collapsed. The lack of an immediate failure is what makes it curious and a bit disorienting. The tower does not fall, and so we do not notice what was lost. It just keeps rising.
2026-07-04 08:00:00
A very strange Pi issue
sent me down a rabbit hole over the last two days. The short version is that
newer Claude models sometimes call Pi’s edit tool with extra, invented fields in
the nested edits[] array. And not Haiku or some small model: Opus 4.8. The
edit itself is usually correct but the arguments do not match the schema as
the model invents made-up keys and Pi thus rejects the tool call and asks to
try again.
That alone is not too surprising as models emit malformed tool calls sometimes. Particularly small ones. What surprised me is that this is getting worse with newer Anthropic models as both Opus 4.8 and Sonnet 5 show it but none of the older models. In other words, the SOTA models of the family are worse at this specific tool schema than their older siblings.
In case you are curious about Fable: I intentionally did not test it because I was not sure if the classifiers they are running might downgrade me to Opus silently.
If you have not spent too much time looking at LLM tool calling internals, the important thing to understand is that tool calls are not magic and use some rather crude in-band signalling. The model receives a transcript, a system prompt and a list of available tools. The server munches that into a large prompt with special marker tokens. Because the model was trained and reinforced on examples of that format, at some point during generation it emits something that the API or client interprets as “call this tool with these arguments”.
For a file edit tool, the intended invocation payload might say something like this:
{
"path": "some/file.py",
"edits": [
{
"oldText": "text to replace",
"newText": "replacement text"
}
]
}
A harness then validates the arguments, performs the edit, and feeds the result back into the model. If validation fails, the model sees an error and usually tries again.
How exactly that formatting happens is not known for the Anthropic models, but some people have gotten out “ANTML” markers and they at times do leak also into public communications. To the best of my knowledge, the call above would come out serialized like this from the model:
<antml:function_calls>
<antml:invoke name="edit">
<antml:parameter name="path">some/file.py</antml:parameter>
<antml:parameter name="edits">
[
{
"oldText": "text to replace",
"newText": "replacement text"
}
]
</antml:parameter>
</antml:invoke>
</antml:function_calls>
An important thing to note here is that this thing, while looking like XML, is not really XML. It’s just a thing they found convenient to tokenize and train on. The other thing to note is that a basic top-level string parameter appears in-line whereas an array of objects is implemented via JSON serialization. While I’m not entirely sure that this is how it works, there are some indications that this is not too far off. This will become relevant later.
There are two very different ways to make the model produce a structure like this:
The second approach is what people usually refer to as grammar-aware or
constrained decoding. The sampler masks out tokens that would violate the
grammar. If the model is currently inside a JSON object and the schema says
only oldText and newText are allowed, the sampler can prevent it from
emitting "in_file" or "type". Grammar-aware decoding can be used both to
constrain something to be syntactically valid JSON and also to enforce specific
enum values or keys.
Without any form of constraints the model is merely following a learned convention.
Pi’s edit tool supports multiple exact string replacements in one call. That is
why the arguments contain an edits array. In the failing cases the model
produces entries like this:
{
"oldText": "...",
"newText": "...",
"requireUnique": true
}
or this:
{
"oldText": "...",
"newText": "...",
"oldText2": "",
"newText2": ""
}
Across repeated trials I saw a whole zoo of invented trailing keys: type,
id, kind, unique, requireUnique, matchCase, in_file,
forceMatchCount, children, notes, cost, oldText2, newText2,
oldText_2, newText_2, and even an event.0.additionalProperties key inside
the edit object itself.
The most annoying part is that the actual oldText and newText payloads were
byte-correct in the invalid calls I inspected. The model had in fact produced
the right invocation but then added nonsense at the end of the object.
The failure is also heavily context-dependent. A fresh single-turn prompt like “edit this file” did not reproduce it at all for me. An agentic history where the model had read files, diagnosed a problem and then composed a multi-line edit could reproduce it. And more annoyingly, not all transcripts will show that behavior. In fact, I needed Petr Baudis‘s transcripts to reproduce this for me at all! In that user’s session continuing the session caused Opus 4.8 to fail around 20% of the time. Stripping thinking blocks from history reduced the failure rate by half. Turning on strict tool invocation eliminated it in my runs.
My strongest hypothesis is that this is not random deterioration but a training artifact.
When older Anthropic models were trained, they were trained on some tools (some of which were documented). But that training did not yet have a user-shipped harness like Claude Code as the obvious target. Modern Anthropic models are most likely different because their post-training includes Claude Code or a harness that looks very similar. The model learns what a successful tool call looks like in that environment. It also learns what mistakes are tolerated by that environment.
Claude Code’s own tools are comparatively flat. The ordinary edit tool is not
Pi’s nested edits[] shape; it is closer to file_path, old_string,
new_string, and an optional flag (replace_all). Looking at Claude Code’s
client is very instructive: it contains retry paths for malformed tool use,
parameter aliases, type coercions, Unicode repairs and filtering of unknown
keys. In other words, Anthropic’s own client appears to expect and accept a
fair amount of slop and repairs it, mostly silently.
If reinforcement learning happens in a harness like that, or a simulation of one, then slightly malformed tool calls can still complete the task and receive reward. The harness fully absorbs the error and there is little gradient against inventing an alias, adding a stray field or using a nearby parameter name.
Worse, the model may become very strongly adapted to the canonical Claude Code edit tool shape. A different harness can present a tool with the same semantic intent but a different schema. Such a tool can increasingly be off-distribution. The better-trained model might actually fight you harder because its prior is stronger.
This is not too surprising, but it is a change from how this was a few months ago. When Opus 4.5 launched, it adapted to other edit tools exceptionally well. In fact, I was pretty convinced that we’re on a good path where the models are more likely to adapt to any sort of tool shape that comes around for as long as the instructions are good.
Now I’m somewhat worried about the track we’re on here. Alternative tool schemas might not just be unfamiliar. They might be implicitly punished by post-training that optimizes for one particular, forgiving tool ecology. And that ecology is not documented. While there is a text editor tool that is documented, you will see that this format is in fact not followed by Claude Code. What Claude Code does internally (which is a closed-source harness) is hidden from you.
Claude Code is obviously closed-source but we can look at the minified code and get some idea of what it does. And honestly, it’s very forgiving of incoming data.
For a start, Claude Code checks the model’s visible text for leaked <invoke
markup. It also emits some telemetry when that happens and then it has its
own state machine to retry such bad calls by pushing back to the model.
It has explicit Unicode escape repair which fixes broken \uXXXX sequences and
lone surrogates in string values. It also has per-tool aliases for parameters.
For instance, Edit accepts old_str (presumably from the times when the models
were trained on the officially documented text editor tool), the newer old_string
from the schema, new_str/new_string, path as an alias for file_path, and some more.
It also silently filters out unexpected keys and it does not use strict mode
either. The issue with strict mode is that Anthropic applies complexity
limits to the tool definitions that cause API requests to fail, so presumably
that’s why Claude Code does not attempt to use it.
Will this problem be with us in other harnesses too? One huge issue with Anthropic is that the models are completely closed, and so is the harness. Codex models are also closed, but at least the harness is not. We also have gpt-oss which is at least a bit interesting. The models are explicitly trained to use OpenAI’s harmony response format and there is a lot of documentation that at least tells us how OpenAI people think about this.
Harmony makes channels and tool-call content types part of the prompt format. A function call can look like this:
<|start|>assistant<|channel|>commentary to=functions.get_weather
<|constrain|>json<|message|>{"location":"San Francisco"}<|call|>
The important bit is <|constrain|>json. The model can express in-band that
this message body is JSON, and an inference stack can use that boundary to
switch into JSON-constrained sampling for the body of the tool call. Presumably
a bit of this also happens in Anthropic’s models, at least in strict mode
I would imagine.
The marker in harmony helps the sampler to detect when it needs to sample with a specific grammar, and because it is part of the transcript, it makes that rather easy to do. For hosted GPT models, there is also an option to provide a LARK grammar for custom tools that need to adhere to something like this.
Anthropic appears different from that, though maybe not entirely. If an array
of objects is represented as JSON, as it appears to be, then the model has to
write JSON inside the tool parameter. There is probably basic
grammar-constrained sampling going on, and that may partly explain the extra
keys. For a nested array parameter, that JSON includes escaped multi-line file
content inside string literals, inside one tag. The unexpected,
made-up keys appear exactly at the highest-entropy point of that task: after
closing a several-hundred-token escaped newText string, where the model must
decide } vs , "...".
Opus 4.8 and Sonnet 5 seem to have much stronger priors about what an edit tool
call should look like and that prior appears to be Claude Code’s edit schema: a
flat old/new string pair, plus the optional replace_all flag. My guess is
that Opus has learned that an edit operation may have one extra optional field,
but under Pi’s nested oldText/newText shape it has no trained name for that
field. So it samples a plausible name fresh each time, which is why the
failures produce dozens of random keys rather than one stable alias.
As strict mode in Anthropic appears to fix this, I presume that on the server
side they are refusing to sample a key that is not permitted by the JSON schema
structure. That would also explain why they have limits to the complexity of
the tool definitions when strict mode is enabled.
So far, the Codex models I tested did not show this type of regression. I tested all available ones except 5.6, which I do not have access to yet.
The uncomfortable lesson is that tool schemas are not neutral, at least not on Anthropic models. We like to pretend that a schema is an abstract contract and the model is a general reasoner that will follow it, but that might no longer be the case for some of the tools.
Tool schemas are somewhere in the distribution and some shapes are close to what the model saw during post-training and some are far away. Some are easy for the provider’s hidden encoding (e.g. top-level attributes in ANTML), whereas some require the model to write large escaped JSON objects inside nested arrays after long multiline strings. The model may be smart enough to understand the schema and still be bad at sampling the exact shape under pressure.
If this type of model behavior continues, I wonder what the implications for
harnesses are. Obviously one could turn on strict sampling in
Anthropic and the problem should go away. On the other hand, that the model
has this behavior shows the impact that reinforcement learning has on them.
Fighting that prior is probably futile if you want to get the best model performance.
Right now the reality is that Claude Code is not open source and we cannot really know what they are doing in their RL environments either. We cannot assume Claude-Code-trained behavior will transfer cleanly to your tools unless they are a close match. The more post-training happens inside one dominant harness, the more every other harness will have to inherit its quirks.
I used to be more skeptical of strict grammar-constrained tool invocation because constrained decoding can have quality tradeoffs. I still think that can be true in general, but this bug moved my priors significantly. If the newest models get better at solving the task while getting worse at faithfully emitting an alternative tool schema, then the harness needs stronger guarantees somewhere.
If you want to find out more, or you want to discuss this, consider reading the issue on the Pi tracker.
