2026-09-02 03:00:00
I just landed a PR that fixed a nasty data race in Quamina. Which shouldn’t have been technically challenging, but I took side-trips into Erlang, unit test footguns, whether or not I should go on accepting LLM-generated PRs, and programming while aging.
Let’s address that last issue first.
In late 2026 we know that:
Claude and its competitors can write perfectly decent code, particularly where the task at hand can be narrowly focused.
They’re also good-to-excellent at detecting vulnerabilities.
The underlying GenAI technology has serious negative externalities in (at least) the domains of environmental impact and intellectual property.
Using this technology might enrich and empower people about whom I feel contempt and fear.
The vast majority of people whom I care about and respect are experiencing real unhappiness about the GenAI big picture.
I’ve never personally put clankers seriously to work and for now I’ll continue not to. I have been accepting Claude-authored PRs but, to the extent I can detect them, I’m stopping. This will have a real cost; some of those PRs yielded big performance boosts. Also, they were built much faster than I could have managed.
But the picture is mixed. Specifically, I have to admit that I don’t really understand the 500 or so lines of clanker code and test that compute epsilon closures for nondeterministic finite automata. Sufficiently so that I’ve written an issue to think it over, understand it, and with any luck, simplify it.
Interestingly, we got to the current epsilon-closure code in a sequence of reasonable-looking PRs that I reviewed, asking for changes in most. But in the big picture my reviewing was ineffective, because it ended with a large lump of code that I don’t understand or, at the end of the day, trust. I wonder how typical that outcome is?
There’s an exception to my new policy: Vulnerability reports. I think it’d be irresponsible to ignore them and penalize Quamina’s users because I don’t like the source. Somebody did a Claude scan which found 14 problems at various levels of criticality. I’ve created issues and hope to retire them all soon.
The ugliest problem the clanker found was a data race in a piece of Quamina called pruner. So I bashed out a quick unit test that ran several thousand read operations in one goroutine and the same number of updates in another. Sure enough, on its first run it instantly exploded in a flurry of data-race complaints.
The problem wasn’t complex and my first cut at a fix worked fine, the unit test running flawlessly. But, also very slowly. I (reasonably, I thought) went looking for contention problems around the mutexes I’d just added. The profiler seemed to be saying that the slowdown was in Quamina’s core matching methods, but that made no sense at all, so I dug deeper and deeper into the Go sync primitives. And found nothing useful.
Concurrency bugs are extra irritating. Also, I’m semi-retired and doing this for fun, so I put it down and ignored it for a few weeks.
There was a bright spot in this sad story. I mistakenly thought I’d found my problem in a statistics-gathering type with several mutexed fields. So, cackling gleefully, I turned them into Erlang-flavored accumulators updated with Go channel messages, a programming idiom that’s always given me a warm glow.
It didn’t make any performance difference but the code was smaller and prettier.
When I got over my grumpiness and went back to the problem, I started by looking at the unit test. Oops. Turns out the read-only thread was (unsurprisingly) running twenty times as fast as the update thread and since they were doing the same number of operations, it finished up instantly and the updater-thread kept thundering away. For reasons that need not concern us, in this specific benchmark scenario a stream of heavy add and delete operations without any stats-gathering read operations leads to massive, horribly complex NFAs. Then the matching runs slow — exactly what the profiler had been trying to tell me, only I thought I was smart enough to know better.
So I forced the reader thread to keep running until the other finished and hey-presto, the race was still fixed and everything ran at typical Quamina speeds, which is to say very fast.
(I didn’t get rid of the Erlangified stats package though, I just liked it too much.)
I’m normally tolerant of a certain amount of sloppy informal YOLO power-coding while constructing unit tests — more are better! — and I think I still am. But this was a painful experience that I’d like to avoid in future. So, is there a lesson here? Maybe there are two: First, believe what the profiler is telling you! Second, if you’re having trouble understanding unit test output, dig around a bit and make sure it’s doing what you think it is.
Even when I’m mad at my software, I’m grateful that I still enjoy working on it. I think it supports my main post-employment objective, which is to keep my brain working. And who knows, the software might help somebody somewhere.
And best of all, when nobody’s paying you to work on something, you can take a few weeks off whenever you’re not in the mood.
2026-08-25 03:00:00
There’s been a lot of discussion recently about the GenAI giants “watermarking” their output, which to say inserting data so you can tell that it’s been generated by ChatGPT or Gemini or Claude or whatever. The technology for watermarking images is as old as dirt, works as advertised, and doesn’t affect image quality. What’s new and controversial is a claim that they can watermark text output cheaply, simply, and without harming the quality of the text.
I was curious how they did this so I dug in a bit. I’m mostly convinced that it works and is pretty clever. However, all the explanations are full of Computer-Science jargon that you need to be a professional developer to understand. So here’s an attempt to explain to non-insiders and then, indirectly, opinions on whether it’s good and what it means and doesn’t mean.
[To professionals: The following is vastly oversimplified but not, I think, misleading.]
LLM software generates output a word at a time. The way it works is, it takes the last few words it’s generated and feeds them to the Large-Language-Model process, and what comes out is a list of plausible next words, and a number which represents just how plausible they are. So if the last few words were “my favorite tropical fruit is” the LLM might come back with “mango”:30, “lychee”:20, “papaya”:10.
You might think that LLMs always choose the most-plausible next word, but that turns out to give lousy results. So they randomly select from the list in a way that respects those numbers. First, note that the numbers add up to 60. So in this case, you’d want it to select “mango” 30 out of 60 attempts, i.e. half the time, “lychee” 20/60 or a third, and “papaya” 10/60 or a sixth.
In that sentence, the word “randomly” is doing a lot of work. Let’s look a little closer at one way you might do this. Your computer program could make a list with 60 entries, fill the first 30 with “mango”, then 20 with “lychee” and 10 with “papaya”. Then you’d ask your computer to give you a random number between 1 and 60. Suppose you got 33, so you pick the 33rd list entry and return “lychee”.
And in that sentence, the phrase “ask your computer” is hard-working. It turns out that computer programmers need random numbers like this all the time (let’s ignore why). They get the numbers from a built-in function called a “PRNG” (let’s ignore what that stands for). The way a PRNG works is you fire one up by giving it a “seed”, which can simply be another number that you get by bashing randomly at the top row on your keyboard or by asking a co-worker to pick a number or whatever.
Once seeded, the PRNG is happy to go on giving you an endless series of random numbers. But (this is important) they depend on the seed. By which I mean that if you fire up a second PRNG with the same seed, it’ll produce exactly the same endless series.
Now we’re almost there. If you think about it, the LLM’s next word depends on, and only on:
The previous few words and the model’s “context”, i.e. the prompts and anything it’s been asked to load up,
The internal workings of the LLM, and
The random number used to select from the list of candidate words. Which in turn depends on the seed that the LLM fired up its PRNG with.
So, suppose you want to test whether some text was generated by an LLM. Well, IF you have access to the LLM, and IF you know which PRNG they used, and IF you know the PRNG’s seed, then you can look at the LLM output and see if the words are the ones that the LLM would have generated. This works even if the output text was edited a bit; if a high proportion of the words are those that the LLM would pick given the preceding ones and the PRNG output, that’s a signal that yes, it was probably LLM-generated.
This only works if you know the details of how the candidate next-words are generated, which PRNG, and what seed was used. It would be technically possible for an LLM provider to share this information, then anyone could check the watermark. In fact, all they have to keep secret is the PRNG seed. So at the end of the day, this “text watermark” means something like “Remember the seed”.
I don’t watch the GenAI sector that closely so I don’t know if any of the providers actually do disclose this stuff. My impression is that generally speaking they’ll provide an API to do this checking for you. Or not, if they decide you’re not entitled to do that, or haven’t paid enough, or whatever.
Here (on X, unfortunately) is an explanation that is slightly less oversimplified than mine above; still, I think, inaccessible to non-professionals, but perhaps useful to some.
Scalable watermarking for identifying large language model outputs is the upstream science, from Nature in October 2024. Strong stuff.
AI Text Watermarking Is Free And Good, by Zvi Mowshowitz, also offers a how-it-works, but then includes commentary around the “… And Good” part of that title. I don’t agree with some of it but the arguments are useful to know and quite clearly written. I’m going to steal a phrase from Mowshowitz’s piece to serve as the title of my next section…
Quite a few people, some very smart, have reacted to this watermarking idea with fear and loathing. Mowshowitz scoffs at them. He (correctly) observes that the hostility is in large part due to people assuming that anything the AI vendors say is probably a lie and that anything they do is probably greedy and dangerous bubble fodder. But he (incorrectly) opines that this take is unreasonable.
So sue me: I too think those businesses lie a lot, that their product vision is damaging to civil society, and that they are inflating what is starting to smell like a truly apocalyptic financial bubble.
I do think the watermarking can work as described and won’t slow things down and won’t degrade the quality of the output.
Sometime in the last year I gave up on making predictions about the impact of GenAI on the world. So don’t ask me what the effect of ubiquitous watermarking will be.
2026-08-18 03:00:00
Fools have said that the Prairies are boring. This cannot be true, because in the absence of hills and trees there’s more distance in each direction you might look. Also, Prairie landscapes include the skies above them which, whether blue, star-spangled, or cloudy, aren’t boring. Herewith photoevidence from the vicinity of Wilcox, Saskatchewan, with which I claim no other connection.
The nearby highway is distinguished by an adjacent railway line that carries really big trains; you really don’t want to be waiting on a crossing while one of them spends tens of minutes reversing back and forth lackadaisically, to what purpose I can’t imagine.
See what I said about the sky not being boring?
This corner of Saskatchewan has less absence of human construction than Prairie-typical. Below, four distinct structures rise from the fields, marked by the clusters of trees which surround every locus of human activity here, there to offer shelter from the endless Prairie wind.
I was out for a drive with my mother, now 96. She asked “How many people wonder what it looks like where the makings of the bread on their table come from?” Below, what it looks like after the harvest. The ground’s less pretty than before the wheat was cut but the dirt/cloud antiphony lured my camera out of the bag.
Prairie photography is right up there with Rock-Concert photography in that there’s all the light and visual drama you could want, just get in a good place and point the camera in a good direction.
Remove distractions, highlight the clouds, and you can’t go wrong.
2026-08-14 03:00:00
It’s been a rough few years, for the earth, for the societies that populate it, and for my family. Like they say, “sandwich generation”. There’ve been weeks when I’ve struggled for a smile. I suspect more than a few folk reading this are also feeling stretched thin these days. Happiness has not entirely departed our lives, and waiting for a happier future is a really bad idea. So we should notice and bookmark those moments and hours when smiles come to our faces.
If you share your life online, write the good times down and share them already. I used to do this a lot; here are three out-takes ranging between 2003 and 2011. Maybe one or two of them will cheer you up?
The Picture of Happiness (2003/06/09)
Chestnut Dusk (2007/10/13)
Thanks (2011/11/14)
I should do that more. Starting now.
We live near a street called Main (which it’s not) and a neighborhood called Mount Pleasant (there’s no mountain but it’s mostly pretty nice). Three weeks back, under the rubric “Pleasant Day”, they blocked cars off nine blocks of Main, and the street was repopulated with retail stands (mostly from the street’s merchants), eat/drink patios, and live music.
It was sunny and everyone was happy, me too for a couple of hours. I also enjoyed taking pictures and editing them.
There was a dog-grooming stand with a big white dreadlocked beast that enjoyed everyone’s attention.
Lucky’s Books and Comics is an excellent bookstore. Their selection echoed the summer-2026 zeitgeist.
There were a couple of stands selling India-flavored garments , they seemed to be doing good business.
Chinese-flavored trinkets too.
Check out the dog.
Who couldn’t smile at a flower truck?
BE GAY DO ART they said.
Or just chill at the honey shop.
She was pretty terrible but everyone enjoyed it.
There’s nothing I can think of that would improve the message on that T-shirt.
Yesterday I was out on an errand with a pretty severe eldercare hangover — man, that can get you down — when I heard I Shot the Sheriff playing. I first thought it was a radio or something, but no, it was Reggae Night with Mostly Marley and they were freaking excellent, razor sharp.
I felt a smile welling up from inside, the first in days. Now I’ve bookmarked it.
2026-08-09 03:00:00
Those first three words in the title name the members of the Dunun family, cylindrical West-African drums that you usually put on a stand to play. Think of those sizes as Papa bear, Mama bear, and Baby bear; the names are a bit onomatopoeic. I bought a set and, separately, stands to put them on. They’re pretty enough that I want to post their pictures, which gives me an excuse to share West-African drumming vibes. And recommend products.
I’ve been studying West-African (and a bit of Afro-Cuban) drumming for decades. This kind of music relies on dununs, djembés, and bells. Djembés are goatskin and played with bare hands, dununs cowhide and with sticks. In class, we play the dununs in traditional style, one drum per person, one hand playing the drum sideways on a stand, the other tapping (with a stick) a bell strapped to its top.
Dunun on a stand (but no bell attached).
Courtesy of Drumskulls (see below).
I’m reasonably proficient on the djembé but nobody would call me a star; I can keep to the beat at a reasonably fast pace and even improvise a bit. On the single sideways dunun, I can get along.
For the last couple of years, I’ve been in an invite-only weekly drum jam; a dozen or fewer of us gather in an old church and create rhythms together for a couple of hours. For me, this has become a major mental-health crutch.
One of our regulars, guy named Mark, brings along three dununs mounted side-by-side vertically so you can play rhythms across all three.
That setup is called “ballet style” because it was was popularized by Les Ballets Africains (that site is worth a visit, as is this documentary).
Anyhow, Mark plays djembé sometimes and welcomes others playing his dunums when he’s not, assuming they ask first.
I’d lacked the nerve to ask — Mark is a pretty elite drummer — but one time a few months ago I felt daring and gave it a try.
Which awoke an unsuspected internal demon. I found playing along with the rhythm effortless, then improvising on top of it; my hands went to surprising places, it seemed of their own accord. Eventually I got excited and was leaping around a bit while playing. After, two of our crew called me out and said that’d been excellent. I love consuming music but very rarely as in never before get compliments about performing it.
I took my turn standing in on Mark’s drums for the next few weeks and had similar fun every time. So I decided to get some. Obviously this involved a lot of research and asking around; I generally prefer to buy used instruments, but couldn’t turn anything up. Mark told me, and others concurred, that a really good source would be Drumskull, where Mark had got his, now 15 years old and still sounding sweet.
There was a small problem, namely that I’m currently boycotting products from the United States of MAGA. But when I emailed and talked with the Drumskullers, it turned out they were old-school hippies from Santa Cruz, which, I decided, earned them an exception.
Then there was a big problem, namely shipping. Somehow Purolator broke the three drums into two shipments with different addresses, both wrong. Multiple phone calls and emails and support-bot interactions occurred and I got seriously bent out of shape. Purolator, you had one job.
Finally I got them, here they are.
I think they’re handsome. Also, they sound wonderful, although Drumskull tells me it’ll take a while for them to break in properly. I also got cases for each from Drumskull and those are superb, I’d feel confident checking them in on an airplane or tossing them into the back of a truck. They also came with three pairs of drumsticks.
I strongly recommend Drumskull. Thanks, guys.
The downside is they’re really freaking heavy, in particular that big Papa-bear drum. By the time I get them out of the house into the car, then out of the car into the church, I’m panting. I’m thinking that on some evenings I could get leave Papa bear behind and get along with the two higher-voiced dununs.
The observant reader will have noticed that in the pictures, the dununs are sitting on nice wooden stands. These are from Shaw Percussion, located in rural Ontario. They’re good people;I recommend them.
The great thing is that they’re adjustable to fit your drums, look nice, and very stable. The bad thing is that, because of being adjustable, they’re heavy and have pointy bits. We have a friend/contractor who’s a gifted carpenter, I may ask him to build a custom set which should be lighter and simpler.
Most cities have an African (and less often Afro-Cuban) drumming community and/or teachers. Most teachers provide drums so the barrier to entry is low. It’s energetic, fun, and will probably change the way you think about rhythm. Also, strikingly gender-balanced both on the teaching and learning side.
If we ever meet at a drumming event feel free to ask for a turn on my dununs.
2026-08-04 03:00:00
The world is, I mean. Bottom-up. Maybe that’s just me not understanding? It doesn’t surprise me that there are people who are stupid and/or evil. The surprise is, the others not voting them out of office, passing legislation outlawing what they’ve been doing, and sending the ones who don’t stop to prison.
But anyhow, I got some amusing photographs at a local garden that are, um, reflective, wanted to blog them, picked the obvious title, then the paragraph above came out. Don’t forgive me. Weird times, bad craziness, etc.
In an upside-down world, reasonable people won’t be. Eventually. I hope.
Calm water helps me calm down. Try pointing your camera at it sometime.