Alex WlchanModify

2026-10-13 14:24:07

I have a lot of digital cross-stitch patterns and until recently they were a mess. My “collection” was a series of several hundred PDFs, disorganised and spread across my disk.

Recently, I was looking for a pattern and it took me several minutes to find it. I had a brief moment of panic, thinking that I’d lost it, and the website this pattern came from is long since gone. Fortunately it did turn up, but it was clear I had to organise my pattern collection.

Some people would use apps like Pattern Keeper or Flosshaven, but I just created another static website on my media drive. This is what it looks like:

The patterns are arranged in a masonry layout with a preview thumbnail for each pattern, which is normally the first page of the PDF. If a pattern has multiple files – for example, different files for colour and black-and-white printing – there’s a list of links below the thumbnail. I can also add a source link to where I bought the pattern, if I want to find more patterns by the same author.

Internally, it’s structured like my other static websites. The metadata for the patterns is stored in a JavaScript file, and then I have an HTML page which renders that metadata in this visual design. My pattern collection doesn’t change very often, so I’m adding new files by hand.

There are no sorting or filtering options, because my collection is small enough that they’re unnecessary. I can scroll through the entire list in less than a minute and find whatever I need.

I’d like to start making my own clothes soon, and I expect to copy this design to store clothing patterns.

This collection now sits alongside my saved bookmarks, podcasts, TV shows, and more. I can keep all my patterns organised, and I won’t lose them in the depths of my hard drive.

Technical tricks

There are a couple of technical pieces I’m quite pleased with in this process:

1. Masonry layout. I love the cascading grid style of a masonry layout, and I think they’re great when you have variable-sized content – like my pattern cards. I’ve been using them for over a decade; screenshots of my early web apps show my experiments with David Desandro’s Masonry.js library.

   Masonry layouts are now a native feature of CSS called “Grid Lanes”, which is simpler and more performant than the JavaScript approach. I’m excited to use these layouts in even more places.

2. Thumbnails of PDFs. Most digital postage patterns are shipped as PDFs, and I’m creating the thumbnail images with my create_thumbnail tool.

   I had to extend my tool to support PDFs, which turned out to be straightforward with the pdf_oxide crate. I’d not come across Oxide before, but it’s a cross-language library for working with PDFs, and I can imagine using it in future.

3. A coloured stripe for the palette. In the metadata for each project, I have the complete list of DMC thread colours and the pattern dimensions. I have a lookup table that converts DMC thread colours to hex codes, and then I’m able to create a linear gradient in CSS to render a coloured stripe of the pattern colours.

   Here’s an example from a pattern with four colours:

   border-bottom: 12px solid transparent;
   border-image:  linear-gradient(
     to right,
     #7a451f 0%,
     #7a451f 25%,
     #985e33 25%,
     #985e33 50%,
     #e4bb8e 50%,
     #e4bb8e 75%,
     #361f0e 75%,
     #361f0e 100%
   ) 1;

Building the HTML, CSS, and JavaScript was very quick – I have notes on how I build my media sites, and I’ve done it plenty of times before. I threw together the initial prototype in less than 15 minutes. The longer part was finding all the PDFs scattered across my disk – but now that’s done, and I know exactly where to find my patterns.

[If the formatting of this post looks odd in your feed reader, visit the original article]

2026-09-26 16:43:00

I’ve been revisiting some of my small Rust utilities this week, and I wanted to improve the test suite. There’s a lot of repetition, and I wanted to switch to parametrised tests, a pattern I use a lot in Python and Go. (These are also called “table-driven tests”.)

Parameterised tests allow you to write the test logic once, then run it with many different inputs and expected results. This reduces repetition, highlights variation, and simplifies adding new cases.

For example, in my thumbnail creator, I want to create a thumbnail with various image formats, and check the thumbnail has the correct filename and dimensions. Rather than copy-pasting test functions, I wanted to declare test cases with named fields, like this:

animated_gif: {
    img_path:          "src/tests/animated_squares.gif",
    target:            TargetDimension::MaxWidth(16),
    expected_filename: "animated_squares.mp4",
    expected_dims:     (16, 16),
},

Supporting this syntax required me to create a custom Rust macro.

Existing approaches

I found three existing approaches to writing this type of test in Rust:

1. Use a for loop over the test cases. This is the simplest approach: define our test cases in an array, then iterate over them one-by-one.

   #[test]
   fn test_fibonacci_numbers() {
       let test_cases = [(1, 1), (6, 8), (10, 55)];
   
       for (input, expected) in test_cases {
           assert_eq!(fibonacci(input), expected);
       }
   }

   This is easy to write and understand, but Rust’s standard assertions don’t give us test case isolation. If one case fails, the test stops immediately and skips the rest.

   (Go’s testing library has t.Error, which marks a test as failed but allows it to continue running. When a test fails, you often have more information than if the test had failed immediately. I now miss this when working in other languages.)

2. Define a macro with positional arguments. Many guides, such as over-code’s article, define custom macros with positional arguments.

   fib_test!(my_tests,
       basic: 1, 1,
       sixth: 6, 8,
       tenth: 10, 55,
   );

   Named test cases are nice, but positional arguments hurt readability if you have more than a few parameters.

3. Use a third-party testing library. Crates like parameterized, rstest, and yare use attributes to declare test cases.

   use rstest::rstest;
   
   #[rstest]
   #[case(1, 1)]
   #[case(6, 8)]
   #[case(10, 55)]
   fn fibonacci_test(#[case] input: u32,#[case] expected: u32) {
       assert_eq!(expected, fibonacci(input))
   }

   If I was working in a larger codebase, I’d compare these libraries and pick my favourite.

   I deliberately skipped them this time, because I only use Rust for tiny personal tools, and I write it as a learning exercise. I’d rather write a basic version myself and learn something new, than use somebody else’s library and miss out on the lesson.

Writing my own macro

Since I couldn’t find a macro that gave me named parameters, I decided to write my own.

Here’s the complete macro invocation for my image thumbnailer, with three test cases:

image_thumbnail_tests! {
    animated_gif: {
        img_path:          "src/tests/animated_squares.gif",
        target:            TargetDimension::MaxWidth(16),
        expected_filename: "animated_squares.mp4",
        expected_dims:     (16, 16),
    },
    static_gif: {
        img_path:          "src/tests/yellow.gif",
        target:            TargetDimension::MaxWidth(16),
        expected_filename: "yellow.gif",
        expected_dims:     (16, 8),
    },
    jpeg: {
        img_path:          "src/tests/noise.jpg",
        target:            TargetDimension::MaxWidth(16),
        expected_filename: "noise.jpg",
        expected_dims:     (16, 32),
    },
}

The goal is to have clear, self-documented test cases. You can see what the test does and how to add a new test case without reading the body of the macro.

Here’s the macro definition, with extra whitespace to make the structure more obvious:

macro_rules! image_thumbnail_tests {
    (
        $(
            $name:ident: {
                img_path:          $img_path:expr,
                target:            $target:expr,
                expected_filename: $expected_filename:expr,
                expected_dims:     $expected_dims:expr,
            },
        )*
    )
    =>
    {
        $(
            #[test]
            fn $name() {
                let path = PathBuf::from($img_path);
                let out_dir = test_dir();
                let thumbnail_path = create_thumbnail(&path, &out_dir, $target).unwrap();

                assert_eq!(thumbnail_path, out_dir.join($expected_filename));
                assert!(thumbnail_path.exists());
                assert_eq!(get_dimensions(&thumbnail_path), $expected_dims);
            }
        )*
    }
}

If we zoom out, the structure of a declarative macro looks like a match expression:

macro_rules! image_thumbnail_tests {
  ( matcher ) => { transcriber }
}

I think of the two halves are input/output, but they’re really called matcher/transcriber:

1. The left-hand side (matcher). The matcher defines a pattern of Rust tokens the input code must match precisely, including commas and brackets (but ignoring whitespace).

   • $(…)* works like a regular expression, matching zero or more instances of the enclosed pattern.
   • $name:ident and $img_path:expr match a Rust identifier and expression, respectively, and bind them to metavariables (similar to named capture groups in regexes).

   I deliberately designed my pattern to resemble Rust’s struct syntax. Macro inputs don’t have to be valid Rust code, but I wanted something that would look familiar in a Rust file.

2. The right-hand side (transcriber). The transcriber is a template for Rust code, which gets substituted in at compile time, replacing metavariables with values captured by the matcher.

   Here, the $(…)* iterates over the matches and creates a corresponding function with a #[test] attribute. This attribute is why we need to use a macro – there’s no way to dynamically generate new test functions in Rust without macros.

When compiled, the macro gets replaced with all the generated code from the transcriber, as if I’d written out the individual tests in full:

#[test]
fn animated_gif() {
    let path = PathBuf::from("src/tests/animated_squares.gif");
    let out_dir = test_dir();
    let thumbnail_path = create_thumbnail(&path, &out_dir, TargetDimension::MaxWidth(16)).unwrap();

    assert_eq!(thumbnail_path, out_dir.join("animated_squares.mp4"));
    assert!(thumbnail_path.exists());
    assert_eq!(get_dimensions(&thumbnail_path), (16, 16));
}

#[test]
fn static_gif() {
    let path = PathBuf::from("src/tests/yellow.gif");
    …

This means all of my test cases run as separate tests, so I get all the benefits of individual test functions without having to copy-paste lots of boilerplate throughout my codebase.

I’ve seen custom macros pop up in the few Rust codebases I’ve read, and it feels good to have a better understanding of how they work. I used to copy-paste the same snippets thoughtlessly from project to project, but now I know what the different brackets and asterisks mean. The next time I write a macro or read somebody else’s, I’ll know what’s going on.

[If the formatting of this post looks odd in your feed reader, visit the original article]

2026-09-15 19:04:15

I listen to a lot of podcasts, and I like how they fit around other tasks. I press play, lock my phone, and put it down. I’m free to wash the dishes, fold the laundry, or shop for groceries.

Unfortunately, more and more information is only published as a video. Technical talks, conference sessions, video essays – they don’t work in an audio-only podcast app. I could convert these videos to MP3 files, but that breaks down the moment a video isn’t pure spoken word. If a speaker says, “Look at this slide” or holds up a diagram, an audio-only file leaves me stranded.

I don’t want to give up the podcast player I like, nor stare at a screen for an hour – but I do want the information in these videos.

To solve this, I’m abusing my podcast player’s chapter support. This gives me the best of both worlds: I can listen to a video as audio-first, and glance at my lock screen if I need a moment of visual context.

The idea: Chapters every few seconds

MP3 files can have ID3 metadata, and ID3 metadata can include chapters. A chapter covers a particular time range, and it can have an associated title, description, and cover art.

My podcast app of choice is Overcast, which can’t play videos, but it does have robust chapter support. I can jump between chapters, navigate a table of contents, and see per-chapter cover art.

To get videos into Overcast, I’m creating MP3 files with a new chapter every few seconds, and the per-chapter cover art is a corresponding frame from the video. As I play the file, I get a slow, stop-motion-like rendition of the original video. If my phone is locked, I can glance at my lock screen and see the current frame in the Now Playing screen.

Overcast is developed by Marco Arment, and I got this idea from Forecast, his app for adding chapters to podcasts. In particular, I was struck by its ability to create chapters that don’t display in the chapter list – ideal if I don’t want a table of contents with hundreds of entries. As I was developing my script, I compared my output to the output from Forecast to ensure I was creating the chapters correctly.

The code: FFmpeg and Mutagen

There are three steps in this process:

1. Convert a video file to an MP3
2. Extract images from the video at a fixed interval
3. Insert the images as hidden chapters in the MP3 file

Let’s go through each in turn.

1. Convert a video file to an MP3

Converting a video file to an MP3 is a single FFmpeg command:

ffmpeg -i video.mp4 audio.mp3

This is consistently the slowest step of the process, and I do wonder if I could use different settings or an alternative encoder to make it go faster – but it’s not slow enough to be worth further investigation.

2. Extract images from the video at a fixed interval

Extracting images from a video needs a more complicated FFmpeg command:

ffmpeg -i video.mp4 \
  -vf 'fps=1/5,scale=iw*sar:ih,scale=min(iw\,945):min(ih\,945):force_original_aspect_ratio=decrease' \
   thumbnail_%04d.jpg

This extracts an image every 5 seconds, downscales any image larger than 945 pixels square (while preserving the original aspect ratio), and saves the results as sequentially numbered JPEG images (thumbnail_0001.png, thumbnail_0002.png, and so on).

The key is the -vf flag, which defines two FFmpeg filters:

• The fps filter selects one frame every 5 seconds (fps=1/5).
• The first scale filter scales the width based on the sample aspect ratio (scale=iw*sar:ih). Without this filter, frames can be stretched and distorted.
• The second scale filter scales the input video, preserving the original aspect ratio (force_original_aspect_ratio=decrease), and ensuring the output images fit within 945×945px or the size of the input video, whichever is smaller. My limit is 945 pixels because that’s the largest size that cover art is shown on my iPhone.

This filter still isn’t completely correct – it sometimes creates images from portrait videos that are smaller than I’m expecting – but it’s good enough. These are only thumbnails for glancing at, and if I want to change it later, I can always do the image resizing outside FFmpeg.

3. Insert the images as hidden chapters in the MP3 file

Inserting the chapters into the MP3 file is more complicated. Although FFmpeg has basic support for ID3 metadata, as far as I know, it can’t insert chapters with per-chapter artwork. Instead, I’m going to reach for Python and the Mutagen library.

Here’s the code to add a chapter to an MP3 file:

from mutagen.id3 import APIC, CHAP, ID3, PictureType

audio = ID3("audio.mp3")

with open("thumbnail_0001.jpg", "rb") as f:
    img_data = f.read()

image_frame = APIC(mime="image/jpeg", type=PictureType.OTHER, data=img_data)

chapter_frame = CHAP(
    element_id="chp1", start_time=0, end_time=5 * 1000, sub_frames=[image_frame]
)

audio.add(chapter_frame)
audio.save()

This creates a single chapter that lasts the first 5 seconds (0 to 5000 milliseconds), and the per-chapter cover art is thumbnail_0001.jpg. If we ran this in a loop, we could add images for every 5 second slice of the original video.

This code is inserting two frames into the ID3 metadata:

• The CHAP (chapter) frame contains the timing information, and it can have subframes for metadata like title, chapter art, or associated URL.
• The APIC (attached picture) subframe contains information about a picture, which can either be a blob of image data or a URL to an image on the web.

Normally, you’d also insert a CTOC frame which defines a table of contents, but I don’t want a TOC with hundreds of 5-second chapters, so I’m deliberately not doing this here. This is allowed by the ID3 spec – you’re not required to insert a CTOC frame if you’re using chapters, and you can have chapters that aren’t listed in your table of contents.

To work out which frames I needed, I used Forecast to create some chapters by hand, and I inspected their frames. In particular, loading an MP3 and calling Mutagen’s pprint() method shows a human-readable list of frames, and then I could drill into the individual fields:

from mutagen.id3 import ID3

audio = ID3("audio.mp3")
print(audio.pprint())

I wrapped all this code in a project called glancecast, which allows you to convert a video file with a single command, with optional flags to set the frame length and chapter art size:

$ python3 glancecast.py interesting_talk.mp4
interesting_talk.mp3

The process takes a minute or so to complete, most of which is spent transcoding the video file to MP3. The resulting MP3s are usually 40 to 50 MB in size, which is very reasonable.

The outcome: How it looks in practice

Here’s what one of these “glanceable” podcasts looks like in Overcast and on my lock screen:

It’s not a lot of extra information, but enough that I can quickly glance down and get the gist of what a speaker is saying. Both views update with a new frame every few seconds, or I can put my phone in my pocket and ignore the screen.

I’ve used this approach for half a dozen videos so far, and I’m happy with the results. I expect to keep using it, because I have a long queue of videos I’ve been meaning to watch.

If you’d like to try this, check out glancecast for the full code and instructions.

[If the formatting of this post looks odd in your feed reader, visit the original article]

2026-08-20 17:49:28

I’m working on a project that generates multiple .tar.gz archives, and I need to combine them into one final file. I thought I could just cat the bytes together, but that doesn’t work. This seemingly simple task exposed my flawed understanding of tar and gzip.

To my fix my code, I first had to fix my mental model – and that took me into tape drives, patent laws, and end-of-file markers.

tar stands for tape archive

tar is a file archiver that combines multiple files and their metadata – filenames, timestamps, directory structure – into a single file.

It was originally designed for magnetic tapes, and the file structure is informed by the physical constraints of that medium:

1. Sequential reads. Magnetic tapes are most efficient when you start at the beginning, and play forward to the end of the tape.

2. Append-only writes. Early tapes could only append data to the end of a record, not replace existing data.

3. Fixed data sizes. Tapes have a fixed capacity, and early tapes had fixed data block sizes.

Internally, a tar archive is a sequence of files, each broken into fixed-size blocks. Files have a header block (with metadata like filename and file size) and data blocks (the file contents). After the files, there are two or more blocks filled entirely with zeroes. These form an end-of-file (EOF) marker that tells a reader to disregard everything else in the archive.

This structure mirrors physical tape: you can read files sequentially or append new ones to the end. That sequential design is why tar remains popular for streaming over a network – you can process incoming files immediately, without waiting to download the complete archive.

Knowing this structure helps me understand aspects of tar that I previously found confusing:

• File sizes must be declared upfront. You need to write the file size in the header before you write any data blocks. When I use Python’s TarFile.addfile API, I often forget to set tarinfo.size, so Python writes 0 to the header and creates an empty archive.

• Archives can contain duplicate filenames. You can’t edit or delete existing blocks on tape, so you update a file by appending a new version with the same filename. When you unpack the archive, the later file overwrites the earlier one.

• Everything after the EOF marker is ignored. Because physical tapes have fixed capacities, the EOF marker signals where data ends and empty tape begins. While tools like GNU tar have an --ignore-zeros flag to keep reading past EOF markers, I want to build archives that can be read with the default settings.

I tried a naïve approach of cat-ing tar archives, but that fails because readers stop at the first EOF marker. Instead, I’m combining archives using Python’s tarfile module. I unpack each archive, then copy its members into a new archive which will have a single EOF marker:

import tarfile

def combine_tars(output_file, input_files):
    """
    Combine multiple tar archives into a single archive.
    """
    with tarfile.open(output_file, "w") as out:
        for f in input_files:
            with tarfile.open(f, "r") as src:
                for member in src.getmembers():
                    out.addfile(member, src.extractfile(member))

combine_tars("numbers.tar", ["one.tar", "two.tar", "three.tar"])

This is more code than concatenating raw bytes, but it creates a tar archive that doesn’t need special settings to read.

gzip compresses a single stream of data

gzip is a stream compressor that takes a single file or data stream, and makes it smaller. The compression is lossless, so you can reverse it to retrieve the original file.

Unlike tar, gzip was a response to patent laws, not physical hardware. Reading RFC 1952 which defines the gzip file format, three design constraints reflect the time in which it was created:

1. Patent-free. The gzip tool was written as a free software replacement for compress, a comprssion tool whose underlying LZW algorithm was protected by patents at the time.

2. Streamable. Compressing or decompressing a gzip file must only use a small, bounded amount of memory. In the early 1990s, when RAM was even more scarce and expensive than it is today, the ability to process data in small, continuous chunks was essential.

3. Portable. A gzip file should be independent of the CPU, OS, filesystem, and other aspects of the computer it was created on. We take this sort of portability for granted today, but it wasn’t always a given.

Internally, a gzip file is a sequence of one or more “members”. Each member has a header (with metadata like original filename and modification time), the compressed data, and a trailer (with a CRC32 checksum and uncompressed size). The file ends after the final trailer – gzip doesn’t have EOF markers.

Conceptually, it’s tempting to see members as an analogue for files, but that’s not how gzip works. Tools treat multiple members as part of the same data stream, and you can’t list or extract them individually. When you uncompress a multi-member gzip file, you only get a single stream back.

Because members come one after another and there’s no EOF marker, you can concatenate gzip files by just cat-ing bytes:

echo "one uno eins"    | gzip > one.gz
echo "two duo zwei"    | gzip > two.gz
echo "three tres drei" | gzip > three.gz

cat one.gz two.gz three.gz > numbers.gz

gunzip --uncompress --to-stdout numbers.gz

How do you combine tar.gz archives?

tar and gzip are firm friends. tar combines a directory tree into a single stream; gzip makes that stream smaller. Because they both support sequential reads, .tar.gz is very popular for streaming data over a network – you can start processing individual files before you download the entire archive.

My mistake was trying to combine .tar.gz files using cat. gzip happily combines the compressed members into a single stream, but when tar tries to read the decompressed stream, it finds the first archive’s EOF marker and stops reading.

To combine .tar.gz files safely, I have to extract the underlying members and write them to a new file. That means modifying my Python function above from plain read/write (r/w) to gzip-compressed read/write (r:gz/w:gz):

import tarfile

def combine_tar_gzs(output_file, input_files):
    """
    Combine multiple gzip compressed tar archives into a single archive.
    """
    with tarfile.open(output_file, "w:gz") as out:
        for f in input_files:
            with tarfile.open(f, "r:gz") as src:
                for member in src.getmembers():
                    out.addfile(member, src.extractfile(member))

combine_tar_gzs("numbers.tar.gz", ["one.tar.gz", "two.tar.gz", "three.tar.gz"])

This started as a confusing bug, but it became a fun side quest. Now I understand how these formats work, I understand why my original code doesn’t work, and I understand how I can fix it. I can go back to my project, safe in the knowledge that I haven’t missed a secret shortcut or an obvious optimisation.

[If the formatting of this post looks odd in your feed reader, visit the original article]

2026-08-14 00:14:06

I wrote a post for the Tailscale blog about a long-running series of corruption incidents, and how they eventually led us to find an SQLite bug that predates my entire programming career. I’m incredibly proud of this, both the work and the blog post.

Before Tailscale, I was coming from smaller teams where I didn’t get to tackle problems of this scale or complexity. This was exactly the sort of tricky, deep technical challenge I wanted to be part of (though I’d rather it hadn’t been quite so stressful)! I’m glad I got to play a small part in these incidents, and I learnt so much from the more experienced engineers I worked with. I never want to hear the words “SQLite corruption” again, but if I do, I’d want to have Tailscalars at my side.

Writing the blog post has a blast, too. The piece transformed from a rough draft into a solid, engaging piece of writing, thanks to thoughtful feedback from many people at Tailscale. Most of my writing is self-edited, and it’s always a pleasure to work with a dedicated editor.

Please check out the blog post if you haven’t read it already – I think it’s a fascinating technical story, and one readers of this site are bound to enjoy.

[If the formatting of this post looks odd in your feed reader, visit the original article]

2026-07-18 16:00:04

One of my favourite tiny details in this website is my non-breaking spaces. I have code that looks for phrases like “5 cm”, “New York”, or “Objective‑C”, and inserts a non-breaking space/hyphen so they’ll never be split across multiple lines.

This is the sort of typographical nicety that would be handled by a professional typesetter if I was writing a printed book with a fixed layout, but that’s not how websites work. My website is viewed at lots of different sizes, and browsers choose where to insert line breaks. I add these non-breaking characters so browsers know to avoid awkward line breaks.

Previously I was only applying this detail to body text, but today I implemented something similar for <code> elements.

I used a lot of inline code snippets in my last post, and while reviewing it I noticed that several of those snippets had unhelpful line breaks. For example, (?-u:…) was split into (?- and u:…), while the flag --multiline was split with - on one line and -multiline on the other. These line breaks make the post harder to read, with no benefit.

I can understand why they happened – browsers look for characters where they can break lines, and in English that includes hyphens. It’s usually fine to split hyphenated words over multiple lines, but it’s annoying when that happens in code.

I could fix this by replacing the hyphen in my <code> with a non-breaking hyphen, but people copy/paste code snippets and that might change the meaning.

Instead, I wrote a check that looks for <code> elements which are short and contain a line-breaking character, then adds the nowrap CSS class.

import re

def add_nowrap(match: re.Match[str]) -> str:
    """
    Add the `nowrap` class to a `<code>` element if it contains line
    breaking characters.
    """
    contents = match.group("contents")
    if "-" in contents or " " in contents:
        return f"<code class=\"nowrap\">{contents}</code>"
    return match.group(0)

text: str

# Add the `nowrap` class to <code> snippets which are short and
# contain line-breaking characters.
#
# The limit of 15 characters is arbitrary. In longer code snippets,
# wrapping is preferable to avoid leaving excessive whitespace on
# the previous line.
text = re.sub(r"<code>(?P<contents>[^<]{1,15})</code>", add_nowrap, text)

This is paired with a CSS rule that uses the text-wrap property to tell browsers not to wrap across lines:

code.nowrap {
  text-wrap: nowrap;
}

This wasn’t necessary, but I think it makes the site slightly nicer.

[If the formatting of this post looks odd in your feed reader, visit the original article]