MoreRSS

site iconAlex WlchanModify

I‘m a software developer, writer, and hand crafter from the UK. I’m queer and trans.
Please copy the RSS to your reader, or quickly subscribe to:

Inoreader Feedly Follow Feedbin Local Reader

Rss preview of Blog of Alex Wlchan

gobench: a tool for comparing Go benchmark runs across branches

2026-11-02 03:38:34

I’ve recently joined a new team that’s focused on performance, stability, and scalability – a change from my previous projects, where traffic was low enough that performance was rarely a concern.

This codebase has already gone through multiple rounds of optimisation, so there’s no low-hanging fruit left. Any proposed change is inherently complex, and it’s rarely obvious whether a tweak will actually speed things up or slow them down. To measure the impact of our work, we’re leaning on the benchmarking tools in Go’s standard library.

On one of my recent pull requests, I wrote a benchmark, and I pasted the output from two before/after runs. Paul improved this with a short shell script that ran multiple samples, and compared the results with the benchstat command.

$ for N in {1..6}; do for C in 613000c fc12d4c; do echo $N: $C; git checkout -q $C && ./tool/go test -run=^X -bench=BenchmarkPostureAnalyzer ./control/posture/analysis/ | tee -a bench-$C.txt; done; done
$ benchstat bench-fc12d4c.txt bench-613000c.txt

benchstat does statistical analysis on the output from multiple go test -bench=... runs, which is more reliable and easier to read than the raw output. In particular, it puts all the data in a single table, and shows the percentage change against the baseline.

I like this script, but it’s a little fiddly. I have to identify the two commits to compare, it leaves text files in my working directory, and it leaves my repo in a detached head state when it’s done. I wanted something more ergonomic.

I’ve written a new tool gobench to give me a smoother experience. I check out my feature branch, run gobench with the arguments I’d pass to go test, and it prints the final analysis from benchstat.

Under the hood, it creates a Git work tree for the base branch (leaving my working directory as-is), captures the benchmark output in memory (so there are no extra text files), and does an alternating sample of benchmarks.

The alternating samples is a feature I really like from Paul’s original script. I run benchmarks on my laptop rather than a dedicated workstation, and the benchmarks could be sensitive to thermal throttling, background processes, or other things happening on my computer. Alternating between the two branches reduces the effect of this noise.

Let’s look at an example. Suppose my main branch has a recursive implementation of Fibonacci numbers, while my feature branch has an iterative approach. Unsurprisingly, the iterative approach is faster, and gobench makes it easy to quantify the improvement.

$ go run ./cmd/gobench ./example/fib/ -bench=BenchmarkFib
Setting up temporary worktree for main (8f2351d)...
Running 6 alternating samples...
 100% |█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| (12/12, 42 it/min)

== Benchmark results ==
goos: darwin
goarch: arm64
pkg: alexwlchan.net/projects/gotools/example/fib
cpu: Apple M4 Pro
       │ main (8f2351d)  │        faster-fib (0969b03)        │
       │     sec/op      │   sec/op     vs base               │
Fib-12   14572.500n ± 6%   5.455n ± 2%  -99.96% (p=0.002 n=6)

My first version of gobench was an expansion of Paul’s script, but as I started adding features, it became clear that Bash was the wrong tool. I rewrote it as a Go tool that shells out to git, go test, and benchstat.

If you’d like to try gobench yourself, the code is available in my gotools repository, or you can install it with:

$ go install alexwlchan.net/projects/gotools/cmd/gobench@latest

gobench has already saved me from shipping code that looked fast on paper but regressed in practice. As I keep working on performance-critical Go code, having low-friction benchmarks will make it easier to test my ideas early and often.

[If the formatting of this post looks odd in your feed reader, visit the original article]

Verifying my web archives with Playwright

2026-11-01 22:11:41

To protect against link rot and disappearing content, I save local copies of web pages I like into my web archive.

Ideally, this archive is completely self-contained. I want a local copy of each page’s HTML and all the assets it depends on (CSS, images, scripts, and so on). I don’t want my archived pages fetching resources from the original website, which could disappear at any time.

Initially, I was verifying self-containedness by hand. I’d open each saved page in a web browser, and check the browser dev tools. I’d check the Console tab for errors and failed requests, and watch the Network tab for requests going out to the internet. That approach was tedious and error-prone, and I wanted something better.

I now automate the process using Playwright, a testing framework that allows you to control web browsers. I open each page using Playwright, and check if it’s making external requests.

Detecting external requests in Playwright

When you open a page in Playwright, you can subscribe to request events which tell you when the page issues a request.

In this Python snippet, we start Playwright, start a WebKit browser, and open a new page. We tell the page to append external requests to a list, and then load example.com. We see that example.com makes two requests on page load: one for https://example.com, and one for a JavaScript file.

from contextlib import closing
from playwright.sync_api import sync_playwright

with (
    sync_playwright() as p,
    closing(p.webkit.launch()) as browser,
    closing(browser.new_page()) as page,
):
    requests = []
    page.on("request", lambda msg: requests.append(msg))

    page.goto("https://example.com")

print("requests:", requests)
# requests: [<Request url='https://example.com/' method='GET'>,
#            <Request url='https://example.com/s.js' method='GET'>]

In a more complex scenario, we might use Playwright to interact with the page after loading – for example, clicking a button or filling in a form, and then we could see the requests triggered by each action.

Once we have a list of requests made by the page, we can inspect the url property to check if any of them were fetching from an external website. I wrote a function is_external_request to carry out this check:

from playwright.sync_api import Request
import urllib.parse


def is_external_request(req: Request, original_url: str) -> bool:
    """
    Check if a request has a different origin than the original URL.

    A request is considered internal if:
      - it matches the original URL exactly
      - it has the same scheme, host, and port as the original URL
      - it uses a `blob:` scheme (for example, in-memory media representations)

    """
    if req.url == original_url:
        return False

    # Compare the scheme/host of the request to that of the original URL.
    org_url = urllib.parse.urlsplit(original_url)
    req_url = urllib.parse.urlsplit(req.url)

    if org_url.scheme == req_url.scheme and org_url.netloc == req_url.netloc:
        return False

    # Ignore blob: URLs, which are in-memory representation of some data,
    # and not an external request.
    #
    # Playwright detects blob: URLs when loading media files; for example
    # using the <audio> or <video> tag.
    if req_url.scheme == "blob":
        return False

    # Fail open: if we can't decide if a request was internal or external,
    # assume it was external.
    return True

assert_page_is_self_contained(ctx, url)

I’ve wrapped these two steps in a function assert_page_is_self_contained(ctx, url), which is part of my chives utility library.

The function takes a Playwright Browser (or BrowserContext) and a URL, then:

  1. Opens a new page in the browser
  2. Loads the URL in the page
  3. Checks what happened when the page completes loading

It throws an exception if there are any:

  • Uncaught page exceptions or errors
  • Console warnings or errors
  • Non-GET requests
  • Requests targeting external resources

The function tries to trigger important network requests that might be skipped on the initial page load. For example, it evaluates the body’s offsetHeight to force a synchronous reflow, which causes the browser to request any lazily loaded images or fonts that might otherwise be missed.

Here’s an example which verifies a file example.html from the working directory:

from chives.browser_fixtures import assert_page_is_self_contained, file_uri
from contextlib import closing
from playwright.sync_api import sync_playwright

with (
    sync_playwright() as p,
    closing(p.webkit.launch()) as browser,
):
    assert_page_is_self_contained(browser, url=file_uri("example.html"))

I originally passed a Page as the first argument, but changing the function to take a Browser or BrowserContext lets it manage its own pages. It gets a fresh page for every verification, and closes it when it’s done. (The first time I ran this, I created a new page for every verification, and never closed them – my computer ground to a crawl as my script opened hundreds of browser tabs.)

If you create a new context, you can emulate different browser settings, like location, locale, and window size. Here’s an example where we use contexts to test the same page in light/dark mode:

with (
    sync_playwright() as p,
    closing(p.webkit.launch()) as browser,
):
    with closing(browser.new_context(color_scheme="light")) as light_ctx:
        assert_page_is_self_contained(light_ctx, url=file_uri("example.html"))

    with closing(browser.new_context(color_scheme="dark")) as dark_ctx:
        assert_page_is_self_contained(dark_ctx, url=file_uri("example.html"))

In my web archive, I check every page in light/dark mode, and at 1×/2× pixel densities. It takes at least a second to run each verification, so I cache the results. As I save new pages, they’re added to a verification queue, and I’m warned of any missing assets.

Unsurprisingly, the first run flagged many missing fonts and images, which I added to my local archive. Every saved page now passes cleanly.

Browser is best-effort, but may still miss things

Even with automated testing, it’s difficult to guarantee a completely self-contained archive. Different instances of a browser might load different assets, and a single page load might miss different variations:

  • Responsive images – web pages can offer different sizes of an image, and the browser chooses a specific size based on the current window dimensions
  • Appearance settings – a browser might load different fonts or images depending on light/dark mode
  • Interaction-dependent assets – a web page might have assets that are only required on hover or user action

You can mitigate some of this by testing with multiple contexts or adding post-load interactions, but every step makes the process take longer. Fortunately, most of the pages I save are simple, blog-style pages where I’ve stripped out the JavaScript. For my specific use case, initial load verification is more than good enough.

Other uses for this code

There are other projects where it could be useful to intercept and audit the requests made by a web page:

  • Verifying your Content Security Policy (CSP). The CSP header restricts what resources can be loaded by a web page, and you could check that your configuration has the correct effect.
  • Auditing login screens. You could check that sensitive pages don’t load any external scripts, which mitigates the risk of credential theft via a compromised third-party.
  • Testing analytics and tracking. You could script user flows, like a shopping cart checkout, and confirm that tracking events fire correctly.

I don’t do much front-end web development, but Playwright has become a key tool in my toolbox. I read a lot about people using AI agents and MCPs to interact with websites, but I prefer Playwright. It’s more fiddly and requires explicit instructions, but it’s constrained, deterministic, and I never have to worry about it doing something unexpected.

[If the formatting of this post looks odd in your feed reader, visit the original article]

Positive alerts in Prometheus

2026-10-27 02:38:45

At work, we use Prometheus to monitor our services. Every few seconds, Prometheus scrapes a /metrics endpoint and records the data in a time series database. The metrics are then used to trigger alerts when services aren’t behaving correctly.

Recently we had a flapping alert where an overloaded service was intermittently failing. We wanted to silence/adjust the alert until we could fix the service, to avoid alert fatigue, but it would be easy to forget to revert the adjustment once the service was fixed.

We solved this using a “positive alert” that would tell us when the service was consistently passing the original alert – that is, when it was no longer flapping.

Let’s walk through an example. One of our alerts checks that Prometheus can successfully scrape our services, which you can check with the automatically generated up time series. A successful scrape gets a 1, a failed scrape gets a 0.

Here’s an alert that warns if more than 5% of scrapes fail within a half hour window:

groups:
  - name: services
    rules:
    - alert: PrometheusTargetIntermittentlyDown
      expr: avg_over_time(up[30m]) < 0.95
      for: 1h
      labels:
        severity: warning
      annotations:
        summary: Prometheus is struggling to scrape metrics from this service.

One of our services was struggling to serve the /metrics endpoint that Prometheus scrapes. When the service had a lot of work in its queue, the worker thread would starve the web server and the metrics request would fail. The scrape rate would dip below 95%, the alert would fire, then the service would empty its queue and the rate would recover, the alert would clear, and so we went round and round. Dozens of notifications which were nothing but noise.

We use Alertmanager Silences when we want to pause an alert for a short period, but it wasn’t clear if or when we’d be able to fix this service. If the silence expired before the problem was fixed, the noise would return. If the problem was fixed before the silence expired, we could unknowingly regress the fix.

Instead, we modified the alert to exclude the overloaded service, then added a “positive alert” to tell us if the service had more than 95% uptime for a fortnight:

groups:
  - name: services
    rules:
    - alert: PrometheusTargetIntermittentlyDown
      expr: avg_over_time(up{instance!="overloaded_service"}[30m]) < 0.95
      for: 1h
      labels:
        severity: warning
      annotations:
        summary: Prometheus is struggling to scrape metrics from this service.
    
    - alert: PrometheusTargetUpAvg
      expr: avg_over_time(up{instance="overloaded_service"}[30m]) >= 0.95
      for: 14d
      labels:
        severity: info
      annotations:
        summary: This service is no longer overloaded; remove the exception.

This worked exactly as intended. A few weeks ago, one of my colleagues made changes to improve the overloaded service, and Prometheus was able to crawl it consistently. The alert fired at the weekend as it ticked over a fortnight of continuous uptime, and we removed the exception.

[If the formatting of this post looks odd in your feed reader, visit the original article]

Organising my cross-stitch patterns

2026-10-13 14:24:07

I have a lot of digital cross-stitch patterns and until recently they were a mess. My “collection” was a series of several hundred PDFs, disorganised and spread across my disk.

Recently, I was looking for a pattern and it took me several minutes to find it. I had a brief moment of panic, thinking that I’d lost it, and the website this pattern came from is long since gone. Fortunately it did turn up, but it was clear I had to organise my pattern collection.

Some people would use apps like Pattern Keeper or Flosshaven, but I just created another static website on my media drive. This is what it looks like:

A collection of cross-stitch patterns shown as a grid of cards. Each card has a thumbnail showing a picture of what the pattern looks like, then below it is the title and a list of links. There's also a stripe of colours below the thumbnail, showing the colours used in the pattern.

The patterns are arranged in a masonry layout with a preview thumbnail for each pattern, which is normally the first page of the PDF. If a pattern has multiple files – for example, different files for colour and black-and-white printing – there’s a list of links below the thumbnail. I can also add a source link to where I bought the pattern, if I want to find more patterns by the same author.

Internally, it’s structured like my other static websites. The metadata for the patterns is stored in a JavaScript file, and then I have an HTML page which renders that metadata in this visual design. My pattern collection doesn’t change very often, so I’m adding new files by hand.

There are no sorting or filtering options, because my collection is small enough that they’re unnecessary. I can scroll through the entire list in less than a minute and find whatever I need.

I’d like to start making my own clothes soon, and I expect to copy this design to store clothing patterns.

This collection now sits alongside my saved bookmarks, podcasts, TV shows, and more. I can keep all my patterns organised, and I won’t lose them in the depths of my hard drive.

Technical tricks

There are a couple of technical pieces I’m quite pleased with in this process:

  1. Masonry layout. I love the cascading grid style of a masonry layout, and I think they’re great when you have variable-sized content – like my pattern cards. I’ve been using them for over a decade; screenshots of my early web apps show my experiments with David Desandro’s Masonry.js library.

    Masonry layouts are now a native feature of CSS called “Grid Lanes”, which is simpler and more performant than the JavaScript approach. I’m excited to use these layouts in even more places.

  2. Thumbnails of PDFs. Most digital postage patterns are shipped as PDFs, and I’m creating the thumbnail images with my create_thumbnail tool.

    I had to extend my tool to support PDFs, which turned out to be straightforward with the pdf_oxide crate. I’d not come across Oxide before, but it’s a cross-language library for working with PDFs, and I can imagine using it in future.

  3. A coloured stripe for the palette. In the metadata for each project, I have the complete list of DMC thread colours and the pattern dimensions. I have a lookup table that converts DMC thread colours to hex codes, and then I’m able to create a linear gradient in CSS to render a coloured stripe of the pattern colours.

    Here’s an example from a pattern with four colours:

    border-bottom: 12px solid transparent;
    border-image:  linear-gradient(
      to right,
      #7a451f 0%,
      #7a451f 25%,
      #985e33 25%,
      #985e33 50%,
      #e4bb8e 50%,
      #e4bb8e 75%,
      #361f0e 75%,
      #361f0e 100%
    ) 1;

Building the HTML, CSS, and JavaScript was very quick – I have notes on how I build my media sites, and I’ve done it plenty of times before. I threw together the initial prototype in less than 15 minutes. The longer part was finding all the PDFs scattered across my disk – but now that’s done, and I know exactly where to find my patterns.

[If the formatting of this post looks odd in your feed reader, visit the original article]

Parametrised tests in Rust with named parameters

2026-09-26 16:43:00

I’ve been revisiting some of my small Rust utilities this week, and I wanted to improve the test suite. There’s a lot of repetition, and I wanted to switch to parametrised tests, a pattern I use a lot in Python and Go. (These are also called “table-driven tests”.)

Parameterised tests allow you to write the test logic once, then run it with many different inputs and expected results. This reduces repetition, highlights variation, and simplifies adding new cases.

For example, in my thumbnail creator, I want to create a thumbnail with various image formats, and check the thumbnail has the correct filename and dimensions. Rather than copy-pasting test functions, I wanted to declare test cases with named fields, like this:

animated_gif: {
    img_path:          "src/tests/animated_squares.gif",
    target:            TargetDimension::MaxWidth(16),
    expected_filename: "animated_squares.mp4",
    expected_dims:     (16, 16),
},

Supporting this syntax required me to create a custom Rust macro.

Existing approaches

I found three existing approaches to writing this type of test in Rust:

  1. Use a for loop over the test cases. This is the simplest approach: define our test cases in an array, then iterate over them one-by-one.

    #[test]
    fn test_fibonacci_numbers() {
        let test_cases = [(1, 1), (6, 8), (10, 55)];
    
        for (input, expected) in test_cases {
            assert_eq!(fibonacci(input), expected);
        }
    }

    This is easy to write and understand, but Rust’s standard assertions don’t give us test case isolation. If one case fails, the test stops immediately and skips the rest.

    (Go’s testing library has t.Error, which marks a test as failed but allows it to continue running. When a test fails, you often have more information than if the test had failed immediately. I now miss this when working in other languages.)

  2. Define a macro with positional arguments. Many guides, such as over-code’s article, define custom macros with positional arguments.

    fib_test!(my_tests,
        basic: 1, 1,
        sixth: 6, 8,
        tenth: 10, 55,
    );

    Named test cases are nice, but positional arguments hurt readability if you have more than a few parameters.

  3. Use a third-party testing library. Crates like parameterized, rstest, and yare use attributes to declare test cases.

    use rstest::rstest;
    
    #[rstest]
    #[case(1, 1)]
    #[case(6, 8)]
    #[case(10, 55)]
    fn fibonacci_test(#[case] input: u32,#[case] expected: u32) {
        assert_eq!(expected, fibonacci(input))
    }

    If I was working in a larger codebase, I’d compare these libraries and pick my favourite.

    I deliberately skipped them this time, because I only use Rust for tiny personal tools, and I write it as a learning exercise. I’d rather write a basic version myself and learn something new, than use somebody else’s library and miss out on the lesson.

Writing my own macro

Since I couldn’t find a macro that gave me named parameters, I decided to write my own.

Here’s the complete macro invocation for my image thumbnailer, with three test cases:

image_thumbnail_tests! {
    animated_gif: {
        img_path:          "src/tests/animated_squares.gif",
        target:            TargetDimension::MaxWidth(16),
        expected_filename: "animated_squares.mp4",
        expected_dims:     (16, 16),
    },
    static_gif: {
        img_path:          "src/tests/yellow.gif",
        target:            TargetDimension::MaxWidth(16),
        expected_filename: "yellow.gif",
        expected_dims:     (16, 8),
    },
    jpeg: {
        img_path:          "src/tests/noise.jpg",
        target:            TargetDimension::MaxWidth(16),
        expected_filename: "noise.jpg",
        expected_dims:     (16, 32),
    },
}

The goal is to have clear, self-documented test cases. You can see what the test does and how to add a new test case without reading the body of the macro.

Here’s the macro definition, with extra whitespace to make the structure more obvious:

macro_rules! image_thumbnail_tests {
    (
        $(
            $name:ident: {
                img_path:          $img_path:expr,
                target:            $target:expr,
                expected_filename: $expected_filename:expr,
                expected_dims:     $expected_dims:expr,
            },
        )*
    )
    =>
    {
        $(
            #[test]
            fn $name() {
                let path = PathBuf::from($img_path);
                let out_dir = test_dir();
                let thumbnail_path = create_thumbnail(&path, &out_dir, $target).unwrap();

                assert_eq!(thumbnail_path, out_dir.join($expected_filename));
                assert!(thumbnail_path.exists());
                assert_eq!(get_dimensions(&thumbnail_path), $expected_dims);
            }
        )*
    }
}

If we zoom out, the structure of a declarative macro looks like a match expression:

macro_rules! image_thumbnail_tests {
  ( matcher ) => { transcriber }
}

I think of the two halves are input/output, but they’re really called matcher/transcriber:

  1. The left-hand side (matcher). The matcher defines a pattern of Rust tokens the input code must match precisely, including commas and brackets (but ignoring whitespace).

    • $(…)* works like a regular expression, matching zero or more instances of the enclosed pattern.
    • $name:ident and $img_path:expr match a Rust identifier and expression, respectively, and bind them to metavariables (similar to named capture groups in regexes).

    I deliberately designed my pattern to resemble Rust’s struct syntax. Macro inputs don’t have to be valid Rust code, but I wanted something that would look familiar in a Rust file.

  2. The right-hand side (transcriber). The transcriber is a template for Rust code, which gets substituted in at compile time, replacing metavariables with values captured by the matcher.

    Here, the $(…)* iterates over the matches and creates a corresponding function with a #[test] attribute. This attribute is why we need to use a macro – there’s no way to dynamically generate new test functions in Rust without macros.

When compiled, the macro gets replaced with all the generated code from the transcriber, as if I’d written out the individual tests in full:

#[test]
fn animated_gif() {
    let path = PathBuf::from("src/tests/animated_squares.gif");
    let out_dir = test_dir();
    let thumbnail_path = create_thumbnail(&path, &out_dir, TargetDimension::MaxWidth(16)).unwrap();

    assert_eq!(thumbnail_path, out_dir.join("animated_squares.mp4"));
    assert!(thumbnail_path.exists());
    assert_eq!(get_dimensions(&thumbnail_path), (16, 16));
}

#[test]
fn static_gif() {
    let path = PathBuf::from("src/tests/yellow.gif");
    …

This means all of my test cases run as separate tests, so I get all the benefits of individual test functions without having to copy-paste lots of boilerplate throughout my codebase.

I’ve seen custom macros pop up in the few Rust codebases I’ve read, and it feels good to have a better understanding of how they work. I used to copy-paste the same snippets thoughtlessly from project to project, but now I know what the different brackets and asterisks mean. The next time I write a macro or read somebody else’s, I’ll know what’s going on.

[If the formatting of this post looks odd in your feed reader, visit the original article]

Abusing ID3 chapters to turn videos into glanceable podcasts

2026-09-15 19:04:15

I listen to a lot of podcasts, and I like how they fit around other tasks. I press play, lock my phone, and put it down. I’m free to wash the dishes, fold the laundry, or shop for groceries.

Unfortunately, more and more information is only published as a video. Technical talks, conference sessions, video essays – they don’t work in an audio-only podcast app. I could convert these videos to MP3 files, but that breaks down the moment a video isn’t pure spoken word. If a speaker says, “Look at this slide” or holds up a diagram, an audio-only file leaves me stranded.

I don’t want to give up the podcast player I like, nor stare at a screen for an hour – but I do want the information in these videos.

To solve this, I’m abusing my podcast player’s chapter support. This gives me the best of both worlds: I can listen to a video as audio-first, and glance at my lock screen if I need a moment of visual context.

The idea: Chapters every few seconds

MP3 files can have ID3 metadata, and ID3 metadata can include chapters. A chapter covers a particular time range, and it can have an associated title, description, and cover art.

My podcast app of choice is Overcast, which can’t play videos, but it does have robust chapter support. I can jump between chapters, navigate a table of contents, and see per-chapter cover art.

To get videos into Overcast, I’m creating MP3 files with a new chapter every few seconds, and the per-chapter cover art is a corresponding frame from the video. As I play the file, I get a slow, stop-motion-like rendition of the original video. If my phone is locked, I can glance at my lock screen and see the current frame in the Now Playing screen.

Overcast is developed by Marco Arment, and I got this idea from Forecast, his app for adding chapters to podcasts. In particular, I was struck by its ability to create chapters that don’t display in the chapter list – ideal if I don’t want a table of contents with hundreds of entries. As I was developing my script, I compared my output to the output from Forecast to ensure I was creating the chapters correctly.

The code: FFmpeg and Mutagen

There are three steps in this process:

  1. Convert a video file to an MP3
  2. Extract images from the video at a fixed interval
  3. Insert the images as hidden chapters in the MP3 file

Let’s go through each in turn.

1. Convert a video file to an MP3

Converting a video file to an MP3 is a single FFmpeg command:

ffmpeg -i video.mp4 audio.mp3

This is consistently the slowest step of the process, and I do wonder if I could use different settings or an alternative encoder to make it go faster – but it’s not slow enough to be worth further investigation.

2. Extract images from the video at a fixed interval

Extracting images from a video needs a more complicated FFmpeg command:

ffmpeg -i video.mp4 \
  -vf 'fps=1/5,scale=iw*sar:ih,scale=min(iw\,945):min(ih\,945):force_original_aspect_ratio=decrease' \
   thumbnail_%04d.jpg

This extracts an image every 5 seconds, downscales any image larger than 945 pixels square (while preserving the original aspect ratio), and saves the results as sequentially numbered JPEG images (thumbnail_0001.png, thumbnail_0002.png, and so on).

The key is the -vf flag, which defines two FFmpeg filters:

  • The fps filter selects one frame every 5 seconds (fps=1/5).
  • The first scale filter scales the width based on the sample aspect ratio (scale=iw*sar:ih). Without this filter, frames can be stretched and distorted.
  • The second scale filter scales the input video, preserving the original aspect ratio (force_original_aspect_ratio=decrease), and ensuring the output images fit within 945×945px or the size of the input video, whichever is smaller. My limit is 945 pixels because that’s the largest size that cover art is shown on my iPhone.

This filter still isn’t completely correct – it sometimes creates images from portrait videos that are smaller than I’m expecting – but it’s good enough. These are only thumbnails for glancing at, and if I want to change it later, I can always do the image resizing outside FFmpeg.

3. Insert the images as hidden chapters in the MP3 file

Inserting the chapters into the MP3 file is more complicated. Although FFmpeg has basic support for ID3 metadata, as far as I know, it can’t insert chapters with per-chapter artwork. Instead, I’m going to reach for Python and the Mutagen library.

Here’s the code to add a chapter to an MP3 file:

from mutagen.id3 import APIC, CHAP, ID3, PictureType

audio = ID3("audio.mp3")

with open("thumbnail_0001.jpg", "rb") as f:
    img_data = f.read()

image_frame = APIC(mime="image/jpeg", type=PictureType.OTHER, data=img_data)

chapter_frame = CHAP(
    element_id="chp1", start_time=0, end_time=5 * 1000, sub_frames=[image_frame]
)

audio.add(chapter_frame)
audio.save()

This creates a single chapter that lasts the first 5 seconds (0 to 5000 milliseconds), and the per-chapter cover art is thumbnail_0001.jpg. If we ran this in a loop, we could add images for every 5 second slice of the original video.

This code is inserting two frames into the ID3 metadata:

  • The CHAP (chapter) frame contains the timing information, and it can have subframes for metadata like title, chapter art, or associated URL.
  • The APIC (attached picture) subframe contains information about a picture, which can either be a blob of image data or a URL to an image on the web.

Normally, you’d also insert a CTOC frame which defines a table of contents, but I don’t want a TOC with hundreds of 5-second chapters, so I’m deliberately not doing this here. This is allowed by the ID3 spec – you’re not required to insert a CTOC frame if you’re using chapters, and you can have chapters that aren’t listed in your table of contents.

To work out which frames I needed, I used Forecast to create some chapters by hand, and I inspected their frames. In particular, loading an MP3 and calling Mutagen’s pprint() method shows a human-readable list of frames, and then I could drill into the individual fields:

from mutagen.id3 import ID3

audio = ID3("audio.mp3")
print(audio.pprint())

I wrapped all this code in a project called glancecast, which allows you to convert a video file with a single command, with optional flags to set the frame length and chapter art size:

$ python3 glancecast.py interesting_talk.mp4
interesting_talk.mp3

The process takes a minute or so to complete, most of which is spent transcoding the video file to MP3. The resulting MP3s are usually 40 to 50 MB in size, which is very reasonable.

The outcome: How it looks in practice

Here’s what one of these “glanceable” podcasts looks like in Overcast and on my lock screen:

The Overcast now playing screen, where a significant chunk of the screen is a region for square album artwork, showing a letterboxed video frame. The left-hand of the frame shows the speaker; the right-hand shows their slide. The lock screen now playing controls, where the same artwork is visible in a floating pane above my wallpaper.
Maggie Appleton presented this talk over two years ago and it’s been on my “talks to watch” list ever since. Once I put it in Overcast? I listened to it in less than a day.

It’s not a lot of extra information, but enough that I can quickly glance down and get the gist of what a speaker is saying. Both views update with a new frame every few seconds, or I can put my phone in my pocket and ignore the screen.

I’ve used this approach for half a dozen videos so far, and I’m happy with the results. I expect to keep using it, because I have a long queue of videos I’ve been meaning to watch.

If you’d like to try this, check out glancecast for the full code and instructions.

[If the formatting of this post looks odd in your feed reader, visit the original article]