original image
2025-08-03 22:49:06
The standard Mac filesystem, APFS, has a feature called space-saving clones. This allows you to create multiple copies of a file without using additional disk space – the filesystem only stores a single copy of the data.
Although cloned files share data, they’re independent – you can edit one copy without affecting the other (unlike symlinks or hard links). APFS uses a technique called copy-on-write to store the data efficiently on disk – the cloned files continue to share any pieces they have in common.
Cloning files is both faster and uses less disk space than copying. If you’re working with large files – like photos, videos, or datasets – space-saving clones can be a big win.
Several filesystems support cloning, but in this post, I’m focusing on macOS and APFS.
For a recent project, I wanted to clone files using Python.
There’s an open ticket to support file cloning in the Python standard library.
In Python 3.14, there’s a new Path.copy() function which adds support for cloning on Linux – but there’s nothing yet for macOS.
In this post, I’ll show you two ways to clone files in APFS using Python.
Table of contents
There are two main benefits to using clones rather than copies.
Because the filesystem only has to keep one copy of the data, cloning a file doesn’t use more space on disk. We can see this with an experiment. Let’s start by creating a random file with 1GB of data, and checking our free disk size:
$ dd if=/dev/urandom of=1GB.bin bs=64M count=16
16+0 records in
16+0 records out
1073741824 bytes transferred in 2.113280 secs (508092550 bytes/sec)
$ df -h -I /
Filesystem Size Used Avail Capacity Mounted on
/dev/disk3s1s1 460Gi 14Gi 43Gi 25% /
My disk currently has 43GB available.
Let’s copy the file, and check the free disk space after it’s done. Notice that it decreases to 42GB, because the filesystem is now storing a second copy of this 1GB file:
$ # Copying
$ cp 1GB.bin copy.bin
$ df -h -I /
Filesystem Size Used Avail Capacity Mounted on
/dev/disk3s1s1 460Gi 14Gi 42Gi 25% /
Now let’s clone the file by passing the -c flag to cp.
Notice that the free disk space stays the same, because the filesystem is just keeping a single copy of the data between the original and the clone:
$ # Cloning
$ cp -c 1GB.bin clone.bin
$ df -h -I /
Filesystem Size Used Avail Capacity Mounted on
/dev/disk3s1s1 460Gi 14Gi 42Gi 25% /
When you clone a file, the filesystem only has to write a small amount of metadata about the new clone. When you copy a file,it needs to write all the bytes of the entire file. This means that cloning a file is much faster than copying, which we can see by timing the two approaches:
$ # Copying
$ time cp 1GB.bin copy.bin
Executed in 260.07 millis
$ # Cloning
$ time cp -c 1GB.bin clone.bin
Executed in 6.90 millis
This 43× difference is with my Mac’s internal SSD. In my experience, the speed difference is even more pronounced on slower disks, like external hard drives.
If you use the Duplicate command in Finder (File > Duplicate or ⌘D), it clones the file.
cp -c on the command lineIf you use the cp (copy) command with the -c flag, and it’s possible to clone the file, you get a clone rather than a copy.
If it’s not possible to clone the file – for example, if you’re on a non-APFS volume that doesn’t support cloning – you get a regular copy.
Here’s what that looks like:
$ cp -c src.txt dst.txt
clonefile() functionThere’s a macOS syscall clonefile() which creates space-saving clones.
It was introduced alongside APFS.
Syscalls are quite low level, and they’re how programs are meant to interact with the operating system.
I don’t think I’ve ever made a syscall directly – I’ve used wrappers like the Python os module, which make syscalls on my behalf, but I’ve never written my own code to call them.
Here’s a rudimentary C program that uses clonefile() to clone a file:
#include <stdio.h>
#include <stdlib.h>
#include <sys/clonefile.h>
int main(void) {
const char *src = "1GB.bin";
const char *dst = "clone.bin";
/* clonefile(2) supports several options related to symlinks and
* ownership information, but for this example we'll just use
* the default behaviour */
const int flags = 0;
if (clonefile(src, dst, flags) != 0) {
perror("clonefile failed");
return EXIT_FAILURE;
}
printf("clonefile succeeded: %s ~> %s\n", src, dst);
return EXIT_SUCCESS;
}
You can compile and run this program like so:
$ gcc clone.c
$ ./a.out
clonefile succeeded: 1GB.bin ~> clone.bin
$ ./a.out
clonefile failed: File exists
But I don’t use C in any of my projects – can I call this function from Python instead?
cp -c using subprocess
The easiest way to clone a file in Python is by shelling out to cp -c with the subprocess module.
Here’s a short example:
import subprocess
# Adding the `-c` flag means the file is cloned rather than copied,
# if possible. See the man page for `cp`.
subprocess.check_call(["cp", "-c", "1GB.bin", "clone.bin"])
I think this snippet is pretty simple, and a new reader could understand what it’s doing.
If they’re unfamiliar with file cloning on APFS, they might not immediately understand why this is different from shutil.copyfile, but they could work it out quickly.
This approach gets all the nice behaviour of the cp command – for example, if you try to clone on a volume that doesn’t support cloning, it falls back to a regular file copy instead.
There’s a bit of overhead from spawning an external process, but the overall impact is negligible (and easily offset by the speed increase of cloning).
The problem with this approach is that error handling gets harder.
The cp command fails with exit code 1 for every error, so you need to parse the stderr to distinguish different errors, or implement your own error handling.
In my project, I wrapped this cp call in a function which had some additional checks to spot common types of error, and throw them as more specific exceptions.
Any remaining errors get thrown as a generic subprocess.CalledProcessError.
Here’s an example:
from pathlib import Path
import subprocess
def clonefile(src: Path, dst: Path):
"""Clone a file on macOS by using the `cp` command."""
# Check a couple of common error cases so we can get nice exceptions,
# rather than relying on the `subprocess.CalledProcessError` from `cp`.
if not src.exists():
raise FileNotFoundError(src)
if not dst.parent.exists():
raise FileNotFoundError(dst.parent)
# Adding the `-c` flag means the file is cloned rather than copied,
# if possible. See the man page for `cp`.
subprocess.check_call(["cp", "-c", str(src), str(dst)])
assert dst.exists()
For me, this code strikes a nice balance between being readable and returning good errors.
clonefile() function using ctypes
What if we want detailed error codes, and we don’t want the overhead of spawning an external process?
Although I know it’s possible to make syscalls from Python using the ctypes library, I’ve never actually done it.
This is my chance to learn!
Following the documentation for ctypes, these are the steps:
Import ctypes and load a dynamic link library.
This is the first thing we need to do – in this case, we’re loading the macOS link library that contains the clonefile() function.
import ctypes
libSystem = ctypes.CDLL("libSystem.B.dylib")
I worked out that I need to load libSystem.B.dylib by looking at other examples of ctypes code on GitHub.
I couldn’t find an explanation of it in Apple’s documentation.
I later discovered that I can use otool to see the shared libraries that a compiled executable is linking to.
For example, I can see that cp is linking to the same libSystem.B.dylib:
$ otool -L /bin/cp
/bin/cp:
/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1351.0.0)
This CDLL() call only works on macOS, which makes sense – it’s loading macOS libraries.
If I run this code on my Debian web server, I get an error: OSError: libSystem.B.dylib: cannot open shared object file: No such file or directory.
Tell ctypes about the function signature.
If we look at the man page for clonefile(), we see the signature of the C function:
int clonefile(const char * src, const char * dst, int flags);
We need to tell ctypes to find this function inside libSystem.B.dylib, then describe the arguments and return type of the function:
clonefile = libSystem.clonefile
clonefile.argtypes = [ctypes.c_char_p, ctypes.c_char_p, ctypes.c_int]
clonefile.restype = ctypes.c_int
Although ctypes can call C functions if you don’t describe the signature, it’s a good practice and gives you some safety rails.
For example, now ctypes knows that the clonefile() function takes three arguments.
If I try to call the function with one or two arguments, I get a TypeError.
If I didn’t specify the signature, I could call it with any number of arguments, but it might behave in weird or unexpected ways.
Define the inputs for the function. This function needs three arguments.
In the original C function, src and dst are char* – pointers to a null-terminated string of char values.
In Python, this means the inputs need to be bytes values.
Then flags is a regular Python int.
# Source and destination files
src = b"1GB.bin"
dst = b"clone.bin"
# clonefile(2) supports several options related to symlinks and
# ownership information, but for this example we'll just use
# the default behaviour
flags = 0
Call the function. Now we have the function available in Python, and the inputs in C-compatible types, we can call the function:
import os
if clonefile(src, dst, flags) != 0:
errno = ctypes.get_errno()
raise OSError(errno, os.strerror(errno))
print(f"clonefile succeeded: {src} ~> {dst}")
If the clone succeeds, this program runs successfully. But if the clone fails, we get an unhelpful error: OSError: [Errno 0] Undefined error: 0.
The point of calling the C function is to get useful error codes, but we need to opt-in to receiving them.
In particular, we need to add the use_errno parameter to our CDLL call:
libSystem = ctypes.CDLL("libSystem.B.dylib", use_errno=True)
Now, when the clone fails, we get different errors depending on the type of failure.
The exception includes the numeric error code, and Python will throw named subclasses of OSError like FileNotFoundError, FileExistsError, or PermissionError.
This makes it easier to write try … except blocks for specific failures.
Here’s the complete script, which clones a single file:
import ctypes
import os
# Load the libSystem library
libSystem = ctypes.CDLL("libSystem.B.dylib", use_errno=True)
# Tell ctypes about the function signature
# int clonefile(const char * src, const char * dst, int flags);
clonefile = libSystem.clonefile
clonefile.argtypes = [ctypes.c_char_p, ctypes.c_char_p, ctypes.c_int]
clonefile.restype = ctypes.c_int
# Source and destination files
src = b"1GB.bin"
dst = b"clone.bin"
# clonefile(2) supports several options related to symlinks and
# ownership information, but for this example we'll just use
# the default behaviour
flags = 0
# Actually call the clonefile() function
if clonefile(src, dst, flags) != 0:
errno = ctypes.get_errno()
raise OSError(errno, os.strerror(errno))
print(f"clonefile succeeded: {src} ~> {dst}")
I wrote this code for my own learning, and it’s definitely not production-ready.
It works in the happy case and helped me understand ctypes, but if you actually wanted to use this, you’d want proper error handling and testing.
In particular, there are cases where you’d want to fall back to shutil.copyfile or similar if the clone fails – say if you’re on an older version of macOS, or you’re copying files on a volume which doesn’t support cloning.
Both those cases are handled by cp -c, but not the clonefile() syscall.
In my project, I used cp -c with a wrapper like the one described above.
It’s a short amount of code, pretty readable, and returns useful errors for common cases.
Calling clonefile() directly with ctypes might be slightly faster than shelling out to cp -c, but the difference is probably negligible.
The downside is that it’s more fragile and harder for other people to understand – it would have been the only part of the codebase that was using ctypes.
File cloning made a noticeable difference. The project involving copying lots of files on an external USB hard drive, and cloning instead of copying full files made it much faster. Tasks that used to take over an hour were now completing in less than a minute. (The files were copied between folders on the same drive – cloned files have to be on the same APFS volume.)
I’m excited to see how file cloning works on Linux in Python 3.14 with Path.copy(), and I hope macOS support isn’t far behind.
[If the formatting of this post looks odd in your feed reader, visit the original article]
2025-07-30 17:26:41
Two weeks ago, I was at Queer Craft – a fortnightly meet-up at Slipstitch, a haberdashery and yarn shop near Alexandra Palace. I was working on a cross stitch piece of a train in a snowy landscape, chatting to my friends, admiring their creations, and gently snacking on a variety of baked goods.
This week, I wasn’t there, because Slipstitch closed its doors on Saturday.
I can’t remember exactly when I first came across Slipstitch, but I remember why. Slipstitch was sometimes the target of stickering campaigns from TERFs and anti-trans campaigners, who objected to Rosie’s vocal support of trans people. (Rosie is the shop’s owner and a dear friend.)
This discussion spilled onto Twitter, and at some point Rosie’s tweets appeared in my timeline. I saw the shop account calling out the stickers and re-affirming its support of trans people, and since I’m trans and I do lots of cross-stitch, I decided to check it out. I looked around the online store, thinking I might buy some thread – then I found an event called “Queer Craft”, and booked.
Turning up the next Monday, I was a bit nervous – would I be queer enough? Would I be crafty enough? For whatever reason, my mental image of a craft meetup is people doing knitting or crochet – does anybody bring cross-stitch to these things?
My nerves were quickly put at ease – Rosie welcomed me enthusiastically, and I settled in. I sat down at the table, put on a name badge, and took out my cross-stitch. As I stitched away, I started chatting to strangers who would soon become friends.
Queer Craft was every other Monday, and I began making regular trips to Muswell Hill for two hours of crafting and conversation. We’d admire each other’s work, and share tips and advice if somebody was struggling. The group was always generous with knowledge, equipment, and sympathy for unexpected snags – but the conversation often drifted away from our crafts.
As we got to know each other more, we developed in-jokes and stories and familiar topics. We’d talk about Taskmaster and the careers of cutlery salesmen. We’d discuss what it’s like to grow up in Cornwall. We’d chat about theatre shows and West End drama. Everyone made jokes about how I’m a spy. (I’m not a spy.) Rosie would tell us about the wrong way to make coffee. We passed around many, many photos of our pets.
I know that Rosie was always keen for Queer Craft to be welcoming to newcomers and not too “clique”-y – I suspect the rate of in-jokes made that difficult, but I admire the effort. (I wonder if that’s the fate of all groups designed to help strangers meet? Either it fizzles out, or a group of regulars gradually forms that makes it harder for new people to join.) I confess I was never too worried about this, because I was too busy having a nice time with my friends.
Craft is such a personal hobby, and we saw glimpses of each other’s lives through the things we were making, especially when they were gifts for somebody else. Somebody making a baby blanket for a friend, or a stuffed toy for a parent, or some socks for a partner. Everyone poured so much time and love into their work. It felt very intimate and special.
I’m always a bit nervous about how visibly trans to be, but that was never an issue at Queer Craft – something I should have known from the start, thanks to Rosie’s vocal support of her trans staff and customers. Everyone treated transness as so self-evidently normal, it didn’t bear comment.
Sometimes I was femme, and sometimes I was masc, and nobody batted an eyelid. (Apart from the time Rosie took one look at my officewear and described it as “corporate drag”, a burn so savage I may never recover.) That sense of casual, unconditional trans acceptance feels like it’s getting more common – but it’s still nice every time.
These friendships have spilled out of Slipstitch and into the world beyond. One Queer Craft regular is a published author, and some of us went to celebrate her at a book event. Several others are in choirs, and I’ve been to see them sing. Last year, I invited people round to my house for “Queer Crafternoon”, where we all crafted in my living room and ate scones with jam and cream.
Nine weeks ago, Rosie told us that she’d decided to close the physical shop. The world is tough for small businesses, and tougher for the person running them. I’m sad, but I could see how much stress it was putting on Rosie, and I respect her decision to put her own wellbeing first, and to close the doors on her own terms.
On Saturday, Rosie held a party to mark the closing of the space. The shelves were empty, the room anything but. There were Queer Craft friends, regulars from the other Meet & Make group, people who’d come to Rosie’s classes, regular customers, and other friends of the shop. The community around Slipstitch is much more than just the queers who met on Monday evenings. The shop was the busiest I’ve ever seen it, and it was lovely to see so many people there to celebrate and mourn.
I’m sure Queer Craft and our friendships will continue in some shape or form, but sitting here now, I can’t help but be a little upset about what we’ve lost. Of course, there are other haberdasheries, and there are other queer craft groups – it’s hardly a unique idea – but Slipstitch was the haberdashery where I shopped, it’s where our group met, and I’m sad to know it’s gone.
In her final newsletter before the closure, Rosie wrote “[Slipstitch] never wanted for community”. I think that’s a lovely sentiment, and one that rung true in my experience – it always felt like such a friendly, welcoming space, and I’m glad I found it. I hope the friendships forged in Queer Craft will survive a long time after the physical shop is gone. I know that Rosie wants Slipstitch to continue as an idea, if not a physical venue, and I’m excited to see what happens next.
Her words also made me reflect on the fragility of our community spaces – those places where we can meet strangers and bond over a common interest. They’re getting scarcer and scarcer. As every bit of land is forced into more and more commercialisation, we’re running out of places to just hang out and meet people. We often talk about how hard it is to make friends as an adult – and that’s in part because the spaces where we might do so are dwindling.
These community spaces are precious for queer people, yes, but for everyone else too. I’m sad that the shop has closed, and I’m sad that this iteration of Queer Craft is over, and I’m sad that this is a trend. These spaces are rare, and getting rarer – we shouldn’t take them for granted.
[If the formatting of this post looks odd in your feed reader, visit the original article]
2025-07-26 04:12:14
Today was my last day at the Flickr Foundation. At 5pm I closed my laptop, left the office for the last time, and took a quiet walk along Regent’s Canal. I saw an adorable family of baby coots, and a teenage coot who was still a bit fluffy and raggedy around the edges.
I’ve got another job lined up, but I’m taking a short break before I start.
My new role is still in software engineering, but in a completely different field. I’m stepping away from the world of libraries, archives, and photography. I’ve met some amazing people, and I’m very proud of everything we accomplished in digital preservation and cultural heritage. I’ll always treasure those memories, but I’m also excited to try something new.
For the last few years, I’ve been among the more senior engineers in my team. In my next role, I’ll be firmly middle of the pack, and I’m looking forward to learning from people who have more wisdom and experience than me.
But first: rest.
[If the formatting of this post looks odd in your feed reader, visit the original article]
2025-07-25 05:59:10
I minify all the HTML on this website – removing unnecessary whitespace, tidying up attributes, optimising HTML entities, and so on. This makes each page smaller, and theoretically the website should be slightly faster.
I’m not going to pretend this step is justified by the numbers. My pages are already pretty small pre-minification, and it only reduces the average page size by about 4%. In June, minification probably saved less than MiB of bandwidth.
But I do it anyway. I minify HTML because I like tinkering with the website, and I enjoy finding ways to make it that little bit faster or more efficient. I recently changed the way I’m minifying HTML, and I thought this would be a good time to compare the three approaches I’ve used and share a few things I learned about HTML along the way.
I build this website using Jekyll, so I’ve looked for Jekyll or Ruby-based solutions.
Table of contents
This is a Jekyll layout that compresses HTML. It’s a single HTML file written in pure Liquid (the templating language used by Jekyll).
First you save the HTML file to _layouts/compress.html, then reference it in your highest-level layout.
For example, in _layouts/default.html you might write:
---
layout: compress
---
<html>
{{ content }}
</html>
Because it’s a single HTML file, it’s easy to install and doesn’t require any plugins. This is useful if you’re running in an environment where plugins are restricted or disallowed (which I think includes GitHub Pages, although I’m not 100% sure).
The downside is that the single HTML file can be tricky to debug, it only minifies HTML (not CSS or JavaScript), and there’s no easy way to cache the output.
The htmlcompressor gem is a Ruby port of Google’s Java-based HtmlCompressor. The README describes it as an “alpha version”, but in my usage it was very stable and it has a simple API.
I start by changing my compress.html layout to pass the page content to a compress_html filter:
---
---
{{ content | compress_html }}
This filter is defined as a custom plugin; I save the following code in _plugins/compress_html.rb:
def run_compress_html(html)
require 'htmlcompressor'
options = {
remove_intertag_spaces: true
}
compressor = HtmlCompressor::Compressor.new(options)
compressor.compress(html)
end
module Jekyll
module CompressHtmlFilter
def compress_html(html)
cache = Jekyll::Cache.new('CompressHtml')
cache.getset(html) do
run_compress_html(html)
end
end
end
end
Liquid::Template.register_filter(Jekyll::CompressHtmlFilter)
I mostly stick with the default options; the only extra rule I enabled was to remove inter-tag spaces. Consider the following example:
<p>hello world</p> <p>my name is Alex</p>
By default, htmlcompressor will leave the space between the closing </p> and the opening <p> as-is.
Enabling remove_intertag_spaces makes it a bit more aggressive, and it removes that space.
I’m using the Jekyll cache to save the results of the compression – most pages don’t change from build-to-build, and it’s faster to cache the results than recompress the HTML each time.
The gem seems abandoned – the last push to GitHub was in 2017.
This is a Rust-based HTML minifier, with bindings for a variety of languages, including Ruby, Python, and Node. It’s very fast, and even more aggressive than other minifiers.
I use it in a very similar way to htmlcompressor.
I call the same compress_html filter in _layouts/compress.html, and then my run_compress_html in _plugins/compress_html.rb is a bit different:
def run_compress_html(html)
require 'minify_html'
options = {
keep_html_and_head_opening_tags: true,
keep_closing_tags: true,
minify_css: true,
minify_js: true
}
minify_html(html, options)
end
This is a much more aggressive minifier.
For example, it turns out that the <html> and <head> elements are optional in an HTML5 document, so this minifier removes them if it can.
I’ve disabled this behaviour, because I’m old-fashioned and I like my pages to have <html> and <head> tags.
This library also allows minifying inline CSS and JavaScript, which is a nice bonus. That has some rough edges though: there’s an open issue with JS minification, and I had to tweak several of my if-else statements to work with the minifier. Activity on the GitHub repository is sporadic, so I don’t know if that will get fixed any time soon.
After I minify HTML, but before I publish the site, I run HTML-Proofer to validate my HTML.
I’m not sure this has ever caught an issue introduced by a minifer, but it gives me peace of mind that these tools aren’t mangling my HTML. (It has caught plenty of issues caused by my mistakes!)
There are two key metrics for HTML minifiers:
Speed: this is a dead heat.
When I built the site with a warm cache, it takes about 2.5s whatever minifier I’m using.
The htmlcompressor gem and minify-html library are much slower if I have a cold cache, but that’s only a few extra seconds and it’s rare for me to build the site that way.
File size: the Ruby and Rust-based minifiers achieve slightly better minification, because they’re more aggressive in what they trim. For example, they’re smarter about removing unnecessary spaces and quoting around attribute values.
Here’s the average page size after minification:
| Approach | Average HTML page size |
|---|---|
| Without minification | 14.9 KiB |
| Compress HTML in Jekyll 3.2.0 | 14.3 KiB |
| htmlcompressor 0.4.0 | 14.0 KiB |
| minify-html 0.16.4 | 13.5 KiB |
I’m currently using minify-html. This is partly because it gets slightly smaller page sizes, and partly because it has bindings in other languages. This website is my only major project that uses Ruby, and so I’m always keen to find things I can share in my other non-Ruby projects. If minify-html works for me (and it is so far), I can imagine using it elsewhere.
[If the formatting of this post looks odd in your feed reader, visit the original article]
2025-07-08 23:51:22
About six weeks ago, Glitch announced that they’re shutting down. Glitch was a platform where you could make websites and web apps, with a heavy emphasis on creativity and sharing. You could read the source code for any project to understand how it worked, and remix somebody else’s project to create your own thing.
Unfortunately, Glitch is shutting down project hosting today. If you had an app on Glitch, it’s about to stop running, but you can set up redirects to another copy of it running elsewhere.
I’ve created redirects for all of my apps, and moved them to the web server that runs this site. This was pretty straightforward, because all of my “apps” were static websites that I can upload to my server, and they get served like the rest of my site:
Checkbox text adventure is a silly game where you can click, lick, and smell a checkbox. I made it with Twine as a response to Nolen Royalty (eieio)’s game One Million Checkboxes.
How long is my data? is an app that measures your data in the world’s most practical unit: the length of shelving you’d need to store it on 3½″ floppy disks.
Looped squares lets you draw looped shapes based on the Mac’s command ⌘ icon.
Marquee rocket is a perverse experiment in trying to use the <marquee> tag to make an interactive web app.
Rainbow hearts and Rainbow valknuts are two Pride-themed apps that draw geometric shapes based on different flags.
UK stations map lets you plot a map of all the railway stations you’ve visited in the UK.
Not all of my Glitch apps made the jump – I deleted a couple of very early-stage experiments, and I have yet to spin up new copies of my Chinese vocabulary graph or the dominant colours web app. I might port them later, but they’re not static websites so they’re a bit more complicated to move.
Glitch felt like a throwback to the spirit of the early web – the platonic ideal of “view source” and “anyone can make a website”. I always liked the idea of Glitch, and I enjoyed making the fun apps that I hosted there. I’m sad to see it close – another space for playful creativity crushed by the commercial tide of the web.
[If the formatting of this post looks odd in your feed reader, visit the original article]
2025-06-11 21:52:05
Last week, the Swift.org website got a redesign. I don’t write much Swift at the moment, but I glanced at the new website to see what’s up and OOH COOL BIRD!
When you load the page, there’s a swooping animation as the bird appears:
I was curious how the animation worked. I thought maybe it was an autoplaying video with no controls, but no, it’s much cooler than that! The animation is implemented entirely in code – there are a few image assets, and then the motion uses JavaScript and the HTML5 canvas element.
I’ve never done anything with animation, so I started reading the code to understand how it works. I’m not going to walk through it in detail, but I do want to show you what I learnt.
All the code from the Swift.org website is open source on GitHub, and the JavaScript file that implements this animation was written by three engineers: Federico Bucchi, Jesse Borden, and Nicholas Krambousanos.
Table of contents
- What are the key steps in this animation?
- Only draw part of a curved path with a dash pattern
- Mask an image with a
globalCompositeOperation- Animate the brush stroke with Anime.js
- Start the animation with a
MutationObserver- Skip the animation if you have
(prefers-reduced-motion: reduce)- Closing thoughts
Most of the animation is made up of five “swoop” images, which look like strokes of a paintbrush. These were clearly made by an artist in a design app like Photoshop.
These images are gradually revealed, so it looks like somebody actually painting with a brush. This is more complex than a simple horizontal wipe, the sort of animation you might do in PowerPoint. Notice how, for example, the purple swoop doubles back on itself – if you did a simple left-to-right wipe, it would start as two separate swoops before joining into one. It would look very strange!
Each swoop is animated in the same way, so let’s focus on the purple one, just because it’s the most visually interesting.
The animation is applying a mask to the underlying image, and the mask gradually expands to show more and more of the image. The mask matches the general shape of the brush stroke, so as it expands, it reveals more of the image. I wrote about masking with SVG four years ago, and the principle is similar here – but the Swift.org animation uses HTML5 canvas, not SVG.
The best way to explain this is with a quick demo: as you drag the slider back and forth, you can see the mask get longer and shorter, and that’s reflected in the final image.
original image+mask→final imageanimation progress:
We can break this down into a couple of steps:
Let’s go through each of these in turn.
Alongside the graphical image of a brush stroke, the artist supplied an SVG path for the mask:
M-34 860C-34 860 42 912 102 854C162 796 98 658 50 556C2 454 18 48 142 88C272 130 290 678 432 682C574 686 434 102 794 90C1009 83 1028 280 1028 280
If you’re not familiar with SVG path syntax, I really recommend Mathieu Dutour’s excellent SVG Path Visualizer tool. You give it a path definition, and it gives you a step-by-step explanation of what it’s doing, and you can see where each part of the path appears in the final shape.
We can draw this path on an HTML5 canvas like so:
const canvas = document.querySelector('canvas');
const ctx = canvas.getContext('2d');
ctx.lineWidth = 100;
ctx.lineCap = 'round';
ctx.strokeStyle = 'black';
const path = new Path2D(
"M-34 860C-34 860 42 912 102 854C162 796 98 658 50 556C2 454 18 48 142 88C272 130 290 678 432 682C574 686 434 102 794 90C1009 83 1028 280 1028 280"
);
ctx.stroke(path);
The way Swift.org draws a partial path is a really neat trick: they’re using a line dash pattern with a variable offset. It took me a moment to figure out what their code was doing, but then it all clicked into place.
First they set a line dash pattern using setLineDash(), which specifies alternating lengths of lines and gaps to draw the line.
Here’s a quick demo:
ctx.setLineDash([100])
The path starts in the lower left-hand corner, and notice how it always starts with a complete dash, not a gap.
You can change this by setting the lineDashOffset property, which causes the patern to start on a gap, or halfway through a dash.
Here’s a demo where you can set both variables at once:
ctx.setLineDash([75])ctx.lineDashOffset = 0;
I find the behaviour of lineDashOffset a bit counter-intuitive: as I increase the offset, it looks like the path is moving backward.
I was expecting increasing the offset to increase the start of the first dash, so the line would move in the other direction.
I’m sure it makes sense if you have the right mental model, but I’m not sure what it is.
If you play around with these two variables, you might start to see how you can animate the path as if it’s being drawn from the start. Here are the steps:
Set the dash length to the exact length of the path. This means every dash and every gap is the same length as the entire path.
(The length of the purple swoop path is 2776, a number I got from the Swift.org source code. This must have been calculated with an external tool; I can’t find a way to calculate this length in a canvas.)
Set the dash offset to the exact length of the path. This means the entire path is just a gap, which makes it look like there’s nothing there.
Gradually reduce the dash offset to zero. A dash becomes visible at the beginning of the path, and the closer the offset gets to zero, the more of that dash is visible. Eventually it fills the entire path.
Here’s one more demo, where I’ve set up the line dash pattern, and you can adjust the progress. Notice how the line gradually appears:
const progress = 0.0;const pathLength = 2776 ctx.setLineDash([pathLength]); ctx.lineDashOffset = pathLength * (1 - progress);
Now we have a way to draw part of a path, and as we advance the progress, it looks it’s being drawn with a brush. The real code has a couple of extra styles – in particular, it sets a stroke width and a line cap – but it’s the way the animation uses the dash pattern that really stood out to me.
Once we have our path, how do we use it to mask an image?
globalCompositeOperation
The masking uses a property of HTML5 canvas called globalCompositeOperation.
If you’ve already drawn some shapes on a canvas, you can control how new shapes will appear on top of them – for example, which one appears on top, or whether to clip one to fit inside the other.
I’m familiar with the basic idea – I wrote an article about clips and masks in SVG in 2021 that I still look back on fondly – but I find this feature a bit confusing, especially the terminology. Rather than talking about clips or masks, this property is defined using sources (shapes you’re about to draw on the canvas) and destinations (shapes that are already on the canvas). I’m sure that naming makes sense to somebody, but it’s not immediately obvious to me.
First we need to load the bitmap image which will be our “source”.
We can create a new img element with document.createElement("img"), then load the image by setting the src attribute:
const img = document.createElement("img");
img.src = url;
In the Swift.org animation, the value of globalCompositeOperation is source-in – the new shape is only drawn where the new shape and the old shape overlap, and the old shape becomes transparent.
Here’s the code:
// The thick black stroke is the "destination"
ctx.stroke(path)
// The "source-in" mode means only the part of the source that is
// inside the destination will be shown, and the destination will
// be transparent.
ctx.globalCompositeOperation = 'source-in'
// The bitmap image is the "source"
ctx.drawImage(img, 0, 0)
and here’s what the result looks like, when the animation is halfway complete:
destination+source→final image
There are many different composite operations, including ones that combine colours or blend pixels from both shapes. If you’re interested, you can read the docs on MDN, which includes a demo of all the different blending modes.
This is a bit of code where I can definitely understand what it does when I read it, but I wouldn’t feel confident writing something like this myself.
It’s too complex a feature to wrap my head around with a single example, and the other examples I found are too simple and unmotivating.
(Many sites use the example of a solid red circle and a solid blue rectangle, which I find completely unhelpful because I can produce the final result in a dozen other ways.
What’s the real use case for this property?
What can I only do if I use globalCompositeOperation?)
Then again, perhaps I’m not the target audience for this feature. I mostly do simple illustrations, and this is a more powerful graphics operation. I’m glad to know it’s there, even if I’m not sure when I’ll use it.
Now we can draw a partial stroke and use it as a mask, how do we animate it?
Before I started reading the code in detail, I tried to work out how I might create an animation like this myself.
I haven’t done much animation, so the only thing I could think of was JavaScript’s setTimeout() and setInterval() functions.
Using those repeatedly to update a progress value would gradually draw the stroke.
I tried it, and that does work!
But I can think of some good reasons why it’s not what’s used for the animation on Swift.org.
The timing of setTimeout() and setInterval() isn’t guaranteed – the browser may delay longer than expected if the system is under load or you’re updating too often.
That could make the animation jerky or stuttery.
Even if the delays fire correctly, it could still look a bit janky – you’re stepping between a series of discrete frames, rather than smoothly animating a shape.
If there’s too much of a change between each frame, it would ruin the illusion.
Swift.org is using Julian Garnier’s Anime.js animation library.
Under the hood, this library uses web technologies like requestAnimationFrame() and hardware acceleration – stuff I’ve heard of, but never used.
I assume these browser features are optimised for doing smooth and efficient animations – for example, they must sync to the screen refresh rate, only drawing frames as necessary, whereas using setInterval() might draw lots of unused frames and waste CPU.
Anime.js has a lot of different options, but the way Swift.org uses it is fairly straightforward.
First it creates an object to track the state of the animation:
const state = { progress: 0 };
Then there’s a function that redraws the swoop based on the current progress. It clears the canvas, then redraws the partial path and the mask:
function updateSwoop() {
// Clear canvas before next draw
ctx.clearRect(0, 0, canvas.width, canvas.height);
// Draw the part of the stroke that we want to display
// at this point in the animation
ctx.lineDashOffset = swoop.pathLength * (1 - state.progress);
ctx.stroke(new Path2D(swoop.path));
// Draw the image, using "source-in" to apply a mask
ctx.globalCompositeOperation = 'source-in'
ctx.drawImage(img, 0, 0);
// Reset to default for our next stroke paint
ctx.globalCompositeOperation = 'source-out';
}
Finally, it creates a timeline, and adds an animation for each swoop.
When it adds the animation, it passes five things:
state objectprogress: 1)in(1.8) means the animation will start slowly and gradually speed upupdateSwoop function as a callback for every time the animation updatesconst tl = anime.createTimeline()
tl.add(
state,
{ progress: 1, duration: 1000, ease: 'in(1.8)', onUpdate: updateSwoop }
);
You may have wondered why the state is an object, and not a single value like const progress = 0.
If we passed a numeric value to tl.add(), JavaScript would pass it by value, and any changes wouldn’t be visible to the updateSwoop() function.
By wrapping the progress value in an object, JavaScript will pass by reference instead, so changes made inside tl.add() will be visible when updateSwoop() is called.
Now we can animate our swoop, as if it was a brush stroke. There’s one final piece: how do we start the animation?
MutationObserver
If I want to do something when a page loads, I normally watch for the DOMContentLoaded event, for example:
window.addEventListener("DOMContentLoaded", () => {
runAnimation();
});
But the Swift.org animation has one more thing to teach me, because it does something different.
In the HTML, it has a <div> that wraps the canvas elements where it draws all the animations:
<div class="animation-container">
<canvas id="purple-swoop" width="1248" height="1116"></canvas> <canvas id="purple-swoop" width="1248" height="1116"></canvas>
<canvas id="white-swoop-1" width="1248" height="1116"></canvas>
<canvas id="orange-swoop-top" width="1248" height="1116"></canvas>
<canvas id="orange-swoop-bottom" width="1248" height="1116"></canvas>
<canvas id="white-swoop-2" width="1248" height="1116"></canvas>
<canvas id="bird" width="1248" height="1116"></canvas>
</div>
Then it uses a MutationObserver to watch the entire page for changes, and start the animation once it finds this wrapper <div>:
// Start animation when container is mounted
const observer = new MutationObserver(() => {
const animContainer = document.querySelector('.animation-container')
if (animContainer) {
observer.disconnect()
heroAnimation(animContainer)
}
})
observer.observe(document.documentElement, {
childList: true,
subtree: true,
})
It achieves the same effect as watching for DOMContentLoaded, but in a different way.
I don’t think there’s much difference between DOMContentLoaded and MutationObserver in this particular case, but I can see that MutationObserver is more flexible for the general case.
You can target a more precise element than “the entire document”, and you can look for changes beyond just the initial load.
I suspect the MutationObserver approach may also be slightly faster – I added a bit of console logging, and if you don’t disconnect the observer, it gets called three times when loading the Swift.org homepage.
If the animation container exists on the first call, you can start the animation immediately, rather than waiting for the rest of the DOM to load.
I’m not sure if that’s a perceptible difference though, except for very large and complex web pages.
This step completes the animation. When the page loads, we can start an animation that draws the brush stroke as a path. As the animation continues, we draw more and more of that path, and the path is used as a mask for a bitmap image, gradually unveiling the purple swoop.
(prefers-reduced-motion: reduce)
There’s one other aspect of the animation on Swift.org that I want to highlight. At the beginning of the animation sequence, it checks to see if you have the “prefers reduced motion” preference. This is an accessibility setting that allows somebody to minimise non-essential animations.
const isReduceMotionEnabled = window.matchMedia(
'(prefers-reduced-motion: reduce)',
).matches
Further down, the code checks for this preference, and if it’s set, it skips the animation and just renders the final image.
I’m already familiar with this preference and I use it on a number of websites. sites, but it’s still cool to see.
Thanks again to the three people who wrote this animation code: Federico Bucchi, Jesse Borden, and Nicholas Krambousanos. They wrote some very readable JavaScript, so I could understand how it worked. The ability to “view source” and see how a page works is an amazing feature of the web, and finding the commit history as open source is the cherry on the cake.
I really enjoyed writing this post, and getting to understand how this animation works. I don’t know that I could create something similar – in particular, I don’t have the graphics skills to create the bitmap images of brush strokes – but I’d feel a lot more confident trying than I would before. I’ve learned a lot from reading this code, and I hope you’ve learned something as well.
[If the formatting of this post looks odd in your feed reader, visit the original article]