Daniel LemireModify

2026-10-16 06:46:54

Redis is an in-memory database. Salvatore Sanfilippo started it in 2009, and it is now one of the most widely deployed pieces of server software. It is often used as a cache in front of a slower database.

You talk to it over the network with short text commands such as SET user:42 "Alice" and GET user:42. Each key maps to a value, and the value can be more than a string: Redis has lists, hashes, sets, sorted sets, streams and more, each with its own commands. 

We often represent sets of integers as bitmaps (or bitsets). The simplest bimtap is a single machine word of, say, 8 bits. It can be used to represent all sets of integers smaller than 8. By adding more words, you can represent sets of integers smaller than 16, or any value you like. You can then do fast operations over bitmaps such as intersections and unions. They translate into a few bit-wise operations such as AND, OR, XOR. 

Redis has supported bitmaps for a long time. You call SETBIT key offset 1, and Redis stores a plain bitmap inside an ordinary Redis string: bit i of the bitmap is bit i of the string. It is simple and fast. But the string takes one byte per eight bit positions, up to the largest offset you ever set, however many bits are actually set. Setting a single bit at offset four billion allocates 512 MB, and every BITCOUNT or BITOP has to scan all of it.

A compressed bitmap avoids this cost: it stores a set of integers in a form that takes little room when the set is sparse, and it computes counts, intersections and unions without decompressing. Roaring is probably the most widely used compressed bitmap format. Rather than compressing a long string of bits, Roaring adapts its storage to the data: sparse regions are stored as short lists of integers, dense regions as plain bitsets, and long stretches of consecutive values as runs. It is often both smaller and faster than an uncompressed bitset. Roaring is used by many systems you already know: Elasticsearch, Apache Spark, ClickHouse …

Antonio Viggiano has brought Roaring into Redis itself. It should ship in Redis 8.12. There is a new type: TYPE key returns bitmap and OBJECT ENCODING key returns bitmap-roaring. It is opt-in: CONFIG SET bitmap-default-roaring yes.

I built Redis at the merge commit and ran the server on one core of an Intel Xeon Gold 6548N. For each workload, I create two bitmaps, a and b, once as strings and once as Roaring bitmaps, with the same content, and I time the commands. My workloads:

• 0.1% random: 100,000 random ids out of 100 million (think: the users who used a given feature today);
• 50% random: 50 million random ids out of 100 million;
• runs: ten runs of one million consecutive ids in a range of 100 million (think: cohorts of users who signed up in the same week);
• hashed 32-bit ids: 100,000 random 32-bit values, so the string is 512 MB.

You can see that Roaring can be much faster than a standard uncompressed bitmap in some cases. The hashed ids are even more compelling. The string must span all four billion bit positions, and each BITOP scans 512 MB. 

Compressed bitmaps like Roaring are not always better. There are many tradeoffs. But turning them on will soon take one configuration setting. Check out the Roaring bitmap website for more information.

2026-10-12 21:06:39

Python used to have limited parallelism, due to its global interpreter lock
(GIL). Only one thread could run Python code at a time. Python 3.14 comes in a
free-threaded version (python3.14t), without the global interpreter lock.
Several threads can now run Python code at the same time.

Let us pick a realistic problem. GitHub publishes everything that happens on
its public repositories: pushes, issues, pull requests, stars, and so forth.
The GH Archive project collects these events
as JSON, one event per line (NDJSON), one file per hour. I took eight hours of
March 13, 2024: 7.6 GB of JSON once decompressed, 2.2 million events. I want
to know which programming languages get the most new pull requests. So I
count the events by type, and for each pull request that was opened, I look
up the language of the repository. Each event is a large JSON
object, about 3.4 kB on average: I need only a few of its fields.

The answer, for these eight hours: 65,269 pull requests were opened, mostly
toward TypeScript (11,565), JavaScript (7,778), Python (7,057) and Java
(4,668) repositories. About 5,857 targeted repositories without a language.

With fastsimdjson, our Python
binding for simdjson, the code looks as follows. The function map_many
calls my function on each event, using as many threads as I ask for.

import collections, threading
import fastsimdjson

local = threading.local()
parts = []

def one_event(e):
    acc = getattr(local, "acc", None)
    if acc is None:  # first event seen by this thread
        acc = local.acc = (collections.Counter(), collections.Counter())
        parts.append(acc)
    t = e["type"]
    acc[0][t] += 1
    if t == "PullRequestEvent":
        p = e["payload"]
        if p["action"] == "opened":
            acc[1][p["pull_request"]["base"]["repo"]["language"]] += 1

data = open("events.json", "rb").read()
fastsimdjson.map_many(one_event, data, format="lines", threads=64, lazy=True)

types, languages = collections.Counter(), collections.Counter()
for t, lang in parts:
    types.update(t)
    languages.update(lang)

Each thread keeps its own counters. The first event a thread sees creates
the pair and appends it to parts. Later events on that thread update the
same pair. Nothing in the counters is shared, so the threads do not wait
for each other. map_many returns only once every event has been counted.
The main thread then walks parts, one pair per thread, and
Counter.update adds the counts. A few dozen event types and about a
hundred languages: the merge is tiny next to the parsing.

With lazy=True, my function receives a lazy view of the event instead of a
dictionary: the event is parsed, but Python objects are created
only for the values I access. The lazy approach was first developed by
Tyler Kennedy in another simdjson binding,
pysimdjson.

How does it compare? I tried the usual suspects: the standard library
(json), orjson and
msgspec. msgspec deserves special
mention: it can decode JSON into typed structures that declare only the fields
you need, skipping the rest. It is a lot like our lazy views, except that you
must declare a schema beforehand. I gave msgspec this advantage. In each case,
I verify that we get exactly the same counts.

I use a server with two Intel Xeon Gold 6548N processors (64 cores),
Python 3.14.6, and the current versions of the libraries (fastsimdjson 0.6.0,
orjson 3.13.0, msgspec 0.22.0). The data is decompressed in memory
before the timer starts. I report the best of seven runs, and one run
for the slow methods: the single-thread parsers, and the standard
library on 64 threads. Let us first use a single core.

The standard library needs 42 seconds to go through the eight hours. With
fastsimdjson and lazy views, it takes 3.6 seconds. Notice how the methods
that materialize everything (json, orjson and fastsimdjson without lazy views)
are slow: most of the time goes into creating Python objects that we
discard right away.

Now let us use all 64 cores, still in one process. On free-threaded Python,
I use a thread pool for json and msgspec, and map_many(..., threads=64)
for fastsimdjson. The threads share the input.

We go through 7.6 GB of JSON in about 0.12 seconds, at 66 GB/s, in one
Python process. That is 31 times faster than with one core. It is faster
than msgspec using 64 threads (25 GB/s).

So, can you parse JSON at tens of gigabytes per second in Python? Yes.

fastsimdjson 0.6.0 is available on PyPI.
My benchmark is available.

2026-10-10 16:56:11

WebSocket is a standard protocol to hook up a client and a server in a persistent manner. It is commonly used for live chat, notifications, multiplayer games, collaborative editing, and streaming dashboards. Many servers are written in JavaScript and use Node.js, Bun or Deno as an engine.

A few years ago, I wrote a simple WebSocket benchmark in JavaScript. Two clients exchange messages through a server: client 1 sends a message to the server, the server forwards it to client 2, client 2 answers, and the server forwards the answer back to client 1. I found that Bun could be twice as fast as Node.js. That result was wrong. The Bun server sent each message back to the sender, so a round trip was two messages instead of four. Since then, Anthropic bought Bun. Also, Cloudflare bought Deno.

My benchmark had an issue: my clients were written in JavaScript, and they were slow. So I wrote a client in C. Doing so by hand in 2023 was expensive, but in 2026, my AI friends can write anything in C.

I run a single ping-pong test: a single pair of clients chats through the server.

I compare three servers against a small C relay:

• Node.js 26 with uWebSockets.js, a native addon written in C++;
• Bun 1.4.2 with its built-in server (Bun.serve);
• Deno 2.9.7 with its built-in server (Deno.serve).

My test machine is a Linux server with an Intel Xeon Gold 6548N (Emerald Rapids). Everything runs locally over the loopback interface. The server is restricted to a single core, and the client runs on another core. I repeated the whole benchmark three times: the results are stable, within a few percent.

I report the number of round trips per second and the time per round trip (median and 99th percentile).

A JavaScript WebSocket server can get close to a simple C server in this test. Node.js 26 with uWebSockets.js is quite a bit faster than Bun and Deno: 49,000 round trips per second, against 40,000 for Deno and 37,000 for Bun. The 99th percentile is 24.8 µs, against 32.0 µs and 46.4 µs.

I repeated the same ping-pong test with the Javascript ws package, version 8.22.0, on all three runtimes. 

Node.js is ahead, then Bun, then Deno. On Node.js, ws is close to uWebSockets.js (45,000 against 49,000). On Bun, ws beats Bun.serve (41,000 against 37,000). On Deno, ws is behind Deno.serve (34,000 against 40,000).

My source code is available.

2026-10-09 11:15:06

When a Python program starts, it needs to load all its dependencies (import). With the upcoming version of Python, it is possible to use lazy imports instead.

lazy import json
lazy from decimal import Decimal

In this instance, the name json is no longer the module, but a mere placeholder object. The actual import only happens when you use json. If you never use json, the module is never loaded.

How much does it help? Let us measure.

I wrote a toy command-line tool. It imports sixteen modules: json, csv, decimal, sqlite3, asyncio, email.parser, http.client, urllib.request, xml.etree.ElementTree, zipfile, tarfile, statistics and some popular third-party packages (numpy, pandas, requests, rich). The tool has three paths:

• --version prints a string and needs nothing;
• mean reads a small CSV file with the csv module and calls statistics.fmean;
• stats loads the same file with pandas and prints a summary.

The lazy version of the tool is identical, except that I prefix the imports with lazy. It is a one-word change per line.

I use Python 3.15 (3.15.0b4) on an Intel Xeon Gold 6548N (Emerald Rapids), with numpy 2.5, pandas 3.0 and requests 2.34.

Printing the version number goes from 295 ms to 20 ms. If you subtract the interpreter startup (11.7 ms), the cost of the imports goes from 283 ms to about 8 ms: a 35-fold reduction. The mean command, which needs a few standard modules, is twelve times faster.

There is a downside: errors move. With an eager import, a missing module fails at startup. With a lazy import, it fails at the first use, perhaps deep inside a function, perhaps hours later in a long-running server.

My source code is available.

2026-10-07 10:07:05

I write a lot about data parallelism. About how our processors have special instructions that can do several operations at once. They are often critical to get the best performance out of a CPU.

Compilers are sometimes able to take advantage of these instructions. But it is hard to tell when it will work.

For Java programmers, you can write directly for data parallelism since JDK 16. With enough skill, you might multiply the performance of a few key functions.

The Vector API is in the incubator module jdk.incubator.vector. It still not officially supported, but you can enable it by passing --add-modules=jdk.incubator.vector. I expect that it will soon become a mainstream feature in Java.

Roman Snytsar wrote a book about it: Mastering SIMD with Java Vector API, and the subtitle is Unlocking Single-Core Performance Through SIMD Optimization. I had the honor of being the technical reviewer. I learned a few things while reading the book and I enjoyed it.

Who has time for books, especially technical books?

What a book offers is time to reflect. Reading technical books today is probably just as relevant as it ever was. The author takes you on a story… and gets you to think. 

Snytsar’s book works from  problems like sum an array, compute a mean and a standard deviation, remove duplicates, merge two sorted arrays, and so forth. A lot of them are similar to the problems you will encounter as a programmer.  He starts from the ordinary solution and rebuilds it with the Java Vector API. He then runs benchmarks. He looks at the assembly code.

The book covers performance issues such as unrolling, asymmetric loops, structural hazards, dependencies. Snytsar shows us that, in some instances, data parallelism can fail to bear fruits. The negative lessons are just as important as the positive ones.

The Java Vector API is a thick layer of abstractions, but Snytsar shows that some tricks work better on some hardware than others.  I liked the chapter on removing duplicates, which is built on compress. It is one instance where the specific hardware matters. And you are unlikely to just find out about it yourself.

It is a book worth buying if you are a Java programmer with a focus on performance. 

The book: Roman Snytsar, Mastering SIMD with Java Vector API, Apress, 2026.

2026-10-06 16:00:20

When you build a program, the compiler turns each source file into an object file. Then a linker stitches all the object files and libraries into one executable.

On Linux, the default linker is usually GNU ld (also called bfd). The GNU binutils also ship gold, an alternative linker that was designed to be faster. Gold was built a Google and first made available in 2008.

The mold linker is a newer linker written by Rui Ueyama, it was first released in 2021. It is meant as a drop-in replacement for GNU ld, and its main selling point is speed. The latest release is version 3.

How much faster is it on a large project? I built Node.js from its main branch on an Intel Xeon Gold 6548N server (Emerald Rapids, two sockets, 64 cores and 128 threads) running Linux with GCC 14.3. The final node executable weighs about 160 MB.

I captured the command that links the node executable and ran it six times with each linker. I report the median.

With multithreading, mold is about 24 times faster than GNU ld and about 14 times faster than gold.

Even if I restrict mold to a single thread, it links Node.js in half a second, five times faster than GNU ld. With only 8 threads, you get nearly all of the benefit.

Using mold is easy. You can pass -fuse-ld=mold to GCC or clang. Or you can wrap your whole build:

mold -run make -j128

The -run option intercepts every call to the default linker and redirects it to mold. You do not need to change the build scripts.

Of course, two seconds saved on a link does not matter much if you build Node.js once. But a developer who edits a file and rebuilds dozens of times a day pays the link cost each time. With mold, the link step becomes nearly free.

My scripts and raw results are available.