2026-09-18 00:48:14
There’s a common saying in discussion of fractals that the length of a coastline depends on how small a device you use to measure it. I thought this was a hypothetical, say as applied to the steps in the construction of the Koch snowflake. But the saying has its roots in actually surveying.
Lewis Fry Richardson (1881–1953) noticed that the length of the coast of Scotland depended on a the size of segments used to measure it. More specifically, he found that the length followed a power law, i.e. that there’s a linear relation between the log of the coastline length and the log of the ruler length.
Here’s a reproduction of Richardson’s plot, taken from [1].
Mandelbrot built on Richardson’s observation and defined the idea of fractal dimension.
I was under the impression that fractals were invented as mathematical novelties that researchers later found applications for. But as is often the case, the applications came first. Or at least some applications came first.
Ideally there’s always a feedback cycle where applications lead to theory and theory leads to applications. As Donald Knuth put it, “The best theory is inspired by practice. The best practice is inspired by theory.”
[1] Eoghan Bradley and Mark McCartney. Four hundred years of the fractal coastline of Scotland. The Mathematical Gazette, November 2019, Vol. 103, No. 558 (November 2019), pp. 518-521
The post Empirical fractal first appeared on John D. Cook.2026-09-17 22:03:51
I recently bought a copy of Los Alamos Rolodex, a book displaying business cards from Los Alamos Nation Labs from 1967 to 1978. You can find some examples of the cards here.
One of the cards in the book is for Eugene Frank, President of B & F Instruments. His card lists his phone number as
(215) MErcury 9-7100
At first glance I thought the “E” in “MErcury” had been accidentally capitalized. Then I realized the intention was that someone would dial ME (i.e. 63) and ingore “rcury”. So the phone number would be (215) 639-7100.
This card was from 1968, the height of the space race. Presumably the card was alluding to the Project Mercury or the planet Mercury, or both.
The telephone keypad mapping (ITU E.161 standard) is a poor attempt at making phone numbers more memorable. For starters, there’s no way to encode 0 or 1 [1]. It’s unlikely a phone number will correspond to anything memorable unless you come up with the word first and then try to obtain the phone number, such as 800 FLOWERS.
Inserting extra letters, as Mr. Frank did, greatly increases the chances of encoding a phone number as a word. But then you need to denote which letters count and which ones are filler, so there’s not much advantage. Still, I wanted to play around with it for fun. I found 109 words [1] containing the letters from a telephone encoding of 4228646. (I’m using the file /usr/share/dict/words on my laptop as my list of words.)
Here are some of the more interesting hits.
There are over 30,000 words containing an encoding of the area code 832. One of these is traditional, and so I could write my phone number as
TraDitionAl semICAThOlIcisM.
Another choice for 832 is intercosmic, so
inTErCosmic GAlaCTOGeNetic
is another possibility.
Galactogentic can refer to the production of milk by the mammary glands or to the formation of galaxies (e.g. the Milky Way). Here intercosmic fits with the later sense.
I got greedy and tried to find a word containing the full phone number, 8324228646, but didn’t find anything.
Here’s my business card in the style of the Los Alamos Rolodex cards, created by Grok, using (832) GlAdiATOrIsM as the phone number.
Now suppose you remembered “gladiatorism” but not which letters were capitalized. Then you’d have to try up to 792, i.e. 12 choose 7, possible numbers, so this really isn’t a practical mnemonic. If you remembered “traditional semicatholicism” without capitalization it would be worse, with over a million possibilities (11 choose 3 times 15 choose 7). Some possibilities are counted twice, since different ways of selecting letters can lead to the same phone number, but still there are too many possibilities to try.
[1] Not only are there no letters for 0 and 1, the letters O and I represent digits. At one point in time the first digit of an exchange (the middle three digits) could not be a 0 or 1, but these digits could appear anywhere else.
[2] I initially found a list of 185 words, but some of these were duplicates: a word can represent a phone number in more than one way.
The post Phone words first appeared on John D. Cook.2026-09-17 00:05:11
I’ve written three posts on cosine similarity lately. The first looked at interpreting cosine similarity. The second looked at an approximation related to the first. The third looked at how ranking according to cosine similarity works better than cosine similarity itself.
Normalized word vectors are points on a high dimensional sphere, and geometry in high dimensions is counterintuitive. See the first post in this series for an explanation.
The set of points within a given angular distance of a point on a hypersphere is called a spherical cap. The ratio of the area of this spherical cap to that of the whole sphere is called cap fraction or concentration ratio. Concentration ratio explains why a modest cosine similarity value corresponds to a tiny portion of the area of the sphere and should be interpreted as a close match.
For this post, I wanted to share a plot of concentration ratio as a function of cosine similarity.
This shows that moderate values of cosine similarity correspond to infinitesimal concentration ratios. And yet, as the third post linked at the top showed, word vectors are very unevenly distributed, and even extremely small regions of the sphere can contain multiple word vectors.
I only included cosine similarity values up to 0.8 because the function plotted above takes a nosedive for larger values, even on a logarithmic scale.
Here’s the Python code to make the plot, using the function cap_fraction from here.
s = np.linspace(0, 0.8, 500)
plt.plot(s, cap_fraction(np.acos(s), 200))
plt.yscale("log")
plt.xlabel("cosine similarity")
plt.ylabel("concentration ratio")
plt.show()
The post Converting between cosine similarity and concentration ratio first appeared on John D. Cook.
2026-09-16 23:06:58
Yesterday I wrote about the canonical example of how vector embeddings of words add:
“king” − “man” + “woman” ≈ “queen”
This should be interpreted as saying that the word vector for king, minus the word vector for man, plus the word vector for woman, is in some sense close to the word vector for queen.
This post will look at another example. Is the expression
“coffee” + “milk” ≈ “latte”
true in some sense?
In this post I will use “foo” to mean the vector embedding of the word foo.
The cosine similarity between “coffee” + “milk” and “latte” is about 0.63. And for reasons given in the previous post, this is a large value of cosine similarity. But there are 11 words that are more similar to “milk” + “coffee” than “latte”. Here are the top 12 matches in order.
There are two questions to resolve. First, why isn’t latte one of the closest words? Second, why is the cosine similarity large even though latte is not one of the best matches?
When word vector arithmetic works, as in the king and queen example, the vectors combine concepts. If you replace the male gender component of king with a female component, you get a vector close to the vector for queen.
But when you add the vectors for milk and coffee, you’re not adding concepts, you’re adding ingredients.
The concepts of milk and coffee are similar in that they’re both common beverages, as are tea and even beer. A latte is a beverage, but it’s not as common as milk, coffee, tea, or beer.
If you divide word vectors by their norm, you get a point on a high-dimensional sphere. In the case of the glove-twitter-200 vector embedding, you get a point on a sphere in 200 dimensions. As explained in the earlier post, a fairly large cosine similarity corresponds to a tiny portion of the sphere’s surface area.
In the example of “king” − “man” + “woman”, the vector “queen” is the closest match (except for “king” itself).
But there are a lot of words whose vectors are within a tiny region around “coffee” + “milk”. And by tiny, I mean a region that accounts for a proportion of the sphere on the order of 10−23.
The glove-twitter-200 vector list contains vectors for 1.2 million words. If these vectors were roughly evenly distributed on the sphere when normalized, you’d expect each patch representing 10−6 of the sphere to contain about a word or two. You wouldn’t expect a patch taking up 10−12 of the sphere to contain more than one word, and you certainly wouldn’t expect a patch taking up 10−23 of the sphere to contain 12 words [1].
Rank order based on cosine similarity is more robust than cosine similarity itself. This is an example of a phenomenon that occurs regularly: a metric whose values are dubious might still rank things well. Naive Bayes is another example. It naively computes probabilities in a way that is blatantly wrong, and yet ranking things by these spurious probabilities works well in some cases.
The cosine similarity between “king” − “man” + “woman” and “queen” is roughly the same as the cosine similarity between “coffee” + “milk” and “latte.” But in the former example, rank order picks out queen as the best match; rank order works like you’d expect, because you’re working with attributes that can be decomposed.
I wouldn’t be surprised if the Anglo-Saxon word for puppy was something like dogchild. The language was full of colorful compound words, such as hronrad (“whale-road”) for the sea and nosethyrl (“nose-hole”) for nostril.
Here are the top ten matches for “dog” + “infant” along with their cosine similarities.
This shows that “puppy” is close to “dog” + “infant”, both in terms of cosine similarity and rank order, though it’s not the closet.
This also shows that you have to take the addition of word vectors with a grain of salt. It’s no surprise that puppy was a good match, but it’s surprising that cat is nearly as good.
[1] I poked around a little to get an idea just how unevenly words are distributed. The closest pair of words is jajaja and jajajaja with a cosine similarity of 0.993. The most isolated word, meaning the word whose nearest neighbor is furthest away, the the Thai word เคยไหม. It’s nearest neighbor is the Russian word боль with a cosine similarity of 0.283.
The glove-twitter-200 vectors were created from a corpus that is about half English and about other languages and strings of symbols that are not words in any language. Presumably เคยไหม would have a much closer neighbor in a corpus containing more Thai words.
I didn’t search the entire corpus, only the 50,000 most frequently occurring vectors, because a full search would require running an O(N²) search with N = 1,200,000.
The post Coffee + milk ≠ latte first appeared on John D. Cook.2026-09-16 20:04:09
The product of four consecutive Fibonacci numbers equals the product of two consecutive integers.
For example,
3 × 5 × 8 × 13 = 39 × 40.
I ran across this theorem in a note [1] that says “The product of any four consecutive Fibonacci numbers is twice a triangular number.” Since triangular numbers have the form n(n + 1)/2, twice a triangular number is the product of two consecutive integers.
The note also gives a way to find the numbers on the right hand side. We have
FnFn+1Fn+2Fn+3 = m(m + 1)
where m equals
Fn+1Fn+2
if n is odd and
FnFn+3
if n is even.
In the example at the top, 3 is the 4th Fibonacci number, so n = 4. Since 4 is even, m is the product of the 4th and 7th Fibonacci numbers, i.e. m = 3 × 13 = 39.
[1] K. B. Subramaniam. On a link between Triangular and Fibonacci numbers. The Mathematical Gazette, Vol. 103, No. 558 (November 2019), p. 489.
The post Fibonacci product first appeared on John D. Cook.2026-09-16 06:00:44
The previous post looked at how to interpret cosine similarity, or equivalently angles between word vectors. In a high-dimensional space, randomly chosen vectors are likely nearly perpendicular, and so relatively large angles, such as 50°, indicate very closely related words.
Another way to look at this, as explained in the previous post, is that in high dimensions, a spherical cap of angular radius θ represents a small portion of a sphere, even for moderately large θ.
The proportion of the area inside the spherical cap, given here, involves the “regularized incomplete beta function” and so it’s hard to have an intuition for the value.
For large dimension n, the approximation
n−1/2 sinn − 1(θ)
gives the proportion of the area inside the cap to within an order of magnitude. It’s easy to see that this function goes to zero quickly as n increases, provided |θ| < π/2.
If you have the cosine similarity c = cos θ rather than θ itself, the approximation becomes
n−1/2 (1 − c²)(n − 1)/2.
Let’s try it on the example from the previous post, in which n = 200 and θ = 49°.
import numpy as np
from scipy.special import betainc
# Fraction of S^{n-1} inside a spherical cap of angular radius theta
# theta is measured from the pole
# Assume 0 < theta < pi/2
def cap_fraction(theta, n):
x = np.sin(theta) ** 2
return 0.5 * betainc(0.5 * (n - 1), 0.5, x)
def cap_fraction_approx(theta, n):
return n**(-0.5) * np.sin(theta)**(n-1)
theta = np.deg2rad(49)
print(cap_fraction(theta, 200))
print(cap_fraction_approx(theta, 200))
This prints 2.03e-26 and 3.37e-26. The order of magnitude is correct as advertised.
The post Simple approximation for spherical cap area first appeared on John D. Cook.
