I have decades of consulting experience helping companies solve complex problems involving applied math, statistics, and data privacy.
The RSS's url is : https://www.johndcook.com/blog/feed/
Please copy to your reader or subscribe it with :
2024-07-26 19:00:08
Imagine in a game of Rock, Paper, Scissors one player is free to play as usual but the other is required to choose each option the same number of times. That is, in 3n rounds of the game, the disadvantaged player much choose Rock n times, Paper n times, and Scissors n times. Obviously the […]
The post A variation on Rock, Paper, Scissors first appeared on John D. Cook.2024-07-23 21:11:37
The previous post looked at the probability that a random n by n matrix over a finite field of order q is invertible. This works out to be This function of q and n comes up in other contexts as well and has a name that we will get to shortly. Pochhammer symbols Leo August […]
The post q-analog of rising powers first appeared on John D. Cook.2024-07-23 02:32:37
If you have n equations in n unknowns over a finite field with q elements, how likely is it that the system of equations has a solution? The number of possible n × n matrices with entries from a field of size q is qn². The set of invertible n × n matrices over a field […]
The post Solvability of linear systems over finite fields first appeared on John D. Cook.2024-07-22 21:27:18
Most people implicitly assume medical tests are infallible. If they test positive for X, they assume they have X. Or if they test negative for X, they’re confident they don’t have X. Neither is necessarily true. Someone recently asked me why medical tests always have an error rate. It’s a good question. A test is […]
The post Why do medical tests always have error rates? first appeared on John D. Cook.2024-07-14 06:14:53
Imagine parallel parking is available along the shoulder of a road, but no parking spaces are marked. The first person to park picks a spot on the shoulder at random. Then another car also chooses a spot along the shoulder at random, with the constraint that the second car can’t overlap the first. This process […]
The post Rényi’s parking constant first appeared on John D. Cook.2024-07-06 23:03:48
When we say that the planets in our solar system orbit the sun, not the earth, we mean that the motions of the planets is much simpler to describe from the vantage point of the sun. The sun is no more the center of the universe than the earth is. Describing the motion of the […]
The post Calculating when a planet will appear to move backwards first appeared on John D. Cook.2024-07-03 00:45:15
Suppose you make an x% improvement followed by a y% improvement. Together do they make an (x + y)% improvement? Maybe. The business principle of kaizen, based on the Japanese 改善 for improvement, is based on the assumption that incremental improvements accumulate. But quantifying how improvements accumulate takes some care. Add or multiply? Two successive […]
The post Do incremental improvements add, multiply, or something else? first appeared on John D. Cook.2024-07-01 20:26:20
I ran across the Clausen function the other day, and when I saw a plot of the function my first thought was that it looks sorta like a sawtooth wave. I wondered whether it also sounds like a sawtooth wave, and indeed it does. More on that shortly. The Clausen function can be defined in […]
The post The Clausen function first appeared on John D. Cook.2024-06-27 21:23:03
Suppose you’re in a boring meeting and you start doodling. You draw a circle, and then you draw a triangle on the outside of that circle. Next you draw a circle through the vertices of the triangle, and draw a square outside that. Then you draw a circle through the vertices of the square, and […]
The post Limit of a doodle first appeared on John D. Cook.2024-06-27 08:23:42
Healthcare providers in the United States are required to have an ID number known as the NPI (National Provider Identifier). This is a 10-digit unique identifier which serves as the primary key in a publicly available database. You can use the NPI number to look up a provider’s name, credentials, their practice location, etc. The […]
The post National Provider Identifier (NPI) and its checksum first appeared on John D. Cook.2024-06-25 23:36:41
How can you possibly solve a mission-critical problem with millions of variables—when the worst-case computational complexity of every known algorithm for that problem is exponential in the number of variables? SAT (Satisfiability) solvers have seen dramatic orders-of-magnitude performance gains for many problems through algorithmic improvements over the last couple of decades or so. The SAT […]
The post Getting some (algorithmic) SAT-isfaction first appeared on John D. Cook.2024-06-25 19:53:31
In the previous post I mentioned that the general strategy for computing a mathematical function using tables is to first reduce the function argument to be within the range of the tabulated values, and then to use interpolation to compute the function at values that are not directly tabulated. The second step is always the […]
The post Computing Γ(z) for complex z with tables first appeared on John D. Cook.2024-06-25 18:15:20
It takes some skill to use tables of mathematical functions; it’s not quite as simple as it may seem. Although it’s no longer necessary to use tables, it’s interesting to look into the details of how it is done. For example, the Handbook of Mathematical Functions edited by Abramowitz and Stegun tabulates sines and cosines […]
The post Calculating trig functions from tables first appeared on John D. Cook.2024-06-20 22:07:06
The previous post looked at two simple approximations for the perimeter of an ellipse. Approximations are necessary since the perimeter of an ellipse cannot be expressed as an elementary function of the axes. Kepler noted in 1609 that you could approximate the perimeter of an ellipse as the perimeter of a circle whose radius is […]
The post Rapidly convergent series for ellipse perimeter first appeared on John D. Cook.2024-06-20 11:43:20
In 1609, Kepler remarked that the perimeter of an ellipse with semiaxes a and b could be approximated either as P ≈ 2π(ab)½ or P ≈ π(a + b). In other words, you can approximate the perimeter of an ellipse by the circumference of a circle of radius r where r is either the geometric mean […]
The post Kepler’s ellipse perimeter approximations first appeared on John D. Cook.2024-06-12 19:20:14
A few days ago I wrote about eigenvector centrality, a way of computing which nodes in a network are most important. Rather than simply looking for the most highly connected nodes, it looks for nodes that are highly connected to nodes that are highly connected. It’s the idea behind Google’s PageRank algorithm. Adjacency matrices One […]
The post Power method and centrality first appeared on John D. Cook.2024-06-11 04:20:29
Anyone with more than casual experience with ChatGPT knows that prompt engineering is a thing. Minor or even trivial changes in a chatbot prompt can have significant effects, sometimes even dramatic ones, on the output [1]. For simple requests it may not make much difference, but for detailed requests it could matter a lot. Industry […]
The post The search for the perfect prompt first appeared on John D. Cook.2024-06-08 21:23:22
A basic question to ask about a network is which nodes are most important. A simple way of answering this question would be to say the most well-connected nodes, i.e. the nodes with the most edges. This approach is known as degree centrality. It’s not a bad place to start. It’s easy to understand and […]
The post Eigenvector centrality first appeared on John D. Cook.2024-06-05 23:46:10
The other day I was looking at how many lumens LED lights put out per watt. I found some data on Wikipedia, and as you might expect the relation between watts and lumens is basically linear, though not quite. If you were to do a linear regression on the data you’d get a relation lumens […]
The post Fitting a line without an intercept term first appeared on John D. Cook.2024-06-04 06:27:10
I read something recently that said in passing that the solutions to the equation tan x = x are the zeros of the Bessel function J3/2. That brought two questions to mind. First, where have I seen the equation tan x = x before? And second, why should its solutions be the roots of a […]
The post Solutions to tan(x) = x first appeared on John D. Cook.2024-06-03 22:33:20
The previous post showed how to compute logarithms using tables. It gives an example of calculating a logarithm to 15 figures precision using tables that only allow 4 figures of precision for inputs. Not only can you bootstrap tables to calculate logarithms of real numbers not given in the tables, you can also bootstrap a […]
The post Computing logarithms of complex numbers first appeared on John D. Cook.2024-06-03 20:44:24
My favorite quote from Richard Feynman is his remark that “nearly everything is really interesting if you go into it deeply enough.” This post will look at something that seems utterly trivial—looking up numbers in a table—and show that there’s much more to it when you dig a little deeper. More than just looking up […]
The post Using a table of logarithms first appeared on John D. Cook.2024-06-01 22:23:24
Excerpt from John Carmack’s review of the book Bullshit Jobs. He talks about how software developers bemoan duct taping systems together, and would rather work on core technologies. He thinks it is some tragic failure, that if only wise system design was employed, you wouldn’t be doing all the duct taping. Wrong. Every expansion in […]
The post Duct tape value creation first appeared on John D. Cook.2024-05-29 00:41:52
Suppose you design an experiment, an A/B test of two page designs, randomizing visitors to Design A or Design B. You planned to run the test for 800 visitors and you calculated some confidence level α for your experiment. You decide to take a peek at the data after only 300 randomizations, even though your […]
The post Condition on your data first appeared on John D. Cook.2024-05-28 23:43:38
Suppose you’re running an A/B test to determine whether a web page produces more sales with one graphic versus another. You plan to randomly assign image A or B to 1,000 visitors to the page, but after only randomizing 500 visitors you want to look at the data. Is this OK or not? Of course […]
The post Can you look at experimental results along the way or not? first appeared on John D. Cook.2024-05-28 04:04:45
In LaTeX, sections are labeled with commands like \label{foo} and referenced like \ref{foo}. Referring to sections by labels rather than hard-coded numbers allows references to automatically update when sections are inserted, deleted, or rearranged. For every reference there ought to be a label. A label without a corresponding reference is fine, though it might be […]
The post One-liner to troubleshoot LaTeX references first appeared on John D. Cook.2024-05-22 09:25:09
I was reading an article [1] that refers to “a well-known trigonometric series” that I’d never seen before. This paper cites [2] which gives the series as Note that the right hand side is not a series in φ but rather in sin φ. Motivation Why might you know sin φ and want to calculate […]
The post A “well-known” series first appeared on John D. Cook.2024-05-21 09:36:28
Probability and cryptography have this in common: really smart people can be confidently wrong about both. I wrote years ago about how striking it was to see two senior professors arguing over an undergraduate probability exercise. As I commented in that post, “Professors might forget how to do a calculus problem, or make a mistake […]
The post Probability, cryptography, and naïveté first appeared on John D. Cook.2024-05-20 23:54:28
Richard Feynman’s Nobel Prize winning discoveries in quantum electrodynamics were partly inspired by his randomly observing a spinning dinner plate one day in the cafeteria. Paul Feyerabend said regarding science discovery, “The only principle that does not inhibit progress is: anything goes” (within relevant ethical constraints, of course). Ideas can come from anywhere, including physical […]
The post Thinking by playing around first appeared on John D. Cook.2024-05-13 21:46:06
The well-known Weierstrass approximation theorem says that polynomials are dense in C [0, 1]. That is, given any continuous function f on the unit interval, and any ε > 0, you can find a polynomial P such that f and P are never more than ε apart. This means that linear combinations of the polynomials 1, […]
The post Approximation by prime powers first appeared on John D. Cook.