2026-07-18 00:12:31
Quick follow up to the previous two posts on ICD-10 codes and HCPCS codes. This post uses Python’s squarify library to create treemaps visualizing how many codes begin with each letter.
Here’s the treemap for HCPCS codes.
And here’s the treemap for ICD-10 codes.
The sizes of the squares are proportional to the number of codes beginning with that letter. Note that they are not necessarily proportional to how often codes are used.
The HCPCS map omits R and U because these are tiny relative to the rest. The ICD-10 map omits U for the same reason.
Here’s the code that was used to create the HCPCS map.
import matplotlib.pyplot as plt
import squarify
# HCPCS
data = {
"G": 2010,
"J": 1232,
"L": 940,
"A": 862,
"E": 671,
"Q": 639,
"C": 619,
"S": 533,
"M": 506,
"V": 212,
"K": 175,
"T": 114,
"H": 94,
"P": 59,
"B": 51,
# "U": 5,
# "R": 3,
}
labels = list(data.keys())
sizes = list(data.values())
# Labels are just the letters (no counts)
display_labels = labels
# Color map — one distinct color per box
colors = plt.cm.tab20.colors[: len(labels)]
fig, ax = plt.subplots(figsize=(12, 8))
squarify.plot(
sizes=sizes,
label=display_labels,
color=colors,
alpha=0.85,
ax=ax,
text_kwargs={"fontsize": 30, "weight": "bold"},
pad=True,
)
ax.axis("off")
plt.tight_layout()
plt.savefig("treemap.png", dpi=72)
plt.show()
The code to create the ICD-10 map differs only in its data.
# ICD-10
data = {
"S": 31052,
"T": 10090,
"M": 6665,
"V": 4086,
"H": 3330,
"O": 2437,
"Y": 1590,
"I": 1427,
"Z": 1411,
"W": 1290,
"C": 1226,
"L": 1000,
"E": 971,
"Q": 894,
"F": 871,
"K": 857,
"N": 836,
"D": 824,
"R": 773,
"G": 700,
"A": 573,
"X": 495,
"B": 495,
"P": 463,
"J": 360,
# "U": 3,
}
The post Visualizing Medical Code Hierarchy first appeared on John D. Cook.
2026-07-17 22:02:52
Since I revisited my old post on ICD code matching, I thought I’d revisit by post on HCPCS codes too.
HCPCS stands for Healthcare Common Procedure Coding System, and is pronounced “hick picks.” When most people say HCPCS, they technically mean HCPCS Level II, and that’s what I mean here.
The format of a HCPCS code is simple: one letter and four digits. In regex terms,
[A-Z]\d{4}
Not all letters are used, so you can get more specific and say
[A-CEGHJ-MP-V][0-9]\d{4}
Some sources say no codes begin with U, but there are currently five codes that begin with U.
When I was doing some research on HCPCS codes recently using AI, I was told there is a D code for dentistry, but that was a hallucination.
HCPCS codes can also have modifiers. These consist of a letter and either a letter or digit:
[A-Z][A-Z0-9]
Not all letters actually appear in modifiers—I, O, W, and Y are missing—so you could be more specific. At the time of writing there are 384 official modifiers.
Modifiers are often stored in a separate column in a database, but in text you’ll see a HCPCS code optionally followed by a dash and a modifier. So a regex to match HCPCS codes with possible modifiers would be
[A-CEGHJ-MP-V][0-9]\d{4}(-[A-Z][A-Z0-9])?
This regex will have some false positives, but it should not have false negatives: every real HCPCS code should match.
Of course you could search against a complete list of HCPCS codes. This would be more accurate and slower. I did a test similar to the one in the previous post and found a search with the regex above took 20 milliseconds, while a search against the list of HCPCS codes took 46 seconds.
However, the regex searched for possible modifiers and the exhaustive search only looked for unmodified HCPCS codes. A complete list of HCPCS codes with possible modifiers would be tedious to create because some combinations of codes and modifiers make no sense. And I imagine that some combinations that would make sense are not used in practice.
The post Regular expressions for HCPCS codes first appeared on John D. Cook.2026-07-17 19:26:19
Seven years ago I wrote a post about regular expressions to match diagnosis codes. I wanted to revisit that post looking at speed and error rates.
Regular expressions usually do not exactly match what you’re looking for and nothing else. They have error false positives and false negatives. But they also have advantages, and context determines whether the advantages make the error rates tolerable.
The post mentioned above gave the following regular expression for ICD-10 diagnosis codes.
[A-TV-Z][0-9][0-9AB]\.?[0-9A-TV-Z]{0,4}
As cryptic as this may look at first glance, it’s straight-forward to interpret. It says that an ICD-10 code
Now suppose you want to scan a text document for ICD-10 codes. One approach would be to use the regex above. Another would be to compare every alphanumeric sequence in the document to a list of ICD-10. Currently this list has 74,719 codes.
I tested both approaches on a 800kb text file. The regex search
egrep -o '[A-TV-Z][0-9][0-9AB]\.?[0-9A-TV-Z]{0,4}' notes.txt
took 18 milliseconds. Searching against the list of codes
grep -w -F -o -f icd10codes.txt notes.txt
took 386 seconds, about six and a half minutes or five orders of magnitude longer.
The regex
[A-TV-Z][0-9][0-9AB]\.?[0-9A-TV-Z]{0,4}
had a false negative rate of zero at the time it was written. I tested the regex against the current list of codes with the following command.
egrep -v '[A-TV-Z][0-9][0-9AB]\.?[0-9A-TV-Z]{0,4}' icd10codes.txt
The -v flag reverses the sense of the search, reporting lines that do not match the regular expression. This returned three matches: U070, U071, and U099. So 3 out of 74,719 valid ICD-10 codes were reported as invalid.
Codes beginning with U are designated for provisional/emergency/special purposes, but these three have become essentially permanent. A change in the application of the ICD-10 standard caused an error in the regular expression.
But the change would also have caused an error in code that did an exhaustive search against the list of ICD-10 codes at the time. In fact, every new code not starting with U would also be reported in error. So the regex is actually more future-proof than an exhaustive search. Presumably the simplified regex
[A-Z][0-9][0-9AB]\.?[0-9A-Z]{0,4}
will remain valid for the foreseeable future.
We’ve looked at false negatives. What about false positives? That depends on context. The false positive rate when searching medical notes is low: a word matching the regex above in a medical record is most likely an ICD-10 code. But the number of conceivable false positives is enormous. If you were searching a file of randomly generated alphanumeric text, the regex matches would overwhelmingly be false positives [1].
The number of strings matching
[A-Z][0-9][0-9AB]\.?[0-9A-Z]{0,4}
would be
26 × 10 × 12 × (1 + 36 + 362 + 363 + 364) = 5,390,127,600.
Out of over five billion strings matching the regular expression, only around 75,000 are valid ICD-10 codes. So a naive theoretical calculation would say the false positive rate is 99.9986%, whereas in practice the false positive rate is very low, though there’s no way to say a priori exactly how low.
[1] You could argue that all positives would be false positives in this context because you’re looking at noise. You couldn’t find an ICD code, though you could find a string of characters that coincides with an ICD code. That may sound like a pedantic distinction, but it matters in the context of evaluating deidentification quality: you want to find instances of PHI, not instances of strings that match the character pattern of PHI.
The post Regular expression speed and error rates first appeared on John D. Cook.2026-07-14 22:24:12
I’ve been thinking about ICD-10 codes; they come up a lot in my work.
The ICD-10-CM standard is divided into 21 chapters, which generally correspond to the first letter of a code. However, a chapter may contain blocks beginning with more than one letter, and codes starting with a single letter, namely D, can span two chapters.
Here’s a diagram I made to visualize the relationship between chapters and initial letters of codes.
Notice there’s no letter U on the diagram. That’s because U is reserved for special/provisional codes.
2026-07-13 03:26:29
A few days ago I wrote a post entitled Does additional data always reduce posterior variance?. In a nutshell, the answer is no, not always.
That led the previous post which looked at posterior means for three Bayesian models, showing how the posterior mean is a weighted average of the prior mean and the mean of the new data. The weights are precisions, which means something different for each model.
For the beta-binomial model, variance may increase when seeing unexpected data (details here), but precision always increases.
For the normal-normal model precision is the reciprocal of variance. Every new data point makes precision go up and posterior variance go down.
The Poisson-gamma model may be the most interesting. As stated in the previous post, if data has a Poisson distribution with parameter λ, and λ has a gamma(α0, β0) prior distribution, then the posterior distribution on λ after observing k events over time t has a gamma(α0 + k, β0 + t) posterior distribution. Therefore the posterior variance is
(α0 + k) / (β0 + t)².
Note the posterior variance is an increasing function of k and a decreasing function of t. This means that the posterior variance increases every time an event is observed, and it decreases quadratically between observations.
Here’s an illustration. I simulated data from a Poisson process with λ and used a gamma(1, 1) prior on λ. Here’s a plot of the posterior variance.
2026-07-13 01:48:14
Common sense says that what you believe after seeing new data should be some sort of compromise between what you believed before and what the new data says. You don’t want to ignore previous information or new information.
How much should new data change your prior beliefs? When prior judgment and new information are in conflict, which one should be given the benefit of the doubt?
Bayesian data models provide a framework for making such decisions quantitative and objective. The choice of a data model is somewhat subjective—whether it’s a Bayesian model or not—but given a Bayesian model, the rules for updating the representation of your beliefs are objective. As some put it, you “turn the Bayesian crank.” A likelihood model and a prior on parameters together specify how new data changes the prior distribution into a posterior distribution.
We will make this more concrete with three examples.
Suppose that data X has a normal distribution with unknown mean μ and known variance σ², and we assume that a priori μ has a normal distribution with mean μ0 and variance σ0².
After observing x, the posterior distribution on μ also has a normal distribution, but with a different mean and variance. Its mean is somewhere between the prior mean and x. We will ignore the change in the variance for this post.
The posterior mean of μ is
This equation becomes more understandable when we introduce precisons τ = 1/σ² and τ0 = 1/σ0².
Then we have
which you can read as saying the posterior mean is the weighted average of the prior mean and x, with the weights given by the precision. Intuitively, you take the weighted mean of your conclusions from previous data and new data, weighting the mean according to how much confidence you have in each.
Now let’s switch over to a different data model. Now assume X is a binary random variable, with probability of success p and probability of failure 1 − p, and we assume p has a beta(a, b) distribution.
After observing s successes and f failures, the posterior mean of the distribution on p becomes
We can rewrite this as
This says that the posterior mean is the weighted average of the prior mean a/(a + b) and the mean of the data s/n. The weights are the prior effective sample size a + b and the sample size of the new data n. In this example (effective) sample size is playing the role that precision played in the normal-normal model above.
Suppose data have a Poisson distribution with parameter λ, and λ has a gamma(α0, β0) prior distribution [1]. And suppose you observe k events over time t. Then the posterior distribution of λ given the data has a gamma(α0 + k, β0 + t) prior distribution and the mean of the posterior distribution is given by
As before, the posterior mean is a weighted average of the prior mean and new data, and the weights are interpretable as some sort of measure of confidence, namely time. The variable t is directly time and the parameter β0 is sort of an effective time, just as a + b is an effective sample size for the beta distribution.
In each example the posterior mean is the weighted average of the prior mean and the mean of the data, with the weights given by a precision. However, precision means something different in each example. In the normal-normal model, precision is the reciprocal of variance, but in the beta-binomial model precision is sample size and in the Poisson-gamma model precision is time.
What all three examples have in common is that they are conjugate models using distributions from the “exponential family” of probability distributions. In technical terms, precision is the multiplicative factor on the sufficient statistic in the exponent of the posterior kernel.
[1] There are multiple conventions for parameterizing the gamma distribution. Here we’re using the shape-rate parameterization, where the mean is α/β.
The post Posterior mean first appeared on John D. Cook.
