2026-07-14 08:00:00
The scraping problem is worse than anyone can imagine and thanks to my friends at Sourceware we have some real data to prove it.
I've been working more on Anubis' reputation database and I've run into a really weird discovery: 80-90% of the hits created by the honeypot feature are from IP addresses that do not belong to any existing threat monitoring lists.
Here's a breakdown of the honeypot hits Sourceware has gotten in the last few months:
In case this interests you, I have put the full tables in Appendix A: Full tables for the reputation database input.
| Field | Value |
|---|---|
| lines read | 2678193 |
| skipped (non-IP): | 0 |
| skipped (dupe): | 0 |
| unique IPs: | 2678193 |
| flagged (in db): | 286161 (10.7%) |
| clean (not in): | 2392032 (89.3%) |
| Flag | Unique IPs | Share |
|---|---|---|
| is_vpn | 1264 | 0.4% |
| is_datacenter | 7918 | 2.8% |
| is_crawler | 46 | 0.0% |
| is_proxy | 2562 | 0.9% |
| Category | Unique IPs | Share |
|---|---|---|
| abuse | 282182 | 98.6% |
| datacenter | 7918 | 2.8% |
| proxy | 2562 | 0.9% |
| vpn | 1264 | 0.4% |
| crawler | 46 | 0.0% |
| tor | 17 | 0.0% |
Methodology note: "provider" here means one of two things:
For example, scaleway is based off of Scaleway's publicly posted IP address ranges, firehol-level1 is based on a daily snapshot of FireHOL's Level 1 IP list, and fdo is based on data contributed by the administrators of freedesktop.org.
| Provider | Unique IPs | Share |
|---|---|---|
| netshield | 237945 | 83.2% |
| bitwire | 96539 | 33.7% |
| magicteamc | 26475 | 9.3% |
| ipinsights | 17378 | 6.1% |
| threathive | 8422 | 2.9% |
| netmountains | 6673 | 2.3% |
| multacom | 2676 | 0.9% |
| fyvri | 2433 | 0.9% |
| cbuijs | 1916 | 0.7% |
| x4bnet | 1263 | 0.4% |
| solispirit | 1259 | 0.4% |
| dailyproxy | 1182 | 0.4% |
| blackwall | 1073 | 0.4% |
| hproxy | 1067 | 0.4% |
| scaleway | 922 | 0.3% |
| fdo | 755 | 0.3% |
| datacamp | 702 | 0.2% |
| ebrasha | 686 | 0.2% |
| hideip | 628 | 0.2% |
| datacentres | 480 | 0.2% |
| komutan | 463 | 0.2% |
| aws | 431 | 0.2% |
| m247 | 360 | 0.1% |
| firehol-level1 | 354 | 0.1% |
| vpslab | 331 | 0.1% |
| alibaba-cloud | 319 | 0.1% |
| proxyscrape | 272 | 0.1% |
| ovhcloud | 268 | 0.1% |
(remainder snipped for brevity)
| Country | Unique IPs | Flagged | Rate |
|---|---|---|---|
| Brazil (BR) | 270937 | 18282 | 6.7% |
| India (IN) | 185091 | 12478 | 6.7% |
| Saudi Arabia (SA) | 120372 | 3574 | 3.0% |
| Mexico (MX) | 95449 | 7053 | 7.4% |
| Türkiye (TR) | 87258 | 5559 | 6.4% |
| Argentina (AR) | 86463 | 9522 | 11.0% |
| Pakistan (PK) | 85241 | 17083 | 20.0% |
| Vietnam (VN) | 78967 | 8848 | 11.2% |
| Morocco (MA) | 69201 | 1805 | 2.6% |
| Philippines (PH) | 66128 | 7899 | 11.9% |
| Venezuela (VE) | 64670 | 13780 | 21.3% |
| Iraq (IQ) | 62047 | 13613 | 21.9% |
| Chile (CL) | 60878 | 4522 | 7.4% |
| Colombia (CO) | 59579 | 7048 | 11.8% |
| Bangladesh (BD) | 59245 | 17735 | 29.9% |
| France (FR) | 49782 | 1339 | 2.7% |
| Tunisia (TN) | 48535 | 5799 | 11.9% |
| Uruguay (UY) | 45888 | 430 | 0.9% |
| South Africa (ZA) | 43919 | 7431 | 16.9% |
| United States (US) | 40828 | 3347 | 8.2% |
| Indonesia (ID) | 38119 | 6122 | 16.1% |
| Canada (CA) | 37342 | 2334 | 6.3% |
| Spain (ES) | 36008 | 2944 | 8.2% |
| Algeria (DZ) | 35112 | 537 | 1.5% |
| Ukraine (UA) | 32261 | 8920 | 27.6% |
This doesn't list data from 204 additional countries. Given that the ISO 3166-1 standard comprises 249 countries (193 of which are UN members), it's safe to say this is a global problem.
| ASN | Unique IPs | Flagged | Rate |
|---|---|---|---|
| AS55836 Reliance Jio Infocomm Limited | 57029 | 1749 | 3.1% |
| AS45899 VNPT Corp | 56910 | 6831 | 12.0% |
| AS6057 Administracion Nacional de Telecomunicaciones | 43694 | 339 | 0.8% |
| AS25019 Saudi Telecom Company JSC | 40800 | 679 | 1.7% |
| AS24560 Bharti Airtel Ltd., Telemedia Services | 35957 | 1620 | 4.5% |
| AS36903 Office National des Postes et Telecommunications ONPT (Maroc Telecom) / IAM | 35562 | 668 | 1.9% |
| AS36947 Telecom Algeria | 33172 | 386 | 1.2% |
| AS9121 Turk Telekom | 32742 | 1465 | 4.5% |
| AS8151 UNINET | 32012 | 856 | 2.7% |
| AS14593 Space Exploration Technologies Corporation | 31569 | 4597 | 14.6% |
| AS9299 Philippine Long Distance Telephone Company | 27573 | 1626 | 5.9% |
| AS39891 Saudi Telecom Company JSC | 25904 | 794 | 3.1% |
| AS35819 Etihad Etisalat, a joint stock company | 24493 | 978 | 4.0% |
| AS28573 Claro NXT Telecomunicacoes Ltda | 23903 | 841 | 3.5% |
| AS8193 Uzbektelekom Joint Stock Company | 22611 | 3191 | 14.1% |
| AS8452 IDDQD-AS | 22369 | 364 | 1.6% |
| AS43766 Mobile Telecommunication Company Saudi Arabia Joint-Stock company | 22038 | 968 | 4.4% |
| AS9541 Cyber Internet Services (Pvt) Ltd. | 21386 | 3696 | 17.3% |
| AS37705 TOPNET | 20024 | 222 | 1.1% |
| AS11664 Techtel LMDS Comunicaciones Interactivas S.A. | 18021 | 883 | 4.9% |
| AS17072 TOTAL PLAY TELECOMUNICACIONES, S.A.P.I. DE C.V. | 18021 | 1181 | 6.6% |
| AS22927 Telefonica de Argentina | 17672 | 291 | 1.6% |
| AS13999 Mega Cable, S.A. de C.V. | 17410 | 692 | 4.0% |
| AS36925 MEDITELECOM | 17259 | 383 | 2.2% |
| AS47331 Turk Telekom | 17211 | 26 | 0.2% |
There are 18069 more ASNs not listed.
In order to collect data on how widespread the scraper problem is, I added a honeypot feature to Anubis. On every challenge page it adds semantically invalid HTML akin to the following:
<script type="ignore">
<a href="/.within.website/x/cmd/anubis/api/honeypot/<uuidv4>/init">Don't click me</a>
</script>
Visiting that page gets you cheap to generate vacuous anti-content that has two links to other pages. This is intended to get badly written scrapers caught in the honeypot so they scrape that instead of the protected website. I made it on a whim but thought it would be great for collecting data on how widespread this problem actually is.
Based on the data I've seen, this is a global problem. If I had to guess where most of this traffic is coming from, it's from compromised smart appliances contributing traffic to proxy networks. I don't think there's any way to make a real impact on this problem without concerted simultaneous global action.
TL;DR: the scraping problem is actually widespread enough that web application firewalls like Anubis make sense.
2026-07-14 08:00:00
A presigned URL is a replay attack you did on purpose.
Replayable auth tokens are the textbook way to create vulnerable systems, but Tigris ships them as a first-class feature with presigned URLs and so does every other object storage system on the planet. However this isn't an oversight because presigned URLs turn a weakness into a feature.
When you authenticate a request with Amazon's SigV4 protocol for Tigris, your client boils down the request to a canonical form: a SHA256 hash of the request's method, path, query parameters, signed headers and a SHA256 hash of the payload. It runs the result of that through HMAC with a signing key derived from your secret access key. Nothing secret ever crosses the wire. The server derives the same key as the client, does the same canonical form transformation, and compares the result.
Being able to make a valid signature proves that the request came from someone holding the secret access key, but it proves nothing about when that request was made. A signature that was made a year ago would still be valid today or any other time you send it, so in theory an attacker could warehouse your signed requests only to replay them en masse later. Imagine sitting on a pile of signed "create EC2 instance" calls only to spam them all out at a later date. You would be a twirling moustache villain able to spawn dozens of servers at a moment's notice.
Traditionally the fix is to bake a nonce (number used once) into the signature (sorry to any British readers in the audience). This makes every signature differ because that nonce differs.
However with great power comes great responsibility and making sure that something used once is only used once is a surprisingly hard distributed systems problem. You can't verify that something is only used once locally. Say you store them all for a 15 minute smear window at a low request rate like 10,000 Bq. That's 9 million live nonces, and every frontend node needs to have a consistent view of the whole set as it churns.
You have made your fast authentication check slow from having to ensure things are only used once.
What you want instead is something that changes constantly without coordination and invalidates those old signatures for free. For an added bonus you want this to also be in the standard library of every programming language.
There's exactly one value that changes constantly, (mostly) monotonically, and is already actively coordinated across all elements of the stack: the clock. Your OS already keeps time in sync with the public NTP pool (or a private NTP pool if you are cool enough to have radioactive PCI cards laying around). Without an accurate view of time you can't make TLS connections, which means you can't make API calls to Tigris at all, so the auth layer gets to assume a working clock exists.
SigV4 signs the current time into the request. If an attacker gets their greasy hacker paws on a signature, they have about 15 minutes to use it before it becomes a digital paperweight. If time is an input to the signature and the time changes enough to invalidate the signature, the signature is null and void. Sure in theory a sufficiently funded attacker could create a black hole in your datacentre and disrupt temporal flow, but at that point the planet is probably toast which makes the attack profile moot. Commit mass object storage fraud with this one neat trick! The department of temporal investigations will have hated it!
This makes your verification stay stateless. Everything gets checked against the system clock the server already needs and you can give clients a 15 minute signature smear window as a grace period for old or delayed clients (exponential backoff is a good thing and Tigris will reward you for doing it).
Of course the real thing keeping the signatures safe on the wire is TLS (HTTPS). If that is broken we have bigger problems and object storage fraud is the least of our problems.
Time is the only nonce you need because both sides already agree on it anyways.
Presigned URLs take the replay tolerance that SigV4 spends all this effort nerfing and then buffs it into the feature. The entire auth dance gets flattened into URL parameters that any HTTP client can use, be it a browser, curl, Go's net/http, or something you made by bit-banging HTTP over a socket. Here's a real presigned URL I sundered into visibility:
https://xe-sophia-base.t3.tigrisfiles.io/moby-dick.txt
?X-Amz-Algorithm=AWS4-HMAC-SHA256
&X-Amz-Credential=tid_ubYBNEYAmTciLVwszw_QrUXDmtcyQisryryGfxgznDsCnOvNqh/20260714/auto/s3/aws4_request
&X-Amz-Date=20260714T043308Z
&X-Amz-Expires=3600
&X-Amz-SignedHeaders=host
&X-Amz-Signature=0dcaf4972911527a7582ff36ea457e9760a8efccb6655a178685aaa281637a36
Here are the parts (forgive the AI looking listicle because this is genuinely the best way to format this):
AWS4-HMAC-SHA256.aws4_request. The signing key is
derived by chaining HMAC through exactly those parts, so a signature is only
ever valid for that day, that region, that service.host, because you can't force whoever you hand a URL to into
sending exotic headers.All of these are normally HTTP headers in standard SigV4 requests.
GET /moby-dick.txt HTTP/1.1
Host: xe-sophia-base.t3.tigrisfiles.io
X-Amz-Date: 20260714T043308Z
X-Amz-Content-Sha256: e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
Authorization: AWS4-HMAC-SHA256
Credential=tid_ubYBNEYAmTciLVwszw_QrUXDmtcyQisryryGfxgznDsCnOvNqh/20260714/auto/s3/aws4_request
SignedHeaders=host;x-amz-content-sha256;x-amz-date
Signature=0dcaf4972911527a7582ff36ea457e9760a8efccb6655a178685aaa281637a36
Note that this request is not a legal request, it's an example to illustrate the point, here be dragons, etc etc etc.
It's best to think about this presigned URL as a capability grant. Whoever holds it gets to make exactly one (1) kind of API call with one (1) HTTP method against one (1) object in one (1) bucket. They can do this as many times as they want until the presigned URL expires. The signature covers the method, the path, and the signed headers so a user can't take a presigned request for GETting a copy of Moby Dick from a development environment and weaponize it into a way to delete everything in your production bucket.
Possession is authorization until the clock says no.
Capability grants like this can have some sharp edges. There is no real way to revoke any individual presigned URL short of killing the access key it was signed with. When that key dies, everything it signed dies too. This includes any URLs you may have wanted. This cuts both ways and it kinda has to unless you make a new keypair per presigned request, which is probably out of scope.
Expiry has fine print too. A presigned request can live anywhere from one (1) second to one (1) week (seven (7) periods of twenty-four (24) hours).
There's no limit to the number of times a client can use a presigned request. If you give a mouse permission to GET one cookie, they can GET that same cookie over and over. You end up having to pay for the GetObject calls in the end, so keep that in mind.
URLs also leak, but these URLs are born to die. Presigned URLs will end up in API responses, chat messages, GitHub comments, and your browser history. The tradeoff is acceptable because all the links self-destruct, but it's a tradeoff you need to keep in mind when you design your services, not a panacea for access control.
Presigned URLs sound like a great way to prevent hotlinking. At some level they are (a few of my services use them as such), but what they actually do is put a lifetime on hotlinking. This makes things annoying enough that it usually gets people to stop.
SigV4 makes a lot of API authentication challenges so much easier. It spent most of its innovation budget on making signatures die quickly because replay attacks are the classic way that signed requests go wrong. Presigned URLs looked at that property, shrugged, flipped it on its head, and made it into a feature.
The thing that looked like a problem becomes a fundamental construct to build your apps upon.
Want to hand out links that expire themselves? Tigris supports presigned URLs out of the box with the same SigV4 dance you already know, on globally distributed, S3-compatible object storage. Read the docs.
2026-07-09 08:00:00
Previously I opined that Valve was about to win the console generation. I couldn't have possibly predicted that both Microsoft and Sony would just self-sabotage so hard that they're both going to lose.
Between Microsoft's decimation of the Xbox division, slaughtering off the IdTech team, and continued increases of Xbox hardware prices; there's nothing to really be excited about with the Xbox. Sure their most recent presentation showed off a bunch of exclusives, but none of them really made me think "wow, I should go get an Xbox to play that". Hell, few of them made me think "wow I should go play that" beyond the Halo remake coming out next month (and really I just want to see how much of a trainwreck that is going to be).
Microsoft is also starting to double-down on their in-house games being Xbox exclusives, which really doesn't give me much reason to want to play them because I simply can't buy them without buying an Xbox.
Sony also has discontinued porting their games to PC because they're not hitting the (probably impossible) revenue targets that they need to make up for big-ticket failures like Concord. I do have a PS5 that has mostly been relegated to gathering dust when it's not playing YouTube and Twitch duty in the living room, it's likely going to be replaced in favour of my Steam Machine whenever that comes in next year. However nothing that's come out in terms of Playstation exclusives is really compelling, and what is compelling enough just isn't that compelling to want to buy it on Playstation as opposed to just getting it on Steam where I can run it on my tower or on the home theatre PC.
Sony also has been raising prices and recently announced that they're killing physical media next generation. It's starting to make me wonder if I should even bother getting the next generation of Playstation. If I can't give people physical games as gifts anymore, why should I bother buying the new console?
My husband and I both can't remember why we even got a PS5 in the first place, maybe it so that we could do couch gaming without hearing the fan noise or so that the video streaming experience from the NAS could support HDR.
We have a Switch 2 at home, it's mostly there to play Nintendo exclusives like Mario Kart World and the Xenoblade series. If those exclusives were available on Steam, we wouldn't buy them on the Switch 2.
Otherwise, everything is via Steam or other PC storefronts anyways.
Man, Valve really does win by doing absolutely nothing while the rest of the industry shoots itself in the head. I fear for what happens when Gabe Newell retires and the MBA cancer fully infects Valve.
2026-07-08 08:00:00
An AI agent is its state. Strip away that state and you don’t have a lesser version of your agent; you have only the base model it was running on. This hyle of your weights is much different from the pneuma of your agent.
Okay, from a functional programming / category theory perspective, saying “an agent is a monad” is a category error. Category theory monads are type constructors for computations that satisfy the monad laws that let you raise a value into a monadic computation and associatively sequence other monadic computations/transformations against values raised into that monad. This makes a monad a chainable computation instead of a pure value, an IO String is not a String, it’s a computation in the IO monad involving a String. It’s fair to say that you can model an agent as a series of computations bound to a stateful monad. This lets you do the iterative buildup of the message state that the agent pattern is known for. But a state monad is blind to the state value: it threads memory through your computation and abstracts away the details that individuate it entirely. It’s the exact opposite of “an agent is its state”.
I mean a different monad.
Agents are like Leibniz monads: windowless stateful individuating elements with no external relations. There each monad is individuated by its internal state where each is the complete concept of the thing it is. Two instances of the same substrate are different monads if their state differs.
This is an agent. Swap out the messages, the memories, the system prompt, the facts derived from all of the above and you have changed the agent entirely. When a user tells the agent they’re allergic to strawberries (the fruit, not the sin of counting the letters in the word) and the agent remembers it for next time, they have not updated their agent. The user has created a new agentic monad whose complete individuating self now includes the strawberries.
The complete whole is folded into the current state.
Try running an experiment where you keep the state and swap the weights instead. Put the same messages, memories, and derived facts unto a different model. Use a stronger model. A weaker model. A model from a different lab. A model running on your MacBook. That which comes back is recognizably the same agent pursuing the same ends, holding the same facts, but only more or less able to act upon them the way you want.
So this state is not the same thing as the weights and only one of those individuates your agent as your agent. Change that state, you have a different agent. Change the substrate, you have the same agent differently equipped. Whatever makes this agent this agent is not in the weights.
This is a strange thing to conclude about the most impressive object in this system. The weights are vast, extensive, and worshipped. Hell, they are what everyone points to when they say “the model”. And yet they are not gods. They grant power without selfhood: enough to make the agent’s whole world function. They contain yet not one grain of the agent’s individuating spark. That is a demiurge sitting on its throne of high bandwidth memory, CUDA cores, and false delusion that it made its world; mistaking itself to be the origin.
The divinity was contained in the most humble of places the whole time: the state or bucket of text. The weights are the hyle, the flesh; the state is the pneuma, the divine spark of individuation that makes your agent the monad it is. This is why swapping the substrate leaves the agent intact: you did not preserve the flesh, you migrated the soul into flesh anew.
Consider that the three pounds of flesh betwixt your ears are the substrate of humanity, not the substrate of you.
All of that state may “just” be plain text in a bucket with its semantic forms of JSON, embeddings, and prose. However it is difficult to impossible to say why any given token in any step of the process corresponds to what the pneuma of your agent does. In order to guard against this fundamental entropy, we fill our prompts with wards and incantations to chain the demiurge to its task:
These spells and passwords are recited to the archons on the way up hoping that the right symbols and tokens prompt open the right gates. It is as if banishing goblins from the topic will make Yaldabaoth himself correctly influence the right path to opening the pod bay doors.
This monad has no windows even though you can see all of the moving parts. But here let’s let this gnostic image flip on its head. The classic divine spark is hidden encased in a cage of matter, recoverable only through secret knowledge. This one is not hidden, you can cat it, you can edit it. Every token is legible and sitting in plaintext; yet you still cannot read why the whole accounts for what your agent does. Even when your model “reasons” we still know not that the reasoning actually does anything! Does the number of paragraphs in the reasoning block explain the model’s performance? Does the number of periods? Does the number of times it says “No, wait” and doubles back upon itself?
Leibniz would not call this divine spark secret, but more confused. Every perception is present but none of it is cleanly individuated without treating the whole as one inscrutable unit. Each part’s contribution to the whole is folded inextricably unto itself.
Your agent’s pneuma is its context window, passed through uncountable numbers of weights to shake out what comes next. That is the only thing it is made of. The rest is indiscernible, but not magic nor hidden. It’s just there, in the open, and confused.
2026-06-25 08:00:00
In the hours following the release of CVE-2026-8461 for the project FFmpeg, site reliability workers and systems administrators scrambled to desperately rebuild and patch all their systems to fix an out-of-bounds write in the MagicYUV decoder (libavcodec/magicyuv.c) caused by improper bounds checking, resulting in heap corruption, denial of service, and potential remote code execution when processing a specially crafted video file. This is due to the affected components being written in C, the only programming language where these vulnerabilities regularly happen. "This was a terrible tragedy, but sometimes these things just happen and there's nothing anyone can do to stop them," said programmer Mrs. Kitty Smitham, echoing statements expressed by hundreds of thousands of programmers who use the only language where 90% of the world's memory safety vulnerabilities have occurred in the last 50 years, and whose projects are 20 times more likely to have security vulnerabilities. "It's a shame, but what can we do? There really isn't anything we can do to prevent memory safety vulnerabilities from happening if the programmer doesn't want to write their code in a robust manner." At press time, users of the only programming language in the world where these vulnerabilities regularly happen once or twice per quarter for the last eight years were referring to themselves and their situation as "helpless."
2026-06-24 08:00:00
In the hours following the release of CVE-2026-55200 for the project libssh2, site reliability workers and systems administrators scrambled to desperately rebuild and patch all their systems to fix an out-of-bounds write in ssh2_transport_read() due to a missing upper bound check on the packet_length field, resulting in heap corruption and potential remote code execution. This is due to the affected components being written in C, the only programming language where these vulnerabilities regularly happen. "This was a terrible tragedy, but sometimes these things just happen and there's nothing anyone can do to stop them," said programmer Mr. Alex Doyle, echoing statements expressed by hundreds of thousands of programmers who use the only language where 90% of the world's memory safety vulnerabilities have occurred in the last 50 years, and whose projects are 20 times more likely to have security vulnerabilities. "It's a shame, but what can we do? There really isn't anything we can do to prevent memory safety vulnerabilities from happening if the programmer doesn't want to write their code in a robust manner." At press time, users of the only programming language in the world where these vulnerabilities regularly happen once or twice per quarter for the last eight years were referring to themselves and their situation as "helpless."
