2025-10-29 17:43:00
On the 25th of October Bear had its first major outage. Specifically, the reverse proxy which handles custom domains went down, causing custom domains to time out.
Unfortunately my monitoring tool failed to notify me, and it being a Saturday, I didn't notice the outage for longer than is reasonable. I apologise to everyone who was affected by it.
First, I want to dissect the root cause, exactly what went wrong, and then provide the steps I've taken to mitigate this in the future.
I wrote about The Great Scrape at the beginning of this year. The vast majority of web traffic is now bots, and it is becoming increasingly more hostile to have publicly available resources on the internet.
There are 3 major kinds of bots currently flooding the internet: AI scrapers, malicious scrapers, and unchecked automations/scrapers.
The first has been discussed at length. Data is worth something now that it is used as fodder to train LLMs, and there is a financial incentive to scrape, so scrape they will. They've depleted all human-created writing on the internet, and are becoming increasingly ravenous for new wells of content. I've seen this compared to the search for low-background-radiation steel, which is, itself, very interesting.
These scrapers, however, are the easiest to deal with since they tend to identify themselves as ChatGPT, Anthropic, XAI, et cetera. They also tend to specify whether they are from user-initiated searches (think all the sites that get scraped when you make a request with ChatGPT), or data mining (data used to train models). On Bear Blog I allow the first kinds, but block the second, since bloggers want discoverability, but usually don't want their writing used to train the next big model.
The next two kinds of scraper are more insidious. The malicious scrapers are bots that systematically scrape and re-scrape websites, sometimes every few minutes, looking for vulnerabilities such as misconfigured Wordpress instances, or .env and .aws files, among other things, accidentally left lying around.
It's more dangerous than ever to self-host, since simple mistakes in configurations will likely be found and exploited. In the last 24 hours I've blocked close to 2 million malicious requests across several hundred blogs.
What's wild is that these scrapers rotate through thousands of IP addresses during their scrapes, which leads me to suspect that the requests are being tunnelled through apps on mobile devices, since the ASNs tend to be cellular networks. I'm still speculating here, but I think app developers have found another way to monetise their apps by offering them for free, and selling tunnel access to scrapers.
Now, on to the unchecked automations. Vibe coding has made web-scraping easier than ever. Any script-kiddie can easily build a functional scraper in a single prompt and have it run all day from their home computer, and if the dramatic rise in scraping is anything to go by, many do. Tens of thousands of new scrapers have cropped up over the past few months, accidentally DDoSing website after website in their wake. The average consumer-grade computer is significantly more powerful than a VPS, so these machines can easily cause a lot of damage without noticing.
I've managed to keep all these scrapers at bay using a combination of web application firewall (WAF) rules and rate limiting provided by Cloudflare, as well as some custom code which finds and quarantines bad bots based on their activity.
I've played around with serving Zip Bombs, which was quite satisfying, but I stopped for fear of accidentally bombing a legitimate user. Another thing I've played around with is Proof of Work validation, making it expensive for bots to scrape, as well as serving endless junk data to keep the bots busy. Both of these are interesting, but ultimately are just as effective as simply blocking those requests, without the increased complexity.
With that context, here's exactly went wrong on Saturday.
Previously, the bottleneck for page requests was the web-server itself, since it does the heavy lifting. It automatically scales horizontally by up to a factor of 10, if necessary, but bot requests can scale by significantly more than that, so having strong bot detection and mitigation, as well as serving highly-requested endpoints via a CDN is necessary. This is a solved problem, as outlined in my Great Scrape post, but worth restating.
On Saturday morning a few hundred blogs were DDoSed, with tens of thousands of pages requested per minute (from the logs it's hard to say whether they were malicious, or just very aggressive scrapers). The above-mentioned mitigations worked as expected, however the reverse-proxy—which sits up-stream of most of these mitigations—became saturated with requests and decided it needed to take a little nap.
The big blue spike is what toppled the server. It's so big it makes the rest of the graph look flat.
This server had been running with zero downtime for 5 years up until this point.
Unfortunately my uptime monitor failed to alert me via the push notifications I'd set up, even though it's the only app I have that not only has notifications enabled (see my post on notifications), but even has critical alerts enabled, so it'll wake me up in the middle of the night if necessary. I still have no idea why this alert didn't come through, and I have ruled out misconfiguration through various tests.
This brings me to how I will prevent this from happening in the future.
This should be enough to keep everything healthy. If you have any suggestions, or need help with your own bot issues, send me an email.
The public internet is mostly bots, many of whom are bad netizens. It's the most hostile it's ever been, and it is because of this that I feel it's more important than ever to take good care of the spaces that make the internet worth visiting.
The arms race continues...
2025-10-22 19:23:00
Now that I've been home for about a month and have all my ducks back in a row, it's time I wrote about my trip to Istanbul for MicroConf Europe.
MicroConf is a conference for bootstrapped (non-VC-tracked) founders. And while I don't necessarily view Bear as a business, MicroConf is the closest there is to an Industry Event for someone like me.
First a note on Istanbul:
I arrived a week early to explore the city and see the sights. I'm a bit of a history nerd, so being at the crossroads of where it all happened in Europe—going back thousands of years—was quite spectacular. I get up early, and wandering the empty streets of the old city before the tour groups flooded in was quite special.
Also, the mosques dotting the skyline as viewed from the Bosporus are like nothing I've ever seen. It's amazing to see human effort geared towards creating beautiful buildings. I know it's not economically viable, but imagine if cities were built with beauty and a cohesive aesthetic in mind.
There were, however, a few negative characteristics of the city that grated at me, the main one being the hard separation of what is the tourist area and what isn't. Inside the old city all of the restaurants were clones, serving the same authentic Turkish food at 5x the reasonable price.
And scams were rife. I remember after a mediocre lunch at a mediocre restaurant, looking at the hand-written bill, the per-person cost came to about 2000TRY (roughly $50 at the time). The staff didn't speak English, and I wasn't going to throw all of my toys out of the cot via Google Translate, so I begrudgingly paid the bill and vowed never to eat there again. Similarly with taxis: it was impossible to take one as a foreigner without an attempted scam, to the extent the conference coordinator put out a PSA to use a specific transport company for rides instead of using the local taxis.
It's unfortunate when a city is unwelcoming in this manner, and it left a bad taste in my mouth.
Putting all of that aside, I still had a spectacular time. The main reason I came to this conference was to learn, get inspired, and soak up the vibes from other interesting people building interesting things. And I got exactly what I came for.
The talks and workshops were good, but what made the event shine was the in-between times spent with other attendees. The meals and walks, the hammam and sauna sessions. I found myself engaged from sunrise to sunset, notebook not far away, transcribing those notes during my downtime.
One of the attendees, Jesse Schoberg, runs a blogging platform as well, which focusses on embeddable blogs for organisations. It's called DropInBlog and is a really neat solution and platform. We chatted about what it's like running this kind of service, from bots to discoverability, and enjoyed the sunset on the terrace overlooking the Sea of Marmara. I can't think of a better place to talk shop.
I can't list all of the great conversations I had over those 3 days, but one standout to me was dinner with the ConversionFactory lads: Zach, Nick, and Corey. Not only were they one of the event sponsors, but were just great people to hang out with—and obviously incredibly proficient at their craft. After dinner on the last evening of the conference we crowded into the steam room to take advantage of the hotel's amenities that I'd paid way too much for. I got too hot, and mistaking the basin in the middle of the room for cooling off, managed to splash strong, menthol-infused water on my face. I immediately regretted it. My face, eyes, and nose started burning intensely with the icy-cold blaze of menthol and I was temporarily blinded. I had one of them hose off my face since I couldn't do anything in that state.
Product idea: Mace, but menthol.
One of my friends, Rob Hope, just arrived back from giving a talk at a conference in the US. When I met up with him for dumplings and dinner last week, it came up that, coincidentally, he had also just met the ConversionFactory lads on his most recent trip. I guess they get around.
Will I come back to MicroConf? Without a doubt. This has been inspiring, educational, and also quite validating. People were impressed with my projects, and surprised that I don't track visitors, conversions, and other important metrics. Bear is healthy, paying me a good salary while being aligned with my values, ethos, and lifestyle.
I guess I'm running on vibes, and the vibes are good.
2025-10-13 21:04:00
I read an article yesterday, stating that on average, people spend 4 hours and 37 minutes on their phones per day1, with South Africans coming in fourth highest in the world at a whopping 5 hours and 11 minutes2.
This figure seems really high to me. If we assume people sleep roughly 8 hours per day, that means that one third of their day is spent on their phones. If we also assume people work 8 hours per day (ignoring the fact that they may be using their phones during work hours), that suggests that people spend over half of their free time (and up to 65% of it) glued to their screens.
I never wanted to carry the internet around in my pocket. It's too distracting and pulls me out of the present moment, fracturing my attention. I've tried switching to old-school black and white phones before, but always begrudgingly returned to using a smartphone due to the utility of it. The problem, however, is that it comes with too many attention sinks tucked in alongside the useful tools.
I care about living an intentional and meaningful life, nurturing relationships, having nuanced conversations, and enjoying the world around me. I don't want to spend this limited time I have on earth watching short form video and getting into arguments on Twitter.
This is what I enjoy. Picture taken yesterday in Scarborough, South Africa.
I've written at length about how I manage my digital consumption, from turning off notifications to forgoing social media entirely. The underlying premise here is that if you're trying to lose weight, you shouldn't carry cookies around in your pockets. And my phone is the bag of cookies in this metaphor.
We're wired to seek out distraction, novel information, and entertainment, and avoid boredom at all costs. But boredom is where creativity and self-reflection do their best work. It's why "all the best ideas come when you're in the shower"—we don't usually take our phones with us into the shower (yet).
According to Screen Time on my iPhone, on average I spend 30 minutes per day on it, which I think is reasonable, especially considering the most-used apps are by-and-large utility apps like banking and messages. This isn't because I have more self-control than other people. I don't think I do. It's because I know myself, and have set up my digital life to be a positive force, and not an uninspired time-sink.
There are many apps and systems to incentivise better relationships with our phones, mostly based around time limits. But these are flawed in three ways:
The only way I've found to have a good relationship with my phone is to make it as uninteresting as possible. The first way is to not have recommendation media (think Instagram, TikTok, and all the rest). I'm pro deleting these accounts completely, because it's really easy to re-download the apps on a whim, or visit them in-browser. However some people have found that having them on a dedicated device works by isolating those activities. Something like a tablet at home that is "the only place you're allowed to use Instagram". I can't comment too much on this route, but it seems reasonable.
My biggest time sink over the past few years has been YouTube. The algorithm knew me too well and would recommend video after engaging, but ultimately useless video. I could easily burn an entire evening watching absolute junk—leaving me feeling like I'd just wasted what could have otherwise been a beautiful sunset or a tasty home-cooked lasagne. However, at the beginning of this year I learnt that you can turn off your YouTube watch history entirely, which means no recommendations. Here's what my YouTube home screen now looks like:
Without the recommendations I very quickly run out of things to watch from the channels I'm subscribed to. It's completely changed my relationship with YouTube since I only watch the videos I actually want to watch, and none of the attention traps. You can turn off your YouTube watch history here, and auto delete your other Google history (like historic searches and navigation) here, which I think is just good practice.
I also used my adblocker, AdGuard on Safari which has a useful "block element" feature, to block the recommended videos on the right of YouTube videos. I use this feature to hide shorts as well, since I have no interest in watching them either, and YouTube intentionally makes them impossible to remove. If you're interested in a similar setup, here are the selectors I use to block those elements:
youtube.com###items > ytd-item-section-renderer.style-scope.ytd-watch-next-secondary-results-renderer:last-child
youtube.com###sections
youtube.com##[is-shorts]
youtube.com###secondary
The only media that I do sometimes consume on my phone are my RSS feeds, but it's something I'm completely comfortable with since it's explicitly opt-in by design and low volume.
While I still have the twitch to check my phone when I'm waiting for a coffee, or in-between activities—because my brain's reward system has been trained to do this—I'm now rewarded with nothing. Over time, I find myself checking my phone less and less. Sometimes I notice the urge, and just let it go, instead focusing on the here and now.
I think that while the attention-span-degrading effects of recommendation media are getting most of the headlines, what isn't spoken about as much is the sheer number of hours lost globally to our phones (3.8 million years per day, according to my back-of-the-napkin-math). And while people may argue that this could involve productive work or enjoyable leisure, I suspect that the vast (vast!) majority of that time is short-form entertainment.
My solution may sound overkill to many people, but I can say with absolute certainty that it has turned me into a more present, less distracted, and more optimistic person. I have much more time to spend in nature, with friends, or on my hobbies and projects. I can't imagine trading it in for a tiny screen, ever.
Give it a try.
Happily on the beach for sunset.
2025-10-03 15:30:00
Most people who read my blog and know me for the development of Bear Blog are surprised to learn that I have another software project in the art and design space. It's called JustSketchMe and is a 3D modelling tool for artists to conceptualise their artwork before putting pencil to paper.
It's a very niche tool (and requires some serious explanation to some non-illustrators involving a wooden mannequin and me doing some dramatic poses), however when provided as a freemium tool to the global population of artists, it's quite well used.
Similar to Bear, I make it free to everyone, with the development being funded through a "pro" tier. Conversely, since it is a standalone app it has a bit of a weakness, which is what this post is about.
I noticed, back in 2021, that when Googling "justsketchme" the top 3 autocompletes were "justsketchme crack", "justsketchme pro cracked", and "justsketchme apk". On writing this post, I checked that this still holds true, and it's fairly similar 4 years later.
The meaning of this is obvious. A lot of people are trying to pirate JustSketchMe. However, instead of feeling frustrated (okay, I did feel a bit frustrated at first) I had a bright idea to turn this apparent negative into a positive.
I created two pages with the following titles and the appropriate subtitles to get indexed as a pirate-able version of JustSketchMe:
These pages rank as the first result on Google for the relevant search terms. Then on the page itself I tongue-in-cheek call out the potential pirate. I then acknowledge that we're in financially trying times and give them a discount code.
And you know what?
That discount code is the most used discount code on JustSketchMe! By far! No YouTube sponsor, nor Black Friday special even comes close.
In some ways this is taking advantage of a good search term. In others it's showing empathy and adding delight, creating a positive incentive to purchase to someone who otherwise wouldn't have.
The discount code is PIRACYKILLS. I'll leave it active for a while. 👮🏻♂️
2025-09-19 17:45:00
Hi everyone,
Just some updates about upcoming travel and events; responses to the recent post about social media platforms; and some thoughts about the Bear license update.
I'll be heading to Istanbul next week for Microconf, which is a yearly conference where non-venture track founders get together, explore a new city, and learn from one another. I had meant to go to the one last year in Croatia, but had just gotten back from two months in Vietnam, and the thought of travelling again so soon felt daunting.
I've made two Bear t-shirts for the conference. One light and one dark mode—inspired by the default Bear theme. Let's see if anyone notices!
If you live in Istanbul and want to grab coffee, I'm keen! If you've previously travelled to Istanbul and have recommendations for me, please pop me an email. I have a few days to explore the city.
I received so many great emails from people about my post on slow social media. There are many great projects underway at the moment, and many great projects that unfortunately didn't make it. Some notable standouts to me:
Unfortunately no longer with us:
Here are some projects that are up-and-running. These aren't necessarily all "social networks", nor necessarily viable at scale, but each of them has an element or two that makes them interesting.
There were many other projects in various states of development that I haven't had the time to fully explore yet, but I'll get to them over the next week or so.
Somehow my post about the change in the Bear source code license exploded on Hacker News, Tildes, Lobsters, and Reddit, and has been read over 120,000 times.
The vast majority of the emails and responses I received were positive, but about 10% of the Hacker News crowd got really mean about it without taking the time to understand the context. I guess I can't expect empathy from 120,000 people.
Regardless, if you're interested in reading about the controversy The Grizzly Gazette covered it quite well.
While I don't feed the trolls on Hacker News (and find comments to be a pretty poor place to have nuanced discussions in general), I'd like to respond to a few of the main critiques here.
While Bear (the platform) has a community—and a very good one at that; the source-code part of Bear has never been community oriented. Bear doesn't accept code contributions and the code has been written by me personally. I have not engaged in the exploitation of free developer labour, nor used it being open-source as marketing material.
I suspect that these kinds of comments arose from the (justified, but ultimately misguided) assumption that the Bear project had active contributors and a community surrounding the code itself.
Yes, I shouldn't have released Bear on an MIT license in the beginning. I didn't even think about licenses when I launched Bear in 2020 and just used the default. I also didn't expect free-ride competition to be an issue in this space. So, this is a justifiable criticism, even if it feels like it was made in bad faith.
This was a common criticism, but fails to resolve the main reason for this change: people forking and hosting a clone of Bear under a new name, social elements and all. The AGPLv3 license only specifies that they would need to release their version of the code under the same license. This doesn't dissuade free-ride competition, at least not in this context.
Bear's source code was never meant to be used by people to set up competing services to Bear. It was there to ensure that people understand what's going on under the hood, and to make the platform auditable. I specify this in the CONTRIBUTIONS.md that was last updated 2 years ago.
In summary, Bear is a platform, not a piece of self-hostable software. I think these criticisms are justified sans-context. With context, I don't think the same arguments would have been made. But Hacker News is well known for nasty comments based on the title of the post alone.
Aaand we're done! Lots of updates. Please feel free to email me your thoughts, recommendations, or anything else. If you haven't dug through my past posts, here're a few lesser-read posts that I enjoyed writing:
If you haven't subscribed to my blog, you can do it via the RSS feed or email.
Have a goodie!
2025-09-16 17:44:00
People often assume that I hate social media. And they'd be forgiven for believing that, since I am overtly critical of current social media platforms and the effects they have on individuals and society; and deleted all of my social media accounts back in 2019.
However, the underlying concept of social media is something I resonate with: Stay connected with the people you care about.
It's just that the current form of social media is bastardised, and not social at all. Instead of improving relationships and fostering connection, they're advertisement-funded content mills which are explicitly designed and continually refined to keep you engaged, lonely, and unhappy. And once TikTok figured out that short-form video with a recommendation engine is digital crack, all other social media platforms quickly sprang into action to copy their secret sauce.
Meta basically turned Instagram and Facebook from 'connecting with friends' into 'doom-scrolling random content'. Even Pinterest is starting to look like TikTok! They followed user engagement, but not the underlying preferences of their users. I posit that any for-profit social media will eventually degrade into recommendation media over time.
I don't think most people using these platforms understand that they are the product. Instagram isn't built for you. It's built for marketers. It's built for celebrities to capitalise on their audiences. It's built for politicians and their cronies to sway sentiment. It's built to be as addictive as possible, and to capitalise on your insecurity and uncomfortability.
Imagine that, society and politics are on the rocks all so a fitness influencer can sell you their "Abs in 30 days" training program.
These platforms are the quintessential poster child for late-stage capitalism.
Okay, now that we've established what the problems with current platforms are—what would a non-evil social media platform look like?
I'd love to see everyone running a blog, and subscribing to the people they care about via RSS. But unfortunately this doesn't scale since it requires effort to put your thoughts down in writing longer than 255 characters. I have many friends who don't even know I have a blog, or what an RSS reader is.
So while everyone blogging may be the ideal we can aspire to, let's design a hypothetical social media platform that takes the good aspects of current social media, while creating pro-social incentives.
The platform should be about:
The platform should NOT be about:
In my opinion, as soon as there is the ability for commercial interests to take hold, they will. The "follow" mechanism is a key part of that. I propose that instead of followers we should regress back to the "friend" or "connection" system where there is a symmetric relationship where both people have to agree to the connection. There is no good reason to have "followers" on a platform that is trying to improve relationships. "Following" is purely for egotistical or financial gain and breeds parasocial relationships.
I think there should also be a reasonable cap on the number of connections that can be made. Something like 300 friends sounds right. Any more than that and you're a collector, and not using the platform to foster connection.
This feature alone already removes 90% of the marketing interests in the platform. Do you want to make a connection, but are maxed out? You'll need to unfriend someone first.
The second necessary element would be a chronological feed with posts from your connections. This turns the platform from an engagement engine into a way to keep up with what everyone else is doing, but importantly, gives you a natural "end" to the feed when you start seeing posts you've already viewed. This way when you start scrolling there's an explicit stopping point.
Relatedly, pagination is more humane than infinite-scroll since it gives users a natural breathing point where they can decide whether they want to keep going. Infinite-scroll is such an obvious user-trap, and I view any website doing it as not having its user's best interests at heart.
And finally, there should be a reasonable cap on the number of times a user can post per day. Roughly 5 times per day feels like the upper threshold of what you can post while being intentional about what it is you're posting. This will keep the feed reasonably populated without one or two people completely overwhelming it.
The rest of the platform can be optimised to be as easy-to-use as possible. Something like a mixture between the old Instagram and Twitter, with comments and reactions. No reels or any other recommendation system to keep people engaged to death. And no analytics, since that would be optimising for reach and engagement instead of the stated goal of connection.
Do I expect a platform like this to succeed? Not by the traditional metrics of success. In the real world it would exist alongside the content mills, which are exciting by design and competing for attention. Could it work in niche groups, or amongst intentional people who are sick of the current platforms? Maybe.
Naturally, a project like this would have to be funded somehow, and unfortunately very few people are willing to pay $5 per month for software services, even if they use it every day. However, I suspect that a social media platform like this would be manageable enough that a small team could run it fairly cheaply and profitably if they're creative. Perhaps with nothing but donations.
Who will create this egalitarian social media? Not me, that's for sure. I already have my fair share of work moderating the Bear discovery feed, to the extent I've had to bring on a second moderator (hello Sheena!) to keep it clean of spam and other nasty things that free services on the internet attract.
That being said, I would love to see something like this. I'd love to be able to stay connected with friends and family abroad without having my attention sold to the highest bidder.
If anyone is working on something like this, I'd be happy to consult.
--
edit: I've collated a bunch of responses as well as some neat projects that were brought to my attention in Miscellaneous updates.