2025-03-31 08:00:00
Every ending marks a new beginning, and today, is the beginning of a new chapter for me. Ten years ago I took a leap into the unknown, today I take another. After a decade of working on Sentry I move on to start something new.
Sentry has been more than just a job, it has been a defining part of my life. A place where I've poured my energy, my ideas, my heart. It has shaped me, just as I've shaped it. And now, as I step away, I do so with immense gratitude, a deep sense of pride, and a heart full of memories.
I've known David, Sentry's co-founder (alongside Chris), long before I was ever officially part of the team as our paths first crossed on IRC in the Django community. Even my first commit to Sentry predates me officially working there by a few years. Back in 2013, over conversations in the middle of Russia — at a conference that, incidentally, also led to me meeting my wife — we toyed with the idea of starting a company together. That exact plan didn't materialize, but the seeds of collaboration had been planted.
Conversations continued, and by late 2014, the opportunity to help transform Sentry (which already showed product market fit) into a much bigger company was simply too good to pass up. I never could have imagined just how much that decision would shape the next decade of my life.
For me, Sentry's growth has been nothing short of extraordinary. At first, I thought reaching 30 employees would be our ceiling. Then we surpassed that, and the milestones just kept coming — reaching a unicorn valuation was something I once thought was impossible. While we may have stumbled at times, we've also learned immensely throughout this time.
I'm grateful for all the things I got to experience and there never was a dull moment. From representing Sentry at conferences, opening an engineering office in Vienna, growing teams, helping employees, assisting our licensing efforts and leading our internal platform teams. Every step and achievement drove me.
Yet for me, the excitement and satisfaction of being so close to the founding of a company, yet not quite a founder, has only intensified my desire to see the rest of it.
Walking away from something you love is never easy and leaving Sentry is hard. Really hard. Sentry has been woven into the very fabric of my adult life. Working on it hasn't just spanned any random decade; it perfectly overlapped with marrying my wonderful wife, and growing our family from zero to three kids.
And will it go away entirely? The office is right around the corner afterall. From now on, every morning, when I will grab my coffee, I will walk past it. The idea of no longer being part of the daily decisions, the debates, the momentum — it feels surreal. That sense of belonging to a passionate team, wrestling with tough decisions, chasing big wins, fighting fires together, sometimes venting about our missteps and discussing absurd and ridiculous trivia became part of my identity.
There are so many bright individuals at Sentry, and I'm incredibly proud of what we have built together. Not just from an engineering point of view, but also product, marketing and upholding our core values. We developed SDKs that support a wide array of platforms from Python to JavaScript to Swift to C++, lately expanding to game consoles. We stayed true to our Open Source principles, even when other options were available. For example, when we needed an Open Source PDB implementation for analyzing Windows crashes but couldn't find a suitable solution, we contributed to a promising Rust crate instead of relying on Windows VMs and Microsoft's dbghelp. When we started, our ingestion system handled a few thousand requests per second — now it handles well over a million.
While building an SDK may seem straightforward, maintaining and updating them to remain best-in-class over the years requires immense dedication. It takes determination to build something that works out of the box with little configuration. A lot of clever engineering and a lot of deliberate tradeoffs went into the product to arrive where it is. And ten years later, is a multi-product company. What started with just crashes, now you can send traces, profiles, sessions, replays and more.
We also stuck to our values. I'm pleased that we ran experiments with licensing despite all the push back we got over the years. We might not have found the right solution yet, but we pushed the conversation. The same goes for our commitment to funding of dependencies.
I feel an enormous amount of gratitude for those last ten years. There are so many people I owe thanks to. I owe eternal thanks to David Cramer and Chris Jennings for the opportunity and trust they placed in me. To Ben Vinegar for his unwavering guidance and support. To Dan Levine, for investing in us and believing in our vision. To Daniel Griesser, for being an exceptional first hire in Vienna, and shepherding our office there and growing it to 50 people. To Vlad Cretu, for bringing structure to our chaos over the years. To Milin Desai for taking the helm and growing us.
And most of all, to my wonderful wife, Maria — who has stood beside me through every challenge, who has supported me when the road was uncertain, and who has always encouraged me to forge my own path.
To everyone at Sentry, past and present — thank you. For the trust, the lessons, the late nights, the victories. For making Sentry what it is today.
I'm fully aware it's a gamble to believe my next venture will find the same success as Sentry. The reality is that startups that achieve the kind of scale and impact Sentry has are incredibly rare. There's a measure of hubris in assuming lightning strikes twice, and as humbling as that realization is, it also makes me that much more determined. The creative spark that fueled me at Sentry isn't dimming. Not at all in fact: it burns brighter fueld by the feeling that I can explore new things, beckoning me. There's more for me to explore, and I'm ready to channel all that energy into a new venture.
Today, I stand in an open field, my backpack filled with experiences and a renewed sense of purpose. That's because the world has changed a lot in the past decade, and so have I. What drives me now is different from what drove me before, and I want my work to reflect that evolution.
At my core, I'm still inspired by the same passion — seeing others find value in what I create, but my perspective has expanded. While I still take great joy in building things that help developers, I want to broaden my reach. I may not stray far from familiar territory, but I want to build something that speaks to more people, something that, hopefully, even my children will find meaningful.
Watch this space, as they say.
2025-03-27 08:00:00
Three years ago I shared the As-Any Hack on this blog. That hack is a way to get upcasting to supertraits working on stable Rust. To refresh your memory, the goal was to make something like this work:
#[derive(Debug)]
struct AnyBox(Box<dyn DebugAny>);
trait DebugAny: Any + Debug {}
impl<T: Any + Debug + 'static> DebugAny for T {}
The problem? Even though DebugAny inherits from Any, Rust wouldn't let you use methods from Any on a dyn DebugAny. So while you could call DebugAny methods just fine, trying to use downcast_ref from Any (the reason to use Any in the first place) would fail:
fn main() {
let any_box = AnyBox(Box::new(42i32));
dbg!(any_box.0.downcast_ref::<i32>()); // Compile error
}
The same would happen if we tried to cast it into an &dyn Any? A compile error again:
fn main() {
let any_box = AnyBox(Box::new(42i32));
let any = &*any_box.0 as &dyn Any;
dbg!(any.downcast_ref::<i32>());
}
But there is good news! As of Rust 1.86, this is finally fixed. The cast now works:
[src/main.rs:14:5] any.downcast_ref::<i32>() = Some(
42,
)
At the time of writing, this fix is in the beta channel, but stable release is just around the corner. That means a lot of old hacks can finally be retired. At least once your MSRV moves up.
Thank you so much to everyone who worked on this to make it work!
For completeness' sake here is the extension map from the original block post cleaned up so that it does not need the as-any hack:
use std::any::{Any, TypeId};
use std::cell::{Ref, RefCell, RefMut};
use std::collections::HashMap;
use std::fmt::Debug;
trait DebugAny: Any + Debug {}
impl<T: Any + Debug + 'static> DebugAny for T {}
#[derive(Default, Debug)]
pub struct Extensions {
map: RefCell<HashMap<TypeId, Box<dyn DebugAny>>>,
}
impl Extensions {
pub fn insert<T: Debug + 'static>(&self, value: T) {
self.map
.borrow_mut()
.insert(TypeId::of::<T>(), Box::new(value));
}
pub fn get<T: Default + Debug + 'static>(&self) -> Ref<'_, T> {
self.ensure::<T>();
Ref::map(self.map.borrow(), |m| {
m.get(&TypeId::of::<T>())
.and_then(|b| (&**b as &dyn Any).downcast_ref())
.unwrap()
})
}
pub fn get_mut<T: Default + Debug + 'static>(&self) -> RefMut<'_, T> {
self.ensure::<T>();
RefMut::map(self.map.borrow_mut(), |m| {
m.get_mut(&TypeId::of::<T>())
.and_then(|b| ((&mut **b) as &mut dyn Any).downcast_mut())
.unwrap()
})
}
fn ensure<T: Default + Debug + 'static>(&self) {
if self.map.borrow().get(&TypeId::of::<T>()).is_none() {
self.insert(T::default());
}
}
}
2025-03-23 08:00:00
Sometimes in Rust, you need to convert a string into a value of a specific type (for example, converting a string to an integer).
For this, the standard library provides the rather useful FromStr trait. In short, FromStr can convert from a &str into a value of any compatible type. If the conversion fails, an error value is returned. It's unfortunately not guaranteed that this value is an actual Error type, but overall, the trait is pretty useful.
It has however a drawback: it takes a &str and not a String which makes it wasteful in situations where your input is a String. This means that you will end up with a useless clone if do not actually need the conversion. Why would you do that? Well consider this type of API:
let arg1: i64 = parser.next_value()?;
let arg2: String = parser.next_value()?;
In such cases, having a conversion that works directly with String values would be helpful. To solve this, we can introduce a new trait: FromString, which does the following:
We start by defining a type alias for our error:
pub type Error = Box<dyn std::error::Error + Send + Sync + 'static>;
You can be more creative here if you want. The benefit of using this directly is that a lot of types can be converted into that error, even if they are not errors themselves. For instance a FromStr that returns a bare String as error can leverage the standard library's blanket conversion implementation to Error.
Then we define the FromString trait:
pub trait FromString: Sized {
fn from_string(s: String) -> Result<Self, Error>;
}
To implement it, we provide a blanket implementation for all types that implement FromStr, where the error can be converted into our boxed error. As mentioned before, this even works for FromStr where Err: String. We also add a special case for when the input and output types are both String, using transmute_copy to avoid a clone:
use std::any::TypeId;
use std::mem::{ManuallyDrop, transmute_copy};
use std::str::FromStr;
impl<T> FromString for T
where
T: FromStr<Err: Into<Error>> + 'static,
{
fn from_string(s: String) -> Result<Self, Error> {
if TypeId::of::<T>() == TypeId::of::<String>() {
Ok(unsafe { transmute_copy(&ManuallyDrop::new(s)) })
} else {
T::from_str(&s).map_err(Into::into)
}
}
}
Why transmute_copy? We use it instead of the regular transmute? because Rust requires both types to have a known size at compile time for transmute to work. Due to limitations a generic T has an unknown size which would cause a hypothetical transmute call to fail with a compile time error. There is nightly-only transmute_unchecked which does not have that issue, but sadly we cannot use it. Another, even nicer solution, would be to have specialization, but sadly that is not stable either. It would avoid the use of unsafe though.
We can also add a helper function to make calling this trait easier:
pub fn from_string<T, S>(s: S) -> Result<T, Error>
where
T: FromString,
S: Into<String>,
{
FromString::from_string(s.into())
}
The Into might be a bit ridiculous here (isn't the whole point not to clone?), but it makes it easy to test this with static string literals.
Finally here is an example of how to use this:
let s: String = from_string("Hello World").unwrap();
let i: i64 = from_string("42").unwrap();
Hopefully, this utility is useful in your own codebase when wanting to abstract over string conversions.
If you need it exactly as implemented, I also published it as a simple crate.
Postscriptum:
A big thank-you goes to David Tolnay and a few others who pointed out that this can be done with transmute_copy.
Another note: TypeId::of call requires V to be 'static. This is okay for this use, but there are some hypothetical cases where this is not helpful. In that case there is the excellent typeid crate which provides a ConstTypeId, which is like TypeId but is constructible in const in stable Rust.
2025-02-20 08:00:00
This week I had a conversation with one of our engineers about “shitty code” which lead me to sharing with him one of my more unusual inspirations: Flamework, a pseudo framework created at Flickr.
There are two driving passions in my work. One is the love of creating beautiful, elegant code — making Open Source libraries and APIs that focus on clear design and reusability. The other passion is building quick, pragmatic solutions for real users (who may not even be developers). The latter usually in a setting of building a product, where the product is not the code. Here, speed and iteration matter more than beautiful code or reusability, because success hinges on shipping something people want.
Flamework is in service of the latter, and in crass violation of the former.
Early on, I realized that creating reusable code and directly solving problems for users are often at odds. My first clue came when I helped run the German ubuntuusers website. It was powered by a heavily modified version of phpBB, which despite how messy it was, scaled to a large user base when patched properly. It was messy, but easy to adjust. The abstractions were one layer deep.
Back then, me and a friend tried to replace it by writing my own bulletin board software, Pocoo. Working in isolation, without users, led me down a path of over-engineering. While we learned a lot and ended up creating popular Open Source libraries (like Jinja, Werkzeug and Pygments), Pocoo never became a solid product. Later, my collaborators and I rebuilt ubuntuusers, without the goal of making it into a reusable product. That rewrite shipped successfully and it lives to this very day.
But it took me years to fully realize what was happening here: reusability is not that important when you’re building an application, but it’s crucial when you’re building a library or framework.
If you are unfamiliar with Flamework you should watch a talk that Cal Henderson gave in 2008 at DjangoCon (Why I hate Django). He talked about scale and how Django didn't solve for it. He enumerated all the things important to him: sharding, using custom sequences for primary keys, forgoing joins and foreign keys, supporting database replication setups, denormalizing data to the extreme. This is also were I first learned about the possibility of putting all session data into cookies via signing. It was a memorable talk for me because it showed me that there are shortcomings. Django (which I used for ubuntuusers) had beautiful APIs but at the time solved for little of that Cal needed. The talk really stuck with me.
At the time of the talk, Flamework did not really exist. It was more of an idea and principles of engineering at Flickr.
A few years later, Flamework appeared on GitHub, not as an open-sourced piece of Flickr code but as a reimplementation of those same ideas. You can explore its repository and see code like this:
function _db_update($tbl, $hash, $where, $cluster, $shard){
$bits = array();
foreach(array_keys($hash) as $k){
$bits[] = "`$k`='$hash[$k]'";
}
return _db_write("UPDATE $tbl SET ".implode(', ',$bits)." WHERE $where", $cluster, $shard);
}
Instinctively it makes me cringe. Is that a SQL injection? Well you were supposed to use the PHP addslashes function beforehand. But notice how it caters to sharding and clustering directly in the query function.
Code like this often triggers a visceral reaction, especially in engineers who prize clean design.
How does something like that get created? Cal Henderson described Flickr's principle as “doing the dumbest possible thing that will work.” Maybe “dumb” is too strong — “simple” might be more apt. Yet simplicity can look messy to someone expecting a meticulously engineered codebase. This is not at all uncommon and I have seen it over and over. The first large commercial project that got traction that I ever worked on (Plurk) was also pretty pragmatic and messy inside. My former colleague Ben Vinegar also recently shared a story of early, messy FreshBooks code and how he came to terms with it. Same story at Sentry. We moved fast, we made a mess.
None of this is surprising in retrospective. Perfect code doesn't guarantee success if you haven't solved a real problem for real people. Pursuing elegance in a vacuum leads to abandoned side projects or frameworks nobody uses. By contrast, clunky but functional code often comes with just the right compromises for quick iteration. And that in turn means a lot of messy code powers products that people love — something that's a far bigger challenge.
I have shown Flamework's code to multiple engineers over the years and it usually creates such a visceral response. It blind sights one by seemingly disregarding all rules of good software engineering.
That makes Flamework serve as a fascinating Rorschach test for engineers. Are you looking at it with admiration for the focus on some critical issues like scale, the built-in observability and debugging tools. Or are you judging it, and its creators, for manually constructing SQL queries, using global variables, not using classes and looking like messy PHP4 code? Is it a pragmatic tool, intentionally designed to iterate quickly at scale, or is it a naive mess made by unskilled developers?
Would I use Flamework? Hello no. But I appreciate the priorities behind it. If these ugly choices help you move faster, attract users and validate the product, then a rewrite, or large refactorings later are a small price to pay.
At the end of the day, where you stand on “shitty code” depends on your primary goal:
Both mindsets are valid, but they rarely coexist harmoniously in a single codebase. Flamework is a reminder that messy, simple solutions can be powerful if they solve real problems. Eventually, when the time is right, you can clean it up or rebuild from the ground up.
The real challenge is deciding which route to take — and when. Even with experience, it is can be hard to know when to move from quick fixes to more robust foundations. The principles behind Flamework are also reflected in Sentry's development philosophy. One more poignant one being “Embrace the Duct Tape”. Yet as Sentry matured, much of our duct tape didn't stand the test of time, and was re-applied at moments when the real solution would have been a solid foundation poured with concrete.
That's because successful projects eventually grow up. What let you iterate fast in the beginning might eventually turn into an unmaintainable mess and will be rebuilt from the inside out.
I personally would never have built Flamework, it repulses me a bit. At the same time, I have a enormous respect for the people who build it. Their work and thinking has shaped how I solve problems and think of product engineering.
2025-02-08 08:00:00
The concept of purity — historically a guiding principle in social and moral contexts — is also found in passionate, technical discussions. By that I mean that purity in technology translates into adherence to a set of strict principles, whether it be functional programming, test-driven development, serverless architectures, or, in the case of Rust, memory safety.
Rust positions itself as a champion of memory safety, treating it as a non-negotiable foundation of good software engineering. I love Rust: it's probably my favorite language. It probably won't surprise you that I have no problem with it upholding memory safety as a defining feature.
Rust aims to achieve the goal of memory safety via safe abstractions, a compile time borrow checker and a type system that is in service of those safe abstractions. It comes as no surprise that the Rust community is also pretty active in codifying a new way to reason about pointers. In many ways, Rust pioneered completely new technical approaches and it it widely heralded as an amazing innovation.
However, as with many movements rooted in purity, what starts as a technical pursuit can evolve into something more ideological. Similar to how moral purity in political and cultural discourse can become charged, so does the discourse around Rust, which has been dominated by the pursuit of memory safety. Particularly within the core Rust community itself, discussion has moved beyond technical merits into something akin to ideological warfare. The fundamental question of “Is this code memory safe?”, has shifted to “Was it made memory safe in the correct way?”. This distinction matters because it introduces a purity test that values methodology over outcomes. Safe C code, for example, is often dismissed as impossible, not necessarily because it is impossible, but because it lacks the strict guarantees that Rust's borrow checker enforces. Similarly, using Rust’s unsafe blocks is increasingly frowned upon, despite their intended purpose of enabling low-level optimizations when necessary.
This ideological rigidity creates significant friction when Rust interfaces with other ecosystems (or gets introduced there), particularly those that do not share its uncompromising stance. For instance, the role of Rust in the Linux kernel has been a hot topic. The Linux kernel operates under an entirely different set of priorities. While memory safety is important there is insufficient support for adopting Rust in general. The kernel is an old project and it aims to remain maintainable for a long time into the future. For it to even consider a rather young programming language should be seen as tremendous success for Rust and also for how open Linus is to the idea.
Yet that introduction is balanced against performance, maintainability, and decades of accumulated engineering expertise. Many of the kernel developers, who have found their own strategies to write safe C for decades, are not accepting the strongly implied premise that their work is inherently flawed simply because it does not adhere to Rust's strict purity rules.
Tensions rose when a kernel developer advocating for Rust's inclusion took to social media to push for changes in the Linux kernel development process. The public shaming tactic failed, leading the developer to conclude:
“If shaming on social media does not work, then tell me what does, because I'm out of ideas.”
It's not just the kernel where Rust's memory safety runs up against the complexities of the real world. Very similar feelings creep up in the gaming industry where people love to do wild stuff with pointers. You do not need large disagreements to see the purist approach create some friction. A recent post of mine for instance triggered some discussions about the trade-offs between more dependencies, and moving unsafe to centralized crates.
I really appreciate that Rust code does not crash as much. That part of Rust, among many others, makes it very enjoyable to work with. Yet I am entirely unconvinced that memory safety should trump everything, at least at this point in time.
What people want in the Rust in Linux situation is for the project leader to come in to declare support for Rust's call for memory safety above all. To make the detractors go away.
Hearing this call and discussion brings back memories. I have lived through a purity driven shift in a community before. The move from Python 2 to Python 3 started out very much the same way. There was an almost religious movement in the community to move to Python 3 in a ratcheting motion. The idea that you could maintain code bases that support both 2 and 3 were initially very loudly rejected. I took a lot of flak at the time (and for years after) for advocating for a more pragmatic migration which burned me out a lot. That feedback came both in person and online and it largely pushed me away from Python for a while. Not getting behind the Python 3 train was seen as sabotaging the entire project. However, a decade later, I feel somewhat vindicated that it was worth being pragmatic about that migration.
At the root of that discourse was a idealistic view of how Unicode could work in the language and that you can move an entire ecosystem at once. Both those things greatly clashed with the lived realities in many projects and companies.
I am a happy user of Python 3 today. This migration has also taught me the important lesson not be too stuck on a particular idea. It would have been very easy to pick one of the two sides of that debate. Be stuck on Python 2 (at the risk of forking), or go all in on Python 3 no questions asked. It was the path in between that was quite painful to advocate for, but it was ultimately the right path. I wrote about my lessons of that migration a in 2016 and I think most of this still rings true. That was motivated by even years later people still reaching out to me who did not move to Python 3, hoping for me to embrace their path. Yet Python 3 has changed! Python 3 is a much better language than it was when it first released. It is a great language because it's used by people solving real, messy problems and because it over time found answers for what to do, if you need to have both Python 2 and 3 code in the wild. While the world of Python 2 is largely gone, we are still in a world where Unicode and bytes mix in certain contexts.
Fully committing to a single worldview can be easier because you stop questioning everything — you can just go with the flow. Yet truths often reside on both sides. Allowing yourself to walk the careful middle path enables you to learn from multiple perspectives. You will face doubts and open yourself up to vulnerability and uncertainty. The payoff, however, is the ability to question deeply held beliefs and push into the unknown territory where new things can be found. You can arrive at a solution that isn't a complete rejection of any side. There is genuine value in what Rust offers—just as there was real value in what Python 3 set out to accomplish. But the Python 3 of today isn't the Python 3 of those early, ideological debates; it was shaped by a messy, slow, often contentious, yet ultimately productive transition process.
I am absolutely sure that in 30 years from now we are going to primarily program in memory safe languages (or the machines will do it for us) in environments where C and C++ prevail. That glimpse of a future I can visualize clearly. The path to there however? That's a different story altogether. It will be hard, it will be impure. Maybe the solution will not even involve Rust at all — who knows.
We also have to accept that not everyone is ready for change at the same pace. Forcing adoption when people aren't prepared only causes the pendulum to swing back hard. It's tempting to look for a single authority to declare “the one true way,” but that won't smooth out the inevitable complications. Indeed, those messy, incremental challenges are part of how real progress happens. In the long run, these hard-won refinements tend to produce solutions that benefit all sides—if we’re patient enough to let them take root. The painful and messy transition is here to stay, and that's exactly why, in the end, it works.
2025-02-04 08:00:00
I recently wrote about dependencies in Rust. The feedback, both within and outside the Rust community, was very different. A lot of people, particularly some of those I greatly admire expressed support. The Rust community, on the other hand, was very dismissive on on Reddit and Lobsters.
Last time, I focused on the terminal_size crate, but I also want to show you a different one that I come across once more: rand. It has a similarly out-of-whack value-to-dependency ratio, but in a slightly different way. More than terminal_size, you are quite likely to use it. If for instance if you want to generate a random UUID, the uuid crate will depend on it. Due to its nature it also has a high security exposure.
I don't want to frame this as “rand is a bad crate”. It's not a bad crate at all! It is however a crate that does not appear very concerned about how many dependencies it has, and I want to put this in perspective: of all the dependencies and lines of codes it pulls in, how many does it actually use?
As the name implies, the rand crate is capable of calculating random numbers. The crate itself has seen a fair bit of churn: for instance 0.9 broke backwards compatibility with 0.8. So, as someone who used that crate, I did what a responsible developer is supposed to do, and upgraded the dependency. After all, I don't want to be the reason there are two versions of rand in the dependency tree. After the upgrade, I was surprised how fat that dependency tree has become over the last nine months.
Today, this is what the dependency tree looks like for the default feature set on macOS and Linux:
x v0.1.0 (/private/tmp/x)
└── rand v0.9.0
├── rand_chacha v0.9.0
│ ├── ppv-lite86 v0.2.20
│ │ └── zerocopy v0.7.35
│ │ ├── byteorder v1.5.0
│ │ └── zerocopy-derive v0.7.35 (proc-macro)
│ │ ├── proc-macro2 v1.0.93
│ │ │ └── unicode-ident v1.0.16
│ │ ├── quote v1.0.38
│ │ │ └── proc-macro2 v1.0.93 (*)
│ │ └── syn v2.0.98
│ │ ├── proc-macro2 v1.0.93 (*)
│ │ ├── quote v1.0.38 (*)
│ │ └── unicode-ident v1.0.16
│ └── rand_core v0.9.0
│ ├── getrandom v0.3.1
│ │ ├── cfg-if v1.0.0
│ │ └── libc v0.2.169
│ └── zerocopy v0.8.14
├── rand_core v0.9.0 (*)
└── zerocopy v0.8.14
About a year ago, it looked like this:
x v0.1.0 (/private/tmp/x)
└── rand v0.8.5
├── libc v0.2.169
├── rand_chacha v0.3.1
│ ├── ppv-lite86 v0.2.17
│ └── rand_core v0.6.4
│ └── getrandom v0.2.10
│ ├── cfg-if v1.0.0
│ └── libc v0.2.169
└── rand_core v0.6.4 (*)
Not perfect, but better.
So, let's investigate what all these dependencies do. The current version pulls in quite a lot.
First there is the question of getting access to the system RNG. On Linux and Mac it uses libc, for Windows it uses the pretty heavy Microsoft crates (windows-targets). The irony is that the Rust standard library already implements a way to get a good seed from the system, but it does not expose it. Well, not really at least. There is a crate called fastrand which does not have any dependencies which seeds itself by funneling out seeds from the stdlib via the hasher system. That looks a bit like this:
use std::collections::hash_map::RandomState;
use std::hash::{BuildHasher, Hasher};
fn random_seed() -> u64 {
RandomState::new().build_hasher().finish()
}
Now obviously that's a hack, but it will work because the hashmap's hasher is randomly seeded from good sources. There is a single-dependency crate too which can read from the system's entropy source and that's getrandom. So there at least could be a world where rand only depends on that.
If you want to audit the entire dependency chain, you end up with maintainers that form eight distinct groups:
If I also cared about WASM targets, I'd have to consider even more dependencies.
So let's vendor it. How much code is there? After removing all tests, we end up with 29 individual crates vendored taking up 62MB disk space. Tokei reports 209,150 lines of code.
Now this is a bit misleading, because like many times most of this is within windows-*. But how much of windows-* does getrandom need? A single function:
extern "system" fn ProcessPrng(pbdata: *mut u8, cbdata: usize) -> i32
For that single function (and the information which DLL it needs link into), we are compiling and downloading megabytes of windows-targets. Longer term this might not be necessary, but today it is.
On Unix, it's harder to avoid libc because it tries multiple APIs. These are mostly single-function APIs, but some non-portable constants make libc difficult to avoid.
Beyond the platform dependencies, what else is there?
All of these are great crates, but do I need all of this just to generate a random number?
Then there are compile times. How long does it take to compile? 4.3 seconds on my high-end M1 Max. A lot of dependencies block each other, particularly the part that waits for the derives to finish.
Only after all the code generation finished, the rest will make meaningful progress. In total a release build produces 36MB of compiler artifacts. 12 months ago, it took just under 2 seconds.
The Rust developer community on Reddit doesn't seem very concerned. The main sentiment is that rand now uses less unsafe so that's benefit enough. While the total amount of unsafe probably did not go down, that moved unsafe is is now in a common crate written by people that know how to use unsafe (zerocopy). There is also the sentiment that all of this doesn't matter anyways, because we will will all soon depend on zerocopy everywhere anyways, as more and more dependencies are switching over to it.
Maybe this points to Rust not having a large enough standard library. Perhaps features like terminal size detection and random number generation should be included. That at least is what people pointed out on Twitter.
We already treat crates like regex, rand, and serde as if they were part of the standard library. The difference is that I can trust the standard library as a whole—it comes from a single set of authors, making auditing easier. If these external, but almost standard crates were more cautious about dependencies and make it more of a goal to be auditable, we would all benefit.
Or maybe this is just how Rust works now. That would make me quite sad.
Update: it looks like there is some appetite in rand to improve on this.
Edit: This post originally incorrectly said that getrandom depends on windows-sys. That is incorrect, it only depends on windows-targets.
