MoreRSS

site iconHackerNoonModify

We are an open and international community of 45,000+ contributing writers publishing stories and expertise for 4+ million curious and insightful monthly readers.
Please copy the RSS to your reader, or quickly subscribe to:

Inoreader Feedly Follow Feedbin Local Reader

Rss preview of Blog of HackerNoon

Making Video Models Obey Your Hands

2026-03-05 06:59:59

Text prompts can’t deliver real VR control. This explains how hand+head pose conditioning makes generated worlds respond to your actual movements.

Wearables Do the Tracking, Agents Do the Deciding - Building the Missing Context Layer With OpenClaw

2026-03-05 06:21:33

Tuesday morning, Feb 12. My phone buzzes at 7:30 AM:

I get out of bed before checking anything else.

\ This didn’t come from Whoop. It came from an agent I built on top of my Apple Watch data, the one that knows my calendar, my baselines, and my schedule.

\ Think about it. Dashboards provide metrics, but we actually derive value when that data is connected to the broader context of our lives. What we are doing, when and why

\ If you got a morning message like the one above, what would your agent remind you about today? Tiny questions like that can reveal what really matters right now. For example, imagine your agent reminding you after a rough night of sleep: “Your first meeting is at 10:00, try a short walk before starting work.” Prompts like that one, shaped by your real data and calendar, help you make smarter choices throughout your day.

\ This post shows exactly how I built it.


Two Years With Whoop

Whoop is a great product. I used it for two years, and it taught me things about my body that I couldn’t have learned otherwise.

\ The first two months, I was obsessed. I was tracking diet, journaling workouts, chasing green recovery days. It felt like too much, and it was! But that experience gave me a unique perspective, and it was teaching me counterintuitive things: walk more to recover, design gym routines that leave you better the next day, not wrecked, recovery is always active and almost never passive. I never would have found those patterns on my own.

\ After two years, I’d distilled it to four rules: sleep better, manage stress, actively restore, and train. Those rules are mine. I don’t need the app to remind me of them.

\ So, I canceled my subscription.

\ What I couldn’t replace was something Whoop never had: active intelligence. How can I train and recover better this week while pushing a new feature to our users and preparing to give a speech at an event? When does it make sense to do? And where?

\ That’s the thing about any wearable in the market. They don’t know my context and can’t coach me in a way that works specifically for me. Whoop was giving me a score and suggestion, but it couldn’t say “Your recovery is shit and you will have a tough day, make sure to have a long walk today and go to the sauna tomorrow.” Whoop’s journals track what happened if you don’t feel too lazy to fill them. It doesn’t shape and guide what should be done next.

\ That gap is what I built.


Why OpenClaw (or any other autonomous agent)

I could do it with any other agent or just curling to the Claude API. It would never work so well, though, because it would not have access to the broader context about me.

\ That’s what makes OpenClaw useful here:

Stateful workspace. OpenClaw agents have a persistent workspace. Health files, baselines, and context are all accessible across sessions. No database setup, no state management code.

\ Scheduling built-in. Morning briefing, stress sentinel, and cleanup jobs are all first-class cron jobs in the config. I didn’t write a scheduler.

\ Connectors. Calendar, Telegram, and others are all connected to the same agent. The morning briefing naturally crosses sources: health, calendar, and workload in one prompt.

\ Composable agents. The briefing agent, stress sentinel, and cleanup agent are separate concerns. Each has its own prompt and schedule. No code required to compose them.

\ Memory as a first-class concept. Baselines, user preferences, and learned patterns live in workspace files that the agent can read and update. Persists across restarts.

\ The intelligence flywheel. OpenClaw agents accumulate proprietary context: calendar patterns, baseline trends, past decisions, communication style, and goals. That context compounds. The agent that’s been running for three months knows you better than one that’s been running for a week, and no generic product can replicate that, because that data is yours and lives locally.

\ This is the moat that wearables can’t copy. Not the sensors. The context.


Solution

If you want to replicate my setup, the easiest way is to point your agent to this article and to the following repo: github.com/ihorkatkov/health-agent. Here, my agent and I have laid out in plain text what needs to be done to make it work specifically FOR YOU. It’s hard to build a general AI solution for everybody, so just use the one as the starting point.


How It Works:

The architecture is simple by design, with three main parts.

\ Data layer. Apple Watch tracks the metrics. Health Auto Export is a $5 app that pushes a JSON snapshot to a lightweight webhook server on your home machine every few hours. The server writes timestamped files that your agent reads during the briefing.

\ Context layer. The agent reads those files at 7:30 AM, checks your calendar, and produces a briefing. Instead of averaging your HRV or sleep against generic population data, it compares everything against your own trends. This you-to-you approach catches your real progress, setbacks, and actual needs, because the best baseline for your body is your personal history, not the average of thousands of strangers. This allows the agent to spot when you’re adapting, struggling, or need recovery before you would notice. If you are using an Apple Watch, you can get this data directly from it. Those numbers become the agent’s frame of reference.

\ Morning briefing. The morning report follows a structure I specified in MORNING_REPORT.md. The agent reads health data, tracks goals, checks my calendar, and looks into memories. It crunches data and generates the report, which it sends to Telegram.

\ If your calendar looks overloaded and your recovery is low, it tells you straight. If stress levels rise during the day, it sends a ping to remind you to slow down.


Final Thoughts

Apple Watch tracks vitals. My agent decides what to do with them.

\ Two years of data distilled to four principles: sleep better, manage stress, actively restore, and train. Once internalized, you don’t need the teacher. You need something that applies those rules to your actual Tuesday, your calendar, your dinner at 19:00, and your four meetings.

\ Dashboards show you what happened. Agents decide what to do next.

\ An agent who knows you. $5 and an hour. That’s it.

Be Your Own Coach: How to Navigate Through Complex Problems

2026-03-05 05:29:59

Most days at work require us to navigate and solve complex problems—a production bug impacting a large customer base, facing a roadblock when designing a product with many dependencies, not getting a buy-in on a new design proposal, stakeholders demanding early piloting of new features, and disagreements and conflicts that require careful handling of the situation.

\ These problems often come unannounced and don’t show up with a bang. They silently creep in and require you to navigate the unknowns, challenges, and uncertainty they bring along. Getting paralyzed by these problems, feeling scared, or waiting for someone to rescue you by showing you the way is a common response, but it also takes away the opportunity to stand out and show up strong.

\ Learning to solve complex problems on your own will not only enable you to do more impactful work but also put you in front of people who have the power to allocate work, provide opportunities, and shape your career. These people can lift you up by entrusting you with bigger and better challenges—making your problem-solving skills highly valued, appreciated, and always in demand.

\ Most of the time, the challenge in solving problems isn’t the complexity, but our attitude towards it. We spend more time agonizing about the obstacle in our path and less time finding solutions, staying determined, and navigating whatever shows up our way.

\

Whatever the problem, be part of the solution. Don’t just sit around raising questions and pointing out obstacles.

― Tina Fey

\ But to be a good problem solver, you need to be methodical in your approach. Follow these 5 steps each time you’re tasked with the responsibility to solve a complex problem:

State the Problem

Knowing what you’re trying to solve seems like the most obvious thing, and yet this is where most people make a mistake. They either have a vague idea of the problem or over-complicate it by combining too many issues that are either irrelevant or not important to the core issue at hand.

\ Throwing multiple problems in the same pile distracts you—you keep jumping between ideas, keep going around in circles, and keep chasing a perfect solution that will fix all your problems. But focusing on too many things at the same time prevents you from solving the real problem. Lack of clarity also leads to procrastination, analysis-paralysis, and overthinking—you tend to get busy without actually getting closer to your goal.

\ To navigate complexity, you need to be specific and directed towards the problem you’re trying to solve. Stating the problem clearly helps you achieve that goal. To do this:

  1. Think about the most important problem that needs to be addressed first.
  2. Ensure it’s a single problem and not a convoluted mess of too many issues.
  3. Try to phrase it in 2-3 lines. Anything more and there’s a likelihood that you’re digressing from the main issue.
  4. Validate it by repeating the problem to yourself and analyzing if the key situation is captured well.

\

You can’t hit a bullseye if you don’t know where the goal is and you can’t solve a problem if you don’t know what problem you are trying to solve.

― Sarah K. Ramsey

\ Complexity can be paralyzing when you don’t take the time to define it clearly. By putting it into words, you can avoid solving the wrong problem as well as free up the mental space to build connections, draw insights, and find a way forward. State the problem clearly. Write it down somewhere. Don’t leave it vague or open-ended.

Dig Deeper, Get Rid of Superficial Analysis

Once you have the problem clearly defined, the next step isn’t to get down to problem-solving. Before jumping to solutions, you need to spend time analyzing the problem in more detail. Superficial analysis by rushing through the data or insights can lead you astray and cause you to make a wrong decision or implement a less effective solution.

\ You need to uncover assumptions, validate your hypothesis, acknowledge unpleasant realities, and dig deeper into the underlying root cause. What appears on the surface is often very different from reality—masked by our own biases, beliefs, past experiences, and judgments. Getting rid of these biases and preconceived notions is crucial to staying close to reality and making a more informed decision.

\ To navigate complexity, you need to go beyond your default tendency to lean towards specific conclusions and expand your thinking to include more possibilities. To do this:

  1. Ask questions. Questions give you access to different viewpoints and perspectives and unlock insights you would otherwise miss.
  2. Consciously pay attention to your biases, like confirmation bias, anchoring bias, availability heuristic, or framing effect, which influence how you perceive and interpret information.
  3. Look for data and patterns to identify if the problem at hand is repetitive or an isolated case. Gather other useful signals and metrics that might be put to use later.
  4. Define the amount of time and effort you’ll dedicate to the task and set a target date by which to complete this analysis. Don’t leave it open-ended to avoid analysis-paralysis.

\

Insights shift us toward a new story, a new set of beliefs that are more accurate, more comprehensive, and more useful. Our insights transform us in several ways. They change how we understand, act, see, feel, and desire. They change how we understand.They transform our thinking; our new story gives us a different viewpoint. They change how we act. In some cases insights transform our abilities as well as our understanding.

― Gary Klein

\ Being blinded by the stories in your mind or refusing to see the reality of your situation can turn complexity into chaos. You may overlook the important details or overindex on the wrong assumption. Don’t go with the first thought or the default idea that comes to mind. Dig deeper. Spend more time analyzing.

Design a Solution

This is the most difficult part and often the one with the most complexity. Despite all your due diligence in collecting data and analyzing the problem in detail, no solution can be perfect or risk-free—there are always unknowns and uncertainties involved. There are always competing possibilities. There’s always a risk of failure or the outcome not turning out the way you expected.

\ But you can’t let the fear of making a mistake or failing badly get in the way of strategizing and locking down on the path to take. When designing a solution, you need to choose the best possible option at the moment—list down different choices, review the pros and cons of each choice, and go with the one that seems most promising amongst them. Explore the uncharted territory. Don’t stick to tried-and-tested methods. Go with the hard choice, not the safest option. It may be scary in the beginning, but if successful can yield substantial rewards in the long run.

\ To navigate complexity, you have to look beyond the easy and secure and step into the discomfort zone. If it isn’t uncomfortable, it probably isn’t worth it. However, there are certain steps you can take to minimize the risk and prevent stepping into a panic zone:

  1. Adopt an experimental mindset. Try different solutions by using simple prototypes or running small experiments. Collect data. Base your decision on actual results, not theory or past achievements.
  2. Use a repertoire of mental models to think clearly and ground your decision-making in real insights. For example, the inversion mental model provides an objective way to explore the problem by thinking the opposite of what you seek. Reverse questioning can help you inquire into your own assumptions, beliefs, and, in turn, gain a better perspective to find answers to your original question with greater clarity and understanding. Second order thinking mental model can help you unravel the implications of your decision by thinking about its consequences in the future.
  3. Examine the feasibility of putting it into action by taking resources, timelines, and other constraints into account. A good enough solution that can be implemented in the given timeframe is better than a great solution that will miss the opportunity window.
  4. Consider the implications of not getting the desired results. Think of the best and the worst-case scenarios. Have an alternate plan ready in case you need to fallback to another strategy.

\

Real toughness is experiencing discomfort or distress, leaning in, paying attention, and creating space to take thoughtful action. It’s maintaining a clear head to be able to make the appropriate decision. Toughness is navigating discomfort to make the best decision you can.

― Steve Magness

\ Solutioning is the hardest part because it tests your core skills as well as your risk-taking capacity. Don’t be risk-averse. Quit doing what’s natural, certain, convenient, or automatic. Take the hard road.

Put it into Action

Defining a solution gives you a direction. It solidifies the approach you wish to take. However, without actually acting on it, your work is only half done. Any approach you take will need multiple modifications once you enter the implementation phase. New challenges, new observations, and new learnings will guide you on the changes you need to make. Sitting with an approach and dissecting it will never lead to insights that can be attained by taking action and identifying what works and what needs attention. You can cut through the complexity only by taking action.

\ However, when working on something complex for the very first time, a wave of self-doubt is unavoidable. Thinking about the outcome or the end result can stop you from getting started. The sheer volume and complexity of the task, combined with the possibility of failure makes it all the harder.

\ Instead of major victories with outsized expectations, what if you targeted small wins—small daily goals that give a sense of progress; work that moves you forward, is fulfilling and rewarding, and where each step takes you closer to your destination.

\ Don’t try to achieve the biggest results in the smallest amount of time; take small steps for continual improvement. To do this:

  1. Break down your solution into monthly, bi-weekly, and daily goals.
  2. Set milestones to track progress and catch mistakes early.
  3. Review your approach and fine-tune it based on the gap between achieved outcomes and expected results.
  4. Focus on the process to reach your destination, don’t obsess about the end result.

\

Radical change is like charging up a steep hill—you may run out of wind before you reach the crest, or the thought of all the work ahead makes you give up no sooner than you’ve begun. There is an alternative… another path altogether, one that winds so gently up the hill that you hardly notice the climb. It is pleasant to negotiate and soft to tread. And all it requires is that you place one foot in front of the other.

— Robert Maurer

\ You can’t cut through complexity by designing beautiful solutions, but never really acting on them. Target small daily wins. Small wins may seem trivial individually, but they ignite joy, engagement, and creativity at work, which turn into much bigger accomplishments over a period of time.

Gather Feedback and Improve

When you’re doing something worthwhile, failures and disappointments come along for the ride. Unexpected situations, a change in your circumstances, or not getting the desired results despite putting your best effort can throw your balance off a bit. Things are going to get hard. New problems are going to crop up. Instead of giving up, you must learn to dust yourself off, stand back up, and keep pushing ahead. You need to persevere and stay on the path without quitting. You need to adopt a learning attitude.

\ Success can be achieved only by stretching yourself and practicing courage to step into the unknown, not by playing it safe and staying inside your comfort zone. Give yourself permission to feel disappointed, but don’t let temporary setbacks turn into permanent excuses.

\ When confronted with an obstacle, use feedback to:

  • Try a different strategy if whatever you’ve been doing hasn’t been working.
  • Experiment with multiple options to determine the one that works.
  • Challenge traditional methods and explore new ways of doing things.
  • Practice multi-disciplinary thinking to get ideas from other domains and functions.

\ Anything you gain by solving one complex problem can also offer a great learning lesson to tackle future complexities—every challenge, obstacle, disappointment, achievement, success, failure, and mistake can be valuable when viewed through the lens of growth and improvement. By showing curiosity, you can gather excellent insights into your process—what parts you did well that might add value to future problems, what disastrous mistakes you made that must be discontinued, and what you missed out as opportunities to not let go in the future?

\

When you do take action, every result is an opportunity to reflect and learn valuable lessons. Even if what you take away from your assessment seems to be of small consequence, all of these small improvements taken together make a huge difference in the long term.

― Ken Watanabe

\ Opening your eyes without shutting down your ears can unravel a lot of complexity by making you derive value from hidden learning lessons. Don’t consider complexity as a barrier. Treat it as a challenge.

Summary

  1. When you don’t define the complex problem clearly, leave it open-ended or vague, solving the problem becomes harder because your attention gets spread out instead of focusing on the key areas. Not knowing what you want can also lead to procrastination. Clearly state the problem before getting down to solving it.
  2. Problems don’t show up alone. They come with many assumptions, biases, and preconceived notions. Not validating these assumptions or letting your biases determine the path forward can lead to bad choices and poor decisions. Before solving, pay attention to them. Dig deeper into the data and insights. Draw patterns instead of jumping to conclusions.
  3. Instead of getting stuck in analysis paralysis by chasing a perfect solution, find a good enough solution and move ahead with it. A solution that can be implemented timely is better than no solution at all. List down a few choices, think about their pros and cons, and finalize the one that seems most promising at the moment.
  4. A good solution can be converted into a great one by acting, learning, and iterating on it. Break down your solution into small steps, implement each one, gather insights, and adapt your strategy to make progress and move closer to your goal.
  5. Feedback collected during the entire process can lead to valuable lessons, not only for the problem at hand, but also for any future problems you may encounter. Proactively seek this feedback. Self-analyze and self-introspect to determine areas you did well and the changes you can bring to make things better.

\


This story was previously published here. Follow me on LinkedIn or here for more stories.

The Death of the Data Warehouse: Why Healthcare is Moving to a "Data Mesh"

2026-03-05 05:02:56

Introduction: The Centralized Bottleneck

For over a decade, the "Data Warehouse" has been the holy grail of healthcare architecture. We spent millions building monolithic, centralized silos where clinical, claims, and administrative data were cleaned, transformed, and dumped into a single "source of truth."

But in my two decades of global experience, I have seen a recurring failure: the centralized bottleneck. When every clinical department and pharmacy benefit program must wait on a centralized IT team to model their data, the "source of truth" becomes a "source of delays." We are now witnessing the slow death of the monolithic warehouse and the rise of the Data Mesh—a decentralized, domain-oriented architecture that treats data as a product rather than a byproduct.

Why the Warehouse Fails in Clinical Environments

In a centralized warehouse, data is often disconnected from the "domain experts"—the clinicians and pharmacists who actually understand what the data means.

  1. Semantic Mismatch: A centralized team often renames clinical variables to fit a "standard schema," stripping away the context that doctors and pharmacists need.
  2. The Throughput Wall: As we integrate more real-time data, our warehouse pipelines become brittle. If one feed breaks, the entire reporting suite goes down.
  3. Regulatory Latency: In a centralized model, ensuring HIPAA compliance across the entire warehouse is a massive, high-risk operation.

Enter the Data Mesh: A Decentralized Blueprint

A Data Mesh isn't just a new tool; it’s a shift in ownership. Instead of dumping data into a central "lake" or "warehouse," we treat data as a product owned by the specific domain team that generates it.

The Four Pillars of Healthcare Data Mesh:

  • Domain Ownership: The Pharmacy Benefit Management (PBM) team owns their formulary and claims data. They are responsible for its quality, its schema, and its access controls.
  • Data as a Product: Data isn't an exhaust pipe; it’s an API. If a clinician needs patient history, the PBM domain provides it as a clean, standardized, and versioned data product.
  • Self-Serve Infrastructure: As architects, we provide the platform—the "data highway"—but we stop trying to control every single lane of traffic.
  • Federated Governance: We use automated policies (like machine identity checks) to ensure that even though the data is decentralized, it adheres to the same HIPAA security and interoperability standards across every domain.

Architectural Deep Dive: Implementing the Mesh

To move from a warehouse to a mesh, we stop using monolithic batch processes and start using Event-Driven Architectures. We use tools that allow domains to "broadcast" their data updates in real-time.

Example: The PBM-to-Clinical Mesh Link Instead of a nightly batch job, we use an event bus (like Kafka) to share claim approvals with the clinical decision support system.

\

# A simple decentralized data product interface
class FormularyDataProduct:
    def __init__(self, domain_id):
        self.domain = domain_id
        
    def get_latest_formulary(self):
        # Data is retrieved directly from the PBM domain endpoint
        # No warehouse 'ETL' required here.
        return f"Fetching latest formulary for domain: {self.domain}"
 
# Each domain maintains its own data interface
pbm_mesh = FormularyDataProduct("PBM_DOMAIN")
print(pbm_mesh.get_latest_formulary())

\

The Security Implications of Decentralization

Decentralization is often feared by security teams, but in a Sky Computing context, it actually improves security. By using Machine Identity Management, we can issue "scope-limited" credentials to each domain. If a domain is compromised, the "blast radius" is limited to that domain, preventing a full-system breach.

The "So What?" for the Architect

Moving to a Data Mesh doesn't happen overnight. It requires us to abandon the "God-Complex" of the centralized warehouse and embrace our role as platform architects. We build the rails; the domain teams drive the trains.

This shift allows for:

  • Faster Innovation: The clinical team can build a new AI model without waiting for the warehouse team to finish their sprint.
  • Higher Data Fidelity: The experts who generate the data are the ones checking its quality.
  • Architectural Resilience: If one data product fails, the rest of the healthcare ecosystem remains live and operational.

Summary: The Path Forward

The death of the data warehouse is not a loss; it is an evolution.

As we architect for a more complex, cloud-agnostic world, the centralized model becomes an anchor that holds us back.

  • Domain-Centricity: Data quality is highest when it is maintained by those who understand the clinical context.
  • Data as a Product: Shift from "data integration" to "data exposure" via APIs and event streams.
  • Zero-Trust Security: Use machine identity and federated governance to secure a decentralized landscape.
  • Scalable Infrastructure: Decentralization allows your system to scale at the speed of your clinical departments, not the speed of your central IT pipeline.

We aren't just changing where we store our rows; we are changing how we unlock the value of clinical intelligence.

Scaling JMeter: A Guide to Distributed Performance Testing

2026-03-05 04:52:21

Traditional load testing from a single machine quickly becomes a bottleneck for modern distributed systems. This article describes a scalable benchmarking architecture using Apache JMeter running on distributed cloud workers, Scribe for centralized log aggregation, and TimescaleDB for time-series performance analytics. The approach enables engineering teams to simulate large-scale production traffic while efficiently collecting and analyzing millions of performance events.

Most teams start performance testing the same way.

They install Apache JMeter, create a test plan, and run it from a laptop or a single VM.

For small systems this works fine. But once services begin handling tens of thousands of requests per second, the testing setup itself becomes the bottleneck.

The problem isn’t the application anymore.

It’s the load generator.

In modern distributed systems, realistic performance testing requires infrastructure capable of generating traffic at scale while collecting and analyzing millions of performance events in real time.

In this article, I’ll walk through a distributed performance benchmarking architecture that combines:

  • Apache JMeter for load generation
  • Cloud infrastructure (AWS EC2 or Google Cloud Compute) for scalable workers
  • Scribe-based log aggregation for centralized metric collection
  • TimescaleDB for high-volume time-series analytics

This architecture allows engineering teams to simulate production-scale traffic patterns and analyze performance results efficiently.

Cloud providers such as Amazon Web Services offer managed load testing solutions such as AWS Distributed Load Testing, which simplifies large-scale benchmarking through a managed infrastructure. However, many engineering teams prefer building custom benchmarking systems using open-source tools to gain greater control over workload generation, logging pipelines, and analytics storage. The architecture described in this article demonstrates one such approach using JMeter, distributed compute infrastructure, and time-series analytics.

\

In my work building large-scale data infrastructure and distributed services, performance testing frequently becomes a bottleneck long before the system under test reaches its limits. Designing a scalable benchmarking platform therefore requires treating the testing infrastructure itself as a distributed system.

The Problem with Traditional Load Testing

Most load testing environments fail for three reasons.

1. The Load Generator Becomes the Bottleneck

A single machine generating requests quickly runs into limits:

  • CPU exhaustion
  • Network saturation
  • Thread scheduling overhead

Before the system under test reaches its limits, the load generator collapses first.

2. Test Traffic Isn’t Realistic

Production traffic comes from:

  • Multiple regions
  • Different network latencies
  • Varying traffic spikes

Running all requests from one machine produces unrealistic behavior.

3. Results Are Hard to Analyze

When running distributed tests, each worker produces logs locally. Without centralized aggregation, analyzing results across dozens of workers becomes difficult.

What we need instead is a distributed load generation system with centralized observability.

Architecture Overview

The architecture is built around four key components:

  1. Test Controller
  2. Distributed JMeter Workers
  3. Centralized Logging Pipeline
  4. Time-Series Analytics Database

The data flow looks like this:

Controller → Distributed JMeter Workers → Scribe Logging Layer → TimescaleDB → Analytics / Visualization

Each layer solves a specific scalability problem

Scaling Load Generation with Cloud Workers

Apache JMeter is widely used for performance testing because it supports many protocols and provides flexible scripting capabilities.

To generate large-scale load, we deploy multiple JMeter workers across cloud instances.

Each worker:

  • Runs JMeter in non-GUI mode
  • Executes the same test plan
  • Generates a portion of the overall traffic

Example execution command:

jmeter -n -t test-plan.jmx

Workers can be provisioned using:

  • AWS Auto Scaling Groups
  • Google Cloud Managed Instance Groups

This enables horizontal scaling.

Example load generation capacity:

| Workers | Estimated Throughput | |----|----| | 5 | ~20k requests/sec | | 20 | ~100k requests/sec | | 100 | 500k+ requests/sec |

The exact numbers depend on:

  • Request complexity
  • Instance type
  • Network overhead

Why Centralized Logging Is Critical

When running distributed performance tests, every request generates metrics such as:

  • Request latency
  • Response code
  • Endpoint
  • Timestamp

During a large test run, this can easily produce hundreds of millions of records.

If workers store logs locally:

  • Disk I/O slows down the test
  • Results become fragmented
  • Analysis becomes difficult

Instead, workers should stream metrics to a centralized logging pipeline.

Aggregating Logs with Scribe

A common approach is to use Scribe, a distributed log aggregation system designed for high-throughput environments.

Each JMeter worker emits structured events like this:

{
  "timestamp": "2026-03-01T12:10:22",
  "test_run_id": "benchmark_test_1",
  "worker_id": "worker-08",
  "endpoint": "/api/resource",
  "latency_ms": 118,
  "status_code": 200
}

Scribe collects logs from all workers and forwards them to the storage layer.

Benefits of this architecture:

  • Decouples load generation from storage
  • Avoids disk bottlenecks on worker nodes
  • Enables centralized analysis pipelines

Storing Benchmark Results in TimescaleDB

Performance test data is fundamentally time-series data.

Each request produces a timestamped measurement. Over a 10-minute test, a distributed system may generate hundreds of millions of events.

A traditional relational database can store this data, but time-series workloads benefit from specialized optimizations.

TimescaleDB is a PostgreSQL extension designed for exactly this use case.

Key features include:

Time-Partitioned Storage

TimescaleDB stores data in hypertables, automatically partitioned by time.

This allows efficient storage and querying of massive datasets.

High Write Throughput

Batch inserts enable TimescaleDB to ingest large volumes of benchmark data without becoming a bottleneck.

Built-in Time-Series Analytics

TimescaleDB supports powerful analytical queries such as:

  • Latency percentiles
  • Time-bucket aggregations
  • Rolling averages

These operations are critical when analyzing system performance.

Example Schema

A simple schema for storing benchmark results might look like this:

CREATE TABLE perf_results (
    timestamp TIMESTAMPTZ NOT NULL,
    test_run_id TEXT,
    worker_id TEXT,
    endpoint TEXT,
    latency_ms INT,
    status_code INT
);

Convert the table into a hypertable:

SELECT create_hypertable('perf_results', 'timestamp');

This enables TimescaleDB’s automatic partitioning and performance optimizations.

Example Performance Queries

Once results are stored, engineers can analyze system behavior using SQL.

P95 latency

SELECT percentile_cont(0.95)
WITHIN GROUP (ORDER BY latency_ms)
FROM perf_results
WHERE test_run_id='benchmark_42';

Error rate

SELECT
COUNT(*) FILTER (WHERE status_code >=500) * 1.0 / COUNT(*)
FROM perf_results
WHERE test_run_id='benchmark_42';

Requests per minute

SELECT
time_bucket('1 minute', timestamp) AS minute,
COUNT(*) AS requests
FROM perf_results
GROUP BY minute
ORDER BY minute;

These queries allow teams to quickly evaluate system performance under heavy load.

Visualization and Monitoring

Once performance data is stored in TimescaleDB, it can be visualized using tools such as:

  • Grafana
  • Superset
  • Metabase

Typical dashboards include:

  • requests per second
  • P50 / P95 / P99 latency
  • error rate
  • latency distribution over time

This turns load testing into a repeatable and observable engineering workflow rather than a one-off experiment.

Design Considerations When Building Benchmarking Systems

Through building large-scale benchmarking environments, several design principles consistently emerge:

1. Load generation must scale independently of analytics

The infrastructure generating traffic should remain isolated from the systems used to store and analyze performance data. Coupling these layers can introduce measurement bias.

2. Benchmarking systems should simulate realistic production conditions

This includes:

  • distributed traffic sources
  • realistic request concurrency
  • network variability

A centralized test client rarely captures these dynamics.

3. Observability is as important as traffic generation

Without structured logging and time-series analytics, identifying bottlenecks becomes extremely difficult in high-throughput environments.

These considerations significantly improve the reliability of performance benchmarks.

Example Deployment

A typical deployment on AWS might include:

| Component | Instance | |----|----| | Controller | t3.large | | JMeter workers | 50 × c6i.large | | Scribe collectors | m6i.large | | TimescaleDB | PostgreSQL RDS |

This setup can generate hundreds of thousands of requests per second, depending on the workload.

Lessons Learned

Building a distributed performance benchmarking system highlights several important principles.

Treat benchmarking infrastructure like production infrastructure

Load testing systems must be designed with the same scalability and reliability considerations as production systems.

Decouple load generation and analytics

Separating workers from the analytics pipeline prevents measurement bias and improves reliability.

Store performance metrics as time-series data

Time-series databases such as TimescaleDB make large-scale performance analysis significantly easier.

Final Thoughts

As distributed systems continue to grow in scale, benchmarking infrastructure must evolve alongside them. Treating performance testing as a distributed systems problem — rather than a simple testing task — enables engineers to simulate real production workloads and identify bottlenecks earlier in the development lifecycle.

References

  1. Apache Software Foundation. Apache JMeter User Manual. https://jmeter.apache.org/usermanual/
  2. Apache Software Foundation. Remote Testing in JMeter. https://jmeter.apache.org/usermanual/remote-test.html
  3. Benson, T., Akella, A., & Maltz, D. Scribe: A Scalable and Reliable Logging Infrastructure. USENIX.
  4. Timescale Inc. TimescaleDB Documentation. https://docs.timescale.com/
  5. Kleppmann, M. Designing Data-Intensive Applications. O’Reilly Media, 2017.

\

Why Your Job Is Safe From Robots

2026-03-05 04:30:26

I just survived another holiday season, which means I just survived the same conversation about eight times. It goes something like this: "So you work on robots, are they going to take all our jobs?" My family ranges from 35 to 70 and almost every one of them hit me with some version of this. I'd put on a different mask, adjust what's worthy of the conversation, and nod along. But after hearing it enough I thought deeply about it and wanted to look at prior events in history.

I get it, when Geoffrey Hinton gets on mainstream media to spread warnings, that lands hard on someone who isn't in the thick of it. The media thrives on fear, and you rarely see the promising progress of robotics publicized outside of niche circles. But I am in the thick of it - and from the inside, the picture looks very different than what you might see in the mainstream media.

\

Looms, Riots, and Wardrobes

In 1811 a group of English textile workers called the Luddites started destroying mechanical looms. Their fear was rational - these machines genuinely could do the work of several people. What they couldn't predict was what cheap cloth would do to demand.

Before the power loom, most people owned one or two outfits. Clothing was expensive because it was labor intensive. But as mechanized weaving drove production costs into the floor, people didn't just buy the same amount of cloth for less money. They bought way more of it. Suddenly a working family could afford a wardrobe. This explosion in demand didn't just preserve jobs in textiles - it birthed entirely new ones. Fashion design, retail, garment manufacturing at scale, textile marketing, global logistics to move it all around. The clothing industry today employs orders of magnitude more people than it did before the loom. The machine didn't subtract work. It moved it somewhere the Luddites couldn't have imagined.

This happens so consistently that economists have a name for the fear itself: the lump of labor fallacy. It's the assumption that there is a fixed amount of work in the economy, and if a machine picks some of it up there's less left for us. Sounds logical. Is also wrong, almost every time.

ATMs were supposed to kill the bank teller. Instead, they made it so cheap to run a branch that banks opened more of them. The number of US bank tellers actually grew for decades after ATMs showed up. The job shifted from counting cash to selling financial products, but it didn't vanish. Spreadsheets didn't eliminate accountants either. They multiplied the applications of financial reasoning so dramatically that there are now more people doing analytical work than ever before.

What I See From the Inside

Here's the part that's hard to explain at a holiday dinner: the robots I've worked on don't slot into an existing human's role. They normally make previously impossible things possible. At Zipline, autonomous drones deliver blood and medical supplies to remote clinics in Africa in under 30 minutes. Nobody was doing that job before. The infrastructure simply did not exist. The whole operation is new - drone engineers, flight operators, logistics coordinators, client integration specialists. Hundreds of roles that weren't a thing five years ago, serving people who previously had no access to critical medicine.

Similarly the Neuralink robot moves with such high precision and speed, that it isn’t even possible as a surgeon. This leads to safer and quicker surgeries. On automotive lines automation on the line isn't about cutting heads - it's about building cars at a price point that expands the market. More buyers means more service centers, more charging infrastructure, more downstream jobs of every kind. When Henry Ford put the Model T on an assembly line he didn't shrink the auto industry - he created it.

Having a robot always available to help out, changes the dynamic of the home and means that homes can say constantly clean and organized, instead of waiting for the weekend.

\

The Mechanism Is Always the Same

Automation reduces the cost of producing something. Cheaper production means lower prices. Lower prices mean more people buy it. More demand creates more overall activity, and that means more work for humans - just not the same work.

David Autor at MIT has been studying this pattern for years. His research shows that about 60% of workers in 2018 were employed in job types that didn't exist in 1940. We are terrible at imagining jobs that don't exist yet, but great at imagining the ones that might disappear. This asymmetry is where the fear comes from.

\

Avoiding Stagnation

I'm not going to pretend transitions are painless. People do lose specific jobs to machines and the adjustment can be brutal even when the big picture works out. That's real and worth taking seriously.

But the alternative is worse. The precautionary principle taken too far means blocking the very progress that creates new opportunity. If the Luddites had won, we would still be hand-weaving cloth and most people would still own two shirts. Slowing down doesn't protect anyone, it just delays the inevitable while also delaying the benefits.

I think about this from a first principles perspective. The economy is not a zero sum game. When you make something more efficiently, you free up resources - time, money, labor - that flow into whatever comes next. That "whatever comes next" is where new industries, new companies, and new careers show up. Nobody in 1811 predicted fashion week. Nobody in 1975 predicted that ATMs would lead to more bank branches. And I guarantee nobody today can predict the industries that cheap, capable robots will create 20 years from now.

So no, my robots are not coming for your job. If history is anything to go by, they are more likely to create it.