2025-11-14 01:00:00

Ten months ago, DeepSeek collapsed AI training costs by 90% using distillation - transferring knowledge from larger models to smaller ones at a fraction of the cost.

Distillation works like a tutor training a student : a large model teaches a smaller one.¹ As we’ve shifted from knowledge retrieval to agentic systems, we wondered if there was a parallel technique for tool calling.²

Could a large model teach a smaller one to call the right tools?

The answer is yes, or at least yes in our case. Here’s our current effort :

Every time we used Claude Code, we logged the session - our query, available tools, & which tools Claude chose. These logs became training examples showing the local model what good tool calling looks like.

We wanted to choose the right data so we used algorithms to cherry-pick. We used SemDeDup³ & CaR⁴, algorithms to find the data examples that lead to better results.

Claude Code fired up our local model powered by GPT-OSS 20b⁵ & peppered it with the queries. Claude graded GPT on which tools it calls.

Claude’s assessments were fed into a prompt-optimization system with DSPy⁶ & GEPA⁷. All of that data was then fed to improve the prompt. DSPy searches for existing examples that could improve the prompt, while GEPA mutates or tests different mutations.

Combined, we improved from a 12% Claude match rate to 93% in three iterations by increasing the data volume to cover different scenarios :

DSPy improved accuracy from 0% to 12%, and GEPA pushed it much higher, all the way to 93%, after three phases. The local model now matches Claude’s tool call chain in 93% of cases.

Make no mistake : matching Claude 93% doesn’t mean 93% accuracy. When we benchmarked Claude itself, it only produced consistent results about 50% of the time. This is non-determinism at work.

This proof of concept works for a small set of tools written in the code mode fashion. It suggests there is a potential for tool calling distillation.

If you’ve tried something similar, I’d love to hear from you.

2025-11-12 08:00:00

By Monday lunch, I had burned through my Claude code credits. I’d been warned ; damn the budget, full prompting ahead.

When that still couldn’t solve the issue, I summoned Opus, the biggest & most expensive model, to solve it.

Now two days on, I’ve needed to figure out alternatives. Do I :

• Switch to API billing (how much will that cost?)
• Try another vendor? Gemini’s model is great, but ageing ; at nearly 8 months old, its a capable jalopy. Cursor’s free coding model Composer 1 sprints at problems with aplomb but a bit overwhelmed at times. Codex, the plodding giant is brilliant at large-scale technical challenges.
• Create another Max subscription & switch between them? Can I ask AI to write a script to save me the hassle of changing my identity?
• Stand-up GPT-OSS to run locally? A little bit more latency but potent & twice as fast on llama.cpp compared to Ollama.
• Go back to writing code the old way? The hedonic treadmill moves quickly. I tried to return to the old ways, but it was painful. I’ve already forgotten where the blog server script is. Claude? Do you remember? Claude?

I’m working through the math of which option will cost more. How much is the Max plan subsidized? Will knowing the true API cost of my Claude Code usage increase my willingness to pay?

Switching between tools incurs costs. The tools, the workflow, the prompts that I’ve optimized for Claude code must all be ported (at my expense!) to other tools.

As the capabilities of these models begin to plateau, the costs to shift increase. So does my willingness to pay for Claude to answer me.

2025-11-10 08:00:00

Datadog is becoming a platform company, & its Q3 2025 results underscore how successful this transition is. If nothing else, the consistency around 25% growth for the last 12 quarters exemplifies this point.

Net dollar retention underpins this growth, combined with accelerating new customer account acquisition. One of the biggest changes in the last five quarters is terrific cross-selling across an increasingly large product suite.

Platform Adoption Deepening

At the end of Q3, 84% of customers were using 2 or more products, up from 83% a year ago. 54% of customers were using 4 or more products, up from 49% a year ago. 31% of our customers were using 6 or more products, up from 26% a year ago & 16% of our customers were using 8 or products, up from 12% a year ago.

Datadog’s platform spans six product categories:

• Digital Experience Monitoring: RUM/Real User Monitoring, Synthetics, Product Analytics
• Security: Cloud SIEM, Cloud Security
• Infrastructure Observability: APM, Log Management, Flex Logs
• Incident Response: Incident Management, On-Call
• AI Capabilities: Bits AI, LLM Observability
• Cost Management: Cloud Cost Management

The steady increase in multi-product adoption demonstrates customers consolidating their observability stack onto Datadog, with the highest-tier customers (8+ products) growing 33% year-over-year as a percentage of the base.

New Customer Momentum

New logo annualized bookings more than doubled year-over-year & set a new record driven by an increase in average new logo land size, particularly in enterprise.

The portion of our year-over-year revenue growth that related to new customers was about 25% in Q3, up from 20% in Q2.

New customer acquisition is also accelerating. This is in concert with a move-up market into the enterprise.

AI Native Customer Expansion

We also experienced strong revenue growth for our AI native customers & a broadening contribution to growth among those customers. There, too, we saw an acceleration of growth in our AI cohort in Q3 when excluding our largest customer.

This group represented 12% of our revenue, up from 11% last quarter & about 6% in the year ago quarter.

The AI native cohort is both growing & maturing. Datadog now has 15 AI native customers spending more than $1 million annually, up from essentially zero a year ago, with over 100 spending more than $100,000.

Revenue Growth

Revenue was $886 million, an increase of 28% year-over-year & above the high end of our guidance range.

The combination of these three factors : a broader product suite that is effectively cross-sold, accelerating new customer momentum, & a very fast-growing AI business, has led to outperformance.

Security Suite Accelerating

Security ARR growth was in the mid-50s as a percentage year-over-year in Q3, up from the mid-40s we mentioned last quarter.

We’re starting to see success in including Cloud SIEM in larger deals, & we’ll get back to that in a bit in our customer examples. And we’re seeing positive trends beyond Cloud SIEM, including fast uptake of good security & an increasing number of wins in cloud security.

Security is becoming a meaningful growth driver for Datadog, accelerating from mid-40s to mid-50s percentage growth & expanding beyond Cloud SIEM into broader cloud security use cases.

Enterprise Deal Momentum

First, we landed a 7-figure annualized deal with a leading European telco, our largest ever land deal in Europe. […] They will adopt 11 Datadog products to start.

Next, we landed a 7-figure annualized deal with a Fortune 500 technology hardware company.

Both of these data points confirm a significant move-up market. A million-dollar land deal with 11 products confirms that Datadog is truly selling a suite.

Datadog’s AI Products

In addition to the existing suite, Datadog is pushing heavily into AI with a broader range of AI deployment products.

• Bits AI SRE Agent (Available in preview, announced June 2025) is an autonomous AI agent that investigates alerts & coordinates incident response 24/7, saving customers significant time on mean-time-to-resolution.

• LLM Experiments & Playgrounds (Generally available, launched 2025) helps teams rapidly iterate on LLM applications by testing prompt changes, model swaps, & application changes against production traces.

• Custom LLM-as-a-Judge Evaluations (Generally available) lets customers write natural language evaluation prompts to assess LLM application quality & safety across traces & spans.

• Datadog MCP Server (Available in preview, announced 2025) bridges Datadog with AI agents like Codex, Claude, Cursor, & GitHub Copilot, providing structured access to metrics, logs, traces, & incidents directly from AI coding environments.

• TOTO, Datadog’s open-source time series forecasting model (launched 2025), was trained on 2 trillion data points & became one of Hugging Face’s top downloads across all categories.

If SaaS companies were dog breeds, many would be temperamental. But Datadog demonstrates continued consistency across a broad range of different businesses.

2025-11-06 08:00:00

Just how much are we spending on AI?

Compared to other massive infrastructure projects, AI is the sixth largest in US history, so far.

World War II dwarfs everything else at 37.8% of GDP. World War I consumed 12.3%. The New Deal peaked at 7.7%. Railroads during the Gilded Age reached 6.0%.

AI infrastructure today sits at 1.6%, just above the telecom bubble’s 1.2% & well below the major historical mobilizations.

Companies like Microsoft, Google, & Meta are investing $140B, $92B, & $71B respectively in data centers & GPUs. OpenAI plans to spend $295B in 2030 alone.

If we assume OpenAI represents 30% of the market, total AI infrastructure spending would reach $983B annually by 2030, or 2.8% of GDP.¹

To match the railroad era’s 6% of GDP, AI spending would need to reach $2.1T per year by 2030 (6% of projected $35.4T GDP), a 320% increase from today’s $500B. That would require Google, Meta, OpenAI, & Microsoft each investing $500-700B per year, a 5-7x increase from today’s levels.

And that should give you a sense of how much we were spending on railroads 150 years ago!

Sources

World War I & II:

• NBER: The Economics of World War I
• EH.net: U.S. Economy in World War I
• St. Louis Fed: Which War Saw the Highest Defense Spending?
• EH.net: The American Economy during World War II

New Deal:

• Wikipedia: Works Progress Administration
• Wikipedia: Public Works Administration

Railroads:

• Melville J. Ulmer, “Capital in Transportation, Communications, & Public Utilities: Its Formation & Financing” (1960), via Silicon Valley Capital Partners
• NBER: Railroads & Economic Growth in the Antebellum United States
• American-Rails.com: Railroads In The Gilded Age

Telecom Bubble:

• Richmond Fed: Boom & Bust in Telecommunications

Apollo Program:

• NASA: Apollo Program Budget Appropriations
• The Planetary Society: How much did the Apollo program cost?

Manhattan Project:

• Brookings Institution: The Costs of the Manhattan Project
• Nuclear Secrecy Blog: The price of the Manhattan Project

AI Infrastructure:

• Microsoft, Google/Alphabet, Meta earnings reports & CAPEX guidance (FY24-25)
• Bureau of Economic Analysis: US GDP Data

Methodology

All historical spending figures are adjusted to 2025 dollars using Consumer Price Index (CPI) inflation data. Each figure represents peak single-year spending in the year indicated. Percentages show spending as a share of GDP in that specific year, not as a percentage of today’s GDP.

For example, WWII’s $1,152B represents actual 1944 defense spending ($63B nominal) adjusted for inflation, which consumed 37.8% of 1944’s GDP ($175B). This differs from asking “what would 37.8% of today’s $30.5T GDP cost?” which would yield $11.5T.

2025-11-05 08:00:00

I wrote a post titled Congratulations, Robot. You’ve Been Promoted! in which OpenAI declared that their AI coders were no longer junior engineers but mid-level engineers.

The post triggered the largest unsubscription rate in this blog’s history. It was a 4-sigma event.

A Nassim Taleb black swan, this was something that should happen once every 700 years of a blog author’s career.

Clearly, the post struck a nerve.

In a job market 13% smaller for recent grads than recent years, a subtle fear persists that positive developments in AI accuracy & performance accelerate job losses. Stanford’s research found :

“young workers (ages 22–25) in the most AI-exposed occupations, such as software developers & customer service reps, have experienced a 13% relative decline in employment since generative AI became widely used.”

The whispered question beneath all this data : are AI advances a zero-sum game for jobs? Are we in the modern era hearing the same refrain as Springsteen lamenting the impact of globalization on his hometown :

They’re closing down the textile mill across the railroad tracks. Foreman says “These jobs are going, boys, and they ain’t coming back”

Let’s go to the data.

Software engineering employment grew steadily from 3.2m developers in 2010 to 4.7m in 2022 during the ZIRP (Zero Interest Rate Policy) era. The 2020-2022 period was the hottest tech jobs market of all time, with demand doubling since 2020.

Then the Fed raised rates aggressively, increasing the cost of capital, triggering a 4.3% contraction, hitting younger workers.

But data from layoffs suggest that this trend isn’t accelerating.

Tech companies laid off 264,220 employees in 2023. The 2025 data (annualized from 11 months through November) projects 122,890 layoffs for the full year. There’s no acceleration yet in the data.

The data doesn’t yet show what readers are clearly feeling : a trepidation that AI advances will accelerate job losses.

2025-11-03 08:00:00

OpenAI has committed to spending $1.15 trillion on hardware & cloud infrastructure between 2025 & 2035.¹

The spending breaks down across seven major vendors: Broadcom ($350B), Oracle ($300B), Microsoft ($250B), Nvidia ($100B), AMD ($90B), Amazon AWS ($38B), & CoreWeave ($22B).²

Using some assumptions, we can generate a basic spending plan through contract completion.³

Across these vendors, estimated annual compute spending grows from $6B in 2025 to $173B in 2029, reaching $295B in 2030. We built a constrained allocation model with the boundary conditions defined in the appendix below, but this is just a guess. The actual growth rates are 124% (2027→2028), 54% (2028→2029), & 70% (2029→2030).

Coincidentally, OpenAI announced today they would hit $100B in 2027, earlier than expected.⁴ This gives us another data point to help us understand the business’ trajectory.

OpenAI projects a 48% gross profit margin in 2025, improving to 70% by 2029.⁵ If we assume all infrastructure spending flows through cost of goods sold (COGS), we can calculate the implied revenue needed to support these spending levels at OpenAI’s target margins.

The calculation assumes linear margin improvement from 48% (2025) to 70% (2029), then holds at 70% for 2030-2032. Revenue is calculated as: Spending / (1 - Gross Margin).

These implied revenue figures suggest OpenAI would need to grow from ~$10B in 2024 revenue to $577B by 2029, roughly the size of Google’s revenue in the same year (assuming Google grows from $350B in 2024 at ~12% annually).

If nothing else, the estimated annual spending & commitments convey an absolutely enormous level of potential & ambition.

Appendix: Estimation Methodology

Deal Duration Summary

Average duration of disclosed deals: 5.7 years (rounded to 6 years for estimation purposes)

Deal Structure

Broadcom : OpenAI commits to deploying 10 gigawatts of custom AI accelerators designed by OpenAI & developed in partnership with Broadcom. Estimated value of $350B based on industry benchmarks ($35B per gigawatt). Deployment begins H2 2026. We estimate a 7-year deployment timeline (2026-2032) consistent with the scale & complexity of custom chip manufacturing & data center buildout. The systems will be deployed across OpenAI’s facilities & partner data centers.⁶

Microsoft : OpenAI commits to purchasing an incremental $250 billion in Azure cloud services over an estimated 6 years (2025-2030).⁷

Nvidia : Nvidia invests up to $100 billion in OpenAI for non-voting shares. OpenAI commits to spending on Nvidia chips across at least 10 gigawatts of AI data centers. First deployment begins in H2 2026 using the Nvidia Vera Rubin platform.⁸

AMD : AMD provides OpenAI with warrants to purchase up to 160 million AMD shares (approximately 10% of the company) at one cent per share. In exchange, OpenAI commits to purchasing 6 gigawatts of AMD Instinct GPUs, representing $90 billion in cumulative hardware revenue potential. The first 1 gigawatt deployment starts in H2 2026.⁹

Oracle : OpenAI commits to paying Oracle $60 billion annually for five years (2027-2031) for cloud infrastructure, totaling $300 billion. The contract is part of Oracle’s $500 billion Stargate data center buildout. Larry Ellison stated in Oracle’s earnings call : “The capability we have is to build these huge AI clusters with technology that actually runs faster & more economically than our competitors.”¹⁰

Amazon AWS : OpenAI commits to $38 billion over seven years (2025-2031) for cloud infrastructure & compute capacity on Amazon Web Services. The agreement, signed November 3, 2025, provides immediate access to hundreds of thousands of Nvidia GB200 & GB300 GPUs running on Amazon EC2 UltraServers. All planned capacity is targeted to come online by the end of 2026, with room to expand through 2027 & beyond. Sam Altman stated : “Scaling frontier AI requires massive, reliable compute.”¹¹ OpenAI’s first major partnership with AWS, adding to its multi-cloud infrastructure.

CoreWeave : $22.4 billion in committed spending for data center usage rights through 2029, consisting of $11.9B initial contract, $4B expansion, & $6.5B September 2025 expansion.¹²

Estimation Method

The year-by-year breakdowns above are estimates based on publicly announced deal terms & deployment schedules. Here’s how we calculated them :

It’s hard to model the payments because some of the contracts are hardware spending (Nvidia, AMD, Broadcom) while others are cloud services (Microsoft Azure, Oracle Cloud, AWS), each with different payment structures & deployment timelines. Additionally, some contracts include chip design costs (like Broadcom’s custom AI accelerators), further complicating the spending distribution.

Contract Structures : The estimates reflect accelerating deployment starting after 2027, with 2025-2027 representing the ramp-up period & 2028-2030 showing peak deployment with growth rates of 124%, 54%, & 70% respectively. Oracle’s $300B contract : We assume a ramp-up period in 2027 ($25B) as infrastructure comes online, reaching full $60B annual run rate in 2028-2031, then completing with $35B in 2032. This assumption reflects realistic deployment timelines : Oracle’s massive data center buildout requires initial site preparation & infrastructure scaling before reaching full capacity. All other vendors follow deployment-based patterns starting from small initial commitments ($2B-$4B) & accelerating as large-scale infrastructure deployments come online. The spending curves reflect physical & financial realities : you can’t deploy 10 gigawatts of infrastructure overnight.

Microsoft ($250B total) : Based on incremental Azure services commitment announced in October 2025. Contract duration not disclosed. We estimated 7 years (2025-2031) consistent with AWS’s 7-year contract structure. Spending starts at $2B in 2025 & accelerates after 2027 : $10B (2028), $20B (2029), $60B (2030), with the remaining spend allocated to 2031 as large-scale deployments peak.⁷

Nvidia ($100B total) : Nvidia invests up to $100 billion in OpenAI for non-voting shares. OpenAI commits to spending on Nvidia chips across at least 10 gigawatts of AI data centers. First deployment begins in H2 2026 using the Nvidia Vera Rubin platform.⁸

AMD ($90B total) : Based on 6 gigawatt commitment & H2 2026 deployment start. AMD’s partnership announcement explicitly states “$90 billion in cumulative hardware revenue potential” from this agreement.⁹

Oracle ($300B total) : The most concrete, $60B annually for five years, as stated in multiple Oracle earnings calls & confirmed by CEO Safra Catz. We model this as a ramp-up period in 2027 ($25B) as infrastructure comes online, reaching full $60B annual rate in 2028-2031, then $35B in 2032 to reach the $300B total. This reflects Oracle’s Stargate data center buildout timeline & realistic deployment constraints.¹³

Amazon AWS ($38B total) : Based on announced 7-year agreement signed November 3, 2025. OpenAI commits to $38B over seven years for access to hundreds of thousands of Nvidia GB200 & GB300 GPUs on Amazon EC2 UltraServers. Deployment begins immediately with all capacity targeted for end of 2026.¹¹ We estimated deployment spending with geometric growth : $2B in 2025 (partial year starting November), ramping through 2027-2030 ($4B → $6B → $10B → $11B), then completing with $2B in 2031.

CoreWeave ($22.4B total) : Based on reported $11.9B initial contract, $4B expansion in May 2025, plus $6.5B expansion in September 2025, bringing total contract value to $22.4B.¹⁴ Note : CoreWeave also provides compute capacity to Google Cloud, creating an interesting three-way dynamic where Google resells CoreWeave’s Nvidia-powered infrastructure.¹⁵

These estimates carry ±30-50% error margins. Actual spending depends on deployment pace, hardware costs, & contract amendments.

Accounting Treatment: Design vs. Manufacturing

A critical complication in estimating OpenAI’s cost structure is determining how much of chip-maker deals like Broadcom represent design services versus manufactured hardware, & how each flows through the income statement.

The Broadcom Deal Structure :

OpenAI & Broadcom collaborated for 18 months designing custom AI accelerators optimized for inference. OpenAI designs the chips, Broadcom provides IP licensing & engineering services, & TSMC manufactures using 3nm process technology. The $350B estimated value represents deployment through 2029, but financial terms weren’t disclosed.

Two Different Accounting Treatments :

Phase 1 : Design & Development (R&D Expense)

• Chip design costs, IP licensing, & engineering services from Broadcom
• Under US GAAP, all internal R&D must be expensed as incurred
• Industry benchmarks : Advanced AI chip design ranges $200M-500M for NRE (non-recurring engineering) at 3nm process nodes
• This hits the income statement immediately as R&D expense, not COGS
• Likely represents <5-10% of total deal value

Phase 2 : Manufacturing & Deployment (Capitalized → COGS)

• TSMC wafer fabrication, assembly, testing, packaging
• Broadcom networking & rack integration
• Once manufactured, chips are capitalized as Property, Plant & Equipment
• Depreciation over useful life (typically 3-5 years) flows to Cost of Revenues (COGS)
• Likely represents 90-95% of total deal value (~$35B per gigawatt based on industry benchmarks)

Why This Matters for Gross Margins :

The table showing implied revenue at OpenAI’s target margins assumes all infrastructure spending flows through COGS. This simplification works reasonably well because:

• Hardware manufacturing dominates total costs (90-95%)
• Cloud service deals (AWS, Oracle, Microsoft) are immediate COGS expenses
• Design NRE, while expensive ($200M-500M), is relatively small versus $1T+ in total spending

However, the true accounting is more complex : upfront design costs hit R&D immediately (worsening near-term operating margins), while manufactured chips depreciate over 3-5 years (smoothing COGS impact). Without disclosed contract terms splitting design services from hardware purchases, precise gross margin modeling remains challenging.

Comparison to Cloud Deals :

AWS ($38B/7 years) & Oracle ($60B/year) are cloud services, immediate COGS expenses with no capitalization benefit. The AWS deal alone represents ~$5.4B/year in direct COGS, making it particularly impactful for gross margins despite being a smaller absolute dollar commitment than hardware contracts.