Lenny RachitskyModify

2026-07-06 23:01:17

Sonnet 5 review: I ran 64 generations to find out if it’s worth it

Brought to you by:

• Runway—The creative AI platform for images, video, and more

• Hyperagent—Deploy fleets of agents that handle real work

Claire puts Anthropic’s new Sonnet 5 through a real benchmark. She builds the How I AI Bench live using Claude Code, then blind-tests Sonnet 5 against Sonnet 4.6, Opus 4.8, GPT-5.5, and Gemini 3 Pro across PRDs, prototypes, agentic tasks, and agent personality. She breaks down what won, what failed, and how builders can create their own repeatable benchmark before trusting the next model release.

Biggest takeaways:

1. Sonnet 5 is priced closer to previous Sonnet models than to Opus, but it doesn’t automatically replace either one. At $2 per million input tokens and $10 per million output tokens through the end of summer, Sonnet 5 sits in an interesting middle band. In Claire’s benchmark, it finished near the bottom of her personal preference ranking, which means the cost argument only holds if the quality argument also holds for your specific use case.

2. One-off vibe checks feel useful, but they’re not repeatable—and repeatability is what makes a benchmark actually matter. Claire has tested GPT-5.5, open-weight models like GLM-5.2, and others this way before, but she could never compare results across time. The How I AI Bench fixes that by using frozen inputs, a fixed rubric, and the same tasks every time a new model comes out.

3. Claude Code can read old session history and use it to generate benchmark ideas tailored to a person’s actual work. Claire gave Claude Code a simple prompt asking it to brainstorm eval tasks based on what they’d worked on together, and it pulled from stored sessions on her desktop. Builders can do the same with Codex. That context is sitting there unused for most people.

4. Building an HTML scoring page to rate outputs based on gut feel and export JSON takes maybe 45 minutes with Claude Code, and Claire argues it’s worth every minute. She scored 64 generations across five models by hand, gave each one a 1-to-5 gut score, added loose notes, and found that the human signal turned out to be the most useful part of the whole benchmark.

5. LLM-as-judge evals are too generous and cluster toward the middle of the scale. Claire had both GPT-5.5 and Opus 4.8 judge the outputs, and neither was spiky enough. They missed things she flagged immediately on a visual pass, like broken prototypes and ignored wireframe constraints. Models can’t yet see what the human eye catches in the first screenshot.

6. Claire’s taste and the automated benchmark disagreed almost completely, and she thinks her taste was at least partly right. The LLM judges ranked Gemini 3 Pro highest and Sonnet 4.6 lowest. Claire’s ranking was almost exactly the opposite. When she ran a 70/30 Claire-to-LLM weighted index, Sonnet 4.6 jumped to first. That divergence tells her the eval rubric needs to encode more of what she actually cares about before she can trust the automated scores.

7. Sonnet 4.6 is still Claire’s choice for daily agent work because of its personality, not its benchmark scores. She pays for API credits to run her OpenClaw on Sonnet 4.6 specifically because she likes how it talks to her. No other model in this test matched it on the voice eval, which asked things like “ugh, deploys are red again” and waited to see how the model responded.

8. For builders, Claire recommends GPT-5.5 for PRDs, Sonnet 4.6 for prototypes and chitchat, and Opus 4.8 or Sonnet 5 for codebase navigation. Those are the task-by-task recommendations that came out of the Claire-weighted index. Complex, dense UI work is where Opus 4.8 still earns its price premium; for everything simpler, Sonnet 4.6 holds up.

9. The How I AI Bench is version one, and a lot of it needs to get sharper. The agentic bug-hunting task turned out to be too easy: every model aced it, which means it can’t differentiate between good and great. Claire plans to retire that task, encode more of her taste into the rubric, and keep running the benchmark blind every time a new model drops. The goal is to make this a benchmark the labs actually care about.

How I run autonomous coding agents from my phone with OpenAI Symphony + Linear | Alessio Fanelli (Kernel Labs)

Brought to you by:

• Firecrawl—Power AI agents with clean web data

• Jira Product Discovery—Prioritize with insights, build with confidence

Alessio Fanelli, the founder of Kernel Labs and co-host of Latent Space podcast, shows Claire how he manages autonomous coding agents from his phone using OpenAI Symphony, Linear, and a cloud VPS. He walks through the shift from “agent prompter” to “agent manager,” explains why Linear works as a state machine for async agent work, and shares what he’s learned from tracking token costs, purging skills files, and giving agents better senses. He also demos a very different use case: using Codex with browser access to hunt for underpriced Pokémon cards on eBay.

Biggest takeaways:

1. The shift from “agent prompter” to “agent manager” is the unlock most people are still missing. Alessio described how early agentic workflows felt fine until the second or third intervention, when the friction of local runtimes and clunky interfaces killed momentum. Moving to a cloud VPS with multiple communication channels (Linear, shell, mobile) made real async management possible.

2. Symphony isn’t magic; it’s just a very opinionated Markdown spec that tells the model how to behave. Claire made this point explicitly during the episode, and it’s the most important corrective to the “complex agent orchestration” framing that intimidates people. The whole framework is a Markdown file, and the models are good enough now to lock to it faithfully.

3. Token cost tracking per task is the primitive that most agent setups don’t have, and it should be table stakes. Alessio showed tasks ranging from 15 million to 221 million tokens, and the 221-million-token job (making an app deployable on Vercel) made complete sense in hindsight. Without that ledger, you have no feedback loop for improving your specs or your tooling.

4. Purge your skills files every few months or they become a liability. Models have a strong tendency to add instructions rather than replace them, so a skills file that grows over time eventually contradicts itself. Alessio’s advice is to keep files short, tight, and explicit about what the agent needs to ask for, not exhaustive lists of every possible behavior.

5. AI’s biggest unlocked opportunity is businesses built on heterogeneous data. The category Alessio described—things like trading cards, vintage clothing, and fish inventory—has always been impossible to scale because the data is inconsistent, visual, and contextual. LLMs are the first technology malleable enough to handle that messiness without extensive preprocessing.

6. Giving agents better senses (screenshots, visual diffs, video) extends autonomous runs dramatically. Kernel Labs built Glimpse, a Playwright extension for coding agents, specifically because the bottleneck wasn’t orchestration but rather agents hitting ambiguity in the UI and calling for help. Better tooling at the perception layer keeps the run going.

7. Context offloading is an underrated AI use case, and it’s worth building deliberately. Alessio’s email monitoring setup gave him the certainty that nothing important was slipping through, which removed a low-grade background anxiety. The same logic applies to personal finance, inventory, and any domain where staying on top of information is taking cognitive bandwidth you’d rather spend elsewhere.

8. The Pokémon card demo is the clearest proof that AI is compressing the information advantage that scale used to provide. Finding underpriced PSA-graded cards at $10,000-plus price points was previously a function of having more time, more people, and more domain expertise than competitors. Codex with browser access and a custom pricing skill collapses that advantage to a single well-written prompt.

9. Small businesses are the most interesting AI story, and they’re being underreported. Alessio’s observation from Japan, that small two- and three-person operations are running happily and profitably, points to a different kind of AI opportunity than the enterprise narrative suggests. The leverage AI gives a one-person operation is asymmetric in a way that bigger organizations can’t replicate at the same cost.

If you’re enjoying these episodes, reply and let me know what you’d love to learn more about: AI workflows, hiring, growth, product strategy—anything.

Catch you next week,
Lenny

P.S. Want every new episode delivered the moment it drops? Hit “Follow” on your favorite podcast app.

2026-07-06 20:03:37

Alessio Fanelli, founder of Kernel Labs and co-host of Latent Space podcast, walks us through two very different AI workflows: (1) a fully autonomous coding setup using OpenAI Symphony + Linear, where Linear acts as a state machine and Symphony manages agents through the whole dev lifecycle with zero babysitting; (2) Codex with browser access searching eBay for underpriced Pokémon cards—autonomously browsing, extracting PSA certificate numbers, and flagging deals on $10K–$20K cards for his San Carlos card shop, Merlin Games.

What you’ll learn:

1. Why “agent manager” is a better mental model than “agent prompter”

2. Why local Mac Minis don’t scale, and what a cloud VPS unlocks

3. How to wire Symphony and Linear together as an agent state machine

4. How to track token costs per task (and what 221 million tokens buys you)

5. What Glimpse does, and why better agent senses extend autonomous runs

6. Why your CLAUDE.md probably needs a full purge, not more instructions

7. How Codex scouts underpriced $10K Pokémon cards on eBay at scale

8. The new category of small business that AI just made possible

Brought to you by:

Firecrawl—Power AI agents with clean web data

Jira Product Discovery—Prioritize with insights, build with confidence

In this episode, we cover:

(00:00) Intro

(02:24) Prompter vs. agent manager

(04:31) Live demo: Symphony + Linear

(09:31) Setting up Symphony

(14:15) Purging your skills files

(18:06) The benefits of this system

(19:10) Demo: Using Codex to hunt for Pokémon cards

(24:17) The benefit of AI for small businesses

(28:23) Lightning round

Tools referenced:

• OpenAI Codex: https://openai.com/codex

• OpenAI Symphony (open-source framework): https://github.com/openai/symphony

• Linear (project management/agent state machine): https://linear.app

• PSA (Professional Sports Authenticator) grading: https://www.psacard.com

• TCGplayer (card pricing): https://www.tcgplayer.com

• eBay (used for card price scouting): https://www.ebay.com

Other references:

• Meta Ray-Ban glasses: https://www.ray-ban.com/usa/ray-ban-meta-smart-glasses

• The Monk and the Riddle by Randy Komisar: https://www.amazon.com/Monk-Riddle-Creating-Making-Living/dp/1578516447/ref=sr_1_1

• The Divine Comedy by Dante Alighieri: https://www.amazon.com/dp/0451208633

• AS Roma (football club Alessio and Claire are both fans of): https://www.asroma.com/en

Where to find Alessio Fanelli:

X: https://x.com/FanaHOVA

Latent Space podcast: https://www.latent.space/

Where to find Claire Vo:

ChatPRD: https://www.chatprd.ai/

Website: https://clairevo.com/

LinkedIn: https://www.linkedin.com/in/clairevo/

X: https://x.com/clairevo

Production and marketing by https://penname.co/. For inquiries about sponsoring the podcast, email [email protected].

2026-07-05 00:58:47

👋 Hello and welcome to this week’s edition of ✨ Community Wisdom ✨ a subscriber-only email, delivered every Saturday, highlighting the most helpful conversations in our members-only Slack community.

Read more

2026-07-01 07:22:23

I’ve been testing every major frontier model release since the start of the year, and when Anthropic dropped Sonnet 5, I wanted more than a vibe check. I got tired of one-off tests I couldn’t repeat or compare over time, so I built something better: the How I AI Bench, a repeatable eval harness I constructed live using Claude Code while recording this episode. I ran Sonnet 5 blind against four other frontier models (Sonnet 4.6, Opus 4.8, GPT-5.5, and Gemini 3 Pro) across PRD quality, prototype generation, agentic task completion, and agent personality. The results were not what I expected.

What you’ll learn:

1. What Anthropic claims Sonnet 5 improves over Sonnet 4.6, and where the benchmark data actually backs that up

2. How I built the How I AI Bench in under 45 minutes using Claude Code, starting from my own stored session history

3. Why I combined human vibe scoring (70%) with LLM as judge scoring (30%) instead of trusting either alone

4. How to set up a local HTML scoring page so you can rate AI outputs on gut feel and export those scores as JSON

5. Which model I recommend for PRDs, which for complex prototypes, and which for chatting with an agent daily

Brought to you by:

Runway—The creative AI platform for images, video and more

Hyperagent—Deploy fleets of agents that handle real work

In this episode, we cover:

(00:00) Sonnet 5 is out

(01:55) What Anthropic claims

(04:02) Why I’m done with one-off vibe checks

(05:05) Building the How I AI Bench live with Claude Code

(07:42) The scoring system

(10:43) Agent voice eval

(11:57) Quick recap

(13:58) Results: The How I AI index leaderboard

(21:21) What I’m improving for the next run

(22:16) Generating a Claire-weighted index

(23:53) Model-by-task recommendations

Tools referenced:

• Claude Sonnet 5: https://www.anthropic.com/news/claude-sonnet-5

• Claude Opus 4.8: https://www.anthropic.com/news/claude-opus-4-8

• GPT-5.5 (OpenAI): https://openai.com/index/introducing-gpt-5-5/

• Gemini 3 Pro (Google DeepMind): https://deepmind.google/models/gemini/pro/

• Cursor: https://www.cursor.com/

Other references:

• SWE-bench Pro (agentic coding benchmark referenced): https://www.swebench.com/

Where to find Claire Vo:

ChatPRD: https://www.chatprd.ai/

Website: https://clairevo.com/

LinkedIn: https://www.linkedin.com/in/clairevo/

X: https://x.com/clairevo

Production and marketing by https://penname.co/. For inquiries about sponsoring the podcast, email [email protected].

2026-06-30 21:31:39

👋 Hey there, I’m Lenny. Each week, I answer reader questions about building product, driving growth, and accelerating your career. For more: Lenny’s Podcast | Lennybot | How I AI | My favorite AI/PM courses, public speaking course, and interview prep copilot

Subscribe now

P.S. Get a full free year of Google AI, Cursor, Lovable, Notion, Manus, Replit, Gamma, n8n, Canva, ElevenLabs, Factory, Wispr Flow, Fin, Supabase, Bolt, Linear, PostHog, Framer, Railway, Granola, Warp, Gumloop, Magic Patterns, Mobbin, Stripe Atlas, and ChatPRD, by becoming an Insider subscriber. Yes, this is for real.

For years, the PM job drifted toward coordinating and aligning people. That version of the role is fading. Today, the best PMs at the best companies are prototyping with real code, querying data conversationally with MCP, confidently running coding AI agents, and finding countless ways to increase their leverage with AI.

To help you navigate this shift, I’m excited to announce that I’ve co-developed a course with my frequent collaborator and new Head of Education, Colin Matthews. It’s called Become an AI-Native Builder, and Colin will teach you how to best use the latest AI tools—including Codex, Claude Code, Cursor, and many of the Product Pass products—in your day-to-day work as a PM. You’ll use skills and MCPs to support discovery, create prototypes using your real codebase, ship changes to production using GitHub, and set up evals to automate and improve the quality of your work. This is the course I’ve always wished existed. And it’s not just for PMs. It’s great for designers, ops, researchers, sales, and basically anyone who’s non-technical and wants to get their hands dirty.

The first cohort kicks off on July 13th. If you’re a Lenny’s Newsletter annual subscriber you’ll get $600 off (and Insiders get $1,000 off). Grab your discount code here and sign up today.

In addition to this course, Colin is hosting a handful of free workshops with leaders from OpenAI, Cursor, Linear, Replit, and Lovable. These will be live and hands-on, and you’ll learn and practice new skills alongside other people in the trenches. These workshops are exclusive to paid Lenny’s Newsletter subscribers, and you can sign up for them here.

Colin is one of the most talented instructors of AI that I’ve come across. We’ve done four guest posts together (one is my third-most-popular post of all time) and, like me, he’s low on hype and high on pragmatic advice. He’s taught AI and other technical skills to tens of thousands of PMs at leading companies like OpenAI, Google, Stripe, Figma, Microsoft, and more. He’s a longtime product leader, a founder, and has shipped more than 10 SaaS products solo.

To mark the launch of this course, Colin wrote a guest post that will help you understand what’s possible with AI right now and where you stand on the “ladders of leverage.”

Let’s get into it.

I’ve trained over 30,000 PMs on how to integrate AI into their workflows. Early this year, I noticed a shift. Whereas before I heard from executives that they expected teams to be using AI for just basic prototyping and general productivity, now they’re increasingly looking for their employees to use AI to complete entire tasks. After building bespoke training programs to level up product and design ICs across industries like healthcare, legal, and streaming, I saw two main gaps: knowing what level of AI to reach for and having the technical skills to leverage AI tools to their fullest.

I wanted to share a forward-looking framework on how the most AI-native PMs are operating in mid-2026, and how you can create much more leverage for yourself with AI.

Think of this framework as three ladders, each for a different type of leverage that AI can give you.

Personal leverage helps you check items off your own to-do list at work. Product leverage accelerates your ability to ship the right things more quickly. And systems leverage helps build repeatable steps to consistently outsource work to AI and get high-quality results.

As you ascend each ladder rung, you get an order of magnitude more leverage. On the first rung, you use AI for assistance in your own work. On the second rung, you pass tasks to AI and review the output. At the top of the ladder, AI completes multi-step tasks and checks its own results. You will always apply some level of review at the end, but increasing leverage frees you up for other work.

Not every task, workflow, or company demands that you move up to the highest rung to get the most out of AI. The right rung on each ladder is about the best use of AI for the work in front of you.

I’ll walk through all three ladders, with examples you can start using today. Let’s get into it.

The personal leverage ladder

This is the most common way we all use AI at work: drafting docs, researching, or creating small artifacts. Most PMs are already pretty capable here, but it’s worth detailing the rungs so that you can see where you’re at and where you might go.

• Rung 1: You use AI to write text. You’re using AI to help you with PRDs, Jira tickets, emails, etc. You then copy and paste answers into other tools. Most people are at this rung, or even lower. Don’t feel bad if you’re hanging out here. There’s so much opportunity!

• Rung 2: You use AI to create artifacts. Think slides, basic Excel models, or small prototypes. Instead of generating text for you to copy, AI generates the actual artifact.

• Rung 3: You get AI to complete a full to-do item for you. You’ve connected your LLM to external products like Amplitude, Google Drive, Notion, and Canva so it can pull and push information as needed. You run a prompt or skill to complete a task you might have handed off to a colleague before, like reading through customer support tickets or analyzing A/B test results.

Let’s walk through an example of each rung to illustrate baseline expectations.

At the beginning, you’re simply talking to the AI. For example, if I wanted to create a PRD, I might ask Claude to help me write it with a prompt like this:

The AI has very little context on your company or what a good PRD looks like. You’ll likely talk back and forth until you get a good enough result, then copy-paste to Google Docs or Word to improve it before sharing with your team.

Next is getting the AI to do work instead of just helping you do the work. Continuing with the last example, you could have Claude generate a financial model that shows the cost of hosting an agent yourself vs. using a managed service like Vercel.

Here’s a prompt I recently used:

Create a model that represents costs if we build and host ourselves vs. using managed agents. Do research on the engineering time saved and the compute costs in self-hosted vs. managed. Look at other vendors, like Cloudflare, Vercel, or E2B that provide sandboxes for agents for pricing. Demonstrate both the cost of the pilot and the cost at scale in the model, assuming we have 5M+ agent instances running annually (where an agent instance is per hour).

And here’s what the resulting model looks like:

As mentioned, you should expect the output to need significant revision at this rung, but it’s a step forward from copy-pasting text from AI into a separate document.

You’re at the highest rung of personal leverage if you are able to delegate complete to-do items to AI. To illustrate this, I’ll use a fictional product called Stride. Stride is a Strava clone, where athletes can share their performance for running, swimming, and other activities. Let’s say I wanted to do a retention analysis of users who share their exercises with an attached photo, and compare that against the cohort who don’t share photos. To complete this task, I’ll connect my LLM of choice (Claude) to my product analytics software (PostHog) and give it instructions to run this analysis for me. This was my prompt:

Use PostHog to check if users who use social share features have a higher 30d retention than those who don’t. Show me an html doc as a final output visualizing cohorts and any other useful data. Cite all your sources so I can validate.

And here’s the result:

In the past, this would have been a task you’d fit in between meetings. Instead, you’re handing it off to an LLM to complete end-to-end. I’d recommend including “cite your sources” so you can easily validate if the output is correct. In this case, Claude provides links directly to the source data in PostHog.

Pro tip: To allow models to complete tasks for you (and thus move to this rung in your personal work), you’ll need to connect your LLM to the products you use frequently via MCP. Claude Code, Codex, and Cursor can all connect to tools like Figma, Amplitude, PostHog, Pendo, and more via MCP. This may sound complicated, but it’s really easy to do, and once you create this connection, you’ll never have to touch it again. Simply navigate to your product’s connectors marketplace and add your tools (Claude, ChatGPT, Gemini).

Once you have your connectors set up, try completing a common task using your AI, like:

• Analyzing how a launch went by reviewing recent customer tickets and online sentiment

• Checking how many users actually use a feature through product analytics events

• Summarizing a recording from a customer call and creating a prototype based on their feedback

• Updating your next sprint based on a change in roadmap priorities

You’ll likely find the results disappointing at first, but that’s just because the model doesn’t know how to meet your standards yet. Continue iterating with the model until you have a good result, then lock in the workflow by creating a skill—just ask your LLM to create one in the same chat you completed the task in.

You can repeat this workflow to get reasonable first drafts for almost anything: PRDs, roadmaps, marketing assets, survey analysis, prototypes, Figma mocks, and more. I totally acknowledge that the quality may not be perfect, but we’ll come back to that when we talk about systems leverage.

The product leverage ladder

Product leverage closes the gap between what you want to build and what you can ship, even without strong design or engineering chops. The way to leverage AI for this ladder breaks down into three rungs:

• Rung 1: You create web-based prototypes. These communicate your ideas better than docs, but the prototype itself doesn’t have any value beyond communication. And your prototype’s code is independent from your product—it doesn’t use your existing codebase.

• Rung 2: You create code-based prototypes. Claude Code or Codex accesses your real codebase as the context for generating prototypes, instead of screenshots or complex prompts.

• Rung 3: You get an agent to ship changes to production as pull requests. An engineer picks up the PR, reviews, and merges your code into the product.

Let’s spend a bit more time with each of these.

Using AI tools like Lovable, Replit, and Magic Patterns is an amazing way to quickly and easily create a prototype. You can use this to share a concept or design with stakeholders, customers, or internal team members, allowing you to validate whether your solution is usable and solves the customer problem faster than ever.

Let’s revisit Stride as an example. Here’s what the profile page currently looks like:

Customers have been complaining that they don’t have clear cancellation paths and are confused about their subscription status after a free trial. You can use AI prototyping to create a quick mock to test with customers and stakeholders if one or more of your solutions address this problem:

This helps get to the right solution by testing more concepts faster, but typically the underlying code doesn’t have any value. Web AI prototyping tools have limited context on our real components, pages, and data models, so this prototype is removed from reality. That means that it will take more work to translate the tested prototype to real code.

The next rung up is to ask AI to prototype with your real product. This does not require running the full product on your laptop—or an expert-level coding ability—but you will need some technical skills. First, use a product like Claude Code or Codex to write code and run your app. Second, you’ll need a codebase that contains your UI but not your full backend; more on that later.

You can create the same prototype, this time asking Claude Code or Codex to use your existing codebase:

This time, the cancellation flow is added to the existing settings page, uses real components, and follows the design patterns of the actual product.

To create a codebase or repo that’s easy to use and doesn’t require running the full backend, pair with an engineer and have them use a prompt like this on your main codebase:

Create a new repo that contains all of the base UI elements, styles, routes, pages, and components for [list parts of the product you want included]. Create a mock data store that mimics the API data model and is stored locally. I should be able to run the resulting repo without any environment variables or backend services.

Once complete, clone the new repo to your computer. You should have an easy-to-run version of your UI that you can’t accidentally mess up. Sometimes your engineer will tell you this process is more complicated to get running than a simple prompt. It is very much worth the effort to build prototypes on your real styles and components, so do your best to get over the hump!

The last rung in product leverage is getting an agent to ship code to production. This is a great example of where your technical judgment as a PM is critical. It’s possible that your billing change is only in the UI, and all backend APIs, data elements, and events already exist. It’s also possible that this would require new infrastructure or integrations with another team to ship.

As a PM, it does not make sense to spend time being a worse engineer than the rest of your team. Knowing when you should write a doc, ship a prototype, or create a PR is as important as the technical ability to complete these tasks. PRs are great for copy changes, small UI/UX tweaks, and changes to views that use existing backend code.

The systems leverage ladder

Read more

2026-06-29 23:02:35

GLM-5.2: why I’m replacing Opus in Claude Code with this new model

Brought to you by:

• Mercury—Radically different banking, loved by over 300K entrepreneurs

Claire tests GLM-5.2, the new open-weight model from Z.ai, inside her actual ChatPRD codebase. She runs it through codebase audits, UI redesigns, and a 45-minute autonomous bug-hunting task in Cursor and Claude Code, and breaks down where it surprised her, where it struggled, and why it may be good enough to replace Opus for some coding workflows.

Biggest takeaways:

1. Open-weight models are no longer a hobbyist curiosity—they are production-grade alternatives. GLM-5.2, built by Beijing-based Z.ai, benchmarks near Claude Opus 4.8 and above GPT-5.5 on SWE Bench Pro, with a million-token context window and full support for reasoning mode, function calling, structured output, and context caching. The decision is no longer about capability ceilings but, instead, about cost, control, and vendor dependency. Claire’s live testing confirmed it: this is not a toy.

2. Self-hosting changes the vendor power dynamic in ways that matter at scale. Open-weight means the trained model weights are publicly available, letting teams run inference on their own hardware, fine-tune on proprietary data, and route around any single provider’s API terms. When frontier labs change pricing or policy, teams using open-weight models can switch inference providers without touching a line of application code. The key: you’re not locked in.

3. Getting GLM-5.2 running in Cursor took 30 minutes, and Claire documented the undocumented part. Route your API key through Open Router, override the OpenAI base URL in Cursor’s settings to openrouter.ai/api/v1/cursor (the /cursor suffix isn’t documented anywhere), and add z-ai/glm-5.2 as a custom model. Claude Code requires two environment variable changes and one edit to claude/settings.json. Total time: under an hour, once you have the exact strings.

4. The 45-minute autonomous task revealed both the ceiling and the floor. Claire gave GLM-5.2 a single prompt inside Claude Code: pull the last 72 hours of Sentry errors and Vercel logs, then build a prioritized bug-fix plan. Over 45 minutes, it ran MCP tool calls, authenticated into external services, and produced a dark-mode engineering canvas with 20 Sentry errors, five Vercel log signals, and 14 planned fixes, including two P0s Claire hadn’t spotted through normal monitoring. The model surfaced signal-to-noise issues in their error pipeline that weren’t showing up elsewhere.

5. It hit a wall with React, then recovered. During the long-running task, GLM-5.2 struggled with TypeScript compilation errors before eventually producing clean React output. Claire’s read: HTML and CSS generation is reliable; React under agentic, multi-step pressure is shakier. For teams whose codebase is primarily React (she estimates it covers 98% of her own use), this is the friction point to test before committing the model to critical paths.

6. The cost math is striking: $3.36 for 6 million tokens, including the full 45-minute agentic session. A 72% cache rate helped, but even at full price, open-weight inference through Open Router sits well below Opus or GPT-5.5 rates for equivalent coding capability. For agents accumulating long context windows over extended sessions (the exact workload where frontier model costs compound fastest), open-weight alternatives offer a structurally different cost curve.

7. Claire’s recommendation: put GLM-5.2 in rotation, not in the spotlight. She’s keeping it in Cursor for frontend and design work, and in Claude Code for long-running agentic tasks, alongside closed frontier models rather than as a replacement. The constraint she’s watching: can it handle her React-heavy workload at the same consistency she gets from Composer? If it can, the cost-and-control argument gets much harder to ignore.

No Figma. No Jira. No docs. How Gusto built a new product line with Claude Code | Eddie Kim (CTO)

Brought to you by:

• Magic Patterns—Prototypes that look like your product

• Jira Product Discovery—Prioritize with insights, build with confidence

Eddie Kim is the co-founder and CTO of Gusto. In this episode, he shares how a five-person team used Claude Code, a permanent Zoom room, and almost none of the usual product process—no PM, no Figma, no Jira, no long specs—to build Gusto Cofounder from scratch in just 10 weeks.

Biggest takeaways:

1. A five-person team with no process can outship a large team with full process, if AI handles the engineering. Eddie’s product launched at Gusto’s tier-one level after 10 weeks, starting from zero code. The constraint wasn’t a liability—it was the design. When AI does the building, coordination overhead doesn’t scale the engineering; it just slows it down. The key: strip process to what the team actually needs, then let AI fill the gap.

2. “Zero code to tier-one launch” is now a viable founding path. The team reached a production milestone at Gusto without a line of pre-existing code. This flips the assumption that early teams spend months on infrastructure before shipping anything real. With Claude Code as the primary builder, the initial sprint becomes about direction and judgment, not typing. It compresses the time between idea validation and real user contact from months to weeks.

3. No meetings, no Jira, no text threads. It shipped anyway. The team had no standup cadence, no ticket system, no async thread to resolve blockers. What replaced all of that: shared context held inside the AI loop. When the model carries state and the team is small and aligned, human coordination overhead becomes optional.

4. The technical stack for a production AI agent is shockingly minimal. The entire agent loop ran on Cloudflare Workers with the Vercel AI SDK. Nothing else. No proprietary orchestration layer, no third-party agent framework. Everything else was built in-house. Teams often over-architect before they’ve proven anything; Eddie’s stack is evidence that infrastructure minimalism accelerates the path to learning what the agent actually needs to do.

5. Building agents is not as complicated as the community makes it sound. An agent is an AI SDK running somewhere in the cloud, able to look up files and call tools. That’s the full definition. The complexity people fear (state management, orchestration, reliability) is solvable with the same judgment calls any backend system requires. Eddie’s team shipped one at production quality in 10 weeks without specialist AI infrastructure experience.

6. The “permanent Zoom” model of AI development changes how teams think about context. Claude Code running in a persistent loop means the model has continuous access to the codebase’s current state. That’s closer to having an engineer who never closes their laptop than a chat interface you query on demand. For small teams, this is the equivalent of a senior engineer who is always available, always current, and never needs onboarding after a break.

7. The lesson for founding teams isn’t “use Claude Code.” It’s “design your process for AI as a team member.” Most early teams graft AI tools onto a human-scaled workflow: standups, tickets, PRs reviewed by three people. Eddie’s team treated the AI as a primary contributor from day one and built their coordination model around that assumption. The result: a workflow that gets faster as the AI improves, not one that merely offloads tasks to it.

If you’re enjoying these episodes, reply and let me know what you’d love to learn more about: AI workflows, hiring, growth, product strategy—anything.

Catch you next week,
Lenny

P.S. Want every new episode delivered the moment it drops? Hit “Follow” on your favorite podcast app.