Shortcast
AI Podcast Player

Short podcasts with real voices

Lenny's Podcast: Product | Career | Growth

Anthropic’s first technical PM on token maxing, the jagged edge, and living in the future | Dianne Penn

--% time saved
PodcastLenny's Podcast: Product | Career | Growth
Publisher/creatorLenny Rachitsky
Published
Shortcast updated

About this episode

Dianne Penn is Head of Product for Anthropic’s AI Research and Labs teams. She joined in 2023 as Anthropic’s first technical product manager, when the entire product team was five engineers, and has since helped ship every model from Claude 2 through Fable, and helped incubate Claude Code, MCP, Skills, computer use, tool use, and reasoning. Before Anthropic, she helped build Alexa’s AI at Amazon and, before that, traded high-yield bonds at JP Morgan Chase. In our in-depth conversation, we discuss: 1. What Anthropic’s early days were like 2. The inflection points that turned Anthropic from an underdog into the fastest-growing company in history 3. How exactly Claude got so good at coding 4. The eval-driven development loop her team is pioneering 5. How to find joy in AI when everything is moving this fast 6. Why Claude’s willingness to push back is key to its success 7. Where human judgment remains irreplaceable — Brought to you by: WorkOS —Make your app enterprise-ready, with SSO, SCIM, RBAC, and more Mercury —Radically different banking, now with Command — Episode transcript: https://www.lennysnewsletter.com/p/anthropics-first-technical-pm-on — Archive of all Lenny's Podcast transcripts: https://www.dropbox.com/scl/fo/yxi4s2w998p1gvtpu4193/AMdNPR8AOw0lMklwtnC0TrQ?rlkey=j06x0nipoti519e0xgm23zsn9&st=ahz0fj11&dl=0 — Where to find Dianne Penn: • LinkedIn: linkedin.com/in/dianne-na-penn — Where to find Lenny: • Newsletter: https://www.lennysnewsletter.com • X: https://twitter.com/lennysan • LinkedIn: https://www.linkedin.com/in/lennyrachitsky/ — In this episode, we cover: (00:00) Introduction (02:31) Early Anthropic days (08:55) Big milestones (13:50) Inside the exponential (20:02) Token maxing (23:30) Anthropic Labs and the incubation model (27:30) How the research role works (31:35) How to become a top researcher (35:18) Frontier model safeguards (39:38) Hiring in the AI era (44:16) Building an eval set (47:48) Evals vs PRDs (49:55) The importance of hands-on leadership (52:46) Finding joy in AI (58:10) How Dianne uses Claude (01:01:05) Avoiding overreliance on AI (01:03:50) The constitution that makes Claude better (01:07:11) AI writing and verification (01:11:40) Where human brains will continue to be valuable (01:14:10) Navigating AI with kids (01:16:26) Alignment, the future of the PM role, and burnout (01:21:54) Lightning round and final thoughts — Referenced: • Anthropic: https://www.anthropic.com • Golden Gate Claude: https://www.anthropic.com/news/golden-gate-claude • Dario Amodei’s website: https://darioamodei.com • Scaling Laws and Interpretability of Learning from Repeated Data: https://www.anthropic.com/research/scaling-laws-and-interpretability-of-learning-from-repeated-data • Tokenmaxxing: How Top Builders Use AI To Do The Work Of 400 Engineers: https://www.ycombinator.com/library/Pa-tokenmaxxing-how-top-builders-use-ai-to-do-the-work-of-400-engineers • Garry Tan on X: https://x.com/garrytan • Anthropic co-founder on quitting OpenAI, AGI predictions, $100M talent wars, 20% unemployment, and the nightmare scenarios keeping him up at night | Ben Mann: https://www.lennysnewsletter.com/p/anthropic-co-founder-benjamin-mann • Anthropic’s CPO on what comes next | Mike Krieger (co-founder of Instagram): https://www.lennysnewsletter.com/p/anthropics-cpo-heres-what-comes-next • Introducing Labs: https://www.anthropic.com/news/introducing-anthropic-labs • Louis CK | about airplane Wi Fi: https://www.youtube.com/watch?v=me4BZBsHwZs • What happens after coding is solved? | Fiona Fung (Manager of the Claude Code and Cowork Teams): https://www.lennysnewsletter.com/p/building-the-most-ai-pilled-engineering • The Anthropic Hive Mind: https://steve-yegge.medium.com/the-anthropic-hive-mind-d01f768f3d7b • How to build a company that withstands any era | Eric Ries, Lean Startup author: https://www.lennysnewsletter.com/p/how-to-build-a-company-that-withstands • Fallout on Prime Video: https://www.amazon.com/dp/B0CN4GGGQ2 • Fallout (video game): https://fallout.bethesda.net • Claude Tag: https://www.anthropic.com/news/introducing-claude-tag — Recommended books: • Crucial Conversations: Tools for Talking When Stakes Are High : https://www.amazon.com/dp/0071771328 • How to Raise an Adult: Break Free of the Overparenting Trap and Prepare Your Kid for Success : https://www.amazon.com/How-Raise-Adult-Overparenting-Prepare/dp/1627791779 • Incorruptible: Why Good Companies Go Bad... and How Great Companies Stay Great : https://www.amazon.com/dp/B0FWZZBPZB — Production and marketing by https://penname.co/ . For inquiries about sponsoring the podcast, email [email protected] . — Lenny may be an investor in the companies discussed. To hear more, visit www.lennysnewsletter.com

Loading episode data...

Episode summary

When Anthropic launched, I honestly thought they had no shot against OpenAI, and now here we are; my guest is Diane Penn, head of product for Anthropic’s research and labs, who’s helped ship everything from Claude 2 to Fable and spun up products like Claude Code, MCP, and Skills—welcome, Diane.

In 2023 we were tiny—five product engineers and one person on the API—and still figuring out who we were; a bottoms‑up culture and quick experiments like the one‑day “Golden Gate Claude” sprint helped us find our voice by turning research into playful, public product moments.

What flipped the switch from scrappy to breakout, and what milestones still stand out?

Training Opus 3 was a turning point—small coding‑focused training bets plus a whole company pulling together over winter break built trust and gave us an edge with long‑form code; later, Opus 4.5 landed because the model and a great vehicle—Claude Code—hit together, proving frontier products unlock frontier models and vice versa.

We feel on the hockey‑stick now; what’s it like inside that acceleration, and how should teams prepare?

You need adaptability, first‑principles thinking, and fast eval loops because capabilities emerge in jumps; build products that can surface new strengths quickly, and bring the org along with trust and tight feedback so plans can change the moment the model unlocks something new.

So the product has to catch up with the model, and sometimes even tell users what’s newly possible—right?

Exactly; scaling laws are smooth, but specific skills can spike discontinuously, so without strong evals and safety tests you miss both breakthroughs and risks.

Gary Tan says heavy token spend lets you live in the future today; how do you think about that?

Orient around experimentation outcomes, not spend; the best ideas come from people living in the models daily and working in public so discovery compounds across the team.

What’s labs, and why does it ship so many hits inside an already fast company?

Labs pulls on discontinuous, non‑roadmap bets—think Claude Code, Skills, MCP, Claude Design—by holding strong views on the theme and staying loose on prototypes; small pods iterate, learn from misses, and sometimes wait a model generation for the idea to click.

Help us demystify researchers—what are they actually doing all day, and how do you partner?

They mix long‑range vision—like computer use—with hands‑on iteration across areas such as coding, tool use, and test‑time compute; my team translates raw user pain into actionable, measured work by diagnosing failures, building evals, and speaking the language of training runs and data.

What makes a great researcher—or PM working with research—right now?

First‑principles thinkers who are deeply ambitious yet close to the details win; and regardless of seniority, PMs need to be hands‑on—sweat the tokens, ship, and carve time to keep a live feel for how quickly models are changing.

You’ve said evals are the new PRDs—how does that change product work, and are PRDs really dead?

For model‑driven work, the fastest way to define and measure user value is an eval; we still use PRDs to align large groups and explore ambiguous bets, but we also read transcripts, trace failure trajectories, and turn them into evals—like our early JSON‑schema suite that took “bad at instructions” and made it measurable and fixable.

We saw scrutiny ramp with Fable and Mythos; how did that reshape launch and UX?

As models get sharper, safeguards, red teaming, and pre‑release gates must level up; we also built graceful fallbacks so if we defer access, users still get a great Opus response immediately while we harden the safety package.

How do you personally use Claude as a PM beyond building features?

I use skills to prep tough conversations—like a Crucial Conversations coach—and treat Claude as a sparring partner that pushes back so my thinking gets clearer and my words land better.

Claude’s alignment and “constitution” somehow make it more interesting and helpful—why does that make it better, not smaller?

Useful AI knows when to disagree and act proactively, not just comply; pushback leads to better decisions, which is core to intelligence, agency, and trust.

AI writing still feels AI‑ish; will that change, and does authorship even matter?

We’re actively training for stronger tone and character, and the goal depends on context—let models draft routine updates while you verify, and keep your own voice for judgment‑heavy writing by thinking first, then iterating with Claude.

Where will humans remain most valuable as capabilities surge?

Judgment, persistence, and proactivity still compound through lived experience, and domain depth—like life sciences—remains early on the curve, which is why we’re investing in areas like Claude Science.

As a parent, what do you try to instill for this world?

Curiosity, persistence, and a strong inner voice—help kids form opinions and stand by them while they learn.

This pace can burn people out—how do you stay steady in the storm?

We treat building as a team sport—low ego, hive‑mind collaboration, backing each other up around launches—and I’m lucky to have support at home; nobody can run this marathon alone.

Any final take on the PM role as models get stronger?

We need more product people who go deep with users, turn messy pain into actionable work, and ship with curiosity; deciding what to build and proving it works matters even more now.

Lightning round time—favorite books, show, product, life motto, and a lesson from trading?

Books: How to Raise an Adult and Incorruptible by Eric Ries; show: Fallout for sharp wit and action; product: Claude Tag is already changing how we work; motto: my grandfather’s line that there’s always another level; trading taught me to be authentic, have conviction, and do the gritty follow‑through no matter your title.

This was a joy—PMs are safe, PRDs still live, and building with models beats talking about them; thanks, Diane, and thanks everyone for listening—subscribe, rate, and we’ll see you next time.

Download on the App Store
QR Code - Scan to download

Ready to save time?

Download Shortcast and get started today

Download on the App Store
QR Code - Scan to download