How to measure AI developer productivity in 2025 | Nicole Forsgren

Episode Overview

• The conversation deconstructs traditional engineering productivity metrics, arguing they are obsolete and easily gamed, especially with the rise of AI.
• It emphasizes the importance of "strategic velocity"—moving faster on the right things—over raw speed, which can lead to shipping low-value products more quickly.
• The discussion explores how AI is fundamentally changing developer workflows, shifting the focus from writing code to reviewing it and enabling new, more efficient work patterns.
• It provides a framework for improving Developer Experience (DevEx) by identifying sources of friction and measuring developer satisfaction to drive meaningful improvements.

Key Concepts

• Flawed Productivity Metrics: Traditional output-based metrics like "lines of code" are poor indicators of performance because they don't measure value and can be easily manipulated by AI tools.
• Strategic Velocity Over Raw Speed: The primary goal should be making smart, strategic decisions to ship valuable products, as increased speed without the right direction is counterproductive.
• AI's Impact on Developer Workflow: AI acts as an accelerator for coding but shifts the developer's role from writing to critically reviewing AI-generated code for quality, security, and correctness. It also enables new work patterns by helping developers enter a "flow state" faster.
• Identifying and Removing Friction (DevEx): Common "smells" of inefficiency include frequently breaking builds, flaky tests, and high-friction processes. Improving Developer Experience should be treated like a product by systematically identifying and removing these barriers.
• Measuring Developer Experience: To drive improvements, it's crucial to measure DevEx using targeted satisfaction surveys about specific tools and processes, which are more actionable than broad happiness surveys.

Quotes

• At 0:03 - "most productivity metrics are a lie." - In response to the idea that companies are trying to measure productivity, highlighting the fundamental problem with traditional metrics.
• At 0:21 - "we can ship trash faster every single day." - Emphasizing that increasing speed without a clear strategy to build valuable products is pointless.
• At 0:50 - "Now we can also make a 45 minute work block useful." - Highlighting how AI can change work patterns by helping developers get into a productive flow state much faster.
• At 29:03 - "You can always move faster, but faster for what? Are we making the right business decisions?" - Distinguishing between raw speed and strategic speed, emphasizing the importance of building the right thing.
• At 56:46 - "I don't love a happiness survey, because there are too many things that contribute to happiness." - Explaining the preference for measuring "satisfaction" with specific tools and processes over the broader, less actionable metric of "happiness."

Takeaways

• Abandon vanity metrics like "lines of code" and instead focus on whether your team is shipping valuable, strategic work.
• Treat Developer Experience (DevEx) as a product by systematically identifying and removing friction points like broken builds and slow processes to improve team velocity.
• Leverage AI to rethink workflows by reducing ramp-up time and making shorter work blocks effective, but be prepared for a new emphasis on reviewing and validating AI-generated code.
• To measure developer productivity, use targeted satisfaction surveys focused on specific tools and processes, as they provide more actionable insights than general happiness surveys.