Speaker
Abstract
AI writes most of the code now, so the hard part of engineering has moved from authoring to the part we’re worse at - verifying. With the incentives all pushing toward shipping faster and faster - the rational move is to skim the diff, click through a little manual QA, and ship on vibes. Existing methods to reduce the risk - tighter specs, letting the agent write its own tests, reviewing the diff more thoroughly - don't help, because they all leave a human reviewing code they didn't write, and reviewing code you didn't write is much harder than writing it yourself. On anything old or high-stakes, that's how something implicitly relying on today's behavior becomes a regression a customer notices.
Why do we keep handing reviewers artifacts built for authoring? Instead, change what they review by having the agent generate and maintain a runnable test for every behavior the service exposes to the outside world - each readable in seconds - with everything a reviewer shouldn't care about stripped out: implementation, flakiness, maintenance. Review becomes tractable again: you go down the behaviors and say yes or no. We did this on our customer-facing gateway. The proof that mattered wasn't the 25K existing scenarios or their sub-minute runtime - it was an engineer who made a change the team had been avoiding because few understood its implications and, reading in plain English that nothing else moved, shipped it without so much as flinching.
What makes this possible now and didn't before is that AI dropped the cost by orders of magnitude. What once would have taken a career now takes an afternoon and a few hundred dollars in tokens. And there's a turn we didn't see coming: the plain-English suite's biggest payoff wasn't only safety, but finally being able to read what our systems actually do. I'll show how we built ours and how any team can do the same with tools they already know - a coding agent and Cucumber - together with a harness that keeps it honest. We call ours Telem.
Key Takeaways:
- Reviewing code you didn't write is harder than writing it yourself, so judge the system first by its observable behavior - all of it, verified automatically, not the handful of paths a person can check by hand.
- Give reviewers an artifact built for reviewing, not authoring: have your coding agent generate and maintain a plain-English behavioral spec - one scenario per externally observable behavior - reviewed instead of the diff.
- Before changing code you're afraid of, have an agent capture today's behavior as characterization tests, then make the change against them - the approach was always sound; only the manual cost made it impractical, and that cost is gone.
- Trust comes from the harness, not the prompt: strict validation, full permutation coverage, and faking all I/O, which makes runs deterministic and fast enough to get the whole suite's answer in seconds.
$2,835, Conference (3 days). Current pricing ends September 8th. All pass options.
QCon San Francisco 2026 is a three day conference for senior software engineers, architects and team leads. An international program committee of working engineers selects every session. Patterns and practices, not products and pitches.
From the same track
Wednesday 18 November
10:35 Seacliff ABC Session The Death of the Code Review Laurie Voss Head of Developer Relations @Arize AI, Co-Founder of npm, Inc., Developer for 30+ Years Code review was built for a world where humans wrote all the code. 11:45 Seacliff ABC Session Legacy as Leverage: How Brownfield Work Builds Technical Judgment Shine Garg Founder of Uncharted Path Breakthrough, Former Staff Software Engineer, 15+ Years Building and Scaling Backend, Data, Machine Learning, and Platform Systems Legacy systems are often treated as a necessary evil: work to be endured before moving on to more “real, strategic” projects. 13:35 Seacliff ABC Session Builder-Driven Development Jyothi Nookula Product Leader Director with 13+ Years Driving AI Product & Platform Innovation, Previously @Meta, @Amazon, and @Etsy Details coming soon. 14:45 Seacliff ABC Session You Can't Read the Code Anymore - Verifying Software in the Age of AI Omer van Kloeten Principal Engineer @Forter AI writes most of the code now, so the hard part of engineering has moved from authoring to the part we’re worse at - verifying. With the incentives all pushing toward shipping faster and faster - the rational move is to skim the diff, click through a little manual QA, and ship on vibes. 15:55 Seacliff ABC Session Breaking Down the Interface Between Code and Strategy Details coming soon.