You Can't Read the Code Anymore - Verifying Software in the Age of AI

QCon San Francisco 2026

Session

You Can't Read the Code Anymore - Verifying Software in the Age of AI

Wednesday Nov 18 / 02:45PM PST, Seacliff ABC at Hyatt Regency, San Francisco

Register

$2,835, Conference (3 days)
Current pricing ends September 8th

Abstract

AI writes most of the code now, so the hard part of engineering has moved from authoring to the part we’re worse at - verifying. With the incentives all pushing toward shipping faster and faster - the rational move is to skim the diff, click through a little manual QA, and ship on vibes. Existing methods to reduce the risk - tighter specs, letting the agent write its own tests, reviewing the diff more thoroughly - don't help, because they all leave a human reviewing code they didn't write, and reviewing code you didn't write is much harder than writing it yourself. On anything old or high-stakes, that's how something implicitly relying on today's behavior becomes a regression a customer notices.

Why do we keep handing reviewers artifacts built for authoring? Instead, change what they review by having the agent generate and maintain a runnable test for every behavior the service exposes to the outside world - each readable in seconds - with everything a reviewer shouldn't care about stripped out: implementation, flakiness, maintenance. Review becomes tractable again: you go down the behaviors and say yes or no. We did this on our customer-facing gateway. The proof that mattered wasn't the 25K existing scenarios or their sub-minute runtime - it was an engineer who made a change the team had been avoiding because few understood its implications and, reading in plain English that nothing else moved, shipped it without so much as flinching.

What makes this possible now and didn't before is that AI dropped the cost by orders of magnitude. What once would have taken a career now takes an afternoon and a few hundred dollars in tokens. And there's a turn we didn't see coming: the plain-English suite's biggest payoff wasn't only safety, but finally being able to read what our systems actually do. I'll show how we built ours and how any team can do the same with tools they already know - a coding agent and Cucumber - together with a harness that keeps it honest. We call ours Telem.

Key Takeaways:

  • Reviewing code you didn't write is harder than writing it yourself, so judge the system first by its observable behavior - all of it, verified automatically, not the handful of paths a person can check by hand.
  • Give reviewers an artifact built for reviewing, not authoring: have your coding agent generate and maintain a plain-English behavioral spec - one scenario per externally observable behavior - reviewed instead of the diff.
  • Before changing code you're afraid of, have an agent capture today's behavior as characterization tests, then make the change against them - the approach was always sound; only the manual cost made it impractical, and that cost is gone.
  • Trust comes from the harness, not the prompt: strict validation, full permutation coverage, and faking all I/O, which makes runs deterministic and fast enough to get the whole suite's answer in seconds.
Register

$2,835, Conference (3 days). Current pricing ends September 8th. All pass options.

76% senior dev or higher
1:11 speaker ratio
60+ practitioners

QCon San Francisco 2026 is a three day conference for senior software engineers, architects and team leads. An international program committee of working engineers selects every session. Patterns and practices, not products and pitches.

Share

Current pricing ends September 8th
$2,835, Conference (3 days)

Register