Designing AI Platforms for Reliability: Tools for Certainty, Agents for Discovery

QCon San Francisco 2025

Session AI

Designing AI Platforms for Reliability: Tools for Certainty, Agents for Discovery

Monday Nov 17 / 10:35AM PST, Ballroom BC at Hyatt Regency, San Francisco

Abstract

Modern AI platforms don’t have to choose between deterministic precision and probabilistic exploration—they need both. Deterministic tools provide the certainty required for high-stakes operations like transactions, security, and compliance, while probabilistic agents bring adaptability and discovery to complex, evolving problems. In this talk, we’ll explore how to design platforms that combine these modes effectively: long-running agents grounded by frequent truth checks, tools that guarantee reliable outcomes where variability is unacceptable, and hybrid systems that thrive in uncertainty when the right tool for the job is probabilistic reasoning. Using real-world examples—from detecting anomalous clusters to health agents debating diagnostic hypotheses—we’ll show how this dual-layer approach leads to platforms that are not only more capable, but also more trustworthy.

Interview

Generally my session is about how one weaves together traditional software and platforms with new forms of agentic software that are usually stochastic. This is often framed as an either/or choice. This is one we should reject in favor of “right tool for the job”.

This is critical because nearly every company is doing work to deploy AI, yet often struggling to product ionize it, often due to mismatched expectations around what AI does well and what it does less well with current tools. Getting this right is critical.

The common challenge is often knowing when to apply a stochastic AI driven approach, what the patterns are in agent development (and how they are different), and how you interleave these well so that the whole is more than the sum of the parts. How, for example, do you build in a grounded deep research feature into a product in a manner that doesn’t run off into incoherent reasoning chains?

One immediate implication is that we should be building tool catalogs for AI agents to use. This goes beyond mere “here are all your MCP endpoints”, and more about giving the agents context to properly select what tool to use for what problem in the first place.

What makes QCon stand out is that the presenters are builders, not marketers, not trying to drive hype.

Topics

AI Platform Engineering Architecture
76% senior dev or higher
1:11 speaker ratio
60+ practitioners

QCon San Francisco 2025 is a three day conference for senior software engineers, architects and team leads. An international program committee of working engineers selects every session. Patterns and practices, not products and pitches.

Share

From the same track

Monday 17 November

10:35 Ballroom BC Session AI Designing AI Platforms for Reliability: Tools for Certainty, Agents for Discovery Aaron Erickson Senior Manager and Founder of the DGX Cloud Applied AI Lab @NVIDIA, Previously Engineer @ThoughtWorks, VP of Engineering @New Relic, CEO and Co-Founder @Orgspace 11:45 Ballroom BC Session Fleet Management Confidently Automating Changes Across a Diverse Fleet Casey Bleifer Senior Software Engineer @Netflix 13:35 Seacliff ABC Session AI/ML Secure Software Supply Chain: Risk Prediction at the Speed of Development Bishwajeet Paul Architect, Platform Engineering @JPMorgan Chase - Specializing in Solving Complex Challenges for the Developer Community 14:45 Pacific DEKJ Session CI/CD Keeping the Mainline Green Across Diverse Language Monorepos Dhruva Juloori Senior Software Engineer @Uber, Core Contributor to SubmitQueue (Uber's CI System at Scale), Expert in Machine Learning, Distributed Systems, and Developer Productivity 15:55 Seacliff ABC Session Rust Rust at the Core - Accelerating Polyglot SDK Development Spencer Judge Engineering Manager @Temporal Technologies, previously Senior Software Engineer @Transparent Systems, Senior Software Engineer @ Tableau Software 17:05 Ballroom A Session Vibe Coding Directing a Swarm of Agents for Fun and Profit Adrian Cockcroft Technology Advisor and Consultant @OrionX.net, Previously VP Open Source and Sustainability @Amazon, Cloud Architect @Netflix, Distinguished Engineer @eBay