Speaker
Abstract
Data platform sits at the intersection of infrastructure, governance, operations, and user experience. As AI becomes part of everyday engineering workflows, it creates new ways for platform teams to build and operate these systems: summarizing user needs, debugging failures, improving documentation, routing support, generating routine changes, and making self-service workflows easier to use. This talk explores emerging patterns for building data platform with AI while keeping reliability, trust, and governance at the center.
Key Takeaways:
- A practical pattern for AI assisted platform workflows: bringing together context from support threads, docs, logs, runbooks, and ownership metadata, using AI to summarize and propose next steps, and relying on validation checks before anything changes.
- How AI can improve platform operations through incident summaries, runbook suggestions, owner routing, repeated support pattern detection, and follow up tracking.
- How to design AI assisted self service so engineers can use the data platform more easily while access control, policy, auditability, and review stay in the underlying systems.
- Where to draw the boundary between AI suggestions, automated validation, and human approval for critical data platform workflows.
Topics
$2,995, Conference (3 days). Current pricing ends November 10th. All pass options.
QCon San Francisco 2026 is a three day conference for senior software engineers, architects and team leads. An international program committee of working engineers selects every session. Patterns and practices, not products and pitches.
From the same track
Tuesday 17 November
10:35 Seacliff ABC Session Breaking Down the Walls to Python & Big Data + AI Performance with Transpilation and LLMs Holden Karau Data Engineer @Snowflake & Co-Founder of Fight Health Insurance, Previously @Netfilx, @Apple, and @Google Many of us have a love hate relationship with Apache Spark, especially when it comes to PySpark. Especially in Python, the UDF performance can be a dumpster fire. 11:45 Seacliff ABC Session Data Infrastructure Building a Data Platform with AI Mouli Mukherjee Engineering Manager @OpenAI Data platform sits at the intersection of infrastructure, governance, operations, and user experience. 13:35 Seacliff ABC Session One Does Not Simply Add Synchronous Writes - Bridging Batch and Real Time with Apache Iceberg Paige Elinson Staff Software Engineer @Datadog What happens when a batch ingestion platform needs to behave like an operational database? Datadog’s Reference Tables product was designed for asynchronous uploads of large datasets, but customers increasingly needed synchronous row-level edits, without sacrificing batch scale. 14:45 Seacliff ABC Session Building a StreamHouse: Turning Kafka Streams into a Millisecond Query Layer AI agents have compressed business decisions from minutes to seconds. Data systems need to keep up. 15:55 Seacliff ABC Session From Datasets to Data Products: Powering Trusted AI for Social Impact When an AI agent acts on a nonprofit's behalf, it's only as trustworthy as the data underneath it. At Blackbaud, we build software for the social impact sector, and nonprofits have relied on our data for decades. 17:05 Seacliff ABC Session Iceberg V4: The Next Chapter in Table Formats Daniel Weeks MTS @Databricks, Co-Founder of Tabular and Co-Creator of Apache Iceberg, 20+ Years Designing & Developing Large-Scale Distributed Systems Apache Iceberg V3 adoption is growing and the community is already shaping what comes next. This talk walks through the emerging scope and priorities for the V4 table spec and progress of the proposals currently underway.