Build AI that delivers in production.

When Airbnb launched its AI customer support assistant, the feature made the news — the guardrails built to make it safe to ship did not. Yet an AI agent that talks to millions of customers, remembers a conversation across turns, and can trigger account actions is a

Resilient systems do not stay resilient by standing still. They survive by adapting. But adaptation comes with risk: under sustained pressure, organizations and systems can slowly drift toward failure while still appearing to operate normally.

Run the same task through the same LLM twice, and you may get two materially different answers. This behavior is often dismissed as the stochastic nature of LLMs, but it creates a serious product risk.

Every reliable process in the world is a protocol: auctions, double-entry bookkeeping, brokerage account transfers, or APIs. Protocols reduce ambiguity but the world remains messy.

A voice agent looks like a chatbot with a microphone.

At Duolingo, anyone (engineer or not) can describe an internal tool and get a working app back, deployed against real company systems. That's BuildingHub, and 350 Duolingo employees have created at least one app with it.

As AI agents write more of the code Roblox engineers push, the hard part is no longer generating changes; it’s knowing which ones to trust.

This session will cover practical measures that you can take now to run agents more securely in production.  This will not cover topics like model training and alignment; rather it will focus on deploying an existing agent as safely as possible.

When something breaks, our first instinct is to focus on the thing that failed. But in complex systems, the failure is usually the result of many things going right, changing, and interacting in ways we didn't anticipate.

After deploying and improving hundreds of production agents, each handling millions of requests, we've seen the same patterns emerge in what works (and what doesn't) when trying to make agents better.

BlackRock’s Portfolio Management Group is transforming how investment teams conduct research by combining deep investment expertise with shared AI capabilities.

Most "AI for engineers" content is about typing faster. That's IC-level work. Staff engineers have a different question to answer. When AI raises the floor on what every engineer can ship, what is the staff engineer's job, and where does the leverage actually come from now?

Data platform sits at the intersection of infrastructure, governance, operations, and user experience.

Most teams can now ship an AI prototype by calling a foundation-model API. The hard part is knowing whether that system works when real users, messy data, and business consequences arrive.

Engineering is shifting from synchronous, line-by-line implementation to an asynchronous model in which agents execute, verify, and retry while humans own intent, architecture, constraints, and release decisions. 

Distributed systems are built around assumptions about workload behavior: connections have reasonable lifetimes, retries eventually stop, traffic spikes have recognizable causes, and application code produces somewhat predictable query patterns.

The LinkedIn Feed retrieval layer selects, within 120 ms, the handful of posts each member sees from hundreds of millions of candidates. This split-second decision shapes what over a billion professionals read and engage with every day.

Everyone talks about model quality. Fewer people talk about what happens when that model has to serve real traffic without falling over, or why the system ended up shaped the way it did in the first place.

As organizations race to build AI agents, many teams are encountering challenges that feel new: evaluation uncertainty, feedback loops, behavioral drift, exploration versus exploitation, and maintaining user trust in systems that continuously adapt.

But these challenges are not new.

Agentic tools amplify what’s already in the codebase. The agent works through your code, your architecture decisions, and the documentation around them. It follows what it can read and guesses at the rest.

Once you figure out how to use AI to make your organization productive, you will soon find you have a whole new set of problems. While some of those problems will be novel thanks to LLMs, many of them are just the result of there being "more stuff".

Parasail serves open-weight LLMs on distributed clusters of GPUs managed by Kubernetes. The typical performance optimization loop is to reproduce a problem or measure a regression, iterate on code or config, set up and run benchmarks, and decide whether the change is worth shipping.

As AI agents move from generating code to testing, deploying, and operating it, the important question is no longer “How autonomous is the agent?” It’s “What can this agent safely decide without a human?”

Coding agents produce 741% more code, and only 30% more software ships as a result. The gap is human review: people top out around 400 lines an hour and agents are not slowing down. This talk is for engineering leaders trying to decide what to do about that.

AI writes most of the code now, so the hard part of engineering has moved from authoring to the part we’re worse at - verifying. With the incentives all pushing toward shipping faster and faster - the rational move is to skim the diff, click through a little manual QA, and ship on vibes.

As agents evolve, prompts get unruly. Edge cases are patched with one-line hotfixes, or with stacked instructions of escalating severity when an issue proves stubborn. Over time, those instructions pile up into a brittle artifact, overfit to an aging model.