Abstract
Resilient systems do not stay resilient by standing still. They survive by adapting. But adaptation comes with risk: under sustained pressure, organizations and systems can slowly drift toward failure while still appearing to operate normally. Using Rasmussen’s Dynamic Safety Model as a lens, this talk explores how AI is accelerating that drift at a pace many engineering organizations have not had to manage before.
Major technological upheavals are not new. The recent impact of AI reflects patterns we saw with cloud computing: early adoption created enormous leverage, architectural sprawl followed as systems rapidly evolved, bill shock forced a painful shift toward cost discipline, and platform maturity required new operational guardrails. AI is following a similar path, but faster. Token usage becomes capacity planning. Model selection becomes routing logic. Prompts, policies, and context become deployable artifacts. Agents become distributed actors with permissions and side effects. The architectural surface area is already extensive, and it continues to evolve.
But the more critical architectural shift is this: non-deterministic behavior is becoming part of the production control plane. The question is no longer simply, “Can AI do this task?” It is: “Can this system adapt across models, workloads, costs, latency, and uncertainty without silently drifting outside the limits of acceptable behavior?”
This is an inflection point for an industry that has spent decades building around deterministic interfaces, explicit contracts, repeatable tests, and observable distributed systems. AI does not replace those concerns; it complicates them. Production AI systems are still distributed systems, but now with probabilistic components, faster feedback loops, external model dependencies, and economic pressure embedded directly in the request path.
This session explores resilience engineering for production AI systems as AI and LLMs become embedded across engineering workflows and production architectures. We will look at the forces pushing systems toward the edge and the architecture patterns that help keep them inside a safe operating envelope: model gateways, cost-aware routing, QoS tiers, prompt and policy versioning, circuit breakers, fallback models, agent permissions, human-in-the-loop escalation, and observability for quality, cost, and behavior.
The goal is not to fear AI or romanticize the past. It is to recognize that when the boundaries move faster, resilience depends on our ability to sense, constrain, adapt, and correct before drift becomes failure with real consequences for people, systems, and businesses.
Interview:
What is your session about, and why is it important for senior software developers?
My session uses Jens Rasmussen’s “Dynamic Safety Barriers” model to examine three boundaries affecting organizational and system performance: workload, economic, and acceptable performance. Engineers can influence acceptable performance, while the other boundaries are largely shaped by market forces, new technologies, and executive decisions. Senior developers need to remain aware of how these boundaries are shifting so they can adapt as conditions change.
Why is it critical for software leaders to focus on this topic right now?
AI is challenging our assumptions about how these boundaries behave, and the effects are still unfolding. We are already seeing pressure to reduce the cost of LLM-driven agents, use agents to handle more work, and manage a significant increase in code changes reaching production. Senior engineers need to understand these shifts because failing to adapt could leave them unprepared for what comes next.
What are the common challenges developers and architects face in this area?
Developers and architects are usually most aware of the acceptable-performance boundary because their work directly affects it, and failures produce clear signals through monitoring and alerting. Economic constraints also shape their designs, while workload is often considered too late—sometimes only after an otherwise sound proposal places unsustainable demands on the engineers implementing it. AI can become a convenient way to dismiss these concerns, but doing so risks obscuring the trade-offs engineers must make in their day-to-day work. I explored the implications of this in a recent article I published on Medium that generated a lot of interest.
What's one thing you hope attendees will implement immediately after your talk?
I hope attendees leave with a clearer understanding of these boundaries and a greater sense of urgency about how they may influence their careers and the systems they build. Ideally, they will begin identifying and discussing these forces within their own teams immediately.
What makes QCon stand out as a conference for senior software professionals?
QCon stands out because of the quality of its talks and the care that goes into selecting and developing them. It is regarded as one of the best conferences for senior software professionals, and for good reason.
What was one interesting thing that you learned from a previous QCon?
I last attended QCon in 2019, but I still remember an excellent talk about remaining calm under pressure, whether speaking at a conference or during a company town hall. Another memorable talk explored neurodiversity in the technology industry. It helped me better support a neurodivergent engineer on my team by adapting both the work and the environment to suit his needs.
Speaker
Andrew Hatch
Engineering Leader and SRE Manager @Cisco ThousandEyes, With 25+ Years Building Software, Operations, SRE, and Platform Teams Across Australia, India, and the United States
Andrew Hatch is an engineering leader and SRE manager at Cisco ThousandEyes, with over 25 years in the technology industry across Australia, India, and the United States. He moved to the Bay Area in 2020 to join LinkedIn as an SRE Manager before taking up his current role at ThousandEyes. His work spans software engineering, consulting, operations, and building SRE and platform teams for large-scale systems. Andrew has previously spoken at SREcon on learning from complex systems and the realities of SRE management, and continues to explore how organisations can hire, lead, and learn more effectively in an AI-augmented world.
