Automating the Web With MCP: Infra That Doesn’t Break

Summary

Disclaimer: This summary has been generated by AI. It is experimental, and feedback is welcomed. Please reach out to info@qconsf.com with any comments or concerns.

This presentation by Paul Klein focused on the architecture and implementation of Browserbase’s Model Context Protocol (MCP), which supports stateful browser automation at scale.

Main Points:

  • Architecture of MCP: The talk explained the infrastructure design that supports concurrent and session-isolated operations, which is vital to running parallel agents at production scale.
  • MCP as Runtime Environment: MCPs are emphasized not as mere tools, but as runtime environments with unique constraints.
  • Scaling Headless Browser Fleets: Participants were given actionable patterns for scaling, including managing cookies and sessions securely.
  • Infrastructure Challenges: The challenges of running browsing functions at scale, such as resource provision and multi-region availability, were discussed.
  • Tool Protocols and MCP: The presentation delved into MCP as a protocol that supports enhanced semantics over REST endpoints; this allows better integration with AI models.
  • MCP Implementation: The integration of MCP was discussed, including comparisons to REST APIs and emphasizing the importance of unifying tools across models to improve model interaction and accuracy.
  • Browser as a Universal Tool: The presentation emphasized the browser as a significant tool for web-based operations, enabling a wide range of functionalities from automation to interacting with web pages.

Takeaways:

  • Understanding the importance of infrastructure in AI agent-based web automation.
  • Insights into MCP and its role in enhancing the reliability and efficiency of web-based tool interactions.
  • Lessons learned from production-scale orchestration and building robust infra that withstands varied challenges.

This is the end of the AI-generated content.


Abstract

AI agents are only as strong as the infrastructure beneath them. In this talk, we’ll walk through the architecture behind Browserbase’s model context protocol (MCP), built to support stateful browser automation at scale. From concurrency models to session isolation, this talk dives into what it actually takes to run parallel agents in production, and what we learned the hard way. Expect live demos, real-world lessons, and infra blueprints you can steal.

Main Takeaways:

  • Infra-first thinking is crucial for agent-based web automation.
  • MCPs are not just tools, they’re runtime environments with real constraints.
  • Attendees will leave with:
    • Concrete patterns for scaling headless browser fleets.
    • Security tips for managing cookies and sessions safely.
    • Infra cost models and deployment blueprints.
    • Hard-won lessons from production-scale orchestration.

Interview:

What is the focus of your work these days?

We’re building a stateful, concurrency-safe browser infrastructure for AI agents. Our focus is making multi-agent orchestration reliable, secure, and scalable—without abstracting away the parts developers actually need to control, like context and memory.

What is the motivation behind your talk?

Everyone’s excited about agents but very few talks get into the gritty, technical realities of running browser-based agents at scale. We wanted to share the behind-the-scenes of how infrastructure decisions shape performance, reliability, and developer experience. If you’re building agents that touch the web, this is the stuff no one’s documenting yet—but should be.

Who is the target audience for this session?

Chief Innovation Officers, Engineering Leads, AI Engineers, Technical Leads, etc.


Speaker

Paul Klein

Founder @Browserbase, previously Director of Self-Service & Engineering Manager @Mux, Co-Founder & CTO @Stream Club, Technical Lead @Twilio Inc.

Paul Klein IV is a San‐Francisco‐based serial entrepreneur and engineer. After honing his chops at Twilio during its IPO and founding Stream Club—a live‐streaming platform acquired by Mux in 2021. In 2024 he launched Browserbase to give developers and AI agents fast, reliable, multi‐region headless‐browser infrastructure. In its first 16 months, Klein raised over $67 million (from investors like Notable Capital, Kleiner Perkins, CRV, and Okta Ventures). He views Browserbase as the “last‐mile” interface between large language models and the web, enabling end‐to‐end workflow automation far beyond traditional scraping.

Read more

Date

Tuesday Nov 18 / 05:05PM PST ( 50 minutes )

Location

Ballroom BC

Topics

AI/ML MCP Browser Automation

Slides

Slides are not available

Share

From the same track

Session

Dynamic Moments: Weaving LLMs into Deep Personalization at DoorDash

Tuesday Nov 18 / 03:55PM PST

In this talk, we’ll walk through how DoorDash is redefining personalization by tightly integrating cutting-edge large language models (LLMs) with deep learning architectures such as Two-Tower Embeddings (TTE) and Multi-Task Multi-Label (MTML) models.

Speaker image - Sudeep Das

Sudeep Das

Head of Machine Learning and Artificial Intelligence, New Business Verticals @DoorDash, Previously Machine Learning Lead @Netflix, 15+ Years in Machine Learning

Speaker image - Pradeep Muthukrishnan

Pradeep Muthukrishnan

Head of Growth for New Business Verticals @DoorDash, Previously Founder & CEO @TrustedFor, 15+ Years in Machine Learning

Session AI Agents

From Content to Agents: Scaling LLM Post-Training Through Real-World Applications and Simulation

Tuesday Nov 18 / 02:45PM PST

This talk presents a comprehensive journey through modern AI post-training techniques, from Pinterest's production-scale content discovery systems to enterprise agent training through Veris AI’s simulation.

Speaker image - Faye Zhang

Faye Zhang

Staff Software Engineer @Pinterest, Tech Lead on GenAI Search Traffic Projects, Speaker, Expert in AI/ML with a Strong Background in Large Distributed System

Speaker image - Andi Partovi

Andi Partovi

Co-Founder @Veris AI, Making AI Agents World-Ready

Session AI Agents

Engineering at AI Speed: Lessons from the First Agentically Accelerated Software Project

Tuesday Nov 18 / 10:35AM PST

Claude Code is the first developer tool built specifically to maximize AI development velocity.

Speaker image - Adam Wolff

Adam Wolff

Engineer and Individual Contributor to Claude Code @Anthropic, Previously @Robinhood, @Facebook

Session

Engineering AI for Creativity and Curiosity on Mobile

Tuesday Nov 18 / 11:45AM PST

This talk shares practical lessons from building production-grade AI for creativity and curiosity on mobile devices.

Speaker image - Bhavuk Jain

Bhavuk Jain

Tech Lead @Google

Session AI/ML

Improving Meta Generative Ad Text using Reinforcement Learning

Tuesday Nov 18 / 01:35PM PST

Reinforcement Learning with Performance Feedback (RLPF) unlocks a new way of turning generic GenAI models into customized models fine-tuned for specific tasks. This approach is especially powerful when combined with in-house data and performance metrics.

Speaker image - Alex Nikulkov

Alex Nikulkov

Research Scientist (RL lead for Monetization GenAI) @Meta