How GitHub Copilot Serves 400 Million Completion Requests a Day

GitHub Copilot is the largest LLM powered Code Completion service in the world, serving hundreds of millions of requests per day with an average response time of under 200ms. This is the story of the architecture which powers this product.

Speaker

David Cheney

Lead, Copilot Proxy @GitHub, Open Source Contributor and Project Member for Go Programming Language, Previously @VMware

David is an open source contributor and project member for the Go programming language. David is a well-respected voice within the tech community, speaking on a variety of topics such as software design, performance, and the Go programming language.

Read more

Date

Monday Nov 18 / 03:55PM PST ( 50 minutes )

Location

Ballroom A

Share

Share Share

From the same track

Session

Optimizing Search at Uber Eats

Monday Nov 18 / 11:45AM PST

Uber has an in-house search engine called Search In Action (SIA). As the backbone behind the feed and search capabilities of Uber's Delivery business, SIA plays a crucial role in expanding selection seamlessly for customers which is a strategic advantage to the business.

Speaker image - Janani Narayanan

Janani Narayanan

Applied ML Engineer @Uber, Previously Tech Lead on DynamoDB Control Plane (Early Stage), 10+ Years Tech Industry Experience

Speaker image - Karthik Ramasamy

Karthik Ramasamy

Senior Staff Software Engineer @Uber, 15 Years of Experience in Design and Implementation of Web Applications, Distributed Systems, Search and Analytics Infrastructure

Session

Supporting Diverse ML Systems at Netflix

Monday Nov 18 / 10:35AM PST

Netflix uses data science and machine learning across all facets of the company, powering a wide range of business applications.

Speaker image - David Berg

David Berg

Senior Software Engineer @Netflix, Previously @IBM Almaden Research Center, Ph.D in Computational Neuroscience

Speaker image - Romain Cledat

Romain Cledat

Senior Software Engineer @Netflix, Metaflow Core Contributor, Previously @Facebook and @Intel

Session

Unified Grid: How We Re-Architected Slack for our Largest Customers

Monday Nov 18 / 01:35PM PST

Slack’s enterprise solution allows users to join multiple workspaces within the same organization. However, for years, users could only view channels, messages, and other content from a single workspace at a time.

Speaker image - Ian Hoffman

Ian Hoffman

Staff Software Engineer @Slack

Session

Unconference: Architectures You've Always Wondered About

Monday Nov 18 / 02:45PM PST

Session

Modernizing Legacy Systems - Building an Event-Driven Architecture With a Mainframe

Monday Nov 18 / 05:05PM PST

Details coming soon