Load balancing

Session HTTP

How GitHub Copilot Serves 400 Million Completion Requests a Day

Monday Nov 18 / 03:55PM PST

GitHub Copilot is the largest LLM powered Code Completion service in the world, serving hundreds of millions of requests per day with an average response time of under 200ms. This is the story of the architecture which powers this product.

Speaker image - David Cheney

David Cheney

Lead, Copilot Proxy @GitHub, Open Source Contributor and Project Member for Go Programming Language, Previously @VMware