Keeping the Mainline Green Across Diverse Language Monorepos

The presentation titled "Keeping the Mainline Green Across Diverse Language Monorepos" covers the challenges and solutions related to maintaining a stable and continuously integrating codebase across multiple language repositories at Uber.

Key Points Addressed:

• The Challenge: Ensuring an "always-green" mainline amidst hundreds of changes per hour across diverse monorepos, including languages like Java, Kotlin, Swift, Go, and Python, which bring distinct tooling and integration challenges.
• Current Scale: Uber has 4,500+ engineers across 10+ global centers working on polyglot monorepos, processing 65,000+ changes monthly and supporting over 100,000 deployments per month.
• Impact of a Broken Mainline: A non-stable mainline can lead to delayed rollouts, hampered productivity, and complex rollbacks, impacting business costs, time, and resources.

Solutions and Strategies:

• SubmitQueue System: Uber developed SubmitQueue, a CI scheduling system that provides a single queue illusion, optimizing CI resources and ensuring mainline stability.
• Speculation Trees and Prioritization: The system uses a binary decision tree model to predict build times and success probabilities, efficiently managing queue prioritization even during peak change volumes.
• Machine Learning Integration: Employed for predicting build outcomes, ensuring efficient resource utilization and maintaining high accuracy in success predictions.
• Recent Enhancements: Introduction of bypassing large diffs and improved build time prediction, resulting in significant resource usage reduction in CI and improved change landing times.

Overall, the presentation emphasizes the importance of a scalable, reliable CI system that can handle rapid change integration while maintaining codebase health.

This is the end of the AI-generated content.

At Uber’s scale, ensuring an always-green mainline while processing hundreds of changes per hour is a massive challenge— especially when those changes span multiple language monorepos supporting dozens of business-critical apps. Each language ecosystem brings its own set of tooling, test strategies, and integration requirements, making Continuous Integration of changes while maintaining green mainline increasingly difficult.

In this talk, I’ll share the design and evolution of MergeQueue— Uber’s language-agnostic CI scheduling system that enables mainline stability and production readiness across our diverse stack, including Java, Kotlin, Swift, Go, Python, and more. By leveraging speculative validation, build-time and change success predictions, and conflict resolution, we’ve built a unified, scalable CI engine that abstracts away language-specific complexity while maintaining speed and reliability.