You are viewing content from a past/completed QCon -

Presentation: CI/CD: Lessons from LinkedIn and Mockito

Track: DevOps: You Build It, You Run It

Location: Ballroom BC

Day of week: Monday

Slides: Download Slides

Level: Intermediate

Persona: Chaos/Resiliency/SRE, DevOps Engineer, Technical Engineering Manager

What You’ll Learn

  • How continuous deployment can improve your organization's productivity.
  • Challenges, differences and similarities of CD at LinkedIn (large scale enterprise) and Mockito (OSS software library with huge user base).


LinkedIn and Mockito are two different use cases of implementing continuous delivery at scale. Yet the challenges, benefits and impact on the engineering culture are very similar.

In 2015, LinkedIn’s flagship application adopted a continuous delivery model we called 3x3: deploy to production 3 times a day, with a 3 hour maximum time from commit to production. At LinkedIn scale - hundreds of engineers building products for 500M users - implementing 3x3 was really hard. How did 3x3 change LinkedIn engineering culture and what we have learned on the way?

Mockito is a top 3 Java library with ~2M users. Even with that large user base, since 2014, the Mockito project has taken the surprising approach of publishing a new version of the library from every single pull request. This approach is challenging and innovative in the Java community, and Mockito leverages Shipkit to ship every change to production. Why did the Mockito team adopt continuous delivery in 2014 and what we have learned to date?

Join and learn from Szczepan Faber, the maker of Mockito framework since 2007, and the tech lead of LinkedIn Development Tools since 2015.


What do you do day-to-day?


Since many years my passion is developer productivity at scale. I started this back in 2007, when I attempted to change how Java engineers write unit tests with mocks (Mockito framework). Today I help with engineering productivity at LinkedIn in two ways.

Firstly, my team provides development tools and workflows to 3000 LinkedIn engineers. We manage code review, continuous integration, commit-to-publish pipeline, IDE integrations, dependency management and many more. We maintain cohesive engineering culture because every team is using standard development workflow. I tech lead the efforts of building great tools for developers.

Secondly, I am deeply involved in training and developer education programs. I often design and develop training because it really helps boosting productivity in engineering teams.


What is your motivation for this talk?


In the open source and in the enterprise continuous delivery is not yet adopted as widely as it should given how it helps with productivity. I hope engineering teams will experiment more with continuous delivery and try to push themselves to go into that model.


Who should come to your talk?


Developers and architects who want help teams increase their productivity by shipping to production frequently and automatically.


What can people come take away from this talk?


Learn about the challenges and advantages of adopting continuous delivery at scale. Get excited to reduce manual steps from commit-to-production pipeline. Get practical suggestions, too! Continuous delivery is hard but you really want to push great features to your customers faster.


What keeps you up at night?


There are challenges at LinkedIn that keep me excited and very busy. Reducing commit-to-publish time for every engineering team is a big one. It’s because shipping code faster is probably the most tangible way of increasing developer productivity.

Another challenge is dependency management at scale. How to manage version upgrades, dependency conflicts, large scale changes that require updates to thousands of codebases? In the Open Source I would love to push the community towards continuous delivery model for software libraries. Mockito framework paved the way for this model. Me and other enthusiasts from Mockito community work on a new release automation framework called “Shipkit” - a toolkit for shipping it.

I still contribute to Mockito regularly, and I focus on strategic API extensions that help other frameworks like Spring Boot or Powermock to cleanly integrate with Mockito. My goal is to ensure thoughtful evolution of Mockito API so that it remains the best mocking framework for Java.

Speaker: Szczepan Faber

Mockito Creator, Core Eng Gradle 1.x/2.x, & TechLead @LinkedIn Development Tools

Szczepan regularly gives lectures and instructs classes on development tools, engineering productivity, code review and testing. Since 2015, Szczepan Faber is a Tech Lead for LinkedIn Development Tools, responsible for developer productivity at LinkedIn. In 2011-2015 he was core engineer of Gradle 1.x and 2.x. and instructed numerous classes on build automation. Szczepan created Mockito framework in 2007, currently estimated user base of 2M, and has been giving classes on automated testing since. Szczepan publishes articles on LinkedIn, tweets as @MockitoGuy, makes tools on GitHub.

Find Szczepan Faber at

Last Year's Tracks

  • Monday, 16 November

  • Paths to Production: Deployment Pipelines as a Competitive Advantage

    Deployment pipelines allow us to push to production at ever increasing volume. Paths to production looks at how some of software's most well known shops continuous deliver code.

  • Java, The Platform

    Mobile, Micro, Modular: The platform continues to evolve and change. Discover how the platform continues to drive us forward.

  • Security for Engineers

    How to build secure, yet usable, systems from the engineer's perspective.

  • Modern Data Engineering

    The innovations necessary to build towards a fully automated decentralized data warehouse.

  • Machine Learning for the Software Engineer

    AI and machine learning are more approachable than ever. Discover how ML, deep learning, and other modern approaches are being used in practice by Software Engineers.

  • Inclusion & Diversity in Tech

    The road map to an inclusive and diverse tech organization. *Diversity & Inclusion defined as the inclusion of all individuals in an within tech, regardless of gender, religion, ethnicity, race, age, sexual orientation, and physical or mental fitness.

  • Tuesday, 17 November

  • Architectures You've Always Wondered About

    How do they do it? In QCon's marquee Architectures track, we learn what it takes to operate at large scale from well-known names in our industry. You will take away hard-earned architectural lessons on scalability, reliability, throughput, and performance.

  • Architecting for Confidence: Building Resilient Systems

    Your system will fail. Build systems with the confidence to know when they do and you won’t.

  • Remotely Productive: Remote Teams & Software

    More and more companies are moving to remote work. How do you build, work on, and lead teams remotely?

  • Operating Microservices

    Building and operating distributed systems is hard, and microservices are no different. Learn strategies for not just building a service but operating them at scale.

  • Distributed Systems for Developers

    Computer science in practice. An applied track that fuses together the human side of computer science with the technical choices that are made along the way

  • The Future of APIs

    Web-based API continue to evolve. The track provides the what, how, and why of future APIs, including GraphQL, Backend for Frontend, gRPC, & ReST

  • Wednesday, 18 November

  • Resurgence of Functional Programming

    What was once a paradigm shift in how we thought of programming languages is now main stream in nearly all modern languages. Hear how software shops are infusing concepts like pure functions and immutablity into their architectures and design choices.

  • Social Responsibility: Implications of Building Modern Software

    Software has an ever increasing impact on individuals and society. Understanding these implications helps build software that works for all users

  • Non-Technical Skills for Technical Folks

    To be an effective engineer, requires more than great coding skills. Learn the subtle arts of the tech lead, including empathy, communication, and organization.

  • Clientside: From WASM to Browser Applications

    Dive into some of the technologies that can be leveraged to ultimately deliver a more impactful interaction between the user and client.

  • Languages of Infra

    More than just Infrastructure as a Service, today we have libraries, languages, and platforms that help us define our infra. Languages of Infra explore languages and libraries being used today to build modern cloud native architectures.

  • Mechanical Sympathy: The Software/Hardware Divide

    Understanding the Hardware Makes You a Better Developer