Legacy Modernization: Architecting Real-Time Systems Around a Mainframe

Designing systems that take advantage of modern platforms, tools, and techniques is critical for building scalable, evolvable applications that underpin businesses of all stripes. Leveraging those when your data is captured in a mainframe, which does not scale well, is challenging. Technical and architectural debt is often a stifling element and the momentum to modernize can be both hard to gain and to maintain. 

This talk discusses how National Grid built an event driven system using change data capture on top of a DB2 mainframe to power a cloud native GraphQL API hosted in Azure. 

National Grid uses mainframes to power their customer billing capabilities. These mainframes encompass decades of data and capabilities in a monolithic, inelastic system. Many business and customer channels rely on this data and functionality. The motivating application for our project is the customer web portal. Earlier iterations of the portal relied on direct synchronous calls to the mainframe. This constraint historically caused problems for scaling to their customer base’s needs on the web. Availability and performance were key challenges. 

Our architecture works by disconnecting the needs of the web application from the constraints of the mainframe. This ‘system of reference’ is hosted in the cloud with modern tech and is far better suited to adapting to business and (by extension) customer needs. It is a foundation which can, and is, being built upon to modernize National Grid’s technology portfolio, with no direct dependency on the legacy systems underlying them.

  • Change data capture enables near-real time synchronization
  • The ‘Customer Graph’ system-of-reference provides capability to scale independently from mainframe and be resilient to on-prem volatility
  • Asynchronous boundaries between cloud/on-prem resources decouples applications from connectivity/availability issues 
  • APIs developed with domain-driven design, providing multi-channel support for diverse business applications

Speaker

Jason Roberts

Lead Software Consultant @Thoughtworks, 15+ years in Software Development, Azure Solutions Architect Expert

With over 15 years of experience in software development, I am passionate about creating innovative solutions that deliver value. I have a strong background in systems thinking, event driven programming, cloud architecture, and evolutionary systems. I use these skills to lead the design and implementation of scalable, reliable, and maintainable systems and applications. At ThoughtWorks I work with clients to understand their needs, challenges, and opportunities, and lead teams to deliver high-quality software.

Read more

Speaker

Sonia Mathew

Director, Product Engineering @National Grid, 20+ Years in Tech

Sonia has spent over 20 years experience in Software Development leading teams , she is passionate about using technology to deliver value .She has a background in leading large transformational projects. She currently works at National Grid working on programs using cloud native architecture to serve Customer needs.

Read more

From the same track

Session Architecture

Optimizing Search at Uber Eats

Monday Nov 18 / 11:45AM PST

Uber has an in-house search engine called Search In Action (SIA). As the backbone behind the feed and search capabilities of Uber's Delivery business, SIA plays a crucial role in expanding selection seamlessly for customers which is a strategic advantage to the business.

Speaker image - Janani Narayanan

Janani Narayanan

Applied ML Engineer @Uber, Previously Tech Lead on DynamoDB Control Plane (Early Stage), 10+ Years Tech Industry Experience

Speaker image - Karthik Ramasamy

Karthik Ramasamy

Senior Staff Software Engineer @Uber, 15 Years of Experience in Design and Implementation of Web Applications, Distributed Systems, Search and Analytics Infrastructure

Session MLOps

Supporting Diverse ML Systems at Netflix

Monday Nov 18 / 10:35AM PST

Netflix uses data science and machine learning across all facets of the company, powering a wide range of business applications.

Speaker image - David Berg

David Berg

Senior Software Engineer @Netflix, Previously @IBM Almaden Research Center, Ph.D in Computational Neuroscience

Speaker image - Romain  Cledat

Romain Cledat

Senior Software Engineer @Netflix, Metaflow Core Contributor, Previously @Facebook and @Intel

Session HTTP

How GitHub Copilot Serves 400 Million Completion Requests a Day

Monday Nov 18 / 03:55PM PST

GitHub Copilot is the largest LLM powered Code Completion service in the world, serving hundreds of millions of requests per day with an average response time of under 200ms. This is the story of the architecture which powers this product.

Speaker image - David Cheney

David Cheney

Lead, Copilot Proxy @GitHub, Open Source Contributor and Project Member for Go Programming Language, Previously @VMware

Session Architecture

Unified Grid: How We Re-Architected Slack for our Largest Customers

Monday Nov 18 / 01:35PM PST

Slack’s enterprise solution allows users to join multiple workspaces within the same organization. However, for years, users could only view channels, messages, and other content from a single workspace at a time.

Speaker image - Ian Hoffman

Ian Hoffman

Staff Software Engineer @Slack, Previously @Chairish

Session

Unconference: Architectures You've Always Wondered About

Monday Nov 18 / 02:45PM PST