Presentation: Snowflake Architecture: Building a Data Warehouse for the Cloud

Track: Architectures You've Always Wondered About

Location: Ballroom A

Duration: 4:10pm - 5:00pm

Day of week:

Slides: Download Slides

This presentation is now available to view on InfoQ.com

Watch video with transcript

Abstract

At Snowflake, we wanted to architect a data warehouse from the ground up to leverage all the benefits of the cloud. Unlike shared-storage architectures that tie storage and compute together, we built a single integrated system with fully independent scaling for compute, storage and services. In the storage layer, we split data into micro-partitions and extract metadata for efficient query processing. At the compute layer, multiple virtual warehouses in separate compute clusters can simultaneously operate on the same data, giving high availability, performance isolation, scalability and concurrency. Virtual warehouses can also be automatically scaled up and down based on workload and performance.

This talk will cover the three pillars of the Snowflake architecture: 

  • Separating compute and storage to leverage abundant cloud compute resources
  • Building an ACID compliant database system on immutable storage
  • Delivering a scalable multi-tenant data warehouse system as a service

Speaker: Thierry Cruanes

Co-founder Snowflake Computing @SnowflakeDB

Thierry co-founded Snowflake and currently serves as Chief Technical Officer. Thierry is a leading expert in query optimization and parallel execution. He spent 13 years at Oracle focused on the optimization and parallelization layers in Oracle databases including as lead for the optimization group. Before Oracle, he spent seven years at the IBM European Center of Applied Mathematics working on data mining technologies. Thierry has a PhD in Computer Science with a focus in Database Systems and holds over 40 patents.

Find Thierry Cruanes at

Similar Talks

Linux Foundation's Project EVE: A Cloud-Native Edge Computing Platform

Qcon

Co-founder, VP Product and Strategy @ZededaEdge & Member Board Of Directors for LF Edge @linuxfoundation

Roman Shaposhnik

AWS Cloud Development Kit (CDK)

Qcon

Developer Tooling Advocate @AWSCloud & CDK Core Contributor

Richard Boyd

Helm 3: A Mariner's Delight

Qcon

Principal Program Manager @Microsoft & K8s Release Lead for 1.16

Lachlan Evenson