Skip to main content

2. Core Architecture

Snowflake's architecture is a hybrid of traditional shared-disk and shared-nothing architectures, specifically designed for the cloud. This design is what enables Snowflake's unique elasticity, performance, and fully managed "serverless" experience.


πŸ›οΈ Multi-Cluster Shared Data Architecture​

To overcome the limitations of traditional Shared-Disk and Shared-Nothing architectures, Snowflake pioneered a hybrid Multi-Cluster Shared-Data Architecture.

This cloud-native design merges the data-sharing convenience of Shared-Disk (Shared-Data) databases with the isolated high-performance execution of Shared-Nothing systems:

How the Hybrid Architecture Works:​

  • The Shared Component (Shared-Disk/Shared-Data): The Storage Layer is a centralized, cloud-native storage repository. All virtual warehouses read and write to the same single, shared source of truth on cloud storage. There is no physical copying or partitioning of data between warehouses.
  • The Shared-Nothing Component: Each Virtual Warehouse behaves as an isolated Shared-Nothing MPP cluster. Warehouses do not share memory, CPU, or local cache SSDs. The compute nodes pull data files from storage and cache them on their dedicated local disks, processing queries locally at high speeds.
  • The Result: Because storage and compute are completely decoupled:
    • You get the simplicity and convenience of a single database storage location.
    • Compute clusters scale up, down, or suspend dynamically in seconds.
    • Multiple query engines can query the exact same data simultaneously with zero performance contention.

🧊 The Three-Layer Architecture​

Snowflake resolves the issues of traditional architectures by physically separating compute and storage into a three-layer architecture.

Architecture of Cloud Data Warehousing

🧱 1. Storage Layer​

Role: Persistent, centralized storage of all data types.

  • The centralized repository where all data is physically stored (typically on AWS S3, Azure Blob, or Google Cloud Storage).
  • Handles both structured (tables, columns) and semi-structured data (JSON, Avro, Parquet, etc.).
  • Fully managed by Snowflake β€” users don’t need to worry about file formats, folders, indexes, partitions, or backups. Data is stored in a proprietary, columnar format (micro-partitions).
  • Provides automatic compression, built-in encryption at rest, and intelligent organization.

πŸ”’ Data is abstracted away from users β€” zero infrastructure management required.

βš™οΈ 2. Compute Layer (Virtual Warehouses)​

Role: Executes all data operations (queries, transformations, loading).

  • A Virtual Warehouse is an independent MPP (Massively Parallel Processing) compute cluster.
  • Fully managed and elastic β€” no server provisioning or cluster setup needed.
  • Supports on-demand scaling:
    • Vertically (choose warehouse size from XS to 6XL)
    • Horizontally (multi-cluster warehouses for high concurrency)
  • Warehouses can be started, paused, and resized independently, with auto-suspend and resume to save cost.

🧠 Fully isolated compute environments can operate on the same data in the Storage Layer without contention or performance degradation.

☁️ 3. Cloud Services Layer​

Role: The "Brain" of the architecture.

  • Handles all non-compute operations, fully abstracted from the user β€” making Snowflake truly serverless and maintenance-free.
  • Includes:
    • User authentication & access control
    • Query parsing & optimization
    • Transaction management
    • Metadata services
    • Advanced features like Time Travel, Fail-safe, and Zero-Copy Cloning
  • Always-on, even when compute is paused, ensuring continuous availability and governance.

🧩 This control layer coordinates all activity between storage and compute, delivering reliability, performance, and security.


βš–οΈ Comparing the Architectural Concepts​

Many learners confuse the Three-Layer Architecture with the Multi-Cluster Shared-Data Architecture. Here is a clear breakdown:

AspectThree-Layer ArchitectureMulti-Cluster Shared Data Architecture
DescribesOverall Snowflake platform designHow compute and storage interact
FocusStorage + Compute + ServicesMultiple compute clusters sharing data
ScopeHigh-level architectureCore architectural model
Components3 layersShared storage + many compute clusters
PurposeExplain entire systemExplain scalability and concurrency

πŸ›Ά Evolution: From Legacy Data Lakes (Hadoop) to Snowflake​

To appreciate the value of Snowflake's Multi-Cluster Shared-Data Architecture, it helps to look at how it evolved from legacy big-data solutions like Apache Hadoop:

πŸ’­ The Hadoop Era (Legacy Data Lakes)​

Once the standard for big data processing, Hadoop promised massive scale but came with significant operational challenges:

  • Complex Infrastructure: Required setting up and managing heavy, disjointed tool ecosystems (MapReduce, Hive, Pig, HDFS).
  • Coupled Scaling: Scaling storage often required purchasing more compute-heavy server nodes, leading to inefficient resource allocation.
  • High Skill Barriers: Required complex programming frameworks (like MapReduce in Java/Scala) instead of standard SQL.
  • Resource Contention: Running heavy ETL batch processes severely degraded performance for concurrent analytics queries.

❄️ Snowflake: The Modern Data Lake​

Snowflake combines the schema flexibility of a data lake with the performance, reliability, and security of a data warehouse:

  • Decoupled Cloud-Native Storage: Leverages low-cost, infinitely scalable cloud storage (AWS S3, Azure Blob, GCP) as a single source of truth.
  • Familiar SQL Access: Queries both structured and semi-structured data formats (JSON, Parquet, Avro, XML) using standard ANSI SQL.
  • Zero Resource Contention: Multiple teams can query the exact same shared dataset simultaneously using independent Virtual Warehouses.
  • Direct External Integration: Allows querying external cloud data lakes directly using Snowflake compute without needing to migrate the data first.

πŸŽ“ Interview Prep: Explaining Snowflake's Architecture​

When interviewers ask: "Explain Snowflake Architecture", they usually expect a structured answer covering both concepts.

πŸ“‹ What to Cover:​

  1. Three-Layer Architecture:
    • Database Storage Layer: Centralized storage of all data.
    • Query Processing Layer (Virtual Warehouses): Compute resources to execute queries.
    • Cloud Services Layer: The "brain" coordinating transactions, metadata, security, and optimization.
  2. Multi-Cluster Shared Data Architecture:
    • Mention that Snowflake follows a Multi-Cluster Shared Data Architecture, where:
      • Data is stored once in centralized cloud storage (Shared-Data).
      • Multiple independent compute clusters (Shared-Nothing) access the same centralized storage.
      • Compute and storage are completely separated/decoupled.
      • High concurrency and scalability are achieved without resource contention.

πŸ—£οΈ A Concise Interview Answer:​

"Snowflake uses a Multi-Cluster Shared Data Architecture implemented through a Three-Layer Architecture consisting of the Storage Layer, Query Processing Layer (Virtual Warehouses), and Cloud Services Layer. This design completely separates storage and compute, allowing multiple independent compute clusters to access the exact same shared data simultaneously without performance contention, providing high scalability and concurrency."