ποΈ Snowflake Compute & Storage Architecture
π The Power of Separation: Storage & Computeβ
One of Snowflakeβs core architectural innovations is the complete separation of storage and compute, enabling:
- βοΈ Independent scalability
- π High concurrency with no contention
- π° Cost efficiency through usage-based billing
- π Team and workload-level performance isolation
Letβs explore how this design delivers practical benefits in real-world scenarios.
πΎ Storage Layer β Where Data Livesβ
Snowflake uses centralized, cloud-native storage (AWS S3, Azure Blob, GCP) to keep your data highly available and optimized.
πΉ Key Features:β
| Feature | Description |
|---|---|
| Structured & semi-structured data | Supports formats like CSV, JSON, Avro, Parquet |
| Compressed & encrypted | Built-in performance optimization & security |
| Scales independently from compute | No performance trade-offs |
| Single source of truth | Shared across all compute clusters |
π Analogy: Snowflake Storage is your data library β secure, centralized, and always accessible.
βοΈ Virtual Warehouses β Where Processing Happensβ
Virtual Warehouses (VWs) are independent compute engines that power data loading, transformation, and analysis.
πΉ Core Capabilities:β
| Feature | Description |
|---|---|
| Query processing | Executes SQL workloads |
| Performance isolation | Each VW is dedicated to its workload/team |
| Elastic scalability | Resize vertically or scale horizontally (multi-cluster) |
| Auto-suspend/resume | Saves cost with no manual intervention |
| Concurrency at scale | Multi-cluster scaling handles simultaneous usage |
π οΈ Analogy: Virtual Warehouses are workers using data from shared shelves (storage) β working in parallel, without conflict.
π How Storage & Compute Work Togetherβ
| π§© Component | π¬ Role |
|---|---|
| Storage | Stores raw and processed data |
| Virtual Warehouse | Executes SQL, transformations, and data loading tasks |
- Storage is always-on and low cost
- Compute is on-demand, billed only when active
- Multiple VWs can access the same storage simultaneously
- No data duplication or bottlenecks β only centralized access
π₯ Real-World Team Usage Scenarioβ
Each team can use its own Virtual Warehouse, all connecting to the same centralized data storage.
| Team | Task | Virtual Warehouse Configuration |
|---|---|---|
| Marketing | Campaign analytics | Medium, auto-suspend enabled |
| Finance | Daily reporting | Small, scheduled runtime |
| Data Engineering | Nightly ETL/ELT workloads | Large, auto-resume enabled |
| Data Science | Model training, ML workloads | XL, multi-cluster enabled |
- β No slowdown from other teams
- β No data duplication
- β Cost and performance optimized
- β All use consistent, centralized data
βοΈ Scaling Virtual Warehousesβ
1. Vertical Scalingβ
Resize warehouses to meet processing needs (XS β S β M β L β XL...).
2. Horizontal Scaling (Multi-cluster)β
Use multi-cluster mode to handle spikes in concurrent workloads.
CREATE WAREHOUSE analytics_wh
WITH WAREHOUSE_SIZE = 'LARGE'
MAX_CLUSTER_COUNT = 5
MIN_CLUSTER_COUNT = 1
SCALING_POLICY = 'ECONOMY';
βοΈ Auto-Suspend & Resume Exampleβ
Suspend warehouses after inactivity to save costs:
CREATE WAREHOUSE my_wh
WITH WAREHOUSE_SIZE = 'MEDIUM'
AUTO_SUSPEND = 60 -- seconds of inactivity
AUTO_RESUME = TRUE;
π Summary Comparison Tableβ
| π¦ Storage Layer | βοΈ Virtual Warehouse (Compute) |
|---|---|
| Stores data (structured/semi) | Processes data (queries, ETL, ML, etc.) |
| Always available | On-demand, elastic compute |
| Shared across teams | Isolated per workload or team |
| Scales infinitely | Scales vertically & horizontally |
| Low cost | Pay only when in use |
π― Final Takeawayβ
π§© Snowflakeβs separation of storage and compute powers:
β Massive concurrency β each team works in parallel β Performance isolation β no cross-team slowdown β Cost efficiency β compute billed per usage β Centralized data access β consistent and reliable β Elastic performance β scale with confidence
This is the foundation behind Snowflakeβs ability to support diverse workloads β BI, ELT, ML β in a modern, cloud-native, high-performance data platform.