Built-in vs Custom Models Decision Framework
Overview
Snowflake offers three tiers of ML capability. Choosing the right one depends on the problem complexity, team skills, and operational requirements. This lesson provides a structured framework for making that decision.
Comparison Table
| Criteria | Built-in ML | Snowpark ML | External Platform |
|---|---|---|---|
| Interface | SQL only | Python (runs in Snowflake) | Python / notebooks / platform UI |
| Setup time | Minutes | Hours | Days to weeks |
| Algorithms | Classification, Regression, Forecasting, Anomaly Detection | scikit-learn, XGBoost, LightGBM, PyTorch, custom | Unlimited |
| Hyperparameter tuning | Automatic | Manual or library-driven | Manual or platform-managed |
| Feature engineering | Basic (auto-handled) | Full control | Full control |
| Model registry | Schema-level object | Snowflake Model Registry | Platform-specific (MLflow, SageMaker, etc.) |
| Data movement | None | None | Export required |
| Team skill required | SQL | Python + ML fundamentals | ML engineering + platform expertise |
| Governance | Native RBAC | Native RBAC | External IAM + Snowflake |
| Cost model | Warehouse compute | Warehouse compute | Warehouse + external compute |
| Explainability | Limited (feature importance) | Full (SHAP, LIME, etc.) | Full |
Decision Framework
Use the following flowchart to select the right approach:
START
│
├─ Does the problem fit Classification, Regression,
│ Forecasting, or Anomaly Detection?
│ │
│ ├─ YES ──► Is automatic hyperparameter tuning acceptable?
│ │ │
│ │ ├─ YES ──► Is the team SQL-first with no Python stack?
│ │ │ │
│ │ │ ├─ YES ──► USE BUILT-IN ML
│ │ │ │
│ │ │ └─ NO ───► Are there custom feature engineering
│ │ │ or preprocessing requirements?
│ │ │ │
│ │ │ ├─ YES ──► USE SNOWPARK ML
│ │ │ └─ NO ───► USE BUILT-IN ML
│ │ │
│ │ └─ NO ───► USE SNOWPARK ML
│ │
│ └─ NO ──► Does the algorithm exist in scikit-learn / XGBoost / PyTorch?
│ │
│ ├─ YES ──► Can the data stay in Snowflake?
│ │ │
│ │ ├─ YES ──► USE SNOWPARK ML
│ │ └─ NO ───► USE EXTERNAL PLATFORM
│ │
│ └─ NO ──► USE EXTERNAL PLATFORM
When to Use Built-in ML
Built-in ML is the right choice when:
- The problem is well-defined classification, regression, forecasting, or anomaly detection.
- The team wants results in hours, not weeks.
- There is no existing ML infrastructure or Python expertise.
- Governance requires data to remain in Snowflake at all times.
- The use case does not require custom loss functions or exotic architectures.
Typical use cases:
- Customer churn prediction
- Sales and demand forecasting
- Fraud detection (binary classification)
- Anomalous transaction flagging
- Revenue trend analysis
When to Use Snowpark ML
Snowpark ML is the right choice when:
- Custom feature engineering or preprocessing is needed.
- The team requires specific algorithms not available in built-in ML (e.g., gradient boosting with custom objectives).
- Fine-grained hyperparameter control is necessary.
- The model needs to be registered, versioned, and served through the Snowflake Model Registry.
- Explainability beyond feature importance is required (SHAP values, partial dependence plots).
-- Example: Register a Snowpark ML model
-- (Python code running in a Snowpark session)
from snowflake.ml.modeling.xgboost import XGBClassifier
from snowflake.ml.registry import Registry
# Train with full control
model = XGBClassifier(
n_estimators=500,
max_depth=6,
learning_rate=0.05,
scale_pos_weight=10 # handle class imbalance
)
model.fit(train_df)
# Register in Snowflake Model Registry
reg = Registry(session)
mv = reg.log_model(
model,
model_name="fraud_xgb_v2",
sample_input_data=train_df
)
When to Use an External Platform
An external platform is the right choice when:
- The problem requires deep learning, reinforcement learning, or architectures not supported in Snowpark.
- The team already operates SageMaker, Vertex AI, Databricks ML, or similar.
- Training requires GPU clusters beyond what Snowflake provides.
- The model serves real-time inference at sub-millisecond latency outside of Snowflake queries.
Cost Considerations
| Approach | Cost Components | Optimization Levers |
|---|---|---|
| Built-in ML | Warehouse credits during training and inference | Use smaller warehouses; suspend after training |
| Snowpark ML | Warehouse credits; Snowpark-optimized warehouse for large models | Right-size warehouse; batch inference |
| External | Snowflake egress + external compute + storage + serving | Minimize data transfer; use Snowflake as feature store |
Estimating Built-in ML Costs
-- Check credit usage for a training job
SELECT
query_id,
warehouse_name,
total_elapsed_time / 1000 AS seconds,
credits_used_cloud_services
FROM SNOWFLAKE.ACCOUNT_USAGE.QUERY_HISTORY
WHERE query_text ILIKE '%CREATE%SNOWFLAKE.ML%'
AND start_time > DATEADD('hour', -24, CURRENT_TIMESTAMP())
ORDER BY start_time DESC;
Migration Path: Built-in to Custom
Teams often start with built-in ML and graduate to Snowpark ML as requirements grow. The migration path is:
| Step | Action |
|---|---|
| 1 | Start with built-in ML for an initial proof of concept |
| 2 | Evaluate model performance against business thresholds |
| 3 | If accuracy is insufficient, export training data and experiment with Snowpark ML |
| 4 | Use feature engineering and hyperparameter tuning in Snowpark ML |
| 5 | Register the improved model in the Snowflake Model Registry |
| 6 | Replace the built-in model reference with the registered model for inference |
-- Built-in ML prediction (before migration)
SELECT my_model!PREDICT(INPUT_DATA => OBJECT_CONSTRUCT(*)) FROM score_data;
-- Snowpark ML registered model prediction (after migration)
SELECT
MODEL_DB.MODEL_SCHEMA.FRAUD_XGB_V2!PREDICT(
feature_1, feature_2, feature_3
) AS prediction
FROM score_data;
Summary Decision Matrix
| Question | Built-in ML | Snowpark ML | External |
|---|---|---|---|
| Can I solve it in SQL? | Yes | - | - |
| Do I need custom algorithms? | - | Yes | Yes |
| Must data stay in Snowflake? | Yes | Yes | - |
| Do I need GPU training? | - | - | Yes |
| Do I need SHAP explainability? | - | Yes | Yes |
| Is time-to-production critical? | Yes | - | - |
| Do I have an ML engineering team? | - | Yes | Yes |
Best Practices
- Default to built-in ML — Validate whether a built-in function solves the problem before investing in custom infrastructure.
- Benchmark early — Train a built-in model first to establish a performance baseline, then evaluate whether a custom model justifies the added complexity.
- Avoid premature complexity — A well-trained built-in classification model often matches or exceeds a hastily built custom model.
- Plan for growth — Design your data pipeline so that swapping from built-in to Snowpark ML requires changing only the model call, not the entire workflow.
- Document the decision — Record why a particular approach was chosen so the team can revisit the decision as requirements evolve.