Skip to main content

🚀 Deploying Models & Monitoring in Snowflake

Snowflake’s Snowpark API enables data engineers and data scientists to build and deploy ML models within Snowflake. Let’s break down the deployment and monitoring aspects:


1️⃣ Typical Deployment Process​

  • ✅ 1. Model training:
    Models are trained outside Snowflake (locally, in notebooks, or in a pipeline) using libraries like scikit-learn or xgboost.

  • ✅ 2. Model serialization:
    Save the trained model as a .pkl (or similar) file:

import joblib

joblib.dump(model, 'my_model.pkl')
  • ✅ 3. Upload model to Snowflake stage: Use the Snowflake PUT command or UI to upload the file:
PUT file://my_model.pkl @my_stage;
  • ✅ 4. Model inference script: Use a Snowpark script or stored procedure to load the model and make predictions:
import joblib
import pandas as pd
from snowflake.snowpark import Session

def predict(session: Session) -> str:
# Load the data
df = session.table("CUSTOMER_DATA").to_pandas()

# Load the model from stage
session.file.get("@my_stage/my_model.pkl", "my_model.pkl")
model = joblib.load("my_model.pkl")

# Make predictions
predictions = model.predict(df[["AGE", "INCOME"]])
df["PREDICTION"] = predictions

# Save predictions back to Snowflake
session.create_dataframe(df).write.mode("overwrite").save_as_table("PREDICTIONS")
return "Predictions saved to table!"
  • ✅ 5. Register as a Stored Procedure: Deploy this as a stored procedure in Snowflake so it can run server-side.

2️⃣ Automated Deployment via CI/CD​

You can automate model deployment and updates using CI/CD tools (like GitHub Actions or Azure DevOps Pipelines). A typical workflow:

  • ✅ Train the model (locally or via pipeline)
  • ✅ Push updated .pkl file to a Snowflake stage
  • ✅ Run a pipeline step to trigger the stored procedure for inference

This ensures consistent and automated updates of your ML workflows!


3️⃣ Monitoring Model Performance​

Monitoring is essential for understanding model health and data drift.

  • ✅ Metrics you might track:

    • Prediction accuracy (e.g., using actuals vs. predictions stored in Snowflake)
    • Model version (store in a metadata table)
    • Data drift (compare input data distribution to training data)
  • ✅ Example Monitoring Table:

CREATE TABLE model_monitoring (
model_version STRING,
prediction_date TIMESTAMP,
metric_name STRING,
metric_value FLOAT
);
  • ✅ Logging metrics:
from datetime import datetime

monitor_df = pd.DataFrame({
"model_version": ["v1.0"],
"prediction_date": [datetime.now()],
"metric_name": ["accuracy"],
"metric_value": [0.85]
})

session.create_dataframe(monitor_df).write.mode("append").save_as_table("MODEL_MONITORING")

4️⃣ Summary: Deployment & Monitoring​

StepTool/MethodNotes
Model trainingLocal Python / NotebookUse sklearn, XGBoost, etc.
Model packagingjoblib / pickleSave as .pkl or similar
Upload to SnowflakeSnowflake stage (PUT command)Central location for model artifact
InferenceSnowpark Python script / stored procLoad model, make predictions, save results
CI/CD automationGitHub Actions / Azure DevOpsAutomate updates and predictions
MonitoringSnowflake tables for metricsTrack performance, drift, and model health

5️⃣ Ready-to-use Example: Deploying & Monitoring Workflow​

Here’s a minimal Snowpark-based workflow:

import joblib
import pandas as pd
from snowflake.snowpark import Session

def main(session: Session) -> str:
# Load data
df = session.table("CUSTOMER_DATA").to_pandas()

# Load trained model
session.file.get("@my_stage/my_model.pkl", "my_model.pkl")
model = joblib.load("my_model.pkl")

# Predict
predictions = model.predict(df[["AGE", "INCOME"]])
df["PREDICTION"] = predictions

# Save predictions
session.create_dataframe(df).write.mode("overwrite").save_as_table("PREDICTIONS")

# Monitoring
monitor_df = pd.DataFrame({
"model_version": ["v1.0"],
"prediction_date": [pd.Timestamp.now()],
"metric_name": ["accuracy"],
"metric_value": [0.85]
})

session.create_dataframe(monitor_df).write.mode("append").save_as_table("MODEL_MONITORING")

return "Predictions and monitoring updated!"

# If running locally:
if __name__ == "__main__":
from snowflake.snowpark import Session
connection_parameters = {
"account": "<account>",
"user": "<user>",
"password": "<password>",
"warehouse": "<warehouse>",
"database": "<database>",
"schema": "<schema>",
"role": "<role>"
}
session = Session.builder.configs(connection_parameters).create()
print(main(session))
  • ✅ This script loads data, makes predictions, logs monitoring metrics, and saves everything to Snowflake!