04. PARSE_HEADER vs SKIP_HEADER
🔹 Is PARSE_HEADER = TRUE mandatory for Schema Evolution?
This is an important distinction that often comes up in interviews!
For CSV Files → Yes, it is mandatory.
If you want Snowflake to:
- Detect new columns
- Match columns by name
- Automatically add new columns
then PARSE_HEADER = TRUE is required because Snowflake needs to read the header row.
Example:
ID,NAME,CITY
1,John,London
2,David,Paris
File Format & COPY INTO:
CREATE FILE FORMAT CSV_FF
TYPE = CSV
PARSE_HEADER = TRUE;
COPY INTO CUSTOMER
FROM @CUSTOMER_STAGE
FILE_FORMAT=(FORMAT_NAME='CSV_FF')
MATCH_BY_COLUMN_NAME=CASE_INSENSITIVE;
Snowflake reads ID, NAME, CITY and compares them with the target table.
🔹 What if PARSE_HEADER = FALSE for CSV?
Suppose you define the file format without parsing the header:
CREATE FILE FORMAT CSV_FF
TYPE = CSV
PARSE_HEADER = FALSE;
If your file has a header like ID,NAME,CITY, Snowflake treats that header row as a normal data row.
It does not extract the column names, so:
- ❌
MATCH_BY_COLUMN_NAMEcannot work. - ❌ Schema Evolution cannot determine that the third column is
CITY. - ❌ The load will either fail or treat the header as data, depending on your COPY options.
🔹 What about SKIP_HEADER = 1?
This is where many people get confused.
| Option | Purpose |
|---|---|
SKIP_HEADER | Skip the first line when loading data. |
PARSE_HEADER | Read the first line as column names. |
For schema evolution, reading the header is what's important, not just skipping it.
[!IMPORTANT] In Snowflake,
PARSE_HEADER = TRUEandSKIP_HEADERare mutually exclusive for CSV file formats. If you're using schema evolution with CSV, usePARSE_HEADER = TRUErather thanSKIP_HEADER, because Snowflake needs to parse the header instead of simply skipping it.
🔹 What about JSON, Parquet, Avro, and ORC?
JSON Files
{
"id": 1,
"name": "John",
"city": "London"
}
- No
PARSE_HEADERrequired. Snowflake readsid,name, andcitydirectly from the JSON keys.
Parquet Files
Parquet contains embedded schema metadata.
- No
PARSE_HEADERrequired. Snowflake reads the column names directly from the Parquet metadata.
Avro / ORC
Same concept. The schema is stored inside the file itself.
- No
PARSE_HEADERrequired.
🎯 Summary
| File Type | PARSE_HEADER Required? | Why? |
|---|---|---|
| CSV | ✅ Yes | Header row contains column names. |
| JSON | ❌ No | Field names are part of the JSON document. |
| Parquet | ❌ No | Schema is embedded in the file metadata. |
| Avro | ❌ No | Schema is embedded in the file. |
| ORC | ❌ No | Schema is embedded in the file. |
💡 Interview Answer
"For CSV-based schema evolution,
PARSE_HEADER = TRUEis effectively required because Snowflake needs to read the header row to identify and map column names. Without parsing the header,MATCH_BY_COLUMN_NAMEcannot determine the source column names, so automatic schema evolution cannot work.For self-describing formats such as JSON, Parquet, Avro, and ORC,
PARSE_HEADERis not required because the schema or field names are already present within the file itself."