intermediate
Snowflake
10 min readLast updated: 2026-07-08
Overview
Connect Spark to Snowflake's cloud data platform using the Snowflake connector to load and write analytical tables.
What You Will Learn
In this lesson, you will learn:
- Snowflake Connector: Configuring connector packages.
- Account Connection: Setting account URLs, databases, and warehouses.
- Read/Write Operations: Reading and writing Snowflake tables.
Detailed Concept Explanation
To connect Spark to Snowflake, you use the Snowflake connector for Spark. Similar to BigQuery, the connector uses an intermediate storage area (like Amazon S3 or Google Cloud Storage) to stage data before writing it to Snowflake.
Code Examples
Python (PySpark) Implementation
python
from pyspark.sql import SparkSession
spark = SparkSession.builder \
.appName("SnowflakeTest") \
.config("spark.jars.packages", "net.snowflake:snowflake-jdbc:3.13.14,net.snowflake:spark-snowflake_2.12:2.9.2-spark_3.1") \
.getOrCreate()
# Configure Snowflake options
sf_options = {
"sfURL": "myaccount.snowflakecomputing.com",
"sfUser": "myuser",
"sfPassword": "mypassword",
"sfDatabase": "mydb",
"sfSchema": "public",
"sfWarehouse": "mywh"
}
# Read from Snowflake table
sf_df = spark.read.format("net.snowflake.spark.snowflake") \
.options(**sf_options) \
.option("dbtable", "users") \
.load()
sf_df.show()
Expected Output
text
+---+-----+
| id| name|
+---+-----+
| 1|Alice|
+---+-----+
Execution Plan Diagram (Python & Scala)
Execution Plan Diagram
SparkSession.builder
read.format(net.snowflake.spark.snowflake)
options(sf_options)
load()
show()
Scala Implementation
scala
import org.apache.spark.sql.SparkSession
val spark = SparkSession.builder().appName("SnowflakeScala").getOrCreate()
val sfOptions = Map(
"sfURL" -> "myaccount.snowflakecomputing.com",
"sfUser" -> "myuser",
"sfPassword" -> "mypassword",
"sfDatabase" -> "mydb",
"sfSchema" -> "public",
"sfWarehouse" -> "mywh"
)
val sfDF = spark.read.format("net.snowflake.spark.snowflake")
.options(sfOptions)
.option("dbtable", "users")
.load()
sfDF.show()
Expected Output
text
+---+-----+
| id| name|
+---+-----+
| 1|Alice|
+---+-----+