beginner

Temporary Views

By DataPlayArena Technical Review BoardUpdated 2026-07-088 min readRuntime Verified

Overview

Learn how to register a Spark DataFrame as a temporary view, allowing you to query it using standard SQL statements.

What You Will Learn

In this lesson, you will learn:
  • Create Views: Registering local views using createOrReplaceTempView().
  • SQL Execution: Running queries on top of views via spark.sql().
  • Scope Limits: Understanding the lifetime boundaries of temporary vs global views.

Detailed Concept Explanation

If you prefer writing SQL queries over DataFrame transformations, you can register your DataFrame as a temporary view: df.createOrReplaceTempView("view_name")

Once registered, you can query this view using standard SQL via the SparkSession: spark.sql("SELECT * FROM view_name WHERE price > 500")

Scope Boundaries

  1. Temporary Views: Scoped to the active SparkSession. They are automatically deleted when the session terminates.
  2. Global Temporary Views: Scoped to the entire Spark application, making them accessible across different SparkSessions. They are registered under the system namespace global_temp.

Code Examples

Input Dataset Preview

Below is the flight bookings dataset:

flightdestination
AA101London
UA204Paris

Python (PySpark) Implementation

python
from pyspark.sql import SparkSession

spark = SparkSession.builder.appName("TempViews").getOrCreate()
data = [("AA101", "London"), ("UA204", "Paris")]
df = spark.createDataFrame(data, ["flight", "destination"])

# Register the DataFrame as a temporary view
df.createOrReplaceTempView("bookings")

# Query the view using standard SQL
result_df = spark.sql("SELECT * FROM bookings WHERE destination = 'London'")
result_df.show()

Expected Output

text
+------+-----------+
|flight|destination|
+------+-----------+
| AA101|     London|
+------+-----------+

Execution Plan Diagram (Python & Scala)

Execution Plan Diagram
SparkSession.builder
createDataFrame
createOrReplaceTempView(bookings)
spark.sql(SELECT * FROM bookings)
show()

Scala Implementation

scala
import org.apache.spark.sql.SparkSession

val spark = SparkSession.builder().appName("TempViewsScala").getOrCreate()
import spark.implicits._

val df = Seq(("AA101", "London"), ("UA204", "Paris")).toDF("flight", "destination")
df.createOrReplaceTempView("bookings")

val result = spark.sql("SELECT * FROM bookings WHERE destination = 'London'")
result.show()

Expected Output

text
+------+-----------+
|flight|destination|
+------+-----------+
| AA101|     London|
+------+-----------+

SQL Implementation

sql
-- Querying the registered temporary view directly in SQL
SELECT flight 
FROM bookings 
WHERE destination = 'Paris';

Expected Output

Executing this SQL query returns:

flight
UA204

Common Mistakes

  • Accessing Views After Session Close: Trying to query a temporary view after calling spark.stop(). Since temporary views are bound to the active session, they are deleted when the session stops.

Best Practices

  • Use createOrReplace: Always use createOrReplaceTempView() instead of createTempView(). This prevents crashes if your code runs multiple times and attempts to register the same view name.

Interview Perspective

What is the difference between a temporary view and a global temporary view in Spark?

A temporary view is bound to the specific SparkSession that created it. A global temporary view is shared across all sessions within the Spark application. Global views are kept in the system database global_temp and must be queried using their fully qualified name: SELECT * FROM global_temp.view_name.


Interactive Challenges

Challenge 1: Register View (Beginner)

What DataFrame method is used to register a temporary view, replacing it if it already exists?

Related Topics

DPA

DataPlayArena Technical Editorial Board

Curated & maintained by enterprise data platform engineers with 10+ years in distributed systems.

Runtime Verification Standards: Every code example, query, and architecture diagram in this tutorial is compiled and verified against real runtimes (Apache Spark 3.5, Python 3.11, PostgreSQL, and DuckDB). If you spot any errata or want to suggest improvements, please submit feedback.