DataPlayArena
Interview Focus: DataFramesDifficulty: Easy

Sales Summary Report

Estimated duration: 10 mins

Lab Overview & Scenario

Compute total revenue, units sold, and peak order sizes using general summary statistics.

Business Scenario

The operations team needs a quick executive dashboard. Calculate global metrics including total sales revenue, total quantity of items sold, and the single largest transaction amount.

1. Local Raw Mock Dataset

The processing pipeline will ingest the following local mock file structures (e.g. simulated as `dataset.csv`):

order_idquantityprice_per_unit
1250.0
21120.0
3515.0
4380.0
52200.0
View Raw CSV Dataset
order_id,quantity,price_per_unit
1,2,50.0
2,1,120.0
3,5,15.0
4,3,80.0
5,2,200.0

2. Your Tasks

3. Pipeline Skeleton Script

Copy this starter template to write the data transformation logic:

python
# starter.py - Sales Summary Report
from pyspark.sql import SparkSession
from pyspark.sql.functions import col, sum, max

def sales_summary():
    spark = SparkSession.builder.appName("SalesSummary").getOrCreate()
    # TODO: Read dataset.csv, calculate revenue aggregates
    pass

if __name__ == "__main__":
    sales_summary()

4. Interactive Solution & Execution Output

Reveal Solution Pipeline Code
Implement the correct transformations using PySpark to achieve the aggregated values metrics output.

Pipeline Flow

dataset.csv
↓
Read CSV
↓
Add order_value
↓
Select global sums/max
↓
Display Output

Step-by-Step Implementation Guide

Step 1

Load the sales CSV file.

Step 2

Add the column order_value multiplying quantity by price.

Step 3

Apply select() using Spark's aggregate functions: sum up order_value, sum up quantity, and calculate the max() of order_value. Use alias naming for a clean table.

Step 4

Print the summary table.

Common Mistakes

  • Forgetting to alias aggregate columns, resulting in confusing auto-generated names.
  • Grouping by order_id, which calculates metrics per order rather than returning one global report.

Interviewer's Expectations

Why this is asked:

Global database metrics summaries are standard requirements for dashboards.

What they look for:

Clean selection of global aggregate metrics without unnecessary groupBy calls.

Common mistakes:

Adding an extra groupBy step when a global summary is required.

An excellent answer includes:

Explain that global aggregates run in a single final reduce step on the driver node.

✔ What You Learned

  • ✔Aggregate columns without group definitions
  • ✔Compute totals and extremes (max/min)
  • ✔Build executive summaries

Continue Learning

Previous LabConvert CSV to ParquetRelated Theory LessonBuilt-in FunctionsNext LabEmployee Bonus Calculator