intermediate

Dict & Set Comprehensions in Python

By DataPlayArena Technical Review BoardUpdated 2026-07-208 min readRuntime Verified

Overview

Just as list comprehensions provide an expressive syntax for constructing lists, Python provides dictionary comprehensions ({k: v for ...}) and set comprehensions ({v for ...}). These constructs allow data engineers to build lookups, invert index mappings, deduplicate data streams, and group tabular metrics into key-value stores with optimal speed and concise, readable code.

What You Will Learn

In this lesson, you will master:
  • Dictionary Comprehension Syntax: Building {key_expr: val_expr for item in iterable if condition}.
  • Set Comprehension Syntax: Constructing unique collections {val_expr for item in iterable}.
  • Key Inversion: Reversing mapping relationships (values becoming keys).
  • Data Aggregation & Grouping: Creating grouped lookup structures directly from tabular datasets.
  • Duplicate Key Overwriting: Understanding how Python handles duplicate keys generated during comprehension execution.

Dictionary Comprehensions

1. Basic Syntax & Lookup Construction

A dictionary comprehension constructs a key-value mapping in a single pass:

python
users = [
    {"id": 101, "name": "Alice", "role": "admin"},
    {"id": 102, "name": "Bob", "role": "viewer"},
    {"id": 103, "name": "Charlie", "role": "editor"},
]

# Fast O(1) user ID to name lookup table
user_lookup = {user["id"]: user["name"] for user in users}
print(user_lookup)
# Output: {101: 'Alice', 102: 'Bob', 103: 'Charlie'}

# Filtering: Map only privileged users
admin_lookup = {user["id"]: user["name"] for user in users if user["role"] in ("admin", "editor")}
print(admin_lookup)
# Output: {101: 'Alice', 103: 'Charlie'}

2. Inverting Dictionaries (Bidirectional Lookups)

When converting a code-to-description mapping into a description-to-code reverse lookup:

python
status_codes = {
    200: "OK",
    400: "BAD_REQUEST",
    404: "NOT_FOUND",
    500: "INTERNAL_SERVER_ERROR"
}

# Invert keys and values
code_by_status = {status: code for code, status in status_codes.items()}
print(code_by_status)
# Output: {'OK': 200, 'BAD_REQUEST': 400, 'NOT_FOUND': 404, 'INTERNAL_SERVER_ERROR': 500}
Collision Warning on Key Inversion

If the original dictionary contains duplicate values (e.g., {1: "A", 2: "A"}), inverting it will overwrite previously evaluated keys! The resulting dictionary will retain only the last encountered key for that value.


Set Comprehensions

Set comprehensions use the same {} syntax as dictionary comprehensions, but specify a single expression instead of a key: value pair:

python
log_entries = [
    "INFO: Server started",
    "WARN: Disk usage high",
    "ERROR: DB connection timeout",
    "INFO: User login",
    "ERROR: Out of memory"
]

# Extract unique log levels
log_levels = {entry.split(":")[0] for entry in log_entries}
print(log_levels)
# Output: {'INFO', 'WARN', 'ERROR'} (order not guaranteed)

Set comprehensions automatically discard duplicate values during construction, producing a deduplicated hash set without intermediate list allocation.


Practical Applications in Data Engineering

1. Grouping and Aggregating by Category

Combining comprehensions with Python collections:

python
transactions = [
    ("engineering", 450.0),
    ("marketing", 200.0),
    ("engineering", 300.0),
    ("sales", 150.0),
    ("marketing", 120.0),
]

# Extract unique departments using a set comprehension
departments = {dept for dept, _ in transactions}

# Build department totals using dictionary comprehension
dept_totals = {
    dept: sum(amount for d, amount in transactions if d == dept)
    for dept in departments
}
print(dept_totals)
# Output: {'engineering': 750.0, 'marketing': 320.0, 'sales': 150.0}

2. Normalizing and Cleansing Raw Headers

python
raw_payload = {
    "  First_Name  ": "John",
    "LAST_NAME": "Doe",
    "  EMAIL_ADDRESS ": "john.doe@example.com",
    "AGE": 30
}

# Clean all keys to lowercase snake_case
clean_payload = {
    key.strip().lower().replace(" ", "_"): val
    for key, val in raw_payload.items()
}
print(clean_payload)
# Output: {'first_name': 'John', 'last_name': 'Doe', 'email_address': 'john.doe@example.com', 'age': 30}

Common Mistakes & Gotchas

  • Empty {} Ambiguity: Remember that {} always denotes an empty dictionary, never an empty set. To create an empty set, use set().
  • Unhashable Keys in Dictionary Comprehension: The key expression must evaluate to a hashable type. Attempting {list(val): key for key, val in data.items()} triggers TypeError: unhashable type: 'list'.
  • Hidden O(N^2) Nested Iteration: In the aggregation example above, nesting sum(... for ... in transactions) inside a dictionary comprehension iterates through the entire list for every department. For large datasets ($> 100,000$ rows), use collections.defaultdict(float) to achieve $O(N)$ linear time.

Performance & Best Practices

  • Use .items() for Iteration: When transforming an existing dictionary, always iterate over dict.items() to extract both key and value simultaneously, avoiding redundant dict[k] lookups.
  • Prefer Dict Comprehensions over dict(zip(...)): While dict(zip(keys, values)) is common, a dictionary comprehension {k: v for k, v in zip(keys, values)} allows inline filtering and transformation before inserting into the dictionary.

Interview Perspective

Python Interview Question: What happens during a dictionary comprehension if multiple items produce identical keys?

Key Talking Points for Candidates:

  1. Last-Write-Wins Rule: If two iterations evaluate to the same key, Python replaces the existing value with the newly computed value for that key.
  2. Deterministic Outcome: The resulting dictionary will contain the value from the last item encountered in the iteration order.
  3. Mitigation: If all values must be preserved for duplicated keys, you must accumulate them into a list or set using collections.defaultdict(list) instead of a standard dictionary comprehension.

Interactive Practice Challenge

Challenge: Invert Unique Server IP Map

You have a dictionary mapping server hostnames to their IP addresses: {"web-01": "10.0.0.1", "db-01": "10.0.0.5", "cache-01": "10.0.0.8"}. Write a dictionary comprehension that creates a reverse lookup from IP address to hostname, but only includes servers that belong to the 10.0.0.x subnet.

DPA

DataPlayArena Technical Editorial Board

Curated & maintained by enterprise data platform engineers with 10+ years in distributed systems.

Runtime Verification Standards: Every code example, query, and architecture diagram in this tutorial is compiled and verified against real runtimes (Apache Spark 3.5, Python 3.11, PostgreSQL, and DuckDB). If you spot any errata or want to suggest improvements, please submit feedback.