intermediate

Selecting & Filtering Data

By DataPlayArena Technical Review BoardUpdated 2026-07-127 min readRuntime Verified

Overview

Learn how to select specific columns, filter rows based on conditions, and slice arrays using loc and iloc.

Learning Objectives

  • Select specific columns and filter rows using logical checks.
  • Slice datasets using index positions (iloc).
  • Slice datasets using index labels (loc).

Concept Explanation

Use loc to select rows and columns by their labels. Use iloc to select by integer index positions. You can filter rows by passing boolean conditions inside brackets.

Code Examples

Example 1 — Basics

This example introduces the fundamental syntax and concepts.

python
import pandas as pd
# df['name'] # Select name column
# df[df['age'] > 30] # Filter rows by age

Example 2 — Everyday Usage

This example demonstrates a realistic scenario handling business parameters.

python
# Select values using loc and iloc labels
data = {
    'State': ['NY', 'CA', 'TX'],
    'Sales': [500, 700, 600]
}
df = pd.DataFrame(data, index=['a', 'b', 'c'])
print('First Row using iloc:\n', df.iloc[0])
print('Row b using loc:\n', df.loc['b'])
print('Filtering Sales > 550:\n', df[df['Sales'] > 550])

Example 3 — Practical Data Engineering Example

This example shows clean, production-grade code structure following senior development standards.

python
# Complex multi-criteria filter and column slice
data = {
    'name': ['Alex', 'Bob', 'Charlie', 'David'],
    'dept': ['HR', 'IT', 'IT', 'HR'],
    'salary': [50000, 70000, 80000, 45000]
}
df = pd.DataFrame(data)
# Filter rows where dept is IT and salary is >= 70000, keep only name and salary columns
filtered_df = df.loc[(df['dept'] == 'IT') & (df['salary'] >= 70000), ['name', 'salary']]
print(filtered_df)

Visual Flow

The following execution flow represents the step-by-step evaluation inside the interpreter:

text
DataFrame → Apply conditional check → Filter rows → Select columns → Return subset DataFrame

Common Mistakes

Review these common pitfalls when working with this topic:

  • Confusing loc (label-based) with iloc (integer position-based).
  • Using python's logical operators (and, or) instead of pandas bitwise operators (&, |) when filtering.
  • Modifying sliced DataFrames directly, raising a SettingWithCopyWarning.
  • Forgetting brackets when specifying multiple filter criteria.

Best Practices

Enforce these Pythonic best practices in your codebase:

❌ Don't
python
it_employees = df[df['dept'] == 'IT']
it_employees['salary'] = 90000 # Raises SettingWithCopyWarning
✅ Do
python
df.loc[df['dept'] == 'IT', 'salary'] = 90000 # Safe modification using .loc

Performance Notes

Keep these optimization guidelines in mind for performance-sensitive hotpaths:

  • Slicing and filtering return views. To modify a subset safely, copy it explicitly: df.copy().

Quick Revision

Use these key summaries for last-minute revision:

  • Use df[column] or df[[col1, col2]] to select columns.
  • loc uses index labels; iloc uses integer positions.
  • Filter rows using boolean conditions inside brackets: df[condition].
  • Use & (and) and | (or) for multi-criteria filters.
  • Wrap filtering conditions in parentheses: (cond1) & (cond2).
DPA

DataPlayArena Technical Editorial Board

Curated & maintained by enterprise data platform engineers with 10+ years in distributed systems.

Runtime Verification Standards: Every code example, query, and architecture diagram in this tutorial is compiled and verified against real runtimes (Apache Spark 3.5, Python 3.11, PostgreSQL, and DuckDB). If you spot any errata or want to suggest improvements, please submit feedback.