Introduction to Pandas

Overview

Pandas is the standard tool for data manipulation in Python. It provides Series (1D) and DataFrames (2D) to manage tabular data.

Learning Objectives

• Understand the difference between Series and DataFrames.
• Create Series and DataFrames from standard dictionaries.
• Learn why Pandas is essential for data engineering tasks.

Concept Explanation

Pandas wraps NumPy arrays to provide labeled rows and columns. A Series is a one-dimensional labeled array. A DataFrame is a two-dimensional tabular structure (like an Excel sheet or SQL table). Labeled columns make working with datasets intuitive.

Code Examples

Example 1 — Basics

This example introduces the fundamental syntax and concepts.

import pandas as pd
# Create a Series mapping values to index labels
s = pd.Series([10, 20, 30], index=['a', 'b', 'c'])
print(s)

Example 2 — Everyday Usage

This example demonstrates a realistic scenario handling business parameters.

# Create a DataFrame from dictionary list data
data = {
    'Name': ['Alice', 'Bob'],
    'Department': ['HR', 'IT'],
    'Salary': [50000, 60000]
}
df = pd.DataFrame(data)
print(df)

Example 3 — Practical Data Engineering Example

This example shows clean, production-grade code structure following senior development standards.

# Build configuration schema DataFrame and check basic properties
configs = {
    'host': ['localhost', 'dev-db'],
    'port': [5432, 3306],
    'ssl': [True, False]
}
df_conf = pd.DataFrame(configs, index=['pg', 'mysql'])
print('Config table shape:', df_conf.shape)
print('System indexes:', df_conf.index.tolist())

Visual Flow

The following execution flow represents the step-by-step evaluation inside the interpreter:

Source Dict Data → Allocate Labeled Series / Columns → Assemble tabular DataFrame with Row Index

Common Mistakes

Review these common pitfalls when working with this topic:

• Confusing Series (1D) with DataFrames (2D).
• Forgetting to import pandas as pd.
• Assuming index values must be unique integers.
• Confusing DataFrame columns names with data row entries.

Best Practices

Enforce these Pythonic best practices in your codebase:

import pandas
df = pandas.DataFrame(data) # Verbose module name

import pandas as pd
df = pd.DataFrame(data) # Clean standard import alias

Performance Notes

Keep these optimization guidelines in mind for performance-sensitive hotpaths:

• Pandas structures keep data aligned using indexes, ensuring fast lookups and row lookups.

Quick Revision

Use these key summaries for last-minute revision:

• Pandas is the standard tool for tabular data manipulation.
• Series represents 1D columns; DataFrame represents 2D tables.
• DataFrames are made of column Series sharing an index.
• Import pandas as pd by convention.
• DataFrames can be created from dictionaries and lists.