groupby: the split-apply-combine engine

Every "give me sum/mean/count by X" analysis in the world is a groupby.

The three phases

1. Split: rows into buckets by one or more columns.
2. Apply: run a function on each bucket.
3. Combine: stitch the results back together.

The three apply modes

Aggregate — reduces each group to a scalar

df.groupby("region").revenue.sum()
df.groupby(["region","product"]).agg(total_qty=("quantity","sum"))

Named aggregations (2nd form) are the modern, readable way.

Transform — returns same shape as input

df["z_score"] = df.groupby("region").revenue.transform(
    lambda s: (s - s.mean()) / s.std()
)

Use when you want a per-row value normalized within each group — running totals, group means broadcast back, percent-of-total.

Filter — keeps or drops entire groups

df.groupby("region").filter(lambda g: g.revenue.sum() > 500)

Common patterns

• Top-N per group: df.sort_values("revenue", ascending=False).groupby("region").head(3)
• Pivot table: df.pivot_table(index="region", columns="product", values="revenue", aggfunc="sum")
• Rolling within group: df.groupby("id").value.rolling(7).mean() — 7-day rolling mean per user.

Performance

• Prefer builtin ops (.sum(), .mean()) over .apply(lambda x: ...) — 5–50× faster.
• Sort by group cols first if grouping is a bottleneck.

Try it

1. Compute average quantity per (region, product).
2. Rank products by revenue within each region.