Python pandas Essentials

pd.DataFrame({"a": [1, 2], "b": [3, 4]})

From a dict of columns; pd.DataFrame(list_of_dicts) works too.

pd.Series([1, 2, 3], name="n")

One column with an index.

pd.read_csv("people.csv")

Read a CSV; read_excel, read_json, read_parquet, read_sql exist too.

df.to_csv("out.csv", index=False)

Write without the index column.

df.head(2), df.tail(1)

First rows, last rows; sample(n) for random rows.

df.shape, len(df), list(df.columns)

(4, 4), 4, ['name', 'dept', 'salary', 'joined'].

df.dtypes

Column types; object usually means strings. df.info() adds non-null counts.

df.describe()

count/mean/std/min/quartiles/max of numeric columns.

df["salary"]

One column → Series.

df[["name", "salary"]]

Several columns → DataFrame (note the double brackets).

df.loc[0, "name"], df.loc[:, "name":"salary"]

By label: a cell; a column range (inclusive).

df.iloc[0], df.iloc[:2, :2]

By position: first row; first 2 rows × 2 columns.

df[df["salary"] > 100]

Boolean mask keeps matching rows.

df[(df["dept"] == "eng") & (df["salary"] > 100)]

Combine masks with & | ~ and parentheses.

df.query("dept == 'eng' and salary > 100")

The same filter as a string expression.

df[df["dept"].isin(["eng", "ops"])]

Membership filter.

df[df["name"].str.startswith("A")], df["name"].str.upper()

.str gives string methods per row.

df.loc[df["salary"] > 100, "name"]

Filter rows and pick a column in one step.

df["bonus"] = df["salary"] * 0.1

New column, vectorised; df.assign(bonus=...) returns a new frame instead.

df["level"] = df["dept"].map({"eng": "E", "ops": "O"})

Map values via a dict; unmapped → NaN.

df["salary"].apply(lambda x: x * 2)

Row-wise Python function; slower than vectorised ops.

df.rename(columns={"salary": "pay"})

Rename columns (inplace=False by default).

df.drop(columns=["joined"])

Remove columns; drop(index=[0]) removes rows.

df.sort_values("salary", ascending=False)

Sort rows; pass lists for several keys and directions.

df.drop_duplicates(subset="dept")

First row per dept.

df["salary"].sum(), df["salary"].mean(), df["salary"].max()

NaN is skipped by default → 345.0, 115.0, 130.0.

df["dept"].value_counts()

Counts per value, largest first.

df["dept"].unique(), df["dept"].nunique()

Distinct values; how many.

df.groupby("dept")["salary"].mean()

One aggregate per group → Series indexed by dept.

df.groupby("dept").agg(total=("salary", "sum"), n=("name", "count"))

Named aggregations, several at once.

df.pivot_table(values="salary", index="dept", aggfunc="mean")

Spreadsheet-style pivot; add columns= for a second axis.

df.isna().sum()

Missing values per column.

df.dropna(subset=["salary"])

Drop rows where salary is missing.

df["salary"].fillna(df["salary"].median())

Fill with the median.

df["salary"].fillna(0).astype(int)

Cast after filling; NaN cannot be int.

df["joined"] = pd.to_datetime(df["joined"])

Strings → datetime64.

df["joined"].dt.year, df["joined"].dt.day_name()

.dt gives date parts.

df[df["joined"] >= "2022-01-01"]

Compare datetimes with a date string.

df.merge(other, on="dept", how="left")

SQL-style join; how = inner | left | right | outer.

pd.concat([df, df], ignore_index=True)

Stack frames vertically; axis=1 for side by side.

df.set_index("name"), df.set_index("name").reset_index()

A column as the index, and back.

df.melt(id_vars="name", value_vars=["dept", "salary"])

Wide → long: one row per (name, variable, value).

df.pivot(index="name", columns="dept", values="salary")

Long → wide (keys must be unique).

df.to_dict("records")

List of row dicts, handy for JSON APIs; a Series has .tolist() and .to_numpy().

for row in df.itertuples(index=False):
    row.name, row.salary

Row iteration when you must; itertuples is far faster than iterrows.