InterviewsVector

Collapse Repeated Rows into One per Key in Pandas (groupby + agg)

Python

Quick answer

Use groupby().agg() to collapse repeated rows into one per key. df.groupby('id').agg({'value':'sum','name':'first'}) reduces each group to a single row with a different aggregation per column. To gather the repeated values into one cell, use a list aggregation: df.groupby('id')['tag'].agg(list). Call .reset_index() to turn the group key back into a column, and use pivot_table when you want the repeated categories spread across columns instead.

Short answer: Group by the repeated key and aggregate: df.groupby('id').agg({'value':'sum','name':'first'}) gives one row per key. To keep every repeated value, use df.groupby('id')['tag'].agg(list). reset_index() turns the key back into a column; pivot_table reshapes categories into columns.

One row per key with groupby().agg()

Pass a dict to apply a different aggregation to each column in a single pass:

import pandas as pd
 
result = (
    df.groupby("id")
      .agg({"value": "sum", "score": "mean", "name": "first"})
      .reset_index()
)

Each id becomes one row: value summed, score averaged, name taking the first value. reset_index() moves id from the index back to a column.

Keep every repeated value: agg(list)

When you don't want to reduce the values but gather them:

df.groupby("id")["tag"].agg(list).reset_index()
# id -> ['a', 'b', 'a']   (all tags for that id, in one cell)
 
df.groupby("id")["tag"].agg(lambda s: ", ".join(sorted(set(s))))  # unique, joined

Multiple aggregations per column

Named aggregation gives clean output column names:

df.groupby("id").agg(
    total=("value", "sum"),
    avg=("value", "mean"),
    n=("value", "count"),
).reset_index()

Reshape instead of reduce: pivot_table

When the repeated category should become columns rather than stacked rows:

df.pivot_table(index="id", columns="date", values="value", aggfunc="sum").reset_index()
# one row per id, one column per date

Which to use

GoalUse
One row per key, aggregatedgroupby().agg({...})
Keep all repeated valuesgroupby()[col].agg(list)
Category → columnspivot_table(...)

Related guides

Sources

Key takeaways

  • •df.groupby(key).agg({...}) collapses each group to one row, with a chosen aggregation per column.
  • •Collect repeated values into a list cell with df.groupby(key)['col'].agg(list).
  • •Use a dict in agg() to apply different functions to different columns in one pass.
  • •reset_index() turns the group key back into a normal column after aggregating.
  • •pivot_table spreads a repeated category across columns instead of stacking rows.

Frequently asked questions

How do I combine duplicate rows into one in pandas?

Group by the key that repeats and aggregate: df.groupby('id').agg({'value':'sum','name':'first'}). Each group becomes a single row. Choose the aggregation per column — sum, mean, first, max, or a list to keep every value.

How do I collect repeated values into a single cell?

Use a list aggregation: df.groupby('id')['tag'].agg(list) returns one row per id with all that id's tags in a list. Use set or ', '.join for unique values or a joined string.

When should I use pivot_table instead of groupby?

Use groupby+agg to stack aggregated results into rows. Use pivot_table when you want a repeated category to become columns — e.g. one row per id with a column per date — which reshapes rather than just reduces.

By Mohammad Wasi

Software Engineering Leader & Technical Author · Updated August 26, 2026

See all posts

Related Posts