Small BusinessIndependent InvestorTrade BusinessAccountingFinanceLegalMarketingOperationsSalesReal EstateTemplatesPricingDocs
Get started

Step Guide: Deduplicate

Flow Control

Collapse duplicate rows in a list into one

Overview

Deduplicate is one of the two actions on the Filter step. It collapses duplicate rows in a list down to one row per item, using the fields you choose as the identity key. Run it before a For Each loop or an export so you do not process or write the same record twice.

Deduplicate

Collapse duplicate rows in a list to one row per key. You pick the fields that make two rows count as the same, and choose which row to keep when duplicates are found.

  • Remove duplicate contacts after merging two lists, before emailing them
  • Collapse repeated events pulled from more than one feed
  • Clean a scraped or imported list before a For Each loop so each item is processed once

Connect a list from an upstream step, then choose the key fields that identify a duplicate (for example venue and start date, or email). By default matching ignores case, spacing, and punctuation, so "Acme Inc." and "acme inc" count as the same. Pick how to resolve duplicates: keep the first row, keep the last, keep the most complete, or merge the richest values from across the group. The output is the deduplicated list, its count, and how many rows were removed. Rows whose key fields are all blank are left alone.

Tips
  • Choose key fields that truly identify a duplicate: a combination like email plus company is more reliable than a single field
  • Reach for the merge option when duplicates each hold different pieces you want to keep

Frequently Asked Questions

How does Dedupe decide two rows are the same?
By the key fields you choose. It compares the combination of those fields, ignoring case, spacing, and punctuation by default, so small formatting differences still count as a match.
Which row does it keep?
Whichever you choose: the first, the last, the most complete, or a merge that takes the richest value for each field across the duplicates. Merge is the default.
What is the difference between the Filter and Deduplicate actions?
They are the two actions on the same Filter step. Filter removes rows that fail a condition. Deduplicate removes rows that repeat an existing one. Choose Filter to drop unwanted records, and Deduplicate to collapse duplicates. You can switch between them from the action selector at the top of the step.