Remove Duplicate Lines
Paste a list and every repeated line is removed instantly, keeping the first occurrence in its original order. Useful for cleaning email lists, keyword lists, product codes, URLs and log files.
Remove duplicates
How it works
- The text is split into lines.
- Each line is compared with the lines already kept, after trimming spaces and ignoring case if you chose those options.
- Only the first copy of each line is kept; later copies are removed.
- The result can be sorted and copied with one click.
Example
Input: apple, banana, Apple, cherry, banana, (blank),
cherry, date- Default options (trim on, case sensitive): apple, banana, Apple, cherry, date (2 duplicates and 1 empty line removed)
- With “Ignore case”: apple, banana, cherry, date
Doing it in Excel or Google Sheets
| Program | Method |
|---|---|
| Excel | Data → Remove Duplicates, or =UNIQUE(A1:A100) in Excel 365 |
| Google Sheets | Data → Data cleanup → Remove duplicates, or =UNIQUE(A1:A100) |
| Command line | awk '!seen[$0]++' file.txt (keeps order) |
Frequently asked questions
Does it keep the original order?
Yes, unless you choose a sort option. The first time a line appears is the copy that is kept.
Is there a limit on the number of lines?
No fixed limit; lists of tens of thousands of lines process instantly in a modern browser.
Is my data uploaded?
No. Everything happens in your browser, so it is safe for email lists and private data.