Last updated 3 September 2026
Three different questions about the same list
Removing duplicates is the obvious use, but it is not always the interesting one. Keeping only the lines that appeared more than once turns the same list into a report of what repeated — useful for finding double-entered records, overlapping email lists or recurring log entries. Keeping only the lines that appeared exactly once does the opposite, isolating the entries that are unique to the set. All three run off the same count, which is why they sit together here rather than as separate tools. Order is preserved by default because the position of the first occurrence usually carries meaning; sorting is available when it does not.
What counts as a duplicate
The most common surprise is lines that look identical but are not. Trailing whitespace is invisible on screen and makes two otherwise matching lines different strings. So does a mixture of line endings, where a file assembled from both Windows and Unix sources carries different invisible characters at the end of different lines. So does case, if the comparison is case-sensitive — Email@example.com and email@example.com are the same address to a mail server and two different strings to a text comparison.
This is why deduplicating a list often removes fewer lines than expected. Trimming whitespace and normalising case before comparing usually resolves the gap, and it is worth checking the before and after counts rather than assuming the operation worked.
Order, and knowing what you removed
There is a meaningful difference between removing duplicates while preserving the original order and sorting the list first. Preserving order keeps the first occurrence of each line where it was, which matters for a chronological log or a ranked list. Sorting groups identical lines together and makes the duplicates visible, which is more useful when you want to see what was repeated rather than simply eliminate it.
One habit worth keeping for any list that matters: note the line count before and after. If a 5,000-line export drops to 1,200, that is either a genuinely repetitive dataset or a sign that the lines are less unique than you assumed — for instance because a leading column was truncated. Either way it is better to notice immediately than after the deduplicated list has been used for something.
Common questions
Does it keep the first or the last occurrence?
The first. Later copies are dropped, which preserves the original order of the list. If you would rather have the result ordered alphabetically, tick the sort option instead.
What does ignoring case do to the output?
It treats “Apple” and “apple” as the same line for the purpose of counting, and keeps whichever version appeared first. The kept line retains its original capitalisation rather than being lowercased.
What is the difference between the three modes?
The first keeps one copy of everything. The second keeps only lines that appeared at least twice, which turns the tool into a duplicate finder. The third keeps only lines that appeared exactly once, isolating the entries that are unique.
Is my list uploaded anywhere?
No. Everything is processed in your browser and nothing is sent to a server, so it is safe for customer lists, emails or anything else you would not paste into an unknown site.
Why were fewer lines removed than I expected?
Almost always invisible differences. Trailing spaces, mixed Windows and Unix line endings, or differing capitalisation all make lines that look identical count as distinct strings. Trimming whitespace and normalising case before comparing usually resolves it.
Should I sort the list first?
It depends on what you need. Preserving the original order keeps the first occurrence of each line in place, which matters for logs and ranked lists. Sorting groups identical lines together and makes it easier to see what was actually repeated.
What do the four counters above the result mean?
Lines in is what you pasted. Lines out is what survived the mode and options you chose. Duplicates removed is the difference between the two. Most repeated shows the line that appeared most often, which is usually the quickest way to spot a record that was entered several times.