A Collection That Decides for Itself What Counts as a Duplicate

An import reads forty thousand rows and the same customer appears in nine of them. Checking each new row against everything already loaded is a loop inside a loop, and it is the reason the import takes eleven minutes.

A Set does that check for you, and it does it in one step rather than forty thousand. What makes that possible is that it decides duplication by hashing rather than by comparing against everything.

java

Set<String> seen = new HashSet<>();

boolean isNew = seen.add(customerRef);

add returns true if the element was not already there and false if it was, so one call both records and answers. There is no separate contains needed, and no second pass.

So a set's job is answering "have I seen this" quickly, and the answer comes from methods on the elements themselves. Refusing a duplicate is what it does with that answer, rather than the reason it exists.