Removing Duplicates from a JavaScript Array ('Deduping')

For an array of primitive values, deduplication can be as small as [...new Set(values)]. Arrays of objects are different because Set compares object identity, not matching fields. The right method therefore depends on what 'duplicate' means for the data in front of you.

Using the Set Object to Remove Duplicates

One of the simplest and most efficient ways to remove duplicates is by using the Set object. A Set automatically enforces uniqueness, making it ideal for deduping arrays of primitive values. For example:

const numbers = [1, 2, 2, 3, 4, 4, 5];
const uniqueNumbers = [...new Set(numbers)];
console.log(uniqueNumbers);  // [1, 2, 3, 4, 5]

This visits the input once and is a useful default for primitive values. Set also works with objects when duplicate means the same object reference appearing more than once. It does not merge separate objects merely because their fields match.

Removing Duplicates from an Array of Objects

For objects that should be unique by a field such as id, define that rule explicitly. Set compares object references; filter() with findIndex() can instead retain the first object with each matching identifier:

This looks something like this:

const people = [
  { id: 1, name: "Maddie" },
  { id: 2, name: "Bob" },
  { id: 1, name: "Maddie" }
];

const uniquePeople = people.filter(
  (person, index, self) =>
    index === self.findIndex(p => p.id === person.id)
);

console.log(uniquePeople);
// [{ id: 1, name: "Maddie" }, { id: 2, name: "Bob" }]

In this way, we can ensure that only the first occurrence of an object with a given id is retained, effectively deduping the array. However, findIndex iterates through the array for each element, making this method O(n²) in the worst case, which means it can become very slow for large datasets.

Deduping Arrays Using reduce

Another approach to deduping involves using Array.prototype.reduce to build a unique array by checking for existing values as we iterate, like this:

const numbers = [1, 2, 2, 3, 4, 4, 5];

const uniqueNumbers = numbers.reduce<number[]>((acc, num) => {
  if (!acc.includes(num)) {
    acc.push(num);
  }
  return acc;
}, []);

console.log(uniqueNumbers);  // [1, 2, 3, 4, 5]

This method offers more control over how duplicates are handled but is ‑ as with using .filter() ‑ O(n²) in complexity because .includes() checks each element in acc linearly, for every iteration. As a result ‑ again ‑ this approach is not terribly efficient for large arrays.

Performance Comparison: Set vs. filter vs. reduce

Here's an overview of the efficiency of each deduping method depends on the dataset size and the type of values stored in the array:

For a small array, readability may matter more than the difference in runtime. For a larger collection, a Map keyed by the chosen identifier avoids repeatedly scanning earlier values. Check which duplicate should win before adopting this version:

const uniquePeople = Array.from(new Map(people.map(person => [person.id, person])).values());

This Map version keeps the last object for each id, whereas the earlier filter() example keeps the first. Updating an existing key does not change its insertion position, so the output follows the order in which each identifier first appeared. It builds the map in one pass; the exact cost still depends on the implementation and data.

Wrapping Up

Key Takeaways

• Set is a concise way to deduplicate primitive values or repeated references to the same object.
• When working with objects, filter with findIndex is an option but can be slow for large datasets.
• reduce allows for more custom deduplication logic but is inefficient for large arrays.
• Using a Map provides a better alternative for efficiently deduping large objects.

Use Set for primitive values and a Map keyed by a stable identifier when objects should be unique by one field. Reach for reduce when the merge itself needs custom rules. Define equality first; choosing between Set, filter(), and reduce() becomes much easier once the data contract is explicit.