DEV Community

Remove duplicates from an Array the short & sweet way!

Ajmal Hassan on November 22, 2019

tldr; const numbers = [1,2,2,3,3,3,4,4,4,4] console.log([...new Set(numbers)]) //output: [1,2,3,4] // OR console.log(Array.from(new ...
Collapse
 
garetmckinley profile image
Garet McKinley

I'd be curious to see performance comparisons for this vs traditional deduping, like Array.filter(). The two way conversion feels like an expensive operation, but I have no basis for that other than the feeling.

Collapse
 
ajmalhassan profile image
Ajmal Hassan

Infact, Array.filter() is more performant but it's a bit harder to understand it's working when compared to this.

array.filter((item, index)=> array.indexOf(item)===index)

If performance isn't critical then the method in the post is more readable and simpler.

Collapse
 
leonardosnt profile image
Leonardo Santos

I don't think the Array.filter + Array.indexOf approach is more performant. Unless I'm missing something.

Both are linear, so for every item you are filtering, you are also iterating over the array again to find an index. Therefore, in the worst case, it will end up having a quadratic O(n^2) performance, which is really bad.

Collapse
 
garetmckinley profile image
Garet McKinley

Maybe it's just my own ignorance not being familiar with the Set syntax. But the Array.filter method is so widely used that I'd know instantly what that line did. The example in the original post would confuse me to no end if there were no comments. I'd definitely have to google to figure out what's going on and why.

I'm sure the syntax is nice when you're familiar with it, but it's just not something that's done very often, so I'd definitely always leave a comment before it if you're using it in a code base that other people will have to read eventually.

Thread Thread
 
ajmalhassan profile image
Ajmal Hassan

I agree, Set isn't something we commonly use.

Collapse
 
ankit199 profile image
Ankit kumar shukla

is this work with like array data [{id=1,name='a'},{id=1,name='b'},{id=2,name='c'},{id=3,name='d'},{id=2,name='e'},{id=3,name='f'},{id=4,name='ag'},{id=5,name='au'}]

Collapse
 
ajmalhassan profile image
Ajmal Hassan

No, it won't.

The above method only works for Primitive Types (number, string, boolean, null, undefined) in the array.

so,

const primitivesArr = [true, false, true, false, undefined, 1, undefined, 'hello', 1, 'hello']

console.log([...new Set(primitivesArr)])
// output: [true, false, undefined, 1, "hello"]

For objects, you'll have to use filter()

Collapse
 
ankit199 profile image
Ankit kumar shukla

okk ..thank you

Collapse
 
rohansawant profile image
Rohan Sawant

Bonus - This works similarly in Python too, you can convert a List to a Set and that removes the duplicates.

Collapse
 
ajmalhassan profile image
Ajmal Hassan

One thing though, Python has unordered sets so, it may not keep the order of the elements intact.

Collapse
 
sinteticwizard profile image
Ramon Ramirez

excelent

Collapse
 
ajmalhassan profile image
Ajmal Hassan

Glad you liked it.

Collapse
 
sqlrob profile image
Robert Myers

Does this maintain order?

Collapse
 
ajmalhassan profile image
Ajmal Hassan • Edited

Yeah, because javascript uses Ordered Sets unlike say python.