How To Remove Duplicates From A List (Python Recipes)

In this video we will be learning about two different ways we can remove duplicates from lists in Python.

▶ Become job-ready with Python:

▶ Follow me on Instagram:

[Jason-bt]

Be careful when using set() because it's unordered. This isn't just "don't use set() when you need the list to stay in order" because ordered/unordered are not types. Therefore, you may 100% think from the API and understanding of subroutines that you only need an unordered list, but in fact it requires an ordered list, resulting in very serious and difficult-to-track errors (yes, I have encountered it. The external exe executed in the program depends on the order of files in the zip archive I created).

[staywithmeforever]

So many new things I'm learning from you.

[MadToroLoco]

Man thank you for these tips, they are really good

[ali-omuv]

def preserve_order(lst):
seen = set()
return [x for x in lst if not (x in seen or seen.add(x))]

[PR-cjpd]

Is there a speed difference with the 2 proposed methods?

[pseudotasuki]

Maintaining order of dicts is an implementation detail of CPython. It is *not* part of the Python specification.

[shinewaine]

Good tip, i know set() but the dict method is new for me. Thanks.

[MuhammadFaizanMumtaz]

That's so great! I am learning so much from you. A bundle of thanks. May Allah give you more respect and a long life. Ameen

[LV-]

When I run list(set(numbers)) it always maintains the order for me. I even added in several other numbers throughout the original list to see if I could get the numbers out of order

[piedepew]

class Solution:
def removeDuplicate(self, arr):
setarr = set(arr)
ans = []
for i in arr:
if(i in setarr):
ans.append(i)
setarr.discard(i)
return ans

[intron]

Set doesn't work if the items are unhasheable (even lists arent hasheable) .
(Edit: Counter would face the same problem)

[laytonjr]

Because I dislike simplicity:
S = set()
for elt in numbers:
if elt not in S:
S.add(elt)
print(list(S))

[SRIKRISHNAV-ob]

Hello, This video was so helpful. But sir your video is something like follows a basics of python. I suggest to make some videos such as Build a basic Neural Network, CNN, create a Transformers using python.

[neuromorphing]

The content of this channel becomes more and more boring. Why not telling something new?