5
votes

I'm trying to remove duplicates from a nested list only if the first 2 elements are the same, ignoring the third...

List:

L = [['el1','el2','value1'], ['el3','el4','value2'], ['el1','el2','value2'], ['el1','el5','value3']]

Would return:

L = [['el3','el4','value2'], ['el1','el2','value2'], ['el1','el5','value3']]

I found a simple way to do similar here:

dict((x[0], x) for x in L).values()

but this only works for the first element and not the first 2, but that is exactly what i want otherwise.

3

3 Answers

4
votes

If the order doesn't matter, you can use that same method but using a tuple of the first and second elements as the key:

dict(((x[0], x[1]), x) for x in L).values()

Or on Python 2.7 and higher:

{(x[0], x[1]): x for x in L}.values()

Instead of (x[0], x[1]) you can use tuple(x[:2]), use whichever you find more readable.

3
votes

If order matters, use a set with only the first two elements of your nested lists:

seen = set()
seen_add = seen.add
return [x for x in seq if tuple(x[:2]) not in seen and not seen_add(tuple(x[:2]))]

or you could use a collections.OrderedDict() object to keep the order; keep the x[:2] slices as keys (as tuples), and extract the values:

from collections import OrderedDict(

return OrderedDict((tuple(x[:2]), x) for x in seq).values()

In Python 3.6 and up, the standard dict type happens to retain insertion order too:

return list({tuple(x[:2]): x for x in seq}.values())

The list() call is needed to convert the dictionary view object to a list.

2
votes

this should do it:

In [55]: dict((tuple(x[:2]), x) for x in L).values()
Out[55]: [['el1', 'el2', 'value2'], ['el1', 'el5', 'value3'], ['el3', 'el4', 'value2']]