How to ignore duplicate string?

Question

I had some script below

unique=[]

thislist =['idn_type1=：111-22-3333',
'idn_type1=：111-22-3333',
'idn_type1=：222-33-4444',
'idn_type2=：333-44-5555',
'idn_type2=：222-33-4444',
'idn_type3=：222-33-4444',
'idn_type4=：222-33-4444',
'idn_type1=：',
'idn_type1=：444-55-6666',
'idn_type1=：555-66-7777',
'idn_type1=：']

for name in thislist:
      if name not in unique and name.partition('：')[-1] not in unique and name.partition('：')[-1]!='':
            unique.append(name)

for i in range(len(unique)):
    print(unique[i])

And the result is wrong

idn_type1=：111-22-3333
idn_type1=：222-33-4444
idn_type2=：333-44-5555
idn_type2=：222-33-4444
idn_type3=：222-33-4444
idn_type4=：222-33-4444
idn_type1=：444-55-6666
idn_type1=：555-66-7777

But what I want is

idn_type1=：111-22-3333
idn_type1=：222-33-4444
idn_type2=：333-44-5555
idn_type1=：444-55-6666
idn_type1=：555-66-7777

I want idn '222-33-4444' append only once when it meets first with idn_type1. Only idn_type1=：222-33-4444 appended to the new list.

How to modify this script?

You didn't insert what you want to check for into the list, namely name.partition(...). — iBug

Ohad Sharet Ohad Sharet · Accepted Answer · 2022-07-28T14:55:11

you can use set to remember the numbers you already saw

unique = []

thislist = ['idn_type1=：111-22-3333',
            'idn_type1=：111-22-3333',
            'idn_type1=：222-33-4444',
            'idn_type2=：333-44-5555',
            'idn_type2=：222-33-4444',
            'idn_type3=：222-33-4444',
            'idn_type4=：222-33-4444',
            'idn_type1=：',
            'idn_type1=：444-55-6666',
            'idn_type1=：555-66-7777',
            'idn_type1=：']

seen = set()
for name in thislist:
    numbers = name.partition('：')[-1]
    if name not in unique and numbers not in seen and numbers != '':
        unique.append(name)
        seen.add(numbers)

for i in range(len(unique)):
    print(unique[i])

output:

idn_type1=：111-22-3333
idn_type1=：222-33-4444
idn_type2=：333-44-5555
idn_type1=：444-55-6666
idn_type1=：555-66-7777

hope i could help, if you need me to clarify anything feel free to ask in the comments :)

How to ignore duplicate string?

2 Answers