python filter list of dictionaries based on key value

Question

I have a list of dictionaries and each dictionary has a key of (let's say) 'type' which can have values of 'type1', 'type2', etc. My goal is to filter out these dictionaries into a list of the same dictionaries but only the ones of a certain "type". I think i'm just really struggling with list/dictionary comprehensions.

so an example list would look like:

exampleSet = [{'type':'type1'},{'type':'type2'},{'type':'type2'}, {'type':'type3'}]

i have a list of key values. lets say for example:

keyValList = ['type2','type3']

where the expected resulting list would look like:

expectedResult = [{'type':'type2'},{'type':'type2'},{'type':'type3'}]

I know i could do this with a set of for loops. I know there has to be a simpler way though. i found a lot of different flavors of this question but none that really fit the bill and answered the question. I would post an attempt at the answer... but they weren't that impressive. probably best to leave it open ended. any assistance would be greatly appreciated.

Answer 1

You can try a list comp

>>> exampleSet = [{'type':'type1'},{'type':'type2'},{'type':'type2'}, {'type':'type3'}]
>>> keyValList = ['type2','type3']
>>> expectedResult = [d for d in exampleSet if d['type'] in keyValList]
>>> expectedResult
[{'type': 'type2'}, {'type': 'type2'}, {'type': 'type3'}]

Another way is by using filter

>>> list(filter(lambda d: d['type'] in keyValList, exampleSet))
[{'type': 'type2'}, {'type': 'type2'}, {'type': 'type3'}]

Answer 2

Use filter, or if the number of dictionaries in exampleSet is too high, use ifilter of the itertools module. It would return an iterator, instead of filling up your system's memory with the entire list at once:

from itertools import ifilter
for elem in ifilter(lambda x: x['type'] in keyValList, exampleSet):
    print elem

Answer 3

This type of filtering is very easy to do in Pandas, especially as there are a lot of cases where lists of dictionaries work better as Pandas dataframes to begin with.

import pandas as pd

exampleSet = [{'type':'type1'}, {'type':'type2'}, {'type':'type2'}, {'type':'type3'}]
keyValList = ['type2', 'type3']

df = pd.DataFrame(my_list)
df[df['type'].isin(keyValList)]

results in:

    type
1   type2
2   type2
3   type3

and to get it back in dictionary form as desired by OP:

expectedResult = df[df['type'].isin(keyValList)].to_dict('records')
# the result will be [{'type': 'type2'}, {'type': 'type2'}, {'type': 'type3'}]