I have a big dictionary object that has several key value pairs (about 16), but I am only interested in 3 of them. What is the best way (shortest/efficient/most elegant) to achieve that?
The best I know is:
bigdict = {'a':1,'b':2,....,'z':26}
subdict = {'l':bigdict['l'], 'm':bigdict['m'], 'n':bigdict['n']}
I am sure there is a more elegant way than this. Ideas?
You could try:
dict((k, bigdict[k]) for k in ('l', 'm', 'n'))
... or in Python 3 Python versions 2.7 or later (thanks to Fábio Diniz for pointing that out that it works in 2.7 too):
{k: bigdict[k] for k in ('l', 'm', 'n')}
Update: As Håvard S points out, I'm assuming that you know the keys are going to be in the dictionary - see his answer if you aren't able to make that assumption. Alternatively, as timbo points out in the comments, if you want a key that's missing in bigdict to map to None, you can do:
{k: bigdict.get(k, None) for k in ('l', 'm', 'n')}
If you're using Python 3, and you only want keys in the new dict that actually exist in the original one, you can use the fact to view objects implement some set operations:
{k: bigdict[k] for k in bigdict.keys() & {'l', 'm', 'n'}}
A bit of speed comparison for all mentioned methods:
UPDATED on 2020.07.13 (thx to @user3780389): ONLY for keys from bigdict.
IPython 5.5.0 -- An enhanced Interactive Python.
Python 2.7.18 (default, Aug 8 2019, 00:00:00)
[GCC 7.3.1 20180303 (Red Hat 7.3.1-5)] on linux2
import numpy.random as nprnd
...: keys = nprnd.randint(100000, size=10000)
...: bigdict = dict([(_, nprnd.rand()) for _ in range(100000)])
...:
...: %timeit {key:bigdict[key] for key in keys}
...: %timeit dict((key, bigdict[key]) for key in keys)
...: %timeit dict(map(lambda k: (k, bigdict[k]), keys))
...: %timeit {key:bigdict[key] for key in set(keys) & set(bigdict.keys())}
...: %timeit dict(filter(lambda i:i[0] in keys, bigdict.items()))
...: %timeit {key:value for key, value in bigdict.items() if key in keys}
100 loops, best of 3: 2.36 ms per loop
100 loops, best of 3: 2.87 ms per loop
100 loops, best of 3: 3.65 ms per loop
100 loops, best of 3: 7.14 ms per loop
1 loop, best of 3: 577 ms per loop
1 loop, best of 3: 563 ms per loop
As it was expected: dictionary comprehensions are the best option.
Maybe:
subdict=dict([(x,bigdict[x]) for x in ['l', 'm', 'n']])
Python 3 even supports the following:
subdict={a:bigdict[a] for a in ['l','m','n']}
Note that you can check for existence in dictionary as follows:
subdict=dict([(x,bigdict[x]) for x in ['l', 'm', 'n'] if x in bigdict])
resp. for python 3
subdict={a:bigdict[a] for a in ['l','m','n'] if a in bigdict}
You can also use map (which is a very useful function to get to know anyway):
sd = dict(map(lambda k: (k, l.get(k, None)), l))
Example:
large_dictionary = {'a1':123, 'a2':45, 'a3':344}
list_of_keys = ['a1', 'a3']
small_dictionary = dict(map(lambda key: (key, large_dictionary.get(key, None)), list_of_keys))
PS: I borrowed the .get(key, None) from a previous answer :)
Okay, this is something that has bothered me a few times, so thank you Jayesh for asking it.
The answers above seem like as good a solution as any, but if you are using this all over your code, it makes sense to wrap the functionality IMHO. Also, there are two possible use cases here: one where you care about whether all keywords are in the original dictionary. and one where you don't. It would be nice to treat both equally.
So, for my two-penneth worth, I suggest writing a sub-class of dictionary, e.g.
class my_dict(dict):
def subdict(self, keywords, fragile=False):
d = {}
for k in keywords:
try:
d[k] = self[k]
except KeyError:
if fragile:
raise
return d
Now you can pull out a sub-dictionary with
orig_dict.subdict(keywords)Usage examples:
#
## our keywords are letters of the alphabet
keywords = 'abcdefghijklmnopqrstuvwxyz'
#
## our dictionary maps letters to their index
d = my_dict([(k,i) for i,k in enumerate(keywords)])
print('Original dictionary:\n%r\n\n' % (d,))
#
## constructing a sub-dictionary with good keywords
oddkeywords = keywords[::2]
subd = d.subdict(oddkeywords)
print('Dictionary from odd numbered keys:\n%r\n\n' % (subd,))
#
## constructing a sub-dictionary with mixture of good and bad keywords
somebadkeywords = keywords[1::2] + 'A'
try:
subd2 = d.subdict(somebadkeywords)
print("We shouldn't see this message")
except KeyError:
print("subd2 construction fails:")
print("\toriginal dictionary doesn't contain some keys\n\n")
#
## Trying again with fragile set to false
try:
subd3 = d.subdict(somebadkeywords, fragile=False)
print('Dictionary constructed using some bad keys:\n%r\n\n' % (subd3,))
except KeyError:
print("We shouldn't see this message")
If you run all the above code, you should see (something like) the following output (sorry for the formatting):
Original dictionary:
{'a': 0, 'c': 2, 'b': 1, 'e': 4, 'd': 3, 'g': 6, 'f': 5, 'i': 8, 'h': 7, 'k': 10, 'j': 9, 'm': 12, 'l': 11, 'o': 14, 'n': 13, 'q': 16, 'p': 15, 's': 18, 'r': 17, 'u': 20, 't': 19, 'w': 22, 'v': 21, 'y': 24, 'x': 23, 'z': 25}Dictionary from odd numbered keys:
{'a': 0, 'c': 2, 'e': 4, 'g': 6, 'i': 8, 'k': 10, 'm': 12, 'o': 14, 'q': 16, 's': 18, 'u': 20, 'w': 22, 'y': 24}subd2 construction fails:
original dictionary doesn't contain some keysDictionary constructed using some bad keys:
{'b': 1, 'd': 3, 'f': 5, 'h': 7, 'j': 9, 'l': 11, 'n': 13, 'p': 15, 'r': 17, 't': 19, 'v': 21, 'x': 23, 'z': 25}
solution
from operator import itemgetter
from typing import List, Dict, Union
def subdict(d: Union[Dict, List], columns: List[str]) -> Union[Dict, List[Dict]]:
"""Return a dict or list of dicts with subset of
columns from the d argument.
"""
getter = itemgetter(*columns)
if isinstance(d, list):
result = []
for subset in map(getter, d):
record = dict(zip(columns, subset))
result.append(record)
return result
elif isinstance(d, dict):
return dict(zip(columns, getter(d)))
raise ValueError('Unsupported type for `d`')
examples of use
# pure dict
d = dict(a=1, b=2, c=3)
print(subdict(d, ['a', 'c']))
>>> In [5]: {'a': 1, 'c': 3}
# list of dicts
d = [
dict(a=1, b=2, c=3),
dict(a=2, b=4, c=6),
dict(a=4, b=8, c=12),
]
print(subdict(d, ['a', 'c']))
>>> In [5]: [{'a': 1, 'c': 3}, {'a': 2, 'c': 6}, {'a': 4, 'c': 12}]
Using map (halfdanrump's answer) is best for me, though haven't timed it...
But if you go for a dictionary, and if you have a big_dict:
so e.g.:
big_dict = {'a':1,'b':2,'c':3,................................................}
req = ['a','c','w']
{k:big_dict.get(k,None) for k in req )
# or
{k:big_dict[k] for k in req if k in big_dict)
Note that in the converse case, that the req is big, but my_dict is small, you should loop through my_dict instead.
In general, we are doing an intersection and the complexity of the problem is O(min(len(dict)),min(len(req))). Python's own implementation of intersection considers the size of the two sets, so it seems optimal. Also, being in c and part of the core library, is probably faster than most not optimized python statements. Therefore, a solution that I would consider is:
dict = {'a':1,'b':2,'c':3,................................................}
req = ['a','c','w',...................]
{k:dic[k] for k in set(req).intersection(dict.keys())}
It moves the critical operation inside python's c code and will work for all cases.
