python - Compare dictionaries ignoring specific keys

33
votes

How can I test if two dictionaries are equal while taking some keys out of consideration. For example,

equal_dicts(
    {'foo':1, 'bar':2, 'x':55, 'y': 77 },
    {'foo':1, 'bar':2, 'x':66, 'z': 88 },
    ignore_keys=('x', 'y', 'z')
)

should return True.

UPD: I'm looking for an efficient, fast solution.

UPD2. I ended up with this code, which appears to be the fastest:

def equal_dicts_1(a, b, ignore_keys):
    ka = set(a).difference(ignore_keys)
    kb = set(b).difference(ignore_keys)
    return ka == kb and all(a[k] == b[k] for k in ka)

Timings: https://gist.github.com/2651872

6
pythondictionary
I appreciate that you compiled these various answers into a timings gist. However, one thing I noticed when looking them over they aren't all doing the same thing. Some compare both dictionaries keys...some just compare the keys from the first dictionary...some handle key errors and others don't. I wish they were all down to the minimal case, or up to the max to show truly which is faster, but either way, I appreciate your compilation as it was helpful. - Jarvis

6 Answers

30
votes
def equal_dicts(d1, d2, ignore_keys):
    d1_filtered = {k:v for k,v in d1.items() if k not in ignore_keys}
    d2_filtered = {k:v for k,v in d2.items() if k not in ignore_keys}
    return d1_filtered == d2_filtered

EDIT: This might be faster and more memory-efficient:

def equal_dicts(d1, d2, ignore_keys):
    ignored = set(ignore_keys)
    for k1, v1 in d1.iteritems():
        if k1 not in ignored and (k1 not in d2 or d2[k1] != v1):
            return False
    for k2, v2 in d2.iteritems():
        if k2 not in ignored and k2 not in d1:
            return False
    return True
13
votes

Using dict comprehensions:

>>> {k: v for k,v in d1.items() if k not in ignore_keys} == \
... {k: v for k,v in d2.items() if k not in ignore_keys}

Use .viewitems() instead on Python 2.

1
votes

Very very crudely, you could just delete any ignored keys and compare those dictionaries:

def equal_dicts(d1, d2, ignore_keys=()):
    d1_, d2_ = d1.copy(), d2.copy()
    for k in ignore_keys:
        try:
            del d1_[k]
        except KeyError: 
            pass
        try:
            del d2_[k]
        except KeyError: 
            pass

    return d1_ == d2_

(Note that we don't need a deep copy here, we just need to avoid modifying d1 and d2.)

1
votes
def compare_dict(d1, d2, ignore):
    for k in d1:
        if k in ignore:
            continue
        try:
            if d1[k] != d2[k]:
                return False
        except KeyError:
            return False
    return True

Comment edit: You can do something like compare_dict(d1, d2, ignore) and compare_dict(d2, d1, ignore) or duplicate the for

def compare_dict(d1, d2, ignore):
    ignore = set(ignore)
    for k in d1:
        if k in ignore:
            continue
        try:
            if d1[k] != d2[k]:
                return False
        except KeyError:
            return False

    for k in d2:
        if k in ignore:
            continue
        try:
            if d1[k] != d2[k]:
                return False
        except KeyError:
            return False
    return True

Whatever is faster and cleaner! Update: cast set(ignore)

-1
votes

Optimal solution for the case of ignoring only one key

return all(
    (x == y or (x[1] == y[1] == 'key to ignore')) for x, y in itertools.izip(
          d1.iteritems(), d2.iteritems()))
-1
votes

in case your dictionary contained lists or other dictionaries:

def equal_dicts(d1, d2, ignore_keys, equal):
    # print('got d1', d1)
    # print('got d2', d2)
    if isinstance(d1, str):
        if not isinstance(d2, str):
            return False
        return d1 == d2
    for k in d1:
        if k in ignore_keys:
            continue
        if not isinstance(d1[k], dict) and not isinstance(d1[k], list) and d2.get(k) != d1[k]:
            print(k)
            equal = False
        elif isinstance(d1[k], list):
            if not isinstance(d2.get(k), list):
                equal = False
            if len(d1[k]) != len(d2[k]):
                return False
            if len(d1[k]) > 0 and isinstance(d1[k][0], dict):
                if not isinstance(d2[k][0], dict):
                    return False
                d1_sorted = sorted(d1[k], key=lambda item: item.get('created'))
                d2_sorted = sorted(d2[k], key=lambda item: item.get('created'))
                equal = all(equal_dicts(x, y, ignore_keys, equal) for x, y in zip(d1_sorted, d2_sorted)) and equal
            else:
                equal = all(equal_dicts(x, y, ignore_keys, equal) for x, y in zip(d1[k], d2[k])) and equal
        elif isinstance(d1[k], dict):
            if not isinstance(d2.get(k), dict):
                equal = False
            print(k)
            equal = equal_dicts(d1[k], d2[k], ignore_keys, equal) and equal
    return equal