python iterate json file where the json structure and key values are unknown

Question

consider the sample JSON below.

{
"widget": {
    "test": "on",
    "window": {
        "title": "myWidget1",
        "name": "main_window"
    },
    "image": {
        "src": "Images/wid1.png",
        "name": "wid1"
    }
},
"os":{
    "name": "ios"
}

}

Consider the case where we dont know the structure of the JSON and any of the keys. What I need to implement is a python function which iterates through all the keys and sub-keys and prints the key. That is by only knowing the JSON file name, I should be able to iterate the entire keys and sub-keys. The JSON can be of any structure.What I have tried is given below.

JSON_PATH = "D:\workspace\python\sampleJSON.json"
os.path.expanduser(JSON_PATH)

def iterateAllKeys(e):
    for key in e.iterkeys():
        print key
        for child in key.get(key):
            iterateAllKeys(child)

with open(JSON_PATH) as data_file:    
    data = json.load(data_file)

iterateAllKeys(data)

Here, the iterateAllKeys() function is supposed to print all the keys present in the JSON file. But if only the outer loop is present, ie

def iterateAllKeys(e):
    for key in e.iterkeys():
        print key

It will print the keys "widget" and "os". But,

def iterateAllKeys(e):
    for key in e.iterkeys():
        print key
        for child in key.get(key):
            iterateAllKeys(child)

returns an error - AttributeError: 'unicode' object has no attribute 'get'. My understanding is - since the value of 'child' is not a dict object, we cannot apply the 'key.get()'. But is there any alternate way by which I can iterate the JSON file without specifying any of the key names. Thank you.

Answer 1

You can use recursion to iterate through multi level dictionaries like this:

def iter_dict(dic):
    for key in dic:
        print(key)
        if isinstance(dic[key], dict):
            iter_dict(dic[key])

The keys of the first dictionary are iterated and every key is printed, if the item is an instance of dict class, we can use recursion to also iterate through the dictionaries we encounter as items.

Answer 2

You can do this thru auxiliary package like flatten_json.

pip install flatten_json

from flatten_json import flatten

for key in flatten(your_dict).keys():
    print(key)

Output:

widget_test
widget_window_title
widget_window_name
widget_image_src
widget_image_name
os_name

If you want to show only key without whole path then you can do like that:

print(key.split('_')[-1])

Answer 3

First of all your last function:

def iterateAllKeys(e):
    for key in e.iterkeys():
        print key
        for child in key.get(key):
            iterateAllKeys(child)

key is just the key_value of the dictionary. So if anything you should be using e.get(key) or e[key].

 for child in e.get(key):

Now this would not solve your problem, one work-around is using try except, as follows:

def iterateAllKeys(e):
    for key in e.iterkeys():
        print key
        try:
            iterateAllKeys(e[key])
        except:
            print "---SKIP---"

This is maybe not the best work-around, but it certainly works. With your Data it prints the following:

widget
test
---SKIP---
window
name
---SKIP---
title
---SKIP---
image
src
---SKIP---
name
---SKIP---
os
name
---SKIP---