Building a huge weighted network with networkx in python

Question

I have a huge number of dictionaries in the format

d1={'el1':2, 'el3':4, ...,'el12':32}
d2={'el3':5, 'el4':6, ...,'el12':21}

I want to create a single network using networkx in which: every node is one of the keys of the dictionaries that has an attribute that represents the sum of all the values of the node (for example, it would be 9 for el3 considering the two given dictionaries), and there is an edge between two nodes if they appear together in the same dictionary, with a weight attribute equal to the number of times they appear together (for instance it would be 2 for el3 and el12, as they appear together in 2 dictionaries).

I know how to create networks and how to add attributes to nodes and edges in networkx, but I am looking for an efficient way, as I have about 12.000 of those dictionaries.

Corley Brigman Corley Brigman · Accepted Answer · 2013-10-16T13:02:15

not sure how much faster you can get than brute force, but itertools can make adding the edges easier, using permutations/combinations...

d1 = {'el1': 2, 'el3': 4, 'el5': 17, 'el12':32}
d2 = {'el1': 5, 'el3': 9, 'el5': 11, 'el12':6}
d3 = {'el1': 1, 'el6': 2, 'el7': 41, 'el12':13}

d = [d1, d2, d3]

G = nx.DiGraph()
# or just Graph() if not weighted
# If unweighted, you should use combinations() instead, as for a given list
# ['e1', 'e2', 'e3'], permutations(l, 2) will give both ('e1', 'e2') and ('e2','e1')
# whereas combinations will give only one of those. 

for item in d:
    G.add_nodes_from(item)
    for entry in item:
        try: 
            G.node[entry]['weight'] += item[entry]
        except:
            G.node[entry]['weight'] = item[entry]
    for source, target in itertools.permutations(item.keys(), 2):
        G.add_edge(source, target)
        try: 
            G.edge[source][target]['weight'] += 1
        except:
            G.edge[source][target]['weight'] = 1

G.node
{'el1': {'weight': 8},
 'el12': {'weight': 51},
 'el3': {'weight': 13},
 'el5': {'weight': 28},
 'el6': {'weight': 2},
 'el7': {'weight': 41}}
G.edge
{'el1': {'el12': {'weight': 3},
  'el3': {'weight': 2},
  'el5': {'weight': 2},
  'el6': {'weight': 1},
  'el7': {'weight': 1}},
 'el12': {'el1': {'weight': 3},
  'el3': {'weight': 2},
  'el5': {'weight': 2},
  'el6': {'weight': 1},
  'el7': {'weight': 1}},
 'el3': {'el1': {'weight': 2}, 'el12': {'weight': 2}, 'el5': {'weight': 2}},
 'el5': {'el1': {'weight': 2}, 'el12': {'weight': 2}, 'el3': {'weight': 2}},
 'el6': {'el1': {'weight': 1}, 'el12': {'weight': 1}, 'el7': {'weight': 1}},
 'el7': {'el1': {'weight': 1}, 'el12': {'weight': 1}, 'el6': {'weight': 1}}}

Building a huge weighted network with networkx in python

1 Answers