Is h5py capable of converting python dictionaries to hdf5 groups automatically?

Question

I have been using scipy.io to save my structured data (lists and dictionaries filled with ndarrays in different shapes). Since v7.3 mat file is going to replace the old v7 mat format some day, I am thinking about switching to HDF5 to store my data, more specifically h5py for python. However, I noticed that I cannot save my dictionaries as easy as:

import scipy.io as sio
data = {'data': 'Complicated structure data'}
sio.savemat('fileName.mat', data)

Instead, I have to use h5py.create_group one by one to replicated the structure in python dictionary. For very large structures, this is unfeasible. Is there an easy way to automatically convert python dictionaries to hdf5 groups?

Thank you!

-Shawn

Answer 1

I needed to do this kind of thing all the time, and decided it would be neat to make a hdf5 version of pickle: https://github.com/telegraphic/hickle

The motivation was storing python dictionaries of numpy arrays, which sounds like what you're after:

import hickle as hkl
import numpy as np
data = {
        'dataset1' : np.zeros((100,100)),
        'dataset2' : np.random.random((100,100))
        }
hkl.dump(data, 'output_filename.hkl')

You should be able to install it via PyPi (pip install hickle), or download it from github.

Cheers Danny