python - Save/dump a YAML file with comments in PyYAML

57
votes

I have a yaml file that looks like this:

# The following key opens a door
key: value

Is there a way I can load and dump this data while maintaining the comment?

3
pythonyamlpyyaml
I once modified the C libyaml code to emit comments for my own use. Extending this to PyYAML is not going to be easy. – David Heffernan
I thought about this again. Does it make sense to parse and write a yaml file which was edited by hand (and will be edited by hand in the future)? Why not split the file into two parts: One is handcrafted and the other part is pure data (without comments). Related: github.com/guettli/programming-guidelines/blob/master/… – guettli

3 Answers

24
votes

PyYAML throws away comments at a very low level (in Scanner.scan_to_next_token).

While you could adapt or extend it to handle comments in its whole stack, this would be a major modification. Dumping (=emitting) comments seems to be easier and was discussed in ticket 114 on the old PyYAML bug tracker.

As of 2020, the feature request about adding support for loading comments is still stalling.

103
votes

If you are using block structured YAML, you can use the python package¹ ruamel.yaml which is a derivative of PyYAML and supports round trip preservation of comments:

import sys
import ruamel.yaml

yaml_str = """\
# example
name:
  # details
  family: Smith   # very common
  given: Alice    # one of the siblings
"""

yaml = ruamel.yaml.YAML()  # defaults to round-trip if no parameters given
code = yaml.load(yaml_str)
code['name']['given'] = 'Bob'

yaml.dump(code, sys.stdout)

with result:

# example
name:
  # details
  family: Smith   # very common
  given: Bob      # one of the siblings

Note that the end-of-line comments are still aligned.

Instead of normal list and dict objects the code consists of wrapped versions² on which the comments attached.

¹ Install with pip install ruamel.yaml. Works on Python 2.6/2.7/3.3+
² ordereddict is used in case of a mapping, to preserve ordering

4
votes

I have a branch of pyyaml that does exactly this. https://github.com/pflarr/pyyaml

To build a yaml file with comments, you have to create an event stream that includes comment events. Comments are currently only allowed before sequence items and mapping keys.

This only currently works for python3, I haven't ported it to the python2 version of the library, but could easily do so on request. Additionally, this should also be fairly easy to port to the C libyaml as well, as the python code is a simple port of that anyway.