python file to ast and back without libs

Question

I am trying to read python file, make some changes (construct product of functions and inline them in one) and write it back to files.

Python provide ast module, which can parse its code to ast, but has no method for reverse transition, I know there are some libraries, but I can't use them. I've tried to compile tree to code object and them get its source, but now my method produce only part of file. What am I doing wrong?

import ast
import inspect


module_path = "python_file.py"
with file(module_path, "r") as f:
    result = ast.parse(f.read(), module_path)
    print(ast.dump(result, include_attributes=True))
    result = compile(result, module_path, mode='exec')
    result = inspect.getsource(result)
print(result)

Giacomo Alzetta Giacomo Alzetta · Accepted Answer · 2018-08-21T15:25:05

The inspect.getsource function looks up the source code from the source file, so it's not going to work for you.

There is actually a tool that is meant to do what you want, i.e. transform python code: the 2to3 tool!

The tool provides a library, lib2to3 that can be used to write "fixers", i.e. pieces of code that change one part of source code into something else.

Unfortunately it is not so well documented, so you'll have to learn how to write a fixer by looking at the standard ones from the python sources.

When you define a fixer you can also specify a PATTERN so that your fixer will only be called when that kind of subtree is matched.

After that you can run the 2to3 tool and specify to only use the fixers you have defined and you end up with exactly what you want.

You should really take a look at Extendind 2to3 with your own fixers it helped me a lot writing fixers to update dependencies/change versions in setup.py scripts and similar.

python file to ast and back without libs

1 Answers