Compile python AST to method

Question

I have been experimenting with ASTs in python. I want to modify methods by transforming the ASTs at runtime.

I can get the source of an pre-compiled method using inspect.getsource(), and I can modify the AST as required by using AST visitors.

This is probably quite naive, but I'd like to be able to compile the AST and do something akin to:

myClass.method.__func__.__code__ = compile(newAST, '<string>', 'exec')

But compile will only accept an AST with an ast.Module as root. Is there a way to either compile just an ast.FunctionDef, or to retrieve the function code object from the compiled (and otherwise empty) Module code?

Any pointers to info covering this sort of thing would be appreciated. The AST examples Ive seen just deal with simple expressions.

I realised that I needed to just exec the module in a namespace, then I'd have access to the normal structure. So the pattern is:

src = inspect.getsource(myClass.myMethod)
astFromSrc = ast.parse(unindent(src))         # need to remove extra indent from method
transform(astFromSrc.body)                    # transform the AST as you need 
ast.fix_missing_locations(astFromSrc)         # fix up line numbers etc
compiled = compile(astFromSrc, '<string>', 'exec') # compile the module AST

#######  From here is the part I was missing
myScope = {}                                  # make an empty namespace
exec compiled in myScope  # now myScope contains a proper compiled function

# Now replace the original with the modified version
myClass.myMethod.__func__.__code__ = myScope['myMethod'].__code__

But now I have another question: How to slot this into the normal python build process so that the modifications are done once only, and thereafter loaded from the .pyc file?

Since you discovered the answer to your question, post it as an answer and accept it. — All Workers Are Essential

M.D. M.D. · Accepted Answer · 2011-08-15T14:57:12

You should not ask a new question in your own answer but create a separate question for that.

Note that your first and second question also sort of contradict. First you want to do things at runtime and later you want to write them to a file so you don't have to do it over and over again. Please state your endgoal is so we know clearly what you want.

It sounds like you might just as well create a new .py file after modifying the ast of a file that you parsed as is explained in Parse a .py file, read the AST, modify it, then write back the modified source code

you will then obtain your pyc file by compiling the newly created py file.

Compile python AST to method

1 Answers