How to inspect program state in the presence of generators/coroutines?

2
votes

With normal functions calls, the program state is mostly described by a simple call stack. It is printed out as a traceback after an uncaught exception, it can be examined with inspect.stack, and it can be displayed in a debugger after a breakpoint.

In the presence of generators, generator-based couroutines, and async def-based coroutines, I don't think the call stack is enough. What's a good way to mentally visualize the program state? How do I inspect it in run-time?

There are functions inspect.getgeneratorstate and inspect.getcoroutinestate, but they only provide information about whether the generator/coroutine is created, running, suspended, or closed. In the case the state is RUNNING, I want to be able to examine the actual line number the generator or coroutine is currently executing and the stack frames that correspond to the other functions it might have called. In the case it's SUSPENDED, I want to examine other generators / coroutines it sent data to or yielded to.

Edit: I found a related question on SO which pointed me to this excellent article that explains everything I asked about in this question.

1
pythonpython-3.xpython-asynciocoroutinepython-internals
Did you try to display stack on breakpoint or call traceback.print_stack() in exception handler inside an async def based coroutine? – Andrew Svetlov
@AndrewSvetlov Yes. All I see in the stack is the event loop at the bottom of the stack, the event handler code in the middle, and the actual async coroutine code at the top of the stack. I see nothing at all about the other coroutines. It seems like there should be another stack or something but I can't even make a clear picture in my mind. – max

1 Answers

2
votes

You just have to findout all instances fo generators and co-routines, in all "traditional" frames - (either search for them recursively in all objects in all frames, or you mitght try to use the garbage collector (gc) module to get a reference to all these instances)

Generators and co-routines have, respectively, a gi_frame and a cr_frame attribute.