Converting from generator-based to native coroutines

Question

I've used Python for many years but only gradually studied the more obscure features of the language, as most of my code is for data processing. Generators based on yield are part of my routine toolkit, and recently I read about coroutines. I found an example similar to this:

def averager():
    sum = 0.0
    n = 0
    while True:
        value = yield
        sum += value
        n += 1
        print(sum/n)

avg = averager()
next(avg) # prime the coroutine
avg.send(3)
avg.send(4)
avg.send(5)

which prints the average of the values sent to it. I figured something like this might come in useful in data processing pipelines so I resolved to keep it in the back of my head. That is, until I read the following notice in the Python documentation:

Support for generator-based coroutines is deprecated and is scheduled for removal in Python 3.10.

Obviously I'd like to write future-proof code so at this point it's probably useless to start learning generator-based coroutines. My question, then, is: How to implement this example using the native (asyncio) coroutines? I have a much harder time wrapping my head around the native coroutine syntax.

While trying to search for an answer, I found a related question which has a comment and an answer that are basically saying "you can't do it with async, do it with yield-based coroutines instead". But if those are going away, is there going to be any way to do this with coroutines in 3.10+?

@RomanPerekhrest: Well, that's my question, isn't it? I'll take "you can't do it" as an answer, but that would mean that the ability to do something like this with coroutines is going away from Python without any replacement, which would be inconvenient. Of course, there are other ways to achieve the same thing, so it's not a huge deal, but unfortunate nonetheless. — Jsl
I think the deprecation notice might only apply to the asyncio.coroutine decorator. The Python 3.10 docs say "Support for generator-based coroutines is deprecated and is removed in Python 3.11". A generator-based coroutine with the handy consumer decorator from PEP 342 still works in Python 3.11. — Brecht Machiels

RomanPerekhrest RomanPerekhrest · Accepted Answer · 2019-11-01T21:50:27

And there come Asynchronous Generators ...

So we still have that power in asynchronous context.
As for theory - the mentioned PEP 525 provides a great description and definitely worth reading.
I'll just post a prepared illustrative example (for asynchronous averager) with initialization and safe finalization included:

import asyncio

async def coro():
    print('running other coroutine in 3 sec ...')
    await asyncio.sleep(3)  # emulate working


async def averager():
    sum_ = n = 0
    while True:
        v = yield
        sum_ += v
        n += 1
        print(sum_ / n)
        await asyncio.sleep(0.1)


async def main():
    agen = averager()
    await agen.asend(None)
    print(agen.__name__, 'initialized ...')

    await agen.asend(3)
    print('another separate processing here ...')

    await coro()

    await agen.asend(4)
    await agen.asend(14)


loop = asyncio.get_event_loop()
try:
    loop.run_until_complete(main())
finally:
    loop.run_until_complete(loop.shutdown_asyncgens())
    loop.close()

Program output:

averager initialized ...
3.0
another separate processing here ...
running other coroutine in 3 sec ...
3.5
7.0

Converting from generator-based to native coroutines

1 Answers