python - Start async function without importing the asyncio package

Question

Is is possible to start a function like this

async def foo():
    while True:
        print("Hello!")

without importing the asyncio package (and getting the event loop)?

I am looking for a principle similar to Go's goroutines, where one can launch a coroutine with only go statement.

Edit: The reason why I'm not importing the asyncio package is simply because I think it should be possible to launch coroutine without event loop (explicit). I don't understand why async def and similar statements are part of core language (even part of syntax) and the way to launch created coroutines is available only through package.

Answer 1

Of course it is possible to start an async function without explicitly using asyncio. After all, asyncio is written in Python, so all it does, you can do too (though sometimes you might need other modules like selectors or threading if you intend to concurrently wait for external events, or paralelly execute some other code).

In this case, since your function has no await points inside, it just needs a single push to get going. You push a coroutine by sending None into it.

>>> foo().send(None)
Hello!
Hello!
...

Of course, if your function (coroutine) had yield expressions inside, it would suspend execution at each yield point, and you would need to push additional values into it (by coro.send(value) or next(gen)) - but you already know that if you know how generators work.

import types

@types.coroutine
def bar():
    to_print = yield 'What should I print?'
    print('Result is', to_print)
    to_return = yield 'And what should I return?'
    return to_return

>>> b = bar()
>>> next(b)
'What should I print?'
>>> b.send('Whatever you want')
Result is Whatever you want
'And what should I return?'
>>> b.send(85)
Traceback...
StopIteration: 85

Now, if your function had await expressions inside, it would suspend at evaluating each of them.

async def baz():
    first_bar, second_bar = bar(), bar()
    print('Sum of two bars is', await first_bar + await second_bar)
    return 'nothing important'

>>> t = baz()
>>> t.send(None)
'What should I print?'
>>> t.send('something')
Result is something
'And what should I return?'
>>> t.send(35)
'What should I print?'
>>> t.send('something else')
Result is something else
'And what should I return?'
>>> t.send(21)
Sum of two bars is 56
Traceback...
StopIteration: nothing important

Now, all these .sends are starting to get tedious. It would be nice to have them semiautomatically generated.

import random, string

def run_until_complete(t):
    prompt = t.send(None)
    try:
        while True:
            if prompt == 'What should I print?':
                prompt = t.send(random.choice(string.ascii_uppercase))
            elif prompt == 'And what should I return?':
                prompt = t.send(random.randint(10, 50))
            else:
                raise ValueError(prompt)
    except StopIteration as exc:
        print(t.__name__, 'returned', exc.value)
        t.close()

>>> run_until_complete(baz())
Result is B
Result is M
Sum of two bars is 56
baz returned nothing important

Congratulations, you just wrote your first event loop! (Didn't expect it to happen, did you?;) Of course, it is horribly primitive: it only knows how to handle two types of prompts, it doesn't enable t to spawn additional coroutines that run concurrently with it, and it fakes events by a random generator.

(In fact, if you want to get philosophical: what we did above that manually, could also be called an event loop: Python REPL was printing prompts to a console window, and it was relying on you to provide events by typing t.send(whatever) into it.:)

asyncio is just an immensely generalized variant of the above: prompts are replaced by Futures, multiple coroutines are kept in queues so each of them eventually gets its turn, and events are much richer and include network/socket communication, filesystem reads/writes, signal handling, thread/process side-execution, and so on. But the basic idea is still the same: you grab some coroutines, juggle them in the air routing the Futures from one to another, until they all raise StopIteration. When all coroutines have nothing to do, you go to external world and grab some additional events for them to chew on, and continue.

I hope it's all much clearer now. :-)

Answer 2

Python coroutines are a syntactic sugar for generators, with some added restrictions in their behavior (so that their purpose is explicitly different and doesn't mix). You can't do:

next(foo())
TypeError: 'coroutine' object is not an iterator

because it's disabled explicitly. However you can do:

foo().send(None)
Hello
Hello
Hello
...

Which is equivalent to next() for a generator.

Answer 3

Coroutines should be able to

1. run

2. yield control to caller (optionally producing some intermediate results)

3. be able to get some information from caller and resume

So, here is a small demo of async functions (aka native coroutines) which do it without using asyncio or any other modules/frameworks which provide event loop. At least python 3.5 is required. See comments inside the code.

#!/usr/bin/env python

import types

# two simple async functions
async def outer_af(x):
    print("- start outer_af({})".format(x))
    val = await inner_af(x)  # Normal way to call native coroutine.
                             # Without `await` keyword it wouldn't
                             # actually start
    print("- inner_af result: {}".format(val))
    return "outer_af_result"


async def inner_af(x):
    print("-- start inner_af({})".format(x))
    val = await receiver()  # 'await' can be used not only with native
                            # coroutines, but also with `generator-based`
                            # coroutines!
    print("-- received val {}".format(val))
    return "inner_af_result"


# To yiled execution control to caller it's necessary to use
# 'generator-based' coroutine: the one created with types.coroutine
# decorator
@types.coroutine
def receiver():
    print("--- start receiver")
    # suspend execution / yield control / communicate with caller
    r = yield "value request"
    print("--- receiver received {}".format(r))
    return r

def main():
    # We want to call 'outer_af' async function (aka native coroutine)
    # 'await' keyword can't be used here!
    # It can only be used inside another async function.
    print("*** test started")
    c = outer_af(42)  # just prepare coroutine object. It's not running yet.
    print("*** c is {}".format(c))

    # To start coroutine execution call 'send' method.
    w = c.send(None)  # The first call must have argument None

    # Execution of coroutine is now suspended. Execution point is on
    # the 'yield' statement inside the 'receiver' coroutine.
    # It is waiting for another 'send' method to continue.
    # The yielded value can give us a hint about what exectly coroutine
    # expects to receive from us.
    print("*** w = {}".format(w))

    # After next 'send' the coroutines execution would finish.
    # Even though the native coroutine object is not iterable it will
    # throw StopIteration exception on exit!
    try:
        w = c.send(25)
        # w here would not get any value. This is unreachable.
    except StopIteration as e:
        print("*** outer_af finished. It returned: {}".format(e.value))


if __name__ == '__main__':
    main()

Output looks like:

*** test started
*** c is <coroutine object outer_af at 0x7f4879188620>
- start outer_af(42)
-- start inner_af(42)
--- start receiver
*** w = value request
--- receiver received 25
-- received val 25
- inner_af result: inner_af_result
*** outer_af finished. It returned: outer_af_result

Additional comment. Looks like it's not possible to yield control from inside native coroutine. yield is not permitted inside async functions! So it is necessary to import types and use coroutine decorator. It does some black magic! Frankly speaking I do not understand why yield is prohibited so that a mixture of native and generator-based coroutines is required.

Answer 4

No, that is not possible. You need an event loop. Take a look at what happens if you just call foo():

>>> f = foo()
>>> print(f)
<coroutine object foo at 0x7f6e13edac50>

So you get a coroutine object, nothing get's executed right now! Only by passing it to an event loop does it get executed. You can use asyncio or another event loop like Curio.